End-to-end encryption is the foundation of WhatsApp’s private messaging, ensuring that only you and the individual with whom you’re communicating can read the message, and nobody else, not even WhatsApp. It is one of the most popular implementations of end-to-end encryption and is based on the public key cryptography developed in the 1970s. Technically speaking, for end-to-end encryption to be trusted, the “ends” of a conversation must be aware of the authenticity and validity of one another’s encryption keys.
To do so, our most security-conscious users have always had access to our security code verification feature, which is accessible via a user’s contact information. Keys can be validated in person with a rapid scan of a QR code, or remotely with a 60-digit code.
This is one of the most reliable methods for determining whether a connection is secure. In actuality, however, we recognize that double-checking a lengthy code is time-consuming, and our team has been searching for a solution for some time.
We are pleased to introduce a new cryptographic security feature that will automatically validate a secure connection without requiring this lengthy code. In order to accomplish this, we are developing a novel Auditable Key Directory (AKD) that is based on an open-source library. The AKD enables WhatsApp clients to automatically validate the authenticity of a user’s encryption key and enables anyone to validate the directory’s audit proofs.
Our two-pronged approach to key transparency introduces two new components:
- WhatsApp’s server maintains an append-only AKD of public keys associated with user accounts.
- A third-party audit record in which any change to the server directory is recorded in a verifiable, publicly accessible audit record that protects privacy.
With these two additions, WhatsApp users can automatically verify the confidentiality of their conversations. As this feature is carried out, security-minded users who visit the page to verify their security code will observe that the verification process occurs rapidly and automatically.
This is a new WhatsApp service that utilizes public auditing to validate the end-to-end encryption status of private conversations. While this system provides our users with simple and convenient verification tools, those who wish to verify their end-to-end encrypted sessions without using WhatsApp servers are encouraged to use the traditional security code verification process in addition to this new automated process.
The public keys are merely a means for users to encrypt messages. On user devices is the private key, which is used to decrypt messages. No one, including WhatsApp, has access to these private credentials. A list of public keys cannot grant access to anyone’s content on its own.
The operation of the “Verify Security Code” page
Public/private key combinations are central to end-to-end encrypted communication. You use the private key to decrypt communications sent by a third party, and it never leaves your device. However, the public key is what you share with others so they can encrypt messages. This is accomplished by providing the key to WhatsApp, which will retain it on your behalf and distribute it to users who desire to send you a message.
The classic threat that end-to-end encryption was designed to protect against is a person-in-the-middle attack, in which you believe you are communicating with a single user but are actually communicating with a middle-man attacker who provides an incorrect public key so that they can read your messages using the private key. The perpetrator may then re-encrypt the message with the correct public key for your contact and transmit it to the user.
What prevents this now? Each WhatsApp contact has a Security Page with a QR code and a 60-digit number that can be verified outside of WhatsApp to ensure that it matches what the contact sees on their device. It is a unique hash of both your and their public keys, so if either of you has the incorrect value, the hashes will not match. When they match, a secure, end-to-end encrypted conversation is confirmed.
What issue does key transparency resolve?
While providing a high level of security, the QR code scanning/number matching feature requires you to communicate with your contacts outside of WhatsApp – over a video call, in person, etc. Which is:
- Difficult to achieve in 1:1 communications, particularly as users alter their devices (and, consequently, their encryption keys) over time.
- Even more difficult in small groups, as each participant dyad has a unique code (there are no “group” codes);
- It is extremely difficult to perform in sizable groups. Each time a participant joins or departs, enrolls a new companion device, changes their phone, etc., this must be performed again. In a group of 100 individuals, there would be 4,950 pairs of security checks.
Ideally, this would not be a manual procedure and could be validated via an automated sequence.
Enter key transparency: a protocol that establishes an AKD on WhatsApp that keeps a record of public key changes. In addition, we have created a third-party public repository of auditable change logs for the directory, which is updated whenever the directory is expanded. This is essential for transparency and for bolstering our end-to-end encryption guarantee. In effect, this verifies that the same public keys used by a user to communicate with a recipient are also used by everyone else to communicate with the recipient.
Despite the fact that key transparency is not a replacement for QR code scanning, it enhances and supplements it in the following ways:
Two individuals are required to coordinate out-of-band verification when scanning QR codes. Key transparency, on the other hand, only requires a single client to initiate and execute a check against the directory, thereby enhancing the accessibility of the check process.
Key transparency serves as a public key consistency mechanism when manual QR code verification is impractical (such as in a large group communication scenario); it also serves as a lightweight first check of end-to-end encryption, which increases the adoption of end-to-end encryption checks by more users, thereby improving messaging security in general.
If the automatic check indicates that the connection may not be secure, we recommend that users perform the manual security verification check.
The History of crucial openness
The term “key transparency” refers to a protocol wherein the server maintains an append-only record of the mapping between a user’s account and their public identity key. This enables the compilation of inclusion proofs to assert that a particular mapping existed in the directory as of the most recent update.
WhatsApp’s implementation of key transparency is based on the original academic works on key transparency, beginning with CONIKS and SEEMless, and augmented by a recent paper titled Parakeet. This culminated in the creation of the Rust AKD crate, which serves as the foundation for sustaining a key transparency solution and producing inclusion and key history proofs from the directory. This AKD directory is hosted by WhatsApp as infrastructure accessible to all users.
Public keys cannot be used to decrypt a user’s messages or ascertain with whom the user has been communicating. However, they are required to ensure that a message is sent to the intended recipient by encrypting messages that only the owner of the associated private key can decrypt.
A user may have multiple entries as their key is updated over time. At WhatsApp’s scope, this equates to an ever-growing number of billions of entries. When a user deletes their account, we remove all public keys associated with that account, but the fact that a key existed at some point in time is immutable (we can’t say what the key was).
How is key transparency implemented?
Assurance of principle
Multiple factors influenced our decision to improve this project’s accessibility and safety, which stemmed from a fundamental design decision. First, the entire AKD proof generation and verification logic is available as open-source code. This is a Rust-based crate (library) for any entity that wishes to manage an append-only directory with a publicly verifiable log, verify append-only audit proofs, and act as a public auditor of WhatsApp’s key transparency solution. A list of public keys cannot grant access to anyone’s content on its own.
This library enables the system to provide a strong guarantee for the correctness of directory entries without compromising security by exposing it to memory-based attacks. In addition, we maintained our decision to implement Rust in the majority of the internal components described below.
Using AKD on WhatsApp High-volume key modifications
Per minute, WhatsApp processes tens of thousands of key changes (registration, re-registration, etc.). This volume is challenging to manage when attempting to insert into an append-only log.
Therefore, we decided to implement a distributed, high-throughput queue where “pending changes” reside before being gathered in a bundle and incorporated into the next epoch. This permits much larger group inserts and drastically reduces the number of database operations we must perform.
Since AKD modifications are additive based on the preceding epoch, we must ensure that only a single update occurs at any given time. Regardless of the database implementation, a single processor processing each update sequentially would be unable to keep up with the rate of change within WhatsApp. This increases the delay between when a key is inserted or updated and when it is “published” in the directory.
By grouping keys together and defining an epoch as a collection of atomically committed changes, we can benefit from numerous query optimizations due to a large number of shared paths in the database-stored Merkle Tree. The frequency at which new epochs are published and emitted is a parameter that can be adjusted over time.
Public auditing on a grand scale
The general requirement for all transparency solutions is that they be publicly auditable, i.e., anyone can validate the transactions on the directory to confirm that:
The past has not been altered (no existing documents have been deleted or modified).
Changes can only be appended.
When publishing a new change to the AKD, an audit proof of those changes is generated and stored in a public repository for anyone who is interested. These audit records preserve the privacy of all directory users while guaranteeing the immutability and verifiability of the directory’s history.
No one’s actual information is at risk of becoming public, nor are any user behavior patterns revealed. In SEEMless and Parakeet, the academic works on which key transparency is founded, you can learn more about how this privacy guarantee operates.
Important WhatsApp transparency launch
Key transparency solutions strengthen the assurance that end-to-end encryption provides for private instant messaging applications in an open and transparent manner. This technology supports WhatsApp’s dedication to and leadership in the security domain.
WhatsApp already hosts and operates an AKD for all of our users, regardless of the application version or platform they employ. In the future months, Android users who utilize the verify security code function will begin to observe that the verification is automatic. This is a crucial mechanism that enables security-conscious users to rapidly validate an end-to-end encrypted private conversation.
Soon, a whitepaper with a deeper technical focus will be published, detailing potential attacks, additional information on data flows and formats, and more.
APN Settings Team