- Newest
- Most votes
- Most comments
The AI bot seems to be mixing things up and making up the rest. I suggest ignoring its response.
When you change the KMS key for a table from KMS-1 to KMS-2, DynamoDB will generate a new table key (say, T2) for the table. Next, it will use the old table key T1 to decrypt the data encryption keys (DEKs) with which the raw data in the table is encrypted, and finally encrypt the old DEKs with the new table key T2. The DEKs will not change, but they will be re-encrypted with T2 instead of T1. The new table key T2 will be the only thing to be encrypted with the new KMS-2 key. Similarly, the only key to be encrypted with the old KMS-1 key will have been the old table key T1.
As long as the KMS-1 key remains accessible for the time DynamoDB needs to decrypt the old table key T1, generate T2, decrypt all the DEKs and re-encrypt each DEK with T2, DynamoDB will only need access to the new KMS key KMS-2 to encrypt the new table key T2. If your table has a small number of DEKs, the process could be very, very quick, even if the absolute amount of data in the table is reasonably large.
This is explained in this documentation section: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/encryption.usagenotes.html#dynamodb-kms
If you change the KMS key for your table, DynamoDB generates a new table key. Then, it uses the new table key to re-encrypt the data encryption keys.
And in the previous section of the same documentation page: https://docs.aws.amazon.com/amazondynamodb/latest/developerguide/encryption.usagenotes.html#encryption.usagenotes.encryptiontypes
When you transition to a new customer managed key, be sure to keep the original key enabled until the process is complete. AWS will still need the original key to decrypt the data before encrypting it with the new key. The process will be complete when the table's SSEDescription Status is ENABLED and the KMSM*sterKeyArn of the new customer managed key is displayed. At this point the original key can be disabled or scheduled for deletion.
Your understanding of how KMS and DynamoDB encryption work together is not entirely accurate, which explains why you're still able to access the data. Here's what's actually happening:
-
DynamoDB uses envelope encryption. When you add data to a DynamoDB table, it doesn't directly encrypt that data with the KMS key. Instead, it uses a table key, which is then encrypted by the KMS key.
-
When you change the KMS key associated with a DynamoDB table, it doesn't immediately re-encrypt all the data or table keys. The old table keys encrypted with the previous KMS key (KMS-1 in your case) remain in use for existing data.
-
DynamoDB caches the decrypted table keys for performance reasons. These cached keys can be used to decrypt data for a period of time, even if the original KMS key is disabled or deleted.
-
When you access the table data, DynamoDB attempts to decrypt the table key using the currently associated KMS key (KMS-2). If that fails, it may fall back to using the cached decrypted table key, allowing you to still access the data.
-
For new data added after changing the KMS key, DynamoDB will use the new KMS key (KMS-2) to encrypt new table keys.
This behavior ensures that changing or deleting a KMS key doesn't immediately render all your data inaccessible, which could be catastrophic for many applications. However, it's important to note that this is a temporary state. Over time, as DynamoDB refreshes its key cache and as new data is added, you may start to see issues accessing older data encrypted with the deleted KMS key.
To properly re-encrypt all data with a new KMS key, you would need to read all items from the table and write them back, which would cause DynamoDB to re-encrypt them using table keys protected by the new KMS key.
In the future, if you need to change KMS keys, it's recommended to keep the old key active until you've confirmed that all data has been successfully re-encrypted with the new key to avoid potential data loss or access issues.
Sources
DynamoDB encryption at rest usage notes - Amazon DynamoDB
DynamoDB encryption at rest: How it works - Amazon DynamoDB
Community | Demystifying AWS KMS key rotation
DynamoDB decrypted table key caching is said to be for 5 mins. But my observation stretches over several days. I was seeing that the data accessible all 7 days during Pending Deletion phase, and several days after KMS-1 is deleted.
Relevant content
- asked 3 years ago
- asked 3 years ago
Thanks Leo, for your help/expert answer. After reading through your answer, the documentation makes some sense. But otherwise, it is a bit inconsistent and confusing, at least to mere mortals like me. Despite knowing the Envelope encryption method, i was got confused. They say 'decrypt the data, but its actually table key to be precisely'.
Thanks Mikel.
Yes, Mikel, you're quite correct. The statements in documentation about "decrypting the data" and "encrypting it with the new key" are needlessly misleading. They should be corrected to state clearly that what's being re-encrypted are the data encryption keys.