What's the best practice to store encrypted key with data when we store in Andes

Question

Hi,

My team uses KMS key to do column level encryption for our dataset storing in Andes table, we use a cradle job to encrypt and write the output to Andes table. The cradle job will be set to run backfill, every timr it runs, a batch of new data will be encrypted and add to the same Andes table. Since KMS key generate random plain data text key everytime we call it, we'll need to associate each row of data with the key to encrypt it so we'll be able to decrypt using the same key. [KMS documentation](https://docs.aws.amazon.com/kms/latest/APIReference/API_GenerateDataKey.html) recommends to store the encrypted data key with the encrypted data, but didn't more specified guidance on use case like ours. So anyone has handled similar use case? Any recommendation on how to store the encrypted key?

Answer

According to the [AWS Encryption SDK](https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/introduction.html) [FAQ](https://docs.aws.amazon.com/encryption-sdk/latest/developer-guide/faq.html):

**How do I keep track of the data keys that were used to encrypt my data?**
```
The AWS Encryption SDK does this for you. When you encrypt data, the SDK encrypts
 the data key and stores the encrypted key along with the encrypted data in the 
encrypted message that it returns. When you decrypt data, the AWS Encryption 
SDK extracts the encrypted data key from the encrypted message, decrypts it, 
and then uses it to decrypt the data.
```
**How does the AWS Encryption SDK store encrypted data keys with their encrypted data?**
```
The encryption operations in the AWS Encryption SDK return an encrypted message, 
a single data structure that contains the encrypted data and its encrypted data keys. 
The message format consists of at least two parts: a header and a body. The message 
header contains the encrypted data keys and information about how the message 
body is formed. The message body contains the encrypted data. If the algorithm suite 
includes a digital signature, the message format includes a footer that contains the 
signature. For more information, see AWS Encryption SDK message format reference.
```

What's the best practice to store encrypted key with data when we store in Andes

Relevant content