By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

can aws_s3.query_export_to_s3 upload s3 without encryption with the 'aws/s3' key?

0

The default behavior of 'aws_s3.query_export_to_s3' is problematic for cross account s3 object share. If RDS is in acountA, 'aws_s3.query_export_to_s3' will encrypt data with accountA's 'aws/s3' key. Users from other AWS account doesn't have access to AccountA's 'aws/s3' key therefore can not access the data exported by rds.

Now I'm wondering is there a way to make 'aws_s3.query_export_to_s3' to upload unencrypted data and s3 can use the custom key for sse. And we can share the custom KMS key with users from other account so that they can read the s3 data properly

Topics
StorageSecurity, Identity, & ComplianceMigration & ModernizationAnalyticsDatabase
Tags
Amazon Simple Storage ServiceAWS Key Management ServiceAmazon Relational Database Service
Language
English
shaohong
asked a year ago1492 views
1 Answer
1
Accepted Answer

Hello,

I understand that you are trying to export data from RDS to S3 using aws_s3 extension and you have noticed that it always uses a default AWS managed KMS key to encrypt the data even though the default KMS Key is set on S3 bucket for encryption.

According to the official documentation, The upload to Amazon S3 uses server-side encryption by default. If you are using encryption, the Amazon S3 bucket must be encrypted with an AWS managed key. Currently, you can't export data to a bucket that is encrypted with a customer managed key.

[+] Exporting data from an RDS for PostgreSQL DB instance to Amazon S3: https://docs.aws.amazon.com/AmazonRDS/latest/UserGuide/postgresql-s3-export.html#postgresql-s3-export-examples

To start with, I would like you to know that when the "aws_s3.query_export_to_s3" function is used to export the table data from RDS PostgreSQL instance to S3 bucket, the object ( i.e. the file containing the table data ) will be encrypted with the default AWS Managed Key for S3 (SSE-KMS encryption) i.e "aws/s3” irrespective of

  1. whether the RDS PostgreSQL Instance is encrypted or not
  2. OR if the target S3 bucket is encrypted or not

Hence, Currently the option to use S3 managed keys or Customer Managed Keys for encryption is not available. Having said that, I would like to inform you that currently the only workaround available is to encrypt the files again using the desired KMS key. The following blog can be followed for executing this workaround:

[+] Encrypting existing Amazon S3 objects with the AWS CLI: https://aws.amazon.com/jp/blogs/storage/encrypting-existing-amazon-s3-objects-with-the-aws-cli/

I understand that the lack of native feature to achieve your use-case might be causing inconvenience at your end. After checking internally I could see there is already a feature request opened to provide option to the customer to choose their own encryption keys through an optional parameter within the pg command, such that they could encrypt with SSE-S3 or CMK or provide Key ARN as well.

I have added your voice to the feature request to help our internal team to prioritize this request. However, I cannot provide an ETA at this time as all the new releases needs to undergo the stages of software development cycle and rigorous testing before it is rolled out for our customers.

I would recommend you to watch out for updates on the following links where new features and bug fixes from AWS are announced:

[+] https://aws.amazon.com/new/ [+] https://aws.amazon.com/about-aws/whats-new/database/ [+] https://aws.amazon.com/blogs/database/

On behalf of AWS, I sincerely apologize for the inconvenience caused to you because of this limitation.

Wish you an Awesome day ahead and stay safe !

AWS
Akash_M
answered a year ago
  • shaohong
    a year ago

    @Akash_M Thanks a lot for pointing out the query_export_to_s3 documentation where it says "The upload to Amazon S3 uses server-side encryption by default. If you are using encryption, the Amazon S3 bucket must be encrypted with an AWS managed key. Currently, you can't export data to a bucket that's encrypted with a customer managed key."

    If I understand it correctly, this functionality is only using 'sse-s3' option and currently it is impossible to use the 'sse-kms' mechanism mentioned in s3's documentation: https://docs.aws.amazon.com/AmazonS3/latest/userguide/UsingServerSideEncryption.html

    And thanks for confirming such feature request is already in place for 'aws_s3.query_export_to_s3' to use a key other than the 'aws/s3' key (which is managed by s3 and users from external aws account can not access).

    Further about the temporary workaround, if I understand it correctly, we would need some piece of code running in the aws account of the rds to do a re-encryption, with a sharable-key, on the exported data, and then users from external account should be able to access the exported data. I'll give that a try!

    Thanks a lot for the explanation and the tips.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content