- Newest
- Most votes
- Most comments
You're correct that the DynamoDB Import from S3 feature has a limitation - it only creates new tables and doesn't support importing into existing tables. This is indeed a challenge when you want to maintain your CDK-managed infrastructure.
For your scenario where you need to import data into existing CDK-managed tables in account 2, you have a few options:
-
AWS Data Pipeline approach: You can create a pipeline in account 2 to import the data from your S3 bucket into your existing DynamoDB tables. This was the traditional method before newer features were introduced. This approach allows you to import into existing tables but will consume provisioned throughput on your target tables.
-
Temporary solution with table replacement: You could:
- Import the data from S3 to create new tables in account 2
- Update your CDK code to reference these imported tables instead of creating new ones
- Once your CDK recognizes the existing tables, you can continue managing them through CDK
- Custom ETL process: Develop a custom solution using AWS services like Lambda or AWS Glue to read the data from S3 and write it to your existing DynamoDB tables. This gives you more control over the process but requires additional development.
Each approach has trade-offs in terms of development effort, cost, and impact on your infrastructure-as-code practices. The best approach depends on factors like the size of your data, your timeline, and your team's expertise.
If maintaining CDK control is critical, you might want to consider the custom ETL approach, as it allows you to keep your existing table structure while still importing the data from S3.
Sources
Using DynamoDB with Amazon S3 to export and import table data - AWS Prescriptive Guidance
Migrating a DynamoDB table from one account to another - Amazon DynamoDB
Using AWS Data Pipeline - AWS Prescriptive Guidance
In CDK you must create the table while pointing it to the S3 Source using importSource: https://docs.aws.amazon.com/cdk/api/v2/docs/aws-cdk-lib.aws_dynamodb.Table.html#importsource
The alternative approach build the tables with S3 data manually, then import them to the stack using cdk import: https://docs.aws.amazon.com/cdk/v2/guide/ref-cli-cmd-import.html
Relevant content
asked a year ago
