Using AWS Glue to transform data from DynamoDB to Redshift.

Question

Hello everyone,  
  
I'm new with AWS Glue. I'm doing a lab to move data from DynamoDB to Redshift with AWS Glue and my job does not succeed.   
I test several times but none of them works (assume that IAM roles are correct).  
  
With Dynamodb:  
- Do I need to create a connection to dynamodb using JDBC ? (It's weird because at connection access there is no username or password to fill in).  
- I need to create the data target first or just create it while add job?  
  
With Redshift:   
- After creating cluster, should I add table into public schema or just leave it as default?  
- Do I need a S3 Endpoint to complete the lab? ( because It requires a S3 endpoint to test connection successfully)  
  
Any suggestions about this case is a big help for me.  
Thanks,  
natuan

Answer

Need a S3 Endpoint (S3 Gateway)

Answer

I figured it out, in my case I created a data target but not the same database name, just create new job with right database name.

Answer

FYI, there's a solution to move from DDB to Redshift to create source-similar tables without the use of glue:

https://repost.aws/articles/ARlyMMJAeUQLWZd8p3c1Jz6A/automated-change-data-capture-cdc-data-ingestion-from-dynamodb-to-redshift

Using AWS Glue to transform data from DynamoDB to Redshift.

Relevanter Inhalt