Using AWS Glue to transform data from DynamoDB to Redshift.

0

Hello everyone,

I'm new with AWS Glue. I'm doing a lab to move data from DynamoDB to Redshift with AWS Glue and my job does not succeed.
I test several times but none of them works (assume that IAM roles are correct).

With Dynamodb:

  • Do I need to create a connection to dynamodb using JDBC ? (It's weird because at connection access there is no username or password to fill in).
  • I need to create the data target first or just create it while add job?

With Redshift:

  • After creating cluster, should I add table into public schema or just leave it as default?
  • Do I need a S3 Endpoint to complete the lab? ( because It requires a S3 endpoint to test connection successfully)

Any suggestions about this case is a big help for me.
Thanks,
natuan

natuan
demandé il y a 5 ans827 vues
3 réponses
0

I figured it out, in my case I created a data target but not the same database name, just create new job with right database name.

natuan
répondu il y a 5 ans
0

Need a S3 Endpoint (S3 Gateway)

natuan
répondu il y a 5 ans
0

FYI, there's a solution to move from DDB to Redshift to create source-similar tables without the use of glue:

https://repost.aws/articles/ARlyMMJAeUQLWZd8p3c1Jz6A/automated-change-data-capture-cdc-data-ingestion-from-dynamodb-to-redshift

profile pictureAWS
EXPERT
Rajiv_G
répondu il y a 5 mois

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions