What is the best ETL tool for ongoing loads of data into Redshift?

0

What would be the best AWS ETL tool that a customer would use to set up ongoing loads of data into Redshift, that would provide some sort of transform functionality, similar to Microsoft SSIS? E.g. “load data from this file into this table daily as a full replace, compute these columns, etc.”

2 réponses
1
Réponse acceptée

Depending on the customer's anticipated timeframe, this is precisely what Glue (https://aws.amazon.com/glue/ ) is intended to be really good at.

Alternatively, if your customer can persuade their data source to stream its content (what is their data source, btw?), Kinesis with its capability to trigger Lambda functions for the "TL" piece, may be a good fit - see eg https://aws.amazon.com/blogs/big-data/tag/aws-lambda/ .

AWS
EXPERT
répondu il y a 8 ans
profile picture
EXPERT
vérifié il y a 6 mois
0

Is it possible to run an advanced SQL query on the Glue job? I have at least 15 tables in my SQL and the query is quite advanced itself. Doesn't Glue work only with a small number of tables like e.g. 1-3 with simple conditions? Is there an option to run my own query on it, without building the query by using boxes in Job?

répondu il y a 2 ans

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions