Questions tagged with Extract Transform & Load Data
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hi,
**Problem statement:** Currently if we're running a query with multiple temporary tables and want to rerun the whole query without errors, it is necessary to drop the temporary tables. Usually...
1
answers
4
votes
224
views
asked 10 months agolg...
I'm using aws glue's visual etl builder to extract data from one rds postgres DB, transform it, then insert it into another rds postgres db (similar business entities but different schema).
The...
2
answers
0
votes
1070
views
asked 10 months agolg...
What are the pros and cons when it comes to using **AWS Glue** over Redshift's internal functions (such as **COPY** and **INSERT**)? for bulk data loading (In terms of **cost, time, and...
1
answers
0
votes
366
views
asked 10 months agolg...
I'm building a glue job for a data migration with schema changes and mappings between two RDS postgres DBs.
I'm getting an error connecting to glue apis.
```
AnalysisException:...
1
answers
0
votes
297
views
asked 10 months agolg...
Trying to write to my OpenSearch cluster
"An error occurred while calling o265.pyWriteDynamicFrame. Connection error (check network and/or proxy settings)- all nodes failed; tried...
1
answers
0
votes
411
views
asked 10 months agolg...
using aws for data visualization using custom python scripts and the best database for the usecase.lg...
I am sending data into kinesis data streams and thinking of using firehose to send data into Redshift, i want to use this database to run custom Python scrips to visualize data.
1
answers
0
votes
320
views
asked 10 months agolg...
Hi everyone, I was trying to ingest csv data to Timestream Db with AWS SDK boto3.M y credential , region , Database_Name, Table_Name are all correct but still I am unable to connect to endpoint of my...
0
answers
0
votes
81
views
asked a year agolg...
Json data is being considered as string while loading data from postgres to json file by AWS Gluelg...
I want to migrate postgres data to redshift, but I have a lot of jsonb data in postgres so for that I had given SUPER data type in Redshift but the problem here is while loading the data to redshift...
1
answers
0
votes
1013
views
asked a year agolg...
The setup we currently have is a kafka cluster with various topics; 1 topic per data structure, and a default consumer that runs on all of these topics. The consumer simply takes the persisted data...
2
answers
0
votes
269
views
asked a year agolg...
I signed up for the free trial, however, while trying to run ETL Jobs for learning, I was charged for AWS Glue Jobrun. It was a mistake since it is a thing that I cannot oversee. I tried to run the...
2
answers
0
votes
281
views
asked a year agolg...
I am using Data Quality to evaluate the dataset and I am routing the failed rules and failed records. But when i check the s3 bucket for the failed records i am seeing empty files along with the...
0
answers
0
votes
118
views
asked a year agolg...
Hi, I'm running a data pipeline from legacy DB(oracle) to Redshift using AWS Glue.
I want to test the connection to the legacy DB before executing ETL without test query in working python...
1
answers
0
votes
278
views
asked a year agolg...