Questions tagged with Extract Transform & Load Data
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hi! I have been searching and playing around with services and cannot seem to find what I need.
I am using the following architecture to guide me in building out my end-to-end solution:...
2
answers
0
votes
323
views
asked 7 months agolg...
When executing a task the last step is validating the data migrated with the source against target apparently using Athena, I have the following error:
2023-11-07T22:09:04 [VALIDATOR_TARGE ]E: Not...
1
answers
1
votes
773
views
asked 7 months agolg...
I'm looking for an open-source solution that can help us make our python API more accessible.
For simplicity's sake, the data is accessed using Athena and has three string fields A, B, C.
Every...
0
answers
0
votes
150
views
asked 7 months agolg...
When I´m about to start an ETL Job, usually I ask some main questions:
1. Where the original file/table is stored?
2. What should I do to delivery data to my end goal?
If I have already all the data...
1
answers
0
votes
461
views
asked 7 months agolg...
We have a job (Jupyter notebook job) version 4 that we are trying to run in concurrent mode changing some of the parameters and running via AWS CLI
like below
```
aws glue start-job-run --job-name...
1
answers
0
votes
701
views
asked 7 months agolg...
Hello everyone, I just started using Glue so forgive me if the question is stupid or I'm not providing the correct information to solve the problem. I've been facing this issue for the past two days...
2
answers
0
votes
1002
views
asked 8 months agolg...
Hi,
The following CTAS query fails with Col not found error.
```
CREATE table <table_name>
with(
format='PARQUET'
, write_compression='SNAPPY'
, partitioned_by=ARRAY["yearMonth"]
, external_location...
1
answers
0
votes
312
views
asked 8 months agolg...
Hi all,
I'd like to check if anyone has ever had any feedback from AWS regarding the current behavior of job bookmarks when you're sourcing from a database view.
What the documentation explains...
1
answers
0
votes
364
views
asked 8 months agolg...
I have been attempting to follow [the documentation](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-libraries.html) for developing Glue jobs locally.
Upon running the recommended...
2
answers
1
votes
917
views
asked 8 months agolg...
We have a Glue job that accepts the parameter "table_name," with the default value set as "dummy" in the Glue job parameters section. Additionally, the Glue job configuration allows a total of 4...
1
answers
0
votes
522
views
asked 8 months agolg...
Hi everyone.
Should crawler update table schema if datasourse schema is changed?
For example, I have some parquet file with data. One field has datatype "double".
Parquet file is created by Glue Job....
2
answers
0
votes
480
views
asked 8 months agolg...
I am planning to move all the filtered logs from CloudWatch log group through Kinesis Firehose to an S3 bucket in parquet files.
Given that CloudWatch log group always pushes gzipped data to Kinesis...
2
answers
1
votes
586
views
asked 8 months agolg...