Questions tagged with Extract Transform & Load Data
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Glue script job error spark_catalog requires a single-part namespace, but got [glue_catalog, foo]lg...
I am trying to query an iceberg table via the glue data catalog. This works fine in my visual etl job, but when I try to do it in a script, it's throwing an error. It's likely due to some sort of...
1
answers
0
votes
1308
views
asked 6 months agolg...
Hello,
I am upgrading Glue Version from 3.0 to 4.0. Using Hudi as datalake. I am getting below error -
py4j.protocol.Py4JJavaError: An error occurred while calling o973.pyWriteDynamicFrame.
:...
2
answers
0
votes
304
views
asked 6 months agolg...
We would like to know does AWS Glue encrypt data in Cache while processing the data through job.
1
answers
0
votes
163
views
asked 6 months agolg...
background:
Users upload files to an S3 bucket containing their predictions for events under certain circumstances. I want my query results to only show the most recently made prediction for the...
2
answers
0
votes
399
views
asked 6 months agolg...
Hi,
I have created a new AWS Glue visual ETL job with source- PostgreSQL, target- Snowflake.
Output schema option shows correct schema as per the source, but the python script shows all the datatype...
2
answers
1
votes
157
views
asked 6 months agolg...
I am trying to read a table from the same account that i used to create the table , the table is shared to other accounts through lake formation
in the glue job in the source account i get this...
1
answers
0
votes
238
views
asked 6 months agolg...
Hi! I have been searching and playing around with services and cannot seem to find what I need.
I am using the following architecture to guide me in building out my end-to-end solution:...
2
answers
0
votes
157
views
asked 6 months agolg...
When executing a task the last step is validating the data migrated with the source against target apparently using Athena, I have the following error:
2023-11-07T22:09:04 [VALIDATOR_TARGE ]E: Not...
1
answers
1
votes
563
views
asked 6 months agolg...
I'm looking for an open-source solution that can help us make our python API more accessible.
For simplicity's sake, the data is accessed using Athena and has three string fields A, B, C.
Every...
0
answers
0
votes
144
views
asked 7 months agolg...
When I´m about to start an ETL Job, usually I ask some main questions:
1. Where the original file/table is stored?
2. What should I do to delivery data to my end goal?
If I have already all the data...
1
answers
0
votes
408
views
asked 7 months agolg...
We have a job (Jupyter notebook job) version 4 that we are trying to run in concurrent mode changing some of the parameters and running via AWS CLI
like below
```
aws glue start-job-run --job-name...
1
answers
0
votes
617
views
asked 7 months agolg...
Hello everyone, I just started using Glue so forgive me if the question is stupid or I'm not providing the correct information to solve the problem. I've been facing this issue for the past two days...
2
answers
0
votes
901
views
asked 7 months agolg...