Questions tagged with Extract Transform & Load Data
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hi, I'm relatively new to AWS glue and i'm having trouble with Glue ETL error.
What makes it strange is that this error is only on Dev env but not on Test env. Same code & configuration!
Also tried to...
1
answers
0
votes
264
views
asked 5 months agolg...
Hey all, I have been trying to perform a simple S3 to redshift data push using an S3 source node and a Amazon Redshift target node.
I have been getting errors such as
'Failed to connect to IP...
4
answers
0
votes
339
views
asked 5 months agolg...
I have a MySQL database in AWS RDS. I want to import bulk data in to table in database. which contains thousands of rows.
I need to do it from my website with also deployed in AWS. Web is developed...
1
answers
0
votes
229
views
asked 5 months agolg...
I am building a data pipeline to Load data into Redshift from an S3 data lake.
Data are stored in Parquet format on S3 and I would like to load them into the respective Redshift tables using an AWS...
1
answers
0
votes
396
views
asked 5 months agolg...
I need to create a aes encryption ECB UDF in the redshift. To achieve this, i have imported pycryptodome zip file in the S3 with the name Crypto.zip and create library in the redshift.
When i try to...
1
answers
0
votes
248
views
asked 5 months agolg...
Scenario:
Source table: Glue Data Catalog table **study** crawled from MySQL with columns:
* id (int),
* code (varchar),
* desc (varchar)
* and 2 other columns not used in the job.
Target table:...
0
answers
0
votes
97
views
asked 5 months agolg...
I want to add Confluent Cloud Apache Kafka as a Data source in AWS ETL job to read data stream from Kafka topic.
I created a cluster, topic, AWS SQS source connector and AWS S3 sink connector in...
1
answers
0
votes
326
views
asked 6 months agolg...
When i create a rule to compare arrays directly like here https://docs.aws.amazon.com/glue/latest/dg/data-quality-getting-started.html it works perfectly.
When i try use
```python
...
1
answers
0
votes
258
views
asked 6 months agolg...
In my study case, I have data coming from a relational database (which stores data directly from the product application) and it sends files into S3 by using AWS DMS with CDC logs;
Usually, I manage...
1
answers
0
votes
357
views
asked 6 months agolg...
Glue script job error spark_catalog requires a single-part namespace, but got [glue_catalog, foo]lg...
I am trying to query an iceberg table via the glue data catalog. This works fine in my visual etl job, but when I try to do it in a script, it's throwing an error. It's likely due to some sort of...
1
answers
0
votes
1288
views
asked 6 months agolg...
Hello,
I am upgrading Glue Version from 3.0 to 4.0. Using Hudi as datalake. I am getting below error -
py4j.protocol.Py4JJavaError: An error occurred while calling o973.pyWriteDynamicFrame.
:...
2
answers
0
votes
301
views
asked 6 months agolg...
We would like to know does AWS Glue encrypt data in Cache while processing the data through job.
1
answers
0
votes
163
views
asked 6 months agolg...