Questions tagged with Extract Transform & Load Data
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
I need to create a aes encryption ECB UDF in the redshift. To achieve this, i have imported pycryptodome zip file in the S3 with the name Crypto.zip and create library in the redshift.
When i try to...
1
answers
0
votes
269
views
asked 6 months agolg...
Scenario:
Source table: Glue Data Catalog table **study** crawled from MySQL with columns:
* id (int),
* code (varchar),
* desc (varchar)
* and 2 other columns not used in the job.
Target table:...
0
answers
0
votes
101
views
asked 6 months agolg...
I want to add Confluent Cloud Apache Kafka as a Data source in AWS ETL job to read data stream from Kafka topic.
I created a cluster, topic, AWS SQS source connector and AWS S3 sink connector in...
1
answers
0
votes
351
views
asked 6 months agolg...
When i create a rule to compare arrays directly like here https://docs.aws.amazon.com/glue/latest/dg/data-quality-getting-started.html it works perfectly.
When i try use
```python
...
1
answers
0
votes
273
views
asked 6 months agolg...
In my study case, I have data coming from a relational database (which stores data directly from the product application) and it sends files into S3 by using AWS DMS with CDC logs;
Usually, I manage...
1
answers
0
votes
376
views
asked 6 months agolg...
Glue script job error spark_catalog requires a single-part namespace, but got [glue_catalog, foo]lg...
I am trying to query an iceberg table via the glue data catalog. This works fine in my visual etl job, but when I try to do it in a script, it's throwing an error. It's likely due to some sort of...
1
answers
0
votes
1450
views
asked 6 months agolg...
Hello,
I am upgrading Glue Version from 3.0 to 4.0. Using Hudi as datalake. I am getting below error -
py4j.protocol.Py4JJavaError: An error occurred while calling o973.pyWriteDynamicFrame.
:...
2
answers
0
votes
333
views
asked 6 months agolg...
We would like to know does AWS Glue encrypt data in Cache while processing the data through job.
1
answers
0
votes
175
views
asked 6 months agolg...
background:
Users upload files to an S3 bucket containing their predictions for events under certain circumstances. I want my query results to only show the most recently made prediction for the...
2
answers
0
votes
421
views
asked 7 months agolg...
Hi,
I have created a new AWS Glue visual ETL job with source- PostgreSQL, target- Snowflake.
Output schema option shows correct schema as per the source, but the python script shows all the datatype...
2
answers
1
votes
168
views
asked 7 months agolg...
I am trying to read a table from the same account that i used to create the table , the table is shared to other accounts through lake formation
in the glue job in the source account i get this...
1
answers
0
votes
254
views
asked 7 months agolg...
Hi! I have been searching and playing around with services and cannot seem to find what I need.
I am using the following architecture to guide me in building out my end-to-end solution:...
2
answers
0
votes
316
views
asked 7 months agolg...