Questions tagged with Extract Transform & Load Data
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hey all, I have been trying to perform a simple S3 to redshift data push using an S3 source node and a Amazon Redshift target node.
I have been getting errors such as
'Failed to connect to IP...
4
answers
0
votes
329
views
asked 5 months agolg...
I have a MySQL database in AWS RDS. I want to import bulk data in to table in database. which contains thousands of rows.
I need to do it from my website with also deployed in AWS. Web is developed...
1
answers
0
votes
225
views
asked 5 months agolg...
I am building a data pipeline to Load data into Redshift from an S3 data lake.
Data are stored in Parquet format on S3 and I would like to load them into the respective Redshift tables using an AWS...
1
answers
0
votes
379
views
asked 5 months agolg...
I need to create a aes encryption ECB UDF in the redshift. To achieve this, i have imported pycryptodome zip file in the S3 with the name Crypto.zip and create library in the redshift.
When i try to...
1
answers
0
votes
239
views
asked 5 months agolg...
Scenario:
Source table: Glue Data Catalog table **study** crawled from MySQL with columns:
* id (int),
* code (varchar),
* desc (varchar)
* and 2 other columns not used in the job.
Target table:...
0
answers
0
votes
96
views
asked 5 months agolg...
I want to add Confluent Cloud Apache Kafka as a Data source in AWS ETL job to read data stream from Kafka topic.
I created a cluster, topic, AWS SQS source connector and AWS S3 sink connector in...
1
answers
0
votes
317
views
asked 5 months agolg...
When i create a rule to compare arrays directly like here https://docs.aws.amazon.com/glue/latest/dg/data-quality-getting-started.html it works perfectly.
When i try use
```python
...
1
answers
0
votes
249
views
asked 5 months agolg...
In my study case, I have data coming from a relational database (which stores data directly from the product application) and it sends files into S3 by using AWS DMS with CDC logs;
Usually, I manage...
1
answers
0
votes
343
views
asked 5 months agolg...
Glue script job error spark_catalog requires a single-part namespace, but got [glue_catalog, foo]lg...
I am trying to query an iceberg table via the glue data catalog. This works fine in my visual etl job, but when I try to do it in a script, it's throwing an error. It's likely due to some sort of...
1
answers
0
votes
1227
views
asked 6 months agolg...
Hello,
I am upgrading Glue Version from 3.0 to 4.0. Using Hudi as datalake. I am getting below error -
py4j.protocol.Py4JJavaError: An error occurred while calling o973.pyWriteDynamicFrame.
:...
2
answers
0
votes
289
views
asked 6 months agolg...
We would like to know does AWS Glue encrypt data in Cache while processing the data through job.
1
answers
0
votes
159
views
asked 6 months agolg...
background:
Users upload files to an S3 bucket containing their predictions for events under certain circumstances. I want my query results to only show the most recently made prediction for the...
2
answers
0
votes
385
views
asked 6 months agolg...