Questions tagged with Extract Transform & Load Data
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hi all,
I'm trying to connect to an external MariaDB database instance using a AWS Glue Spark script and a JDBC Glue connection.
The code snippet from the Spark script is:
dyf =...
1
answers
0
votes
170
views
asked 7 months agolg...
I'm using DMS to capture CDC from an RDS PostgreSQL Database, then writing the changes to a Kinesis Data Stream and finally using a Glue Streaming Job to process the data and write it to a Hudi Data...
2
answers
0
votes
372
views
asked 7 months agolg...
I am currently using a Glue job to read data from one Amazon S3 source, perform some transformations and write the transformed data into another S3 bucket in parquet format. While writing data to the...
1
answers
0
votes
452
views
asked 7 months agolg...
Hi,
I am trying to migrate a table from Postgres to Redshift using a migration task
Simplified table structure:
| Name | Type |
| --- | --- |
| id | integer |
| time | timestamp with time zone |
|...
0
answers
0
votes
106
views
asked 7 months agolg...
In a glue job that is using bookmarks, I'm including the transformation_ctx parameter in each of the create dynamic frame methods (where I read data).
If I then do a join and a select and then an...
1
answers
0
votes
365
views
asked 7 months agolg...
I have a Glue job that performs a column mapping (a different question question!), the job fails at the final stage where it is time to persist the results back to the...
3
answers
0
votes
504
views
asked 7 months agolg...
My Glue 4.0 jobs have suddenly stopped working with error message below. As it is related to boto3, I am unable to make any changes to library config. Pls advise.
NB: I noticed that urllib3 released...
0
answers
0
votes
95
views
asked 7 months agolg...
I have converted a json format file in parquet, I can see the parquet file and the columns, but while querying with Athena getting error.
HIVE_UNKNOWN_ERROR: Path is not absolute:...
1
answers
0
votes
270
views
asked 7 months agolg...
1. **Spun up an EMR instance:**
emr-6.10.0
Spark 3.3.1, HBASE 2.4.15, Hive 3.1.3, JupyterHub 1.5.0, Hadoop 3.3.3, ZooKeeper 3.5.10, Zeppelin 0.10.1, Phoenix 5.1.2, Presto 0.278,
...
1
answers
1
votes
271
views
asked 7 months agolg...
hi team, can I ask why Glue is generating so many parquet files from my ETL job?
![Enter image description here](/media/postImages/original/IM6V7UVsE-QSi5AEKRNdOqkQ)
![Enter image description...
2
answers
0
votes
317
views
I am using AWS Glue and using the Glue Console to create ETL jobs for data transfer between Salesforce and AWS S3 bucket. I am using third party (Progress DataDirect and CData) connectors to connect...
1
answers
0
votes
258
views
asked 7 months agolg...
Our current setup involves AWS Glue in operation, where data is being extracted from one SQL Server and loaded into another SQL Server through use of AWS Glue Studio for selected tables.
Is there a...
1
answers
0
votes
170
views
asked 7 months agolg...