Questions tagged with Extract Transform & Load Data
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hi Team,
I have a complex nested xml file which I want to read using AWS Glue and convert it to parquet format. I want to use pandas read_xml function to read the xml file. But, I get error lxml not...
1
answers
0
votes
319
views
asked a year agolg...
I read data from s3 using as follow.
```
sec_id_dyf = glueContext.create_dynamic_frame.from_options(
connection_type = 's3',
...
0
answers
0
votes
105
views
asked a year agolg...
Hello, I have been experimenting with Aws glue, and created some crawlers to crawl the data but the behavior wasn't what I expected,
Question 1) I had an S3 bucket
with 3...
1
answers
0
votes
345
views
asked a year agolg...
Looks like attempting to write to a Delta Lake table from a DynamicFrame is not working. The Visual Glue interface generates a script like:
```
s3 =...
2
answers
0
votes
599
views
asked a year agolg...
We are trying to use Glue to query and aggregate some Parquet files in S3.
We get this error related to schema mismatch:
```
An error occurred while calling o106.pyWriteDynamicFrame....
2
answers
0
votes
274
views
asked a year agolg...
Hello,
I am experience an issue when trying to use the Glue ETL on one of tables in my data catalogue. I am using the visual tool with a very simple SQL transformation on the table and when clicking...
0
answers
0
votes
95
views
asked a year agolg...
I have a large postgres table which is having issues replicating on DMS v3.5.1. The same task (and endpoints) work fine with v3.4.7, so I have rolled back to that for now. The job is a full load &...
1
answers
0
votes
984
views
asked a year agolg...
Hello,
I am trying to save job parameters that I would like to pass to individual jobs within the workflow. After adding them and clicking the 'Update' button and reopening the job parameters, all...
1
answers
0
votes
300
views
asked a year agolg...
I am looking to have metadata produced by crawlers during Glue Jobs record both source AND target information on the same data catalog table. From the research I have done, recording source metadata...
1
answers
0
votes
250
views
asked a year agolg...
In a AWS Glue job, I am getting issue when I am trying to execute sql query in below code:
```
source_dataset = glueContext.create_dynamic_frame.from_options(connection_type="oracle",...
2
answers
0
votes
2288
views
asked a year agolg...
Hi,
I have created a JDBC connection (setup VPC, subnet and security group) called 'my_test_connection that connects to an external postgres database. The connection test passed. I was also able crawl...
1
answers
0
votes
562
views
asked a year agolg...
I am using glue to move data from s3 and redshift into a data lake. I would like to use a combination of AWS glue and Athena to create a source to target mapping report. The goal is to make the report...
1
answers
0
votes
284
views
asked a year agolg...