Questions tagged with Extract Transform & Load Data
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hi all,
I'd like to check if anyone has ever had any feedback from AWS regarding the current behavior of job bookmarks when you're sourcing from a database view.
What the documentation explains...
1
answers
0
votes
302
views
asked 6 months agolg...
I have been attempting to follow [the documentation](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-libraries.html) for developing Glue jobs locally.
Upon running the recommended...
2
answers
0
votes
671
views
asked 6 months agolg...
We have a Glue job that accepts the parameter "table_name," with the default value set as "dummy" in the Glue job parameters section. Additionally, the Glue job configuration allows a total of 4...
1
answers
0
votes
416
views
asked 6 months agolg...
Hi everyone.
Should crawler update table schema if datasourse schema is changed?
For example, I have some parquet file with data. One field has datatype "double".
Parquet file is created by Glue Job....
2
answers
0
votes
387
views
asked 6 months agolg...
I am planning to move all the filtered logs from CloudWatch log group through Kinesis Firehose to an S3 bucket in parquet files.
Given that CloudWatch log group always pushes gzipped data to Kinesis...
2
answers
1
votes
494
views
asked 6 months agolg...
I HAVE MULTIPLE CSVS ABOUT A SINGLE PATIENT AND I WOULD LIKE TO KNOW HOW DO I COMBINE ALL THE CSVS BECAUSE ALL THE COLUMNS INSIDE THE CSVS MAKE UP AN ALL THE INFORMATION FOR ONE PATIENT. THE CSV'S ARE...
1
answers
0
votes
359
views
asked 6 months agolg...
Hello,
We are trying to join some dataframes in Glue using Spark und Python.
The dataframes are created from the same source table, but since we are using like 1000 withColumn operations to rename,...
1
answers
0
votes
342
views
asked 7 months agolg...
Hi
I been create Glue Data Connector using its AWS RDS option
and I also create proper IAM role, that have full access to "rds-data", "s3" and "glue"
but whenever I tried to connect (using test...
0
answers
0
votes
114
views
asked 7 months agolg...
# Background
We were getting a "HIVE_CURSOR_ERROR: Failed to read Parquet file" when running trying to run an athena query using `SELECT * FROM mydb`. Our underlying data that we were querying was...
1
answers
0
votes
527
views
asked 7 months agolg...
I want to run my Glue Streaming job locally on Docker container (amazon/aws-glue-streaming-libs:glue_streaming_libs_4.0.0_image_01) to better troubleshoot memory issues, but I encountered this issue...
1
answers
0
votes
252
views
asked 7 months agolg...
Hi all,
I'm trying to connect to an external MariaDB database instance using a AWS Glue Spark script and a JDBC Glue connection.
The code snippet from the Spark script is:
dyf =...
1
answers
0
votes
168
views
asked 7 months agolg...
I'm using DMS to capture CDC from an RDS PostgreSQL Database, then writing the changes to a Kinesis Data Stream and finally using a Glue Streaming Job to process the data and write it to a Hudi Data...
2
answers
0
votes
371
views
asked 7 months agolg...