All Content tagged with AWS Glue

AWS Glue is a serverless data integration service that makes it easy to discover, prepare, and combine data for analytics, machine learning, and application development.

Content language: English

Select tags to filter
Sort by most recent
2010 results
I created a connection in **AWS Glue Data Connections**, but it **failed**. After that, I **deleted the connection**, and the deletion was **successful**. However, the **UI did not refresh** with t...
1
answers
0
votes
30
views
asked 2 months ago
Hi AWS Experts, I am working on an AWS Glue job to transfer data from an on-premise SQL Server to AWS RDS (SQL Server). My goal is to achieve a 20-second execution time, but despite multiple optimiza...
1
answers
0
votes
78
views
asked 2 months ago
I'm trying to do some partition evolutions on my iceberg tables. It seems like `ALTER TABLE ... DROP PARTITION FIELD ...` is not supported by Athena and/or spark sql queries. Is that correct, or am I ...
1
answers
0
votes
72
views
asked 2 months ago
Hi, I'm having a strange issue. I have two S3 buckets or DataCatalog tables to join. When I query them on Athena they sure gives me results when I join them. ![Result of joining the two tables](/medi...
2
answers
0
votes
60
views
asked 2 months ago
Looking on this page: https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-glue-arguments.html I do not see this java-options as an option. We use:  "--java-options" ...
2
answers
0
votes
122
views
asked 2 months ago
I have created a Glue Connection to my PostgreSQL (RDS) instance. I have created and run a Crawler for a table to create a corresponding table in my Glue Data Catalog. I have been unable to read from ...
1
answers
0
votes
198
views
asked 2 months ago
![Custom regex pattern](/media/postImages/original/IMLkdhYjnCSrmf2YAg4OGJJQ) I get error below when I try to use a custom regex pattern to detect sensitive data in Glue Studio Error Category: UNCLASS...
2
answers
0
votes
50
views
asked 2 months ago
profile pictureAWS
published 2 months ago0 votes181 views
This spotlight on AWS Glue equips you with the skills and troubleshooting tips to get the most out of a data integration service.
Can I enable optimization in AWS Glue when I create an iceberg table? Is there an option like a tableProperty? df.writeTo(f"{table_name}") \ .tableProperty("format-version", "...
1
answers
0
votes
92
views
asked 2 months ago
I want to run the two database table in parallel in AWS Glue job. Currently the database table run sequentially one after another and it also increase running time of AWS Glue job
2
answers
0
votes
111
views
asked 2 months ago
Hi, I use Athena to query tables from glue data catalog stored as JSON and with many partitions. However, the number of records is not correct when I select * from tables even though the data scanned...
1
answers
0
votes
75
views
asked 2 months ago
I am working on a process that is as follows: 1. New S3 file is PUT in a bucket/folder (i.e. bucket_name/folder_name/file_name.json) 2. Lambda 1 is triggered by S3 PutObject and sends a message contai...
3
answers
0
votes
262
views
asked 2 months ago