Questions tagged with AWS Glue
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
At every 30 minutes it saying "ICEBERG_VACUUM_MORE_RUNS_NEEDED: Removed 20000 files in this round of vacuum" but when I calculate the my table metadata size it didn't changed before and after the...
2
answers
0
votes
250
views
asked 2 months agolg...
I received this error - ResourceSetupError: Exception when listing images from AWS Glue. There are no logs because the duration of the job was 0s.
Why? =/
0
answers
0
votes
351
views
asked 2 months agolg...
I have a glue job (job_a) that starts through a Lambda. When a file is placed inside an S3 bucket, I am triggering a glue job (job_a) through Lambda. My requirement is, once this glue job (job_a), is...
1
answers
0
votes
336
views
asked 2 months agolg...
We are running into `No space left on device` errors in EMR Serverless for big jobs, even when setting driver / executor drive size to the maximum 200GB.
I tried to make the S3 shuffle storage...
1
answers
0
votes
199
views
asked 2 months agolg...
I am interested particularly in `%additional_python_modules` and I always get this error:
`UsageError: Line magic function `%additional_python_modules` not found.`
The same error is thrown when I...
2
answers
0
votes
127
views
asked 2 months agolg...
Hi,
we have a situation where an application running in a k8 environment of a different account have to access the athena and the glue data catalog in a different account.
since these two accounts...
1
answers
0
votes
200
views
asked 2 months agolg...
Similar to RedShift or Snowflake tables is there a way to perform UPSERT for RDS DBs or non RS/SF DB/tables using Glue Visual?
I know Spark Dataframe through JDBS connections only support Insert /...
1
answers
0
votes
89
views
asked 2 months agolg...
I cannot create an AWS Glue job. I am trying to create one of the Glue sample jobs, or a new job from a blank graph. I have followed all the instructions to set up IAM permissions for my account and...
1
answers
0
votes
140
views
asked 3 months agolg...
I created a custom Glue Connector for a JDBC resource I want to connect to in an ETL job. I created a connection for this connector referencing the credential secret. When I attempt to connect to to...
2
answers
0
votes
93
views
asked 3 months agolg...
I'm trying to learn how to use this.
Not sure what the issue is behind the scenes, but I have 3 simple CSV files that I uploaded to S3.
I'm creating a test ETL pipeline with those three CSV files,...
3
answers
0
votes
134
views
asked 3 months agolg...
I am running a PoC around integrating the Glue lineage into the [DataHub](https://datahubproject.io/). I have based my research on this set of AWS blog posts...
1
answers
0
votes
548
views
asked 3 months agolg...
I have Security Lake enabled with my org level Cloud Trail. Events are coming into the Cloud Trail Management table, `amazon_security_lake_table_us_west_2_cloud_trail_mgmt_1_0`, in the underlying...
2
answers
0
votes
132
views
asked 3 months agolg...