Questions tagged with AWS Glue
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hello team,
So, I built an ETL in python using pyspark. I have a bastion EC2 mysql database that is a copy of a production environment.
Every day it is copying the prod at round 2 oclock, and my...
1
answers
0
votes
261
views
asked 3 months agolg...
I'm running into an issue when reading a Glue Data Catalog data source in an Visual ETL AWS Glue job. An extra column is being added in called 'col40', which is not in the underlying file that was...
1
answers
0
votes
227
views
asked 3 months agolg...
hello, I am creating a dataframe consuming from a Glue Catalog table, this table has fields of type bigint, which can be null. It turns out that when this information is null, the dataframe ignores...
Accepted AnswerAWS Glue
1
answers
0
votes
177
views
asked 3 months agolg...
I was running glue job to process data from MariaDB inside VPC. Recently my glue job get "com.mysql.cj.jdbc.exceptions.CommunicationsException: Communications link failure" although it was running...
1
answers
2
votes
261
views
asked 3 months agolg...
Hello! According to the [documentation](https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-connect-kinesis-home.html), it should be possible to write data to Kinesis from Glue...
2
answers
0
votes
1592
views
asked 3 months agolg...
I am working in the Glue ETL Visual Editor and I've started to encounter this error `QuotaExceededError - Failed to execute 'setItem' on 'Storage'`.
It is preventing me from even starting a Data...
0
answers
0
votes
131
views
asked 3 months agolg...
At every 30 minutes it saying "ICEBERG_VACUUM_MORE_RUNS_NEEDED: Removed 20000 files in this round of vacuum" but when I calculate the my table metadata size it didn't changed before and after the...
2
answers
0
votes
451
views
asked 3 months agolg...
I received this error - ResourceSetupError: Exception when listing images from AWS Glue. There are no logs because the duration of the job was 0s.
Why? =/
0
answers
0
votes
359
views
asked 3 months agolg...
I have a glue job (job_a) that starts through a Lambda. When a file is placed inside an S3 bucket, I am triggering a glue job (job_a) through Lambda. My requirement is, once this glue job (job_a), is...
1
answers
0
votes
380
views
asked 3 months agolg...
We are running into `No space left on device` errors in EMR Serverless for big jobs, even when setting driver / executor drive size to the maximum 200GB.
I tried to make the S3 shuffle storage...
1
answers
0
votes
247
views
asked 3 months agolg...
I am interested particularly in `%additional_python_modules` and I always get this error:
`UsageError: Line magic function `%additional_python_modules` not found.`
The same error is thrown when I...
2
answers
0
votes
158
views
asked 3 months agolg...
Hi,
we have a situation where an application running in a k8 environment of a different account have to access the athena and the glue data catalog in a different account.
since these two accounts...
1
answers
0
votes
219
views
asked 3 months agolg...