Questions tagged with AWS Glue
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
We are running into `No space left on device` errors in EMR Serverless for big jobs, even when setting driver / executor drive size to the maximum 200GB.
I tried to make the S3 shuffle storage...
1
answers
0
votes
248
views
asked 3 months agolg...
I am interested particularly in `%additional_python_modules` and I always get this error:
`UsageError: Line magic function `%additional_python_modules` not found.`
The same error is thrown when I...
2
answers
0
votes
160
views
asked 3 months agolg...
Hi,
we have a situation where an application running in a k8 environment of a different account have to access the athena and the glue data catalog in a different account.
since these two accounts...
1
answers
0
votes
219
views
asked 3 months agolg...
Similar to RedShift or Snowflake tables is there a way to perform UPSERT for RDS DBs or non RS/SF DB/tables using Glue Visual?
I know Spark Dataframe through JDBS connections only support Insert /...
1
answers
0
votes
127
views
asked 4 months agolg...
I cannot create an AWS Glue job. I am trying to create one of the Glue sample jobs, or a new job from a blank graph. I have followed all the instructions to set up IAM permissions for my account and...
1
answers
0
votes
164
views
asked 4 months agolg...
I created a custom Glue Connector for a JDBC resource I want to connect to in an ETL job. I created a connection for this connector referencing the credential secret. When I attempt to connect to to...
2
answers
0
votes
110
views
asked 4 months agolg...
I'm trying to learn how to use this.
Not sure what the issue is behind the scenes, but I have 3 simple CSV files that I uploaded to S3.
I'm creating a test ETL pipeline with those three CSV files,...
3
answers
0
votes
154
views
asked 4 months agolg...
I am running a PoC around integrating the Glue lineage into the [DataHub](https://datahubproject.io/). I have based my research on this set of AWS blog posts...
1
answers
0
votes
595
views
asked 4 months agolg...
I have Security Lake enabled with my org level Cloud Trail. Events are coming into the Cloud Trail Management table, `amazon_security_lake_table_us_west_2_cloud_trail_mgmt_1_0`, in the underlying...
2
answers
0
votes
163
views
asked 4 months agolg...
We have a requirement where we need to register our Avro schema in Glue schema registry from a service running in my onprem cluster (outside AWS ). We have provisioned AWS Glue schema registry for...
1
answers
0
votes
231
views
asked 4 months agolg...
Hi, I am using AWS glue studio to read from a DDB table with direct DDB connection. So far my visual diagram has two nodes:
1. Source DDB table node -> Here preview takes 5 minutes for only 2 rows of...
1
answers
0
votes
293
views
asked 4 months agolg...
I have 2 AWS accounts, Account "A" contain AWS Redshift and Account "B" has external data that crawler from S3.
## What I have done
#### Account A
1. Attached spectrum role to Redshift
![spectrum...
1
answers
0
votes
538
views
asked 4 months agolg...