Questions tagged with AWS Glue
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
In AWS Glue jobs, within the Targets node, I am unable to see the data types such as struct, array or map while changing the schema. Does AWS Glue not support these data types?
1
answers
0
votes
175
views
asked 25 days agolg...
I got the error when write glue dynamic data frame to redshift table .
The code I have used :-
my_conn_op={
"dbtable":"public.xxxx",
...
0
answers
0
votes
428
views
asked 25 days agolg...
Hi,
I've created Glue Crawler to determine data sctructure from XML file uploaded to s3, and write Table into Data Catalog.
I tried 2 approaches:
1. Use Glue default classifier - this is preferred...
1
answers
0
votes
132
views
asked 25 days agolg...
Hello.
I am trying to configure specific iam permission for an user. I need a permission for only read tables from existing Data Catalog.
So, I have configured this policiy:
```
{
"Version":...
0
answers
0
votes
189
views
asked 25 days agolg...
I've successfully set up AWS Glue with an RDS database serving as the data source and a Snowflake database as the data target. In this setup, I've configured AWS Glue crawlers to catalog the metadata...
0
answers
0
votes
442
views
asked a month agolg...
I am running a glue job with python script shell(version 3.9) and glue version is 3.0. I am passing 8 arguments to the glue job and accessing it using getResolvedOptions(args, options). One of the...
2
answers
0
votes
163
views
asked a month agolg...
Hi,
I have created a Glue job to trigger it for S3 event. So I have below design
S3 Bucket ---> SQS ---> Lambda ---> Trigger Glue job from Lambda.
I am facing below error when multiple files are...
2
answers
0
votes
122
views
asked a month agolg...
Hello,
I work as a data engineer and business intelligence specialist for a fintech startup. We've entered into a new agreement with a supplier to provide a technological solution for managing their...
0
answers
0
votes
236
views
asked a month agolg...
Hello, I am relatively new to Glue and encountering some challenges with Glue ETL.
Our setup involves a datalake that retrieves data from a backend database as its source. This datalake is...
1
answers
0
votes
276
views
asked a month agolg...
Hello,
I have parquets files in S3 that i parse using Glue Crawler and query in Athena. I found that some files have two columns "x" and "y" that have a type **int64** while other files have them as...
1
answers
0
votes
230
views
asked a month agolg...
Unable to push Glue job to GitHub. Empty connections list is now allowed if connection is specified.lg...
Hi,
I am trying to Push the Glue job to GitHub repo. I have got added the access permissions to my role as specified in...
3
answers
0
votes
123
views
asked a month agolg...
I am using aws lakeformation workflow to create a data lake following [this](https://docs.aws.amazon.com/lake-formation/latest/dg/getting-started-tutorial-jdbc.html) guide. everything is setup as...
0
answers
0
votes
110
views
asked a month agolg...