Questions tagged with AWS Glue
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
I am running a glue job with python script shell(version 3.9) and glue version is 3.0. I am passing 8 arguments to the glue job and accessing it using getResolvedOptions(args, options). One of the...
2
answers
0
votes
83
views
asked 2 days agolg...
Hi,
I have created a Glue job to trigger it for S3 event. So I have below design
S3 Bucket ---> SQS ---> Lambda ---> Trigger Glue job from Lambda.
I am facing below error when multiple files are...
2
answers
0
votes
41
views
asked 3 days agolg...
Hello,
I work as a data engineer and business intelligence specialist for a fintech startup. We've entered into a new agreement with a supplier to provide a technological solution for managing their...
0
answers
0
votes
134
views
asked 3 days agolg...
Hello, I am relatively new to Glue and encountering some challenges with Glue ETL.
Our setup involves a datalake that retrieves data from a backend database as its source. This datalake is...
1
answers
0
votes
63
views
asked 5 days agolg...
Hello,
I have parquets files in S3 that i parse using Glue Crawler and query in Athena. I found that some files have two columns "x" and "y" that have a type **int64** while other files have them as...
1
answers
0
votes
70
views
asked 6 days agolg...
Unable to push Glue job to GitHub. Empty connections list is now allowed if connection is specified.lg...
Hi,
I am trying to Push the Glue job to GitHub repo. I have got added the access permissions to my role as specified in...
3
answers
0
votes
65
views
asked 6 days agolg...
I am using aws lakeformation workflow to create a data lake following [this](https://docs.aws.amazon.com/lake-formation/latest/dg/getting-started-tutorial-jdbc.html) guide. everything is setup as...
0
answers
0
votes
67
views
asked 6 days agolg...
Trying to connect to redshift database in python shell jobs. Tried with packages like psycopg2, redshift connector and pg. All of them gave similar error, hence I'm assuming problem is to establish...
1
answers
0
votes
115
views
asked 6 days agolg...
I've imported a dataset of JSON objects that all have consistent schema. Glue crawler finished successfully and created a table. I can select * with a limit in Athena, but when I select all rows I get...
0
answers
0
votes
91
views
asked 6 days agolg...
trying out AWS glue for the first time. i want to read some data from s3 and push it to some database in a database within aws. what resources , do i need to create/provision for AWS glue, if doing...
1
answers
0
votes
72
views
asked 6 days agolg...
Hello, I am attempting to update a glue job default argument within a lambda function.
So far the best I have been able to find is to use the boto3 get_job and update_job functions with the desired...
1
answers
0
votes
129
views
asked 7 days agolg...
In our ETL process we are building out a pipeline where someones job is to take input files (ex. csv) and map the columns to existing column names. After the mapping is complete a glue workflow will...
0
answers
0
votes
59
views
asked 7 days agolg...