Questions tagged with AWS Glue
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hi,
I have created a Glue job to trigger it for S3 event. So I have below design
S3 Bucket ---> SQS ---> Lambda ---> Trigger Glue job from Lambda.
I am facing below error when multiple files are...
2
answers
0
votes
114
views
asked 18 days agolg...
Hello,
I work as a data engineer and business intelligence specialist for a fintech startup. We've entered into a new agreement with a supplier to provide a technological solution for managing their...
0
answers
0
votes
219
views
asked 19 days agolg...
Hello, I am relatively new to Glue and encountering some challenges with Glue ETL.
Our setup involves a datalake that retrieves data from a backend database as its source. This datalake is...
1
answers
0
votes
222
views
asked 20 days agolg...
Hello,
I have parquets files in S3 that i parse using Glue Crawler and query in Athena. I found that some files have two columns "x" and "y" that have a type **int64** while other files have them as...
1
answers
0
votes
217
views
asked 21 days agolg...
Unable to push Glue job to GitHub. Empty connections list is now allowed if connection is specified.lg...
Hi,
I am trying to Push the Glue job to GitHub repo. I have got added the access permissions to my role as specified in...
3
answers
0
votes
105
views
asked 21 days agolg...
I am using aws lakeformation workflow to create a data lake following [this](https://docs.aws.amazon.com/lake-formation/latest/dg/getting-started-tutorial-jdbc.html) guide. everything is setup as...
0
answers
0
votes
96
views
asked 21 days agolg...
Trying to connect to redshift database in python shell jobs. Tried with packages like psycopg2, redshift connector and pg. All of them gave similar error, hence I'm assuming problem is to establish...
1
answers
0
votes
163
views
asked 21 days agolg...
I've imported a dataset of JSON objects that all have consistent schema. Glue crawler finished successfully and created a table. I can select * with a limit in Athena, but when I select all rows I get...
1
answers
1
votes
254
views
asked 21 days agolg...
trying out AWS glue for the first time. i want to read some data from s3 and push it to some database in a database within aws. what resources , do i need to create/provision for AWS glue, if doing...
1
answers
0
votes
83
views
asked 22 days agolg...
Hello, I am attempting to update a glue job default argument within a lambda function.
So far the best I have been able to find is to use the boto3 get_job and update_job functions with the desired...
1
answers
0
votes
151
views
asked 22 days agolg...
In our ETL process we are building out a pipeline where someones job is to take input files (ex. csv) and map the columns to existing column names. After the mapping is complete a glue workflow will...
0
answers
0
votes
177
views
asked 22 days agolg...
Hi!
I am trying to use DataZone to share an Iceberg table from the Glue Data Catalog with another AWS account.
I have created the table with Athena in the source account like this:
```sql
CREATE...
0
answers
0
votes
83
views
asked 22 days agolg...