Questions tagged with AWS Glue

Running show tables in Athena console is giving error: [ErrorCategory:USER_ERROR, ErrorCode:DDL_FAILED], Detail:FAILED: Execution Error, return code 1 from org.apache.hadoop.hive.ql.exec.DDLTask....

I have a Glue streaming job that consume a kafka topic as: ``` sourceData = glueContext.create_data_frame.from_options( connection_type="kafka", connection_options={ "connectionName":...

I am currently using a Glue job to read data from one Amazon S3 source, perform some transformations and write the transformed data into another S3 bucket in parquet format. While writing data to the...

Hello guys. i'm new on Glue, but i need to do a job who extract from a rds and load in s3. Script works fine, but i need a incremental load, cause every time i start the job, all data is processed,...

I need to make an update to a table in athena called businesses. The table schema was updated in Github, Lamdba, Glue and the new fields appear in the table in Athena, however they appear empty, for...

We are working on Glue ETL job with visual mode and have multiple S3 (Glue Data Catalog type) source nodes that we want to use Pushdown Predicate to implement daily updating task. - The Glue table...

I'm trying to create a Glue Crawler, but I'm encountering the following error: "The following crawler failed to create: 'name'. Here is the most recent error message: Account XXXX is denied...

In a glue job that is using bookmarks, I'm including the transformation_ctx parameter in each of the create dynamic frame methods (where I read data). If I then do a join and a select and then an...

I am trying to use chromadb in AWS Glue, it's throwing out the below error, RuntimeError: Your system has an unsupported version of sqlite3. Chroma requires sqlite3 >= 3.35.0. I tried to add to...

Hi, I have a question about how athena columns listing works, since in my schema and my table I have for example a column called 'anonymousId' (and I can see it directly on the athena panel at the...

I created an Athena table using Glue ``` import * as glue from '@aws-cdk/aws-glue-alpha' const dataWarehouse = new glue.Table(this, `data_warehouse`, { database: db, tableName:...

We just created a matching job on AWS Identity Resolution. For our initial testing we used a sample data of 2781 records. It took about 807 seconds to finish this job. Assuming we are satisfied with...