All Content tagged with AWS Glue

AWS Glue is a serverless data integration service that makes it easy to discover, prepare, and combine data for analytics, machine learning, and application development.

Content language: English

Select tags to filter
Sort by most recent
2010 results
I have created a compute connection for existing redshift serverless in sagemaker unified studio also added neccesary tag in workgroup as mentioned in the document. ![Enter image description here](/...
1
answers
0
votes
55
views
asked a month ago
when using ATHENA to query a redshift table it fail on column of type timestamptz ICEBERG_BAD_DATA: Field created_at's type INT96 in parquet file s3://AAAAAAAAAAAAAAA-5b5dc388-103a-4130-bab4-c1508e...
2
answers
0
votes
67
views
asked a month ago
In the world of big data processing, ensuring data consistency and fault tolerance is crucial. While AWS Glue provides built-in job bookmarks, sometimes we need more fine-grained control over our proc...
I have Glue schema registry Registry A created in AWS account A and I want to provide access to resources in AWS account to be able to retrieve schemas from Registry A
1
answers
0
votes
68
views
asked a month ago
Hi, I have been looking into a solution option that uses the Athena invoker_principal to get the ARN of the IAM role being used into the SQL query. Is there a way to do the same if EMR or Redshift...
1
answers
0
votes
58
views
asked a month ago
I am trying to get the metadata of a database by running a query in Athena:- SELECT 'DEV' DF_ENVIRONMENT, 'Source Layer' DATA_LAYER, CAST(TABLE_CATALOG AS VARCHAR) DATABASE_NAME, CAST(TABL...
2
answers
0
votes
67
views
asked a month ago
Whenever I run a data quality job on a Glue table that was created via a spark SQL CTAS command in a Glue Job, I get the following error: *Exception in User Class: java.lang.RuntimeException : Failed...
1
answers
0
votes
48
views
asked 2 months ago
Hello, I have an AWS Glue 5.0 job where I am specifying `--additional-python-modules s3://my-dev/other-dependencies /MyPackage-0.1.1-py3-none-any.whl` in my job options. My glue job itself is just a `...
2
answers
0
votes
56
views
asked 2 months ago
Hi AWS, in our organization we monthly generate a report which we need to share with a Partner AWS account. The report is in parquet format stored inside one of the Glue Tables under the Glue Database...
1
answers
0
votes
83
views
profile picture
asked 2 months ago
I launched the stack for US East (N. Virginia) from the AWS Glue user guide (https://docs.aws.amazon.com/glue/latest/dg/monitor-spark-ui-history.html) to use the yaml template to get a better understa...
1
answers
1
votes
100
views
asked 2 months ago
#### **Issue Description:** I am trying to query an **S3 table (`ams_dev_schema.sp_traffic`) registered in AWS Glue** from **Amazon Redshift**. However, when running: ```sql SELECT * FROM ams_dev...
1
answers
0
votes
57
views
asked 2 months ago
I am new to AWS and need guidance on connecting an EC2 instance to AWS Glue when IP whitelisting is required. I am trying to connect to a MySQL database running on an EC2 instance within a VPC. Howeve...
1
answers
0
votes
68
views
asked 2 months ago