Questions tagged with AWS Glue
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Similar to RedShift or Snowflake tables is there a way to perform UPSERT for RDS DBs or non RS/SF DB/tables using Glue Visual?
I know Spark Dataframe through JDBS connections only support Insert /...
1
answers
0
votes
68
views
asked 2 months agolg...
I cannot create an AWS Glue job. I am trying to create one of the Glue sample jobs, or a new job from a blank graph. I have followed all the instructions to set up IAM permissions for my account and...
1
answers
0
votes
116
views
asked 2 months agolg...
I created a custom Glue Connector for a JDBC resource I want to connect to in an ETL job. I created a connection for this connector referencing the credential secret. When I attempt to connect to to...
2
answers
0
votes
72
views
asked 2 months agolg...
I'm trying to learn how to use this.
Not sure what the issue is behind the scenes, but I have 3 simple CSV files that I uploaded to S3.
I'm creating a test ETL pipeline with those three CSV files,...
3
answers
0
votes
107
views
asked 2 months agolg...
I am running a PoC around integrating the Glue lineage into the [DataHub](https://datahubproject.io/). I have based my research on this set of AWS blog posts...
1
answers
0
votes
491
views
asked 2 months agolg...
I have Security Lake enabled with my org level Cloud Trail. Events are coming into the Cloud Trail Management table, `amazon_security_lake_table_us_west_2_cloud_trail_mgmt_1_0`, in the underlying...
2
answers
0
votes
97
views
asked 2 months agolg...
We have a requirement where we need to register our Avro schema in Glue schema registry from a service running in my onprem cluster (outside AWS ). We have provisioned AWS Glue schema registry for...
1
answers
0
votes
137
views
asked 2 months agolg...
Hi, I am using AWS glue studio to read from a DDB table with direct DDB connection. So far my visual diagram has two nodes:
1. Source DDB table node -> Here preview takes 5 minutes for only 2 rows of...
1
answers
0
votes
170
views
asked 2 months agolg...
I have 2 AWS accounts, Account "A" contain AWS Redshift and Account "B" has external data that crawler from S3.
## What I have done
#### Account A
1. Attached spectrum role to Redshift
![spectrum...
1
answers
0
votes
234
views
asked 2 months agolg...
Is it possible to wildcard the include path for a MongoDB crawler. I've tried a number of different options similar to the options available for JDBC and other relational database connections, but...
1
answers
0
votes
99
views
asked 2 months agolg...
I receive a file from external vendor. The file is in ***.dat*** format. Once the file arrives into my S3 bucket, I have to trigger a AWS Glue job to read the file and load into my Redshift table. I...
2
answers
0
votes
138
views
asked 2 months agolg...
My dataframe has 2 columns - name and age. If there is name Manish with 2 rows one with age 16 and another with age 23 , will AWS data quality fail both, pass both or one fail one pass. for below...
1
answers
0
votes
130
views
asked 2 months agolg...