Questions tagged with AWS Glue
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Similar to RedShift or Snowflake tables is there a way to perform UPSERT for RDS DBs or non RS/SF DB/tables using Glue Visual?
I know Spark Dataframe through JDBS connections only support Insert /...
1
answers
0
votes
124
views
asked 3 months agolg...
I cannot create an AWS Glue job. I am trying to create one of the Glue sample jobs, or a new job from a blank graph. I have followed all the instructions to set up IAM permissions for my account and...
1
answers
0
votes
162
views
asked 3 months agolg...
I created a custom Glue Connector for a JDBC resource I want to connect to in an ETL job. I created a connection for this connector referencing the credential secret. When I attempt to connect to to...
2
answers
0
votes
108
views
asked 3 months agolg...
I'm trying to learn how to use this.
Not sure what the issue is behind the scenes, but I have 3 simple CSV files that I uploaded to S3.
I'm creating a test ETL pipeline with those three CSV files,...
3
answers
0
votes
152
views
asked 4 months agolg...
I am running a PoC around integrating the Glue lineage into the [DataHub](https://datahubproject.io/). I have based my research on this set of AWS blog posts...
1
answers
0
votes
592
views
asked 4 months agolg...
I have Security Lake enabled with my org level Cloud Trail. Events are coming into the Cloud Trail Management table, `amazon_security_lake_table_us_west_2_cloud_trail_mgmt_1_0`, in the underlying...
2
answers
0
votes
162
views
asked 4 months agolg...
We have a requirement where we need to register our Avro schema in Glue schema registry from a service running in my onprem cluster (outside AWS ). We have provisioned AWS Glue schema registry for...
1
answers
0
votes
228
views
asked 4 months agolg...
Hi, I am using AWS glue studio to read from a DDB table with direct DDB connection. So far my visual diagram has two nodes:
1. Source DDB table node -> Here preview takes 5 minutes for only 2 rows of...
1
answers
0
votes
291
views
asked 4 months agolg...
I have 2 AWS accounts, Account "A" contain AWS Redshift and Account "B" has external data that crawler from S3.
## What I have done
#### Account A
1. Attached spectrum role to Redshift
![spectrum...
1
answers
0
votes
534
views
asked 4 months agolg...
Is it possible to wildcard the include path for a MongoDB crawler. I've tried a number of different options similar to the options available for JDBC and other relational database connections, but...
1
answers
0
votes
149
views
asked 4 months agolg...
I receive a file from external vendor. The file is in ***.dat*** format. Once the file arrives into my S3 bucket, I have to trigger a AWS Glue job to read the file and load into my Redshift table. I...
2
answers
0
votes
232
views
asked 4 months agolg...
My dataframe has 2 columns - name and age. If there is name Manish with 2 rows one with age 16 and another with age 23 , will AWS data quality fail both, pass both or one fail one pass. for below...
1
answers
0
votes
256
views
asked 4 months agolg...