Questions tagged with Extract Transform & Load Data
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
I am looking for the best way to pass a parameter from one glue job to another within a step function.
Each day, I will receive a file. In the file there will be data for certain dates. The first...
1
answers
0
votes
873
views
asked 5 months agolg...
Hi. I am trying to run an AWS Glue job where I transfer data from S3 to Amazon Redshift. However, I am receiving the following error:
```
Error Category: UNCLASSIFIED_ERROR; An error occurred while...
2
answers
0
votes
1194
views
asked 5 months agolg...
```
df = spark.read.parquet("s3://folder/")
df = df.withColumn('filename', input_file_name())
AmazonS3_node1697616892615 = DynamicFrame.fromDF(df, glueContext, "s3sparkread")
```
if this is the code...
1
answers
0
votes
383
views
asked 5 months agolg...
I'm trying to achieve data change capture using AWS Glue and don't want to use DMS. I'm trying to transfer data between two Oracle RDS instances which are in different AWS Account. Here I am trying to...
1
answers
0
votes
527
views
asked 5 months agolg...
I'm trying to achieve data change capture using AWS Glue and don't want to use DMS. I'm trying to transfer data between two Oracle RDS instances which are in different AWS Account. Here I am trying to...
1
answers
0
votes
494
views
asked 5 months agolg...
is it possible to conveniently mask all columns in a Redshift table? The example in the [docs](https://docs.aws.amazon.com/redshift/latest/dg/t_ddm.html#ddm-example) only masks one column but is there...
1
answers
0
votes
516
views
asked 5 months agolg...
For a DeltaTarget, glue.create_crawler is not recognizing the parameter "CreateNativeDeltaTable"lg...
Unknown parameter in Targets.DeltaTargets[0]: "CreateNativeDeltaTable", must be one of: DeltaTables, ConnectionName, WriteManifest.
However, in the documentation it shows the parameter (see...
1
answers
0
votes
175
views
asked 5 months agolg...
I have a Delta Lake table saved in s3. I am running the following command:
spark.sql("""
CREATE EXTERNAL TABLE db.my_table
USING DELTA
LOCATION 's3://path/to/delta/table
""")
Everything...
2
answers
0
votes
411
views
asked 5 months agolg...
Hello.
I am currently completing the AWS Industry Quest: Financial Industry but I'm unable to do so as 2 modules produce errors that prevent me from continuing. The modules "Securing a Banking Data...
1
answers
0
votes
471
views
asked 5 months agolg...
I have a strange problem.
I created my first Visual ETL job; it runs fine, but there are a few mistakes in the output schema.
The problem is now when I try to open the job, it will not load in...
2
answers
0
votes
350
views
asked 5 months agolg...
Hi folks!
I want to ask can we do insert-update in AWS Glue Visual ETL Job?
I want to use Postgre as DB target and will do insert-update.
Prefers use Visual ETL Job...
1
answers
0
votes
426
views
asked 5 months agolg...
Hi folks!
right now i'm working on project to do ETL Process using AWS Glue. I already create the query script to process the data, but my source data is in Json format. I need help to transform the...
1
answers
0
votes
280
views
asked 5 months agolg...