Questions tagged with Extract Transform & Load Data
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
My dataframe has 2 columns - name and age. If there is name Manish with 2 rows one with age 16 and another with age 23 , will AWS data quality fail both, pass both or one fail one pass. for below...
1
answers
0
votes
254
views
asked 4 months agolg...
I have a glue job that transforms data from glue table. And I encounter the following error. It does not occur for every run of the job.
I have looked at a few documentarians, it seems to be coming...
1
answers
0
votes
377
views
asked 4 months agolg...
Hello
I am using Glue Pyspark to handle ETL, but when I tried running script with bookmark, I found out that if one script handles more than one table and one of them doesn't have changes or...
2
answers
0
votes
407
views
asked 4 months agolg...
When I try and add a new BigQuery connection as a sink for glue I am getting the following error:
InvalidInputException: jdbcEnforceSsl: is not defined in the schema and the schema does not allow...
1
answers
0
votes
186
views
asked 4 months agolg...
I tried AWS Glue data quality dynamic rules in my AWS Glue pipeline. I wrote below rule
RowCount > avg(last(3))
Then I processed 3 csv files with 1000,10000 and 100 rows. Then in 4th run I again...
1
answers
0
votes
255
views
asked 4 months agolg...
It is not at present possible to modify placeholder at all, including deletion. Deleting them is at least in principle possible as it doesn’t actually modify the object, but since the placeholders...
1
answers
0
votes
258
views
asked 4 months agolg...
I am doing a AWS Glue job to read from Redshift (schema_1) and write it back to Redhshift (schema_2). This process is done using below:
```
Redshift_read =...
1
answers
0
votes
496
views
asked 4 months agolg...
Hi,
I have recently started working with AWS Glue. I have created a Visual ETL job and it ran successfully. I noticed it had somehow created an extra S3 bucket instead of using the desired bucket I...
1
answers
0
votes
186
views
asked 4 months agolg...
How do I add a sort / dist key to the glue dynamicframe writer into redshift?
1
answers
0
votes
201
views
asked 4 months agolg...
Error Category: RESOURCE_NOT_FOUND_ERROR; An error occurred while calling o123.pyWriteDynamicFrame. Requested resource not found: Table: datacatalog_table_name not found (Service: AmazonDynamoDBv2;...
1
answers
0
votes
216
views
asked 4 months agolg...
I have crawled the schema of my DynamoDB table using AWS Glue crawler and the table is now shown under the tables section in AWS Glue. However, the table is not being shown under Athena database...
1
answers
0
votes
295
views
asked 4 months agolg...
Good day,
I am trying to build a no/zero code architecture, i was planning to use MWAA for orchestration but i was tasked to look at alternatives keeping it simple
I was hoping Redshift query...
2
answers
0
votes
435
views
asked 4 months agolg...