Questions tagged with Extract Transform & Load Data
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
I do a crawler to load all my S3 csv files to Glue Data Catalog. Now I want to create a glue job to execute ETL (create and drop temporary tables, select and insert data to tables in Data Catalog) But...
0
answers
0
votes
56
views
asked a year agolg...
I can't find a proper way of setting the correct data type for a timestamp attribute on my Athena table **parquet** in order to query for time intervals.
im creating the table via a crawler on parquet...
1
answers
0
votes
494
views
asked a year agolg...
I have written a lambda function to convert json file in raw s3 bucket into parquet file and gets uploaded directly it to the cleansed s3 bucket. I cannot delete json files since i want to convert...
2
answers
0
votes
1297
views
asked a year agolg...
I'm trying to fill Aurora MySQL DB from CSV in an S3 bucket using the manual [Loading data into an Amazon Aurora MySQL DB cluster from text files in an Amazon S3...
1
answers
0
votes
215
views
asked a year agolg...
Hello, I'm creating a Glue Job using Jupyter notebook and I'm currently using Ray as the ETL type.
After running the job once, I noticed I can no longer save my notebook or push it to a repository...
1
answers
0
votes
381
views
asked a year agolg...
Running a glue job to fetch records from Microsoft sql server but glue jobs keeps running and does not show any results. Job is scheduled with G.2X worker with 5 works with auto scheduling.
Logs:-...
2
answers
0
votes
1143
views
asked a year agolg...
I have noticed that my AWS Step Function freezes after previous run has been aborted with some jobs still left in the pending state.
The use case:
I have a step function that increments through the...
0
answers
0
votes
70
views
asked a year agolg...
Join Tables in AWSlg...
I want to join two tables.I have the tables in CSV format stored in S3 bucket
1.Is Amazon Glue studio,the right option?
2.What is the correct procedure?
3.What are the IAM permissions...
2
answers
0
votes
305
views
asked a year agolg...
Hi,
I am using GlueETL version Spark 3.0 with Python version ![Glue Job Details](/media/postImages/original/IMGyPWz2XIS_-4GohAdHXVpw)
The ETL job has only 2 steps. I am using...
1
answers
0
votes
226
views
asked a year agolg...
All,
I recently updated an AVRO Schema. I have checked that the updates made to the schema are backward compatible; and they are. Next, I saved an avro file w/ the new schema to an s3 Bucket and...
0
answers
0
votes
48
views
asked a year agolg...
Hello All,
I am working on Glue pyspark script . In this script I read data from table and store it in pyspark dataframe. Now I want to add new column whose value will be calculated by passing...
2
answers
0
votes
351
views
asked a year agolg...
AWS GLUE - JOB ERRORlg...
Hello,
I'm stuck with this error and I can't find anything help full.
I'm trying to migrate data between s3 to Redshift,
Note: i crawled both and both tables are in my glue databases
but when i'm...
1
answers
0
votes
1630
views
asked a year agolg...