Questions tagged with Extract Transform & Load Data
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hi all, I created a EventBridge rule with the following event pattern that is suppose to match all glue crawler state changes and send them to a lambda function, however, the rule is only only sending...
2
answers
0
votes
720
views
asked 9 months agolg...
Hi,
I am searching for a transformation engine that supports multi-tenancy with the following requirements:
* Each tenant must be transformed every 10 minutes.
* One tenant transformation transforms...
1
answers
0
votes
336
views
asked 9 months agolg...
Hello, we have an S3 bucket with various CSV files and an AWS Glue crawler to update the Data Catalog and finally an AWS Glue job to move the data to RedShift. The handling of data and target table is...
0
answers
0
votes
124
views
asked 9 months agolg...
Hi all. I generated a batch transform. The process finished correctly with no issues. When I check the output data, the directory is empty.
This is the manifest file format (RecordIO)...
1
answers
0
votes
250
views
asked 9 months agolg...
importing form encoded/csv data from data logger using aws api gateway,lamda function and dynamodblg...
greetings, for past few weeks i have been trying to fetch data using a solar data logger using an api gateway that sends data to my api gateway which is integrated to my lamda function but the problem...
4
answers
0
votes
413
views
asked 9 months agolg...
Hello everyone.
Data from the rest api in the form of JSON is loaded daily by lambda into s3-bucket-1.
Then this data should be stored in s3-bucket-2 in the form of a flat parquet table.
I did it in...
0
answers
0
votes
71
views
asked 9 months agolg...
Hey Guys
I want to run my pyspark on EMR Serverless but it has some dependencies/libraries which are needed by the pyspark script to run. Please suggest a optimized approach to import the...
1
answers
0
votes
384
views
asked 9 months agolg...
Hello,
I have gone through the recommended changes provided in [this](https://repost.aws/knowledge-center/glue-crawler-internal-service-exception) article. However, I continue to get the same...
1
answers
0
votes
233
views
asked 9 months agolg...
Hi there I managed to convert csv files to parquet files using glue job, my crawler does see the parquet files in the s3 bucket and crawls it and present me with the proper schema and adds for each...
1
answers
0
votes
478
views
asked 9 months agolg...
Are there any known, recently (~07/18/2023) introduced performance issues with Glue crawlers?
We have recently observed excessive slowness with Glue crawlers that had been running for months without...
0
answers
0
votes
28
views
asked 9 months agolg...
Glue 4 Hudi supportlg...
I am trying to store a data stream from kafka using the hudi format. I am following this doc https://docs.aws.amazon.com/glue/latest/dg/aws-glue-programming-etl-format-hudi.html and I even tried to...
3
answers
0
votes
262
views
asked 9 months agolg...
We are moving our content from one developer to another. I am trying to figure out not to stop my Images and pdf uploads from being rasterized.
My old developer had figured it out, but I can't.
Any...
1
answers
0
votes
200
views
asked 9 months agolg...