Questions tagged with Extract Transform & Load Data
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
We have a number of saved report templates and when we look at the generated csv files, there are a few problems.
1. the columns change order randomly
2. in the csv some of the column headers (such...
2
answers
0
votes
278
views
asked a year agolg...
I am using Amazon Kinesis Firehose for converting files from JSON to Parquet leveraging Glue for Table creation.
When the data is blank the glue schema creates a NULL and the conversion at Kinesis...
1
answers
0
votes
2300
views
asked a year agolg...
Hi,
I'm trying to run a python job on EMR with some dependencies installed with venv as following
```
python -m venv pyspark_venv
source pyspark_venv/bin/activate
pip install pyarrow pandas...
1
answers
0
votes
941
views
asked a year agolg...
Hey all! Wondering if anybody has some experience what the best way is to ETL data from Influx2 to my AWS S3 Data lake. I have been looking for influx2 jdbc connectors (I have used these for PG) but...
0
answers
0
votes
50
views
asked a year agolg...
Hey! I have a setup currently with a crawler that connects to a PostgreSQL database with JDBC, this works and the crawler generates around 20 tables for this database.
I now want to create an ETL job...
1
answers
0
votes
1011
views
asked a year agolg...
I want to extract data from my PostgreSQL on RDS using Aws data glue, transform the data and export the data to s3 bucket. how do i do that. i need an AWS tutorial on this.
2
answers
0
votes
1122
views
asked a year agolg...
I am writing a glue script to take data from s3(PSQL WAL LOGS) to write that data into a hudi data lake.
Whenever I am trying to do that I am getting unable to upsert data with commit time error with...
1
answers
0
votes
816
views
asked a year agolg...
```
df.toDF().write.format("jdbc").\
option("url", "").\
option("dbtable", f"public.{tableName}_staging").\
option("user", "").\
option("password", "").\
...
1
answers
0
votes
373
views
asked a year agolg...
HI,
I'm working with databrew to import some excel file to be then used by Athena. For some columns if I try to add an action I get the following error
`<ACTION>has not allowed characters in...
0
answers
0
votes
99
views
asked a year agolg...
First of all, I have already asked about this problem here in re:Post and on Database Administrators Stack Exchange...
0
answers
0
votes
131
views
asked a year agolg...
Hi, I want to run a job on EMR serverless which reads and writes data from postgresql. I downloaded the jar file and pushed it to s3 and set "spark.jars" in Spark properties in management console....
1
answers
0
votes
1161
views
asked a year agolg...
We have been able to connect to a Microsoft SQL Server DB using both Glue's DynamicFrame and Spark's own JDBC write option due to the Glue connection option.
However, we want to move this workload to...
1
answers
0
votes
693
views
asked a year agolg...