Questions tagged with Extract Transform & Load Data
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
Hi,
I'm trying to run a python job on EMR with some dependencies installed with venv as following
```
python -m venv pyspark_venv
source pyspark_venv/bin/activate
pip install pyarrow pandas...
1
answers
0
votes
909
views
asked a year agolg...
Hey all! Wondering if anybody has some experience what the best way is to ETL data from Influx2 to my AWS S3 Data lake. I have been looking for influx2 jdbc connectors (I have used these for PG) but...
0
answers
0
votes
49
views
asked a year agolg...
Hey! I have a setup currently with a crawler that connects to a PostgreSQL database with JDBC, this works and the crawler generates around 20 tables for this database.
I now want to create an ETL job...
1
answers
0
votes
975
views
asked a year agolg...
I want to extract data from my PostgreSQL on RDS using Aws data glue, transform the data and export the data to s3 bucket. how do i do that. i need an AWS tutorial on this.
2
answers
0
votes
1103
views
asked a year agolg...
I am writing a glue script to take data from s3(PSQL WAL LOGS) to write that data into a hudi data lake.
Whenever I am trying to do that I am getting unable to upsert data with commit time error with...
1
answers
0
votes
777
views
asked a year agolg...
```
df.toDF().write.format("jdbc").\
option("url", "").\
option("dbtable", f"public.{tableName}_staging").\
option("user", "").\
option("password", "").\
...
1
answers
0
votes
362
views
asked a year agolg...
HI,
I'm working with databrew to import some excel file to be then used by Athena. For some columns if I try to add an action I get the following error
`<ACTION>has not allowed characters in...
0
answers
0
votes
92
views
asked a year agolg...
First of all, I have already asked about this problem here in re:Post and on Database Administrators Stack Exchange...
0
answers
0
votes
129
views
asked a year agolg...
Hi, I want to run a job on EMR serverless which reads and writes data from postgresql. I downloaded the jar file and pushed it to s3 and set "spark.jars" in Spark properties in management console....
1
answers
0
votes
1114
views
asked a year agolg...
We have been able to connect to a Microsoft SQL Server DB using both Glue's DynamicFrame and Spark's own JDBC write option due to the Glue connection option.
However, we want to move this workload to...
1
answers
0
votes
674
views
asked a year agolg...
The goal is to create an ETL job that can be altered and executed by non-technical users in our organization, which is why we are sticking to only visuals and not code.
The problem is that the nodes...
1
answers
0
votes
831
views
asked a year agolg...
Hi,
I'm loading a set of files from S3 and filtering the dataset to files from the last hour.
See error in the Cloudwatch log below:
```
Processing parameterized path with the following...
0
answers
0
votes
109
views
asked a year agolg...