Questions tagged with Extract Transform & Load Data
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
I converted a CSV(from S3) to parquet(to S3) using AWS glue and the file which is converted to Parquet was named randomly .How do i choose the name of the file that is to be converted to Parquet from...
1
answers
0
votes
778
views
asked a year agolg...
Macie provides detailed positions of sensitive data in output file. But, I want to extract that data using positions from output file. Also, macie reveal only 10 samples.
Is there any way to get more...
1
answers
0
votes
325
views
asked a year agolg...
I'm writing partitioned parquet data using a Spark data frame and mode=overwrite to update stale partitions. I have this set: spark.conf.set('spark.sql.sources.partitionOverwriteMode','dynamic')
The...
1
answers
0
votes
863
views
asked a year agolg...
How can one set up an Execution Class = FLEX on a Jupyter Job Run , im using the %magic on my %%configure cell like below and also setting the input arguments with --execution_class = FLEX
But still...
2
answers
0
votes
602
views
asked a year agolg...
Hi, I'd appreciate AWS Athena support for TIMESTAMP data type with microsecond precision for all row formats and table engines. Currently, the support is very inconsistent. See the SQL script below....
0
answers
0
votes
155
views
asked a year agolg...
Started getting this error today when querying data from Athena in a table created from parquet files in our S3 bucket:
![Enter image description...
0
answers
0
votes
99
views
asked a year agolg...
Hi community,
I am trying to perform an ETL job using AWS Glue.
Our data is stored in MongoDB Atlas, inside a VPC.
Our AWS is connected to our MongoDB Atlas using VPC peering.
To perform the ETL...
1
answers
1
votes
446
views
asked a year agolg...
In Redshift, I'm trying to update a table using another table from another database. The error details:
SQL Error [XX000]: ERROR: Assert
Detail:
-----------------------------------------------
...
1
answers
0
votes
239
views
asked a year agolg...
I'm attempting to use AWS Data Pipeline to move a CSV file from my computer to AWS Data Lake as a parquet file. I'm unable to finad the exact template to select to migrate from my local...
0
answers
0
votes
84
views
asked a year agolg...
i want to directly move a csv file from my laptop to aws data lake using aws pipeline?
is it possible to so ? if yes how?
1
answers
0
votes
326
views
asked a year agolg...
Hey,
My ETL Glue job is:
1. reading from Data Catalogue (S3 based),
2. selecting specific fields from the input file (which is json)
2. doing some mapping
3. saving output data to Postgres Aurora...
2
answers
0
votes
337
views
asked a year agolg...
Hello, I am running a job to apply an ETL on a semi-colon-separated CSV on S3, however, when I read the file using the DynamicFrame feature of AWS and try to use any method like `printSchema` or...
1
answers
0
votes
1737
views
asked a year agolg...