Unanswered Questions tagged with Extract Transform & Load Data
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
AWS Glue Job Errorlg...
Im trying to convert CSV files in S3 to Parquet in another S3 bucket. So first I read the CSV files using a crawler, load the data into a Table, and then use a Job to convert from the Table to S3 in...
0
answers
0
votes
32
views
asked 18 hours agolg...
I've successfully set up AWS Glue with an RDS database serving as the data source and a Snowflake database as the data target. In this setup, I've configured AWS Glue crawlers to catalog the metadata...
0
answers
0
votes
171
views
asked 10 days agolg...
In our ETL process we are building out a pipeline where someones job is to take input files (ex. csv) and map the columns to existing column names. After the mapping is complete a glue workflow will...
0
answers
0
votes
172
views
asked 17 days agolg...
Why doesn't Glue Job and Glue Workflow have the function of version control and alias likes Labmda.lg...
I tried to develop the data orchestlation with s3, Glue Job and Glue Workflow. After I developed it, I found that Glue Job and Glue Workflow doesn't have the function of version control and alias...
0
answers
0
votes
171
views
asked a month agolg...
Hi team, first post, let me know if it provides a good explanation.
I'd like to know a way to minimize the effort for data ingestion.
We have two options as follows:
(1) csv files from a file...
0
answers
0
votes
298
views
asked a month agolg...
Question:
We currently have approximately 100 tables in delta format, partitioned by yyyy, mm, dd, hh, mm. Our current process involves reading these delta tables via a crawler, cataloging them, and...
0
answers
0
votes
359
views
asked a month agolg...
I have an iceberg table defined like this:
CREATE TABLE IF NOT EXISTS staging (
id STRING,
staging_timestamp BIGINT,
... blah blah blah ...
)
PARTITIONED BY...
0
answers
0
votes
176
views
asked 2 months agolg...
I have multiple Visual ETL configured correctly, but if go back to the previous screen and then try to see the job again, the display editor will lost the configuration and it will highlight some...
0
answers
0
votes
104
views
asked 4 months agolg...
Scenario:
Source table: Glue Data Catalog table **study** crawled from MySQL with columns:
* id (int),
* code (varchar),
* desc (varchar)
* and 2 other columns not used in the job.
Target table:...
0
answers
0
votes
97
views
asked 6 months agolg...
I'm looking for an open-source solution that can help us make our python API more accessible.
For simplicity's sake, the data is accessed using Athena and has three string fields A, B, C.
Every...
0
answers
0
votes
144
views
asked 7 months agolg...
Hi
I been create Glue Data Connector using its AWS RDS option
and I also create proper IAM role, that have full access to "rds-data", "s3" and "glue"
but whenever I tried to connect (using test...
0
answers
0
votes
116
views
asked 8 months agolg...
Hi,
I am trying to migrate a table from Postgres to Redshift using a migration task
Simplified table structure:
| Name | Type |
| --- | --- |
| id | integer |
| time | timestamp with time zone |
|...
0
answers
0
votes
114
views
asked 8 months agolg...