Unanswered Questions tagged with Extract Transform & Load Data
Content language: English
Select up to 5 tags to filter
Sort by most recent
Browse through the questions and answers listed below or filter and sort to narrow down your results.
In our ETL process we are building out a pipeline where someones job is to take input files (ex. csv) and map the columns to existing column names. After the mapping is complete a glue workflow will...
0
answers
0
votes
59
views
asked 7 days agolg...
Why doesn't Glue Job and Glue Workflow have the function of version control and alias likes Labmda.lg...
I tried to develop the data orchestlation with s3, Glue Job and Glue Workflow. After I developed it, I found that Glue Job and Glue Workflow doesn't have the function of version control and alias...
0
answers
0
votes
59
views
asked 17 days agolg...
Hi team, first post, let me know if it provides a good explanation.
I'd like to know a way to minimize the effort for data ingestion.
We have two options as follows:
(1) csv files from a file...
0
answers
0
votes
187
views
asked 18 days agolg...
Question:
We currently have approximately 100 tables in delta format, partitioned by yyyy, mm, dd, hh, mm. Our current process involves reading these delta tables via a crawler, cataloging them, and...
0
answers
0
votes
356
views
asked a month agolg...
I have an iceberg table defined like this:
CREATE TABLE IF NOT EXISTS staging (
id STRING,
staging_timestamp BIGINT,
... blah blah blah ...
)
PARTITIONED BY...
0
answers
0
votes
171
views
asked 2 months agolg...
I have multiple Visual ETL configured correctly, but if go back to the previous screen and then try to see the job again, the display editor will lost the configuration and it will highlight some...
0
answers
0
votes
100
views
asked 3 months agolg...
Scenario:
Source table: Glue Data Catalog table **study** crawled from MySQL with columns:
* id (int),
* code (varchar),
* desc (varchar)
* and 2 other columns not used in the job.
Target table:...
0
answers
0
votes
96
views
asked 5 months agolg...
I'm looking for an open-source solution that can help us make our python API more accessible.
For simplicity's sake, the data is accessed using Athena and has three string fields A, B, C.
Every...
0
answers
0
votes
142
views
asked 6 months agolg...
Hi
I been create Glue Data Connector using its AWS RDS option
and I also create proper IAM role, that have full access to "rds-data", "s3" and "glue"
but whenever I tried to connect (using test...
0
answers
0
votes
115
views
asked 7 months agolg...
Hi,
I am trying to migrate a table from Postgres to Redshift using a migration task
Simplified table structure:
| Name | Type |
| --- | --- |
| id | integer |
| time | timestamp with time zone |
|...
0
answers
0
votes
109
views
asked 7 months agolg...
My Glue 4.0 jobs have suddenly stopped working with error message below. As it is related to boto3, I am unable to make any changes to library config. Pls advise.
NB: I noticed that urllib3 released...
0
answers
0
votes
98
views
asked 8 months agolg...
I was trying to perform Glue ETL transformation and store it in AWS Serverless Redshift database and S3 (both) . However, even the Console generated PySpark sheet fails. Almost none of the methods...
0
answers
0
votes
161
views
asked 8 months agolg...