By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

AWS GLUE JOB NOT DETECTING SCHEMA IN PYTHON SCRIPT

1

Hi, I have created a new AWS Glue visual ETL job with source- PostgreSQL, target- Snowflake. Output schema option shows correct schema as per the source, but the python script shows all the datatype as VARCHAR and the target snowflake table all columns are created with datatype - VARCHAR. Please let me know if there is any option to auto detect the schema in python script. Enter image description here Enter image description here

Topics
AnalyticsDatabase
Tags
AWS GlueExtract Transform & Load Data
Language
English
Nivetha Palaniappan
asked 5 months ago141 views
2 Answers
0

Seems that's a limitation of that target node, I've opened a ticket to request that it honors the schema.

profile pictureAWS
EXPERT
Gonzalo Herreros
answered 5 months ago
  • Thomas
    2 months ago

    Is there a way to transfert data using a script without losing the data type ? Do you have the link of the ticket you speak about?

0

Workaround found : "Just" change the python script manually to fix the output data type

Thomas
answered 2 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content