Redshift varchar(max) not enough to store json data type column from Postgres

0

Redshift varchar(max) not enough to store json data type column from Postgres. Which data type should I use to store the column, I am using glue to perform the ETL and storing to Redshift, Postgres has Json data type but in Redshift the limit is exceeding even by varchar(max). Getting below error in STL_LOAD_ERRORS table-redshift ->String length exceeds DDL length
Is there any workaround or direct solution, as super data type is not supported in Glue, so can't map that to the column.

asked 2 years ago2799 views
2 Answers
0

Hi,

In Redshift for storing JSON data we have introduced the SUPER data type.

However, the SUPER data type only supports up to 1MB of data for an individual SUPER field or object. For more information, see Ingesting and querying semistructured data in Amazon Redshift.

hope this helps,

AWS
EXPERT
answered 2 years ago
0

We are encountering a similar issue where we're utilizing the "super" datatype, which has a maximum length of 65K. However, the column in the Parquet file we receive has a maximum length of 192K. How should we handle this data? Are there alternative datatypes we can use to accommodate such large data sizes?

msve
answered a day ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions