Redshift varchar(max) not enough to store json data type column from Postgres

Question

**Redshift** varchar(max) not enough to store json data type column from **Postgres**. Which data type should I use to store the column, I am using glue to perform the ETL and storing to Redshift, Postgres has Json data type but in Redshift the limit is exceeding even by varchar(max).
Getting below error in STL_LOAD_ERRORS table-redshift ->**String length exceeds DDL length**               
Is there any workaround or direct solution, as super data type is not supported in Glue, so can't map that to the column.

Answer

Hi,

In Redshift for storing JSON data we have introduced the [SUPER data type](https://docs.aws.amazon.com/redshift/latest/dg/r_SUPER_type.html).

However, the SUPER data type only supports up to 1MB of data for an individual SUPER field or object. For more information, see [Ingesting and querying semistructured data in Amazon Redshift](https://docs.aws.amazon.com/redshift/latest/dg/super-overview.html).

hope this helps,

Answer

We are encountering a similar issue where we're utilizing the "super" datatype, which has a maximum length of 65K. However, the column in the Parquet file we receive has a maximum length of 192K. How should we handle this data? Are there alternative datatypes we can use to accommodate such large data sizes?

Redshift varchar(max) not enough to store json data type column from Postgres

相关内容