DMS truncation when redshift as target

0

i have one table in aurora db that i want to migrate to redshift there is one column in that table which store JSONB object and there is one data whose size is around 50KB.

my question is when i perform migration from aurora to redshift via DMS then that JSON object data is getting truncated to 31.5KB even though i have setted the MaxLobSize to 63KB with Limited LOB mode in migration task.

已提問 1 個月前檢視次數 299 次
2 個答案
0

That sounds like a bug. Have you captured anything in the logs that you can share with AWS Support via a support ticket?

Also, there might be other easier ways to get this data synced to Amazon Redshift. Have you tried zero-ETL , refer https://aws.amazon.com/about-aws/whats-new/2024/03/amazon-aurora-zero-etl-integration-redshift-data-filtering-cloudformation/ , towards the bottom it provides a blog link with an example use case too.

profile pictureAWS
已回答 1 個月前
profile picture
專家
已審閱 1 個月前
  • In logs it is showing the truncated to 64KB but our actual data is getting truncated even less than 64KB. but anyways we are using Zero-ETL now as this is new feature which came just couple of days ago in our region. but here as well am facing some other issue when i create ETL integration from RDS my aurora postgres cluster is not getting listed here in the select source.Zero-ETL-integration this is my aurora cluster RDS aurora postgres cluster

  • Aurora PostgreSQL zero-ETL is in preview so there are certain limitations, listed at https://docs.aws.amazon.com/AmazonRDS/latest/AuroraUserGuide/zero-etl.html#zero-etl.reqs-lims-apg , as long as these are satisfied then it should show work for you. Kindly double check these and if still doesn't work then you can open Support Ticket or reach out to your Account Team and someone will be able to assist you further.

0

You can create target table manually in Redshift with "Super" data type. This data type allows you to store up to 16MB of semi-structured JSON data. If you're using AWS Database Migration Service (DMS) to replicate data from a source database to an Amazon Redshift target, the Super data type is supported. You can define the target table column type as Super in the DMS table mapping, the data will be replicated accordingly. For additional details on super data type please refer https://docs.aws.amazon.com/redshift/latest/dg/r_SUPER_type.html

AWS
專家
Nita_S
已回答 1 個月前

您尚未登入。 登入 去張貼答案。

一個好的回答可以清楚地回答問題並提供建設性的意見回饋,同時有助於提問者的專業成長。

回答問題指南