ValidationSettings are not supported for 's3' endpoint

0

we are using Terraform as CICD, I want to migrate a PostgressSQL Database to S3 bucket, I need CDC and Data validation, when I run the terraform, I have this error: ValidationSettings are not supported for 's3' endpoint I don't see anywhere in the AWS documentation that saying S3 endpoint can't do data validation. How to solve this issue? Can I do or not the data validation with S3 as target?

2개 답변
0

While using S3 as the DMS target , validation is not supported. + https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Validating.html

You can only use validation, when you use AWS DMS with the below endpoints as Source or Target:

Oracle
PostgreSQL-compatible database (PostgreSQL or Amazon Aurora PostgreSQL)
MySQL-compatible database (MySQL, MariaDB, or Amazon Aurora MySQL)
Microsoft SQL Server
IBM Db2 LUW

Unfortunately, there is no feature to validate data with S3 as target at this moment. And that's because of the validation works with DMS. Whenever you try to compare records between any source and target using DMS, DMS runs a SELECT query on both Source and Target, and then compares the values retrieved from both ends. Since its not possible on S3 to run a SELECT query for comparison as its not a database, validation is not supported with S3 as target.

You should think of working on a custom solution to compare the records between PostgreSQL and S3.

AWS
D-Rao
답변함 2년 전
0

AWS Database Migration Service (AWS DMS) now supports validation to ensure that data is migrated accurately to S3.

If data validation is enabled for the migration then validation begins immediately after a full load is performed for a table. In the CDC phase, data validation compares the incremental changes as they occur. During data validation, AWS DMS compares each row in the source with its corresponding row at the target, verifies the rows contain the same data, and reports any mismatches.

Amazon S3 target data validation - https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Validating_S3.html

AWS
지원 엔지니어
Arnab_S
답변함 3달 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠