ValidationSettings are not supported for 's3' endpoint
we are using Terraform as CICD, I want to migrate a PostgressSQL Database to S3 bucket, I need CDC and Data validation, when I run the terraform, I have this error: ValidationSettings are not supported for 's3' endpoint I don't see anywhere in the AWS documentation that saying S3 endpoint can't do data validation. How to solve this issue? Can I do or not the data validation with S3 as target?
While using S3 as the DMS target , validation is not supported. + https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Validating.html
You can only use validation, when you use AWS DMS with the below endpoints as Source or Target:
Oracle PostgreSQL-compatible database (PostgreSQL or Amazon Aurora PostgreSQL) MySQL-compatible database (MySQL, MariaDB, or Amazon Aurora MySQL) Microsoft SQL Server IBM Db2 LUW
Unfortunately, there is no feature to validate data with S3 as target at this moment. And that's because of the validation works with DMS. Whenever you try to compare records between any source and target using DMS, DMS runs a SELECT query on both Source and Target, and then compares the values retrieved from both ends. Since its not possible on S3 to run a SELECT query for comparison as its not a database, validation is not supported with S3 as target.
You should think of working on a custom solution to compare the records between PostgreSQL and S3.
How do I create a VPC Endpoint for S3 Interface?asked 4 months ago
Does an S3 bucket have to be provided for a Braket DWave run?asked 2 years ago
S3 bucket permissions to run CloudFormation from different accounts and create Lambda Funtions.asked 5 months ago
Duplicate an existing serverless app infra(built in the console management) using terraformerasked 2 months ago
I need to read S3 data, transform and put into Data Catalog. Should I be using a Crawler?Accepted Answerasked 2 months ago
How can we set s3 lifecycle rule to delete the files form S3 bucket!asked a month ago
ValidationSettings are not supported for 's3' endpointasked 3 months ago
Can we create a Table in Aurora during the infra setup using terraform IaaC code ?Accepted Answerasked a month ago
How can I find what S3 buckets are required by a service?asked 3 months ago
S3 Glacier (2013) Flexible running without Bucket, want to move to Glacier Deep Archiveasked a month ago