AWS data migration service task, Load complete, replication ongoing, but no rows loaded in redshift or shown in Table statistics
I am running some tests to understand how a pipeline can be made from MongoDB Atlas, (MongoDB v 4.4) to AWS Redshift cluster for data analytics and BI, I have loaded sample data to a MongoDB cluster and I am trying to load the sample data to Redshift cluster using a Database migration tasks, I have a connection to both my source endpoint and my target endpoint tested, I have created the DMS replication instance and the Redshift cluster both in the same region, I have also set a security group for DMS instance and allowed inbound access on Redshift port 5439 from the DMS security group, I have also allowed inbound access on port 27017 on the DMS security group as that is the MongoDB port I am trying to connect to, after setting up the task finishes normally but not even a single row appears in the Redshift tables.
I am really not able to understand what wrong am I doing as there I am not even able to see any obvious errors.
Link with some screenshots to the question:
Based on info shared. Seems like test connections to endpoints having no issues.
--> To troubleshoot the issue further, I would suggest investigate DMS task log with detailed debug on for SOURCE_UNLOAD, TASK_MANAGER, TARGET_LOAD. https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.CustomizingTasks.TaskSettings.Logging.html
After modifying the task restart the task.
--> Also make sure DMS version greater than 3.4.5 as officially support for MongoDB 4.4 starting from this version. https://docs.aws.amazon.com/dms/latest/userguide/CHAP_ReleaseNotes.html#CHAP_ReleaseNotes.DMS345
--> Make sure to follow (prerequisites, limitations etc):
What mode used (Document mode or table mode) for migration?
Hope above info can help some, let us know if further questions.
AWS data migration service task, Load complete, replication ongoing, but no rows loaded in redshift or shown in Table statisticsasked 5 months ago
DMS - How to retain non-ASCII values when running DMS task from MySQL source to Redshift targetasked a month ago
Table id found in `stl_load_errors` doesn't correspond to loaded tableAccepted Answerasked 3 years ago
DMS task is creating full row duplicates in targetasked 3 months ago
AWS DMS migration task doesn't automatically increment the source tableAccepted Answerasked 2 years ago
How to offload historical data to Redshift or S3?Accepted Answerasked 2 years ago
How to estimate size and number of DMS instances for a given number of ongoing replicationsAccepted Answerasked 2 years ago
table not replicating in DMS from MySQL RDS to Redshift RDSasked 3 years ago
Redshift Insert into s3 file data into existing tableasked 2 months ago
How do I define a SQLite expression in DynamoDB 'object-mapping' DMS migration task?asked a month ago