AWS data migration service task, Load complete, replication ongoing, but no rows loaded in redshift or shown in Table statistics
I am running some tests to understand how a pipeline can be made from MongoDB Atlas, (MongoDB v 4.4) to AWS Redshift cluster for data analytics and BI, I have loaded sample data to a MongoDB cluster and I am trying to load the sample data to Redshift cluster using a Database migration tasks, I have a connection to both my source endpoint and my target endpoint tested, I have created the DMS replication instance and the Redshift cluster both in the same region, I have also set a security group for DMS instance and allowed inbound access on Redshift port 5439 from the DMS security group, I have also allowed inbound access on port 27017 on the DMS security group as that is the MongoDB port I am trying to connect to, after setting up the task finishes normally but not even a single row appears in the Redshift tables.
I am really not able to understand what wrong am I doing as there I am not even able to see any obvious errors.
Link with some screenshots to the question:
Hi,
Based on info shared. Seems like test connections to endpoints having no issues.
--> To troubleshoot the issue further, I would suggest investigate DMS task log with detailed debug on for SOURCE_UNLOAD, TASK_MANAGER, TARGET_LOAD. https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Tasks.CustomizingTasks.TaskSettings.Logging.html
After modifying the task restart the task.
--> Also make sure DMS version greater than 3.4.5 as officially support for MongoDB 4.4 starting from this version. https://docs.aws.amazon.com/dms/latest/userguide/CHAP_ReleaseNotes.html#CHAP_ReleaseNotes.DMS345
--> Make sure to follow (prerequisites, limitations etc):
https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Source.MongoDB.html https://docs.aws.amazon.com/dms/latest/userguide/CHAP_Target.Redshift.html
What mode used (Document mode or table mode) for migration?
Hope above info can help some, let us know if further questions.
Relevant questions
AWS data migration service task, Load complete, replication ongoing, but no rows loaded in redshift or shown in Table statistics
asked 5 months agoDMS - How to retain non-ASCII values when running DMS task from MySQL source to Redshift target
asked a month agoTable id found in `stl_load_errors` doesn't correspond to loaded table
Accepted Answerasked 3 years agoDMS task is creating full row duplicates in target
asked 3 months agoAWS DMS migration task doesn't automatically increment the source table
Accepted Answerasked 2 years agoHow to offload historical data to Redshift or S3?
Accepted Answerasked 2 years agoHow to estimate size and number of DMS instances for a given number of ongoing replications
Accepted Answerasked 2 years agotable not replicating in DMS from MySQL RDS to Redshift RDS
asked 3 years agoRedshift Insert into s3 file data into existing table
asked 2 months agoHow do I define a SQLite expression in DynamoDB 'object-mapping' DMS migration task?
asked a month ago