Hi,
Current setup:
I setup a DMS replication instance/endpoints and task from an RDS aurora mysql cluster to a redshift cluster
- it's a full-load task for only one database + table (testing purpose)
- the database is created and table "user_details" is filled with 10K rows
Current outcome:
- the database is created on redshift cluster
- user_details table schema is loaded and created correctly
- user_details table **is empty **
- task ended with Error and Table Error on "user_details"
Expected outcome diff:
- user_details table should have 10K rows
- task is successfull
Logs:
I would expect data loaded from this lines in log:
2023-04-24T08:09:14 [SOURCE_UNLOAD ]I: Unload finished for table 'dbsource'.'user_details' (Id = 1). 10000 rows sent. (streamcomponent.c:3543)
2023-04-24T08:09:14 [TARGET_LOAD ]I: Load finished for table 'dbsource'.'user_details' (Id = 1). 10000 rows received. 0 rows skipped. Volume transferred 6584376. (streamcomponent.c:3838)
But also it seems redshift endpoint has hard time with S3 connection (but the log is not enough verbose to be sure)
2023-04-24T07:26:57 [FILE_FACTORY ]E: Write entire file failed: source = '/rdsdbdata/data/tasks/YYY/cloud/1/LOAD00000001.gzip' target = 'dms-zzz/redshift-XXX/YYY/1/LOAD00000001.csv' open type = 3 [1000731] (at_universal_fs_object.c:316)
And also (still not verbose enough to undesrtand):
2023-04-24T07:20:30 [SOURCE_CAPTURE ]W: Got 0 headers at error exception (anw_retry_strategy.cpp:91)
2023-04-24T07:20:30 [SOURCE_CAPTURE ]E: curlCode: 7, Couldn't connect to server [1001705] (transfer_client.cpp:500)
2023-04-24T07:20:30 [SOURCE_CAPTURE ]E: Failed to upload file '/rdsdbdata/data/tasks/YYY/cloud/1/LOAD00000001.gzip' to bucket 'dms-zzz' as 'redshift-XXX/YYY/1/LOAD00000001.csv', status = 4 (FAILED) [1001705] (transfer_client.cpp:514)
2023-04-24T07:20:30 [SOURCE_CAPTURE ]E: Failed to list bucket 'dms-zzz': error code <>: curlCode: 7, Couldn't connect to server [1001713] (s3_dir_actions.cpp:105)
The default bucket is still created though, and AmazonDMSRedshiftS3Role policy is attached to the dms-access-for-endpoint IAM role has stated here
https://repost.aws/knowledge-center/dms-redshift-connectivity-failures
Is there a way to know everything about the service/assumed role/request parameters for the "Couldn't connect to server" within the log ?
This way I could check if it's a permission issue / network issue / etc...
Any help would be much appreciated
@rePost-User-4113219 Did you find a resolution here? We tried adding a VPC endpoint but are still experiencing the same error.