Migration & Transfer

Easily migrate to AWS and see business results faster. We’ve taken our experience with migrations to AWS and developed a broad set of first and third party tools and services to help simplify and accelerate migrations. Our migration tool catalog includes an end-to-end set of tools to ensure your investment achieves your desired business outcomes.

Recent questions

see all
1/18

Copy Data from EFS to EFS in Same Account & Region

Hi Everyone, Testing what should be a simple use case in DataSync. For the moment, just using the console to setup two EFS locations (source and destination) and one task (unscheduled) to copy all data (from task path) from source to destination. Since these two file systems operate in the same region and the same account, I think (hope) I can use a serverless setup that does not require (for example) NFS to EFS with an agent, etc. I have setup only these resources in DataSync: - Location: Source EFS in us-east-1 with correct subnet and SG that has "/" as the mount path - Location: Destination EFS location in us-east-1 with correct subnet and SG that has "/" as the mount path - Task: in us-east-1 that configures source and destination and mostly uses default settings, though has this notable config: Data transfer configuration > Specific files and folders > Include patterns > /home/user/subfolder/ Everything looks good and I am able to manually start the task, which moves through the Task status: *Launching* and then shows *Success*, but I see that only 1 file is transferred (there are 122 files on the source file system that I am attempting to copy). Looking at the destination EC2 instance, I see only a new hidden directory named: `.aws-datasync` created in `/home/user/subfolder/`. I thought perhaps my include pattern was slightly off and so tested these: - /home/user/subfolder/ - /home/user/subfolder - /home/user/subfolder/* - /home/user/subfolder* but the results are always the same: task success but no files (except the hidden dir) transferred ... Any help is much appreciated. Ben
1
answers
0
votes
19
views
bwmills
asked a day ago

RDS "Failed to start" when i changed the parameter group

with the default parameter group, there is no problem to pause and start RDS but with the custom parameter group, i face "Failed to start" error everytime i pause and start RDS after the "Failed to start" error, RDS backup and reboot the database automatically, and it works fine at the end. - i've changed the parameter group from "default.mysql8.0" to the custom parameter group - i haven't changed any parameters of the new parameter group, every parameters is same with the default parameter group this is the log of the lastest pause and start, i have no idea what is the problem, thanks for the any advice `2022-12-03T16:31:42.090150Z 0 [Warning] [MY-011068] [Server] The syntax 'log_s lave_updates' is deprecated and will be removed in a future release. Please use log_replica_updates instead. 2022-12-03T16:31:42.090159Z 0 [Warning] [MY-011069] [Server] The syntax '--m aster-info-repository' is deprecated and will be removed in a future release. 2022-12-03T16:31:42.090164Z 0 [Warning] [MY-011069] [Server] The syntax '--m aster-info-repository' is deprecated and will be removed in a future release. 2022-12-03T16:31:42.090220Z 0 [Warning] [MY-011069] [Server] The syntax '--relay-log-info-file' is deprecated and will be removed in a future release. 2022-12-03T16:31:42.090225Z 0 [Warning] [MY-011069] [Server] The syntax '--relay-log-info-repository' is deprecated and will be removed in a future release. 2022-12-03T16:31:42.090252Z 0 [Warning] [MY-011068] [Server] The syntax 'skip_s lave_start' is deprecated and will be removed in a future release. Please use skip_replica_start instead. 2022-12-03T16:31:42.090259Z 0 [Warning] [MY-011068] [Server] The syntax 's lave_exec_mode' is deprecated and will be removed in a future release. Please use replica_exec_mode instead. 2022-12-03T16:31:42.090265Z 0 [Warning] [MY-011068] [Server] The syntax 's lave_load_tmpdir' is deprecated and will be removed in a future release. Please use replica_load_tmpdir instead. 2022-12-03T16:31:42.090437Z 0 [Warning] [MY-010101] [Server] Insecure configuration for --secure-file-priv: Location is accessible to all OS users. Consider choosing a different directory. 2022-12-03T16:31:42.090470Z 0 [Warning] [MY-010918] [Server] 'default_authentication_plugin' is deprecated and will be removed in a future release. Please use authentication_policy instead. 2022-12-03T16:31:42.090478Z 0 [System] [MY-010116] [Server] /rdsdbbin/mysql/bin/mysqld (mysqld 8.0.30) starting as process 731 2022-12-03T16:31:42.099878Z 0 [Warning] [MY-013907] [InnoDB] Deprecated configuration parameters innodb_log_file_size and/or innodb_log_files_in_group have been used to compute innodb_redo_log_capacity=268435456. Please use innodb_redo_log_capacity instead. 2022-12-03T16:31:42.101840Z 1 [System] [MY-013576] [InnoDB] InnoDB initialization has started. 2022-12-03T16:31:42.104547Z 1 [Warning] [MY-012191] [InnoDB] Scan path '/rdsdbdata/db/innodb' is ignored because it is a sub-directory of '/rdsdbdata/db/ 2022-12-03T16:31:42.513654Z 1 [System] [MY-013577] [InnoDB] InnoDB initialization has ended. 2022-12-03T16:31:42.827606Z 0 [Warning] [MY-013414] [Server] Server SSL certificate doesn't verify: unable to get issuer certificate 2022-12-03T16:31:42.827781Z 0 [System] [MY-013602] [Server] Channel mysql_main configured to support TLS. Encrypted connections are now supported for this channel. 2022-12-03T16:31:42.887455Z 0 [System] [MY-010931] [Server] /rdsdbbin/mysql/bin/mysqld: ready for connections. Version: '8.0.30' socket: '/tmp/mysql.sock' port: 3306 Source distribution. 2022-12-03T16:34:08.192155Z 11 [Warning] [MY-010055] [Server] IP address '125.178.113.15' could not be resolved: Name or service not known 2022-12-03T16:34:08.192271Z 12 [Warning] [MY-010055] [Server] IP address '125.178.113.15' could not be resolved: Name or service not known 2022-12-03T16:34:08.192151Z 10 [Warning] [MY-010055] [Server] IP address '125.178.113.15' could not be resolved: Name or service not known 2022-12-03T16:34:08.192487Z 9 [Warning] [MY-010055] [Server] IP address '125.178.113.15' could not be resolved: Name or service not known `
1
answers
0
votes
5
views
asked 2 days ago
1
answers
0
votes
25
views
asked 7 days ago

AWS DataSync sometimes creates 0 sized S3 objects and doesn't report any error

I am using AWS Datasync to copy/backup data from bucket A (Singapore region) to bucket B (Ohio region). I had done this in steps via multiple sequential back-to-back executions of the DataSync task to make it more manageable for me to monitor. The DataSync task was configured with the option "Log basic information such as transfer errors". One such task execution was run with a particular include filter - say "/folderA*". A few minutes into the execution I realized I wanted to make some changes and therefore cancelled the execution. The execution stopped without any transfer error message in the Cloudwatch log stream. I subsequently started the execution again with the same include filter and this task succeeded. There were no errors in the log stream for this task as well. However, I can now see that some of the files in bucket A weren't transferred correctly. These files show up as 0-sized files in bucket B. Why did this happen? Why were no errors reported during either of the two task executions? How can I rely on AWS Datasync to do the transfer reliably if such issues happen? Note that this was about a 40TB transfer and I found out the issue only because I ran my own sanity checks on the transferred data. PS: Source data is in standard storage class. Destination location is standard storage class, but the destination bucket has a lifecycle rule that transitions incoming data to intelligent tiering immediately
1
answers
0
votes
20
views
asked 11 days ago

Recent articles

see all
1/3

Popular users

see all
1/18

Learn AWS faster by following popular topics

1/1