Skip to content

AWS CLI command to monitor start-remote-move

1

I'm using an SFTP Connector as a client to transfer files to/from an SFTP Remote Server. I've been using successfully the following command to move files between folders in the Remote Server:

aws transfer start-remote-move --connector-id c-1234567f5c7442939 --source-path /data/file1.txt --target-path /data/processed/file1.txt

The command above triggers a remote file move for file1.txt from the folder *data *to its subfolder processed. The command returns the following ID:

{
    "MoveId": "7261e15d-3e46-4e02-8a6a-fc44f86c56af"
}

I expect to use the ID above to monitor the completion of the move operation, as start-remote-move is asynchronous, so it just instruct the connector to initiate the move, it doesn't guarantee success. How can I do that?

P.S. I'm hoping to find something similar to "aws transfer list-file-transfer-results", which is used to monitor the completion of a file transfer initiated by "aws transfer start-file-transfer"

asked a year ago218 views

4 Answers
1
Accepted Answer

hi again,

In addition to using CloudWatch logs and EventBridge events as mentioned above, also running directory- listing on both source and target directories and checking the output in S3 bucket, to confirm the move, the file disappears from source and appears in target.

aws transfer start-directory-listing  \
   --connector-id c-AAAA1111BBBB2222C  \ 
   --output-directory-path /amzn-s3-demo-bucket/example/connector-files  \
   --remote-directory-path /home 

https://docs.aws.amazon.com/transfer/latest/userguide/sftp-connector-list-dir.html

Best

answered a year ago

  • Yes, that's what I ended up doing. Easier (for me) to also do it programmatically compared the CloudWatch and EventBridge solutions. Thanks for your help!

1

Hi! As you rightly noted, "aws transfer list-file-transfer-results" can currently only be used to monitor status of a transfer-ID, and does not support monitoring of a move-ID. To monitor the status of a move operation, you can either -

  1. Use the CloudWatch logs generated by your connector. See documentation on example logs emitted by connector here - https://docs.aws.amazon.com/transfer/latest/userguide/cw-example-logs.html#example-sftp-connector-logs
  2. Use the event emitted by your connector after each file operation. Note that the connector emits an event to your EventBridge account after each file operation along with status. More details on the events can be found in service documentation here - https://docs.aws.amazon.com/transfer/latest/userguide/events-detail-reference.html#event-detail-sftp-connector-events

If you still need an API operation to monitor status of a move operation on-demand, please reach out to the Transfer Family service team via your AWS account team or AWS support and submit a feature request.

AWS

answered a year ago

  • Thank you! I've ended up using start-directory-listing, but I appreciate your suggestions.

    BTW, start-directory-listing lacks a monitoring command as well (monitor status of listing-ID would be great), but it's easy to check if the listing file has been created in my local S3. I wish the AWS API were more consistent.

0

hi,

as mentioned in question

aws transfer start-remote-move

It is used to move or rename a file or directory on a remote SFTP server using an AWS Transfer Family SFTP connector.

can you clarify intention was to aws transfer start-file-transfer or aws transfer start-remote-move

as it give you

{ "MoveId": "7261e15d-3e46-4e02-8a6a-fc44f86c56af" }

you’re trying to monitor it with aws transfer list-file-transfer-results, which is designed for aws transfer start-file-transfer (file uploads/downloads) and requires a TransferId. This mismatch causes the ResourceNotFoundException error, as list-file-transfer-results doesn’t recognize a MoveId.

if your intent was to StartFileTransfer:-

https://docs.aws.amazon.com/transfer/latest/userguide/transfer-files-and-track.html

https://docs.aws.amazon.com/transfer/latest/APIReference/API_StartFileTransfer.html

Best

answered a year ago

  • Thank you, Malini. Sorry if my question was not clear.

    I am using the command "aws transfer start-remote-move" to move files between folders on a remote server. This command works asynchronously, which means it starts the file move but does not wait for it to finish.

    Is there a way to monitor the status of the move after it has started?

    I noticed that the command gives a "MoveId." Can I use this MoveId with another "aws transfer" command or through a program (for example, a Java Lambda function) to check the status of the operation?

    Thank you for your help!

-1

To monitor the completion of a remote move operation initiated with the start-remote-move command in AWS Transfer Family, you can use the ListFileTransferResults API operation. This operation allows you to query the status of individual file transfer operations, including move operations.

You would use the MoveId returned from your start-remote-move command as follows:

aws transfer list-file-transfer-results --connector-id c-1234567f5c7442939 --transfer-id 7261e15d-3e46-4e02-8a6a-fc44f86c56af

This command will provide you with the status and details of your remote move operation. The results from this API operation are available for up to 7 days after the operation.

Additionally, you can monitor SFTP connector operations through Amazon EventBridge. AWS Transfer Family generates events for SFTP connector operations, including "SFTP Connector Remote Move Completed" and "SFTP Connector Remote Move Failed" events that you can subscribe to.

If you need to move multiple files, note that you must use a separate StartRemoteMove call for each file or directory. However, you can improve performance by creating concurrent sessions with remote servers that support them, allowing you to move up to 5 files in parallel.
Sources
Monitoring SFTP connectors - AWS Transfer Family
Move, rename, or delete files or directories on the remote server - AWS Transfer Family
AWS Transfer Family events - Amazon EventBridge

answered a year ago

  • This answer is completely wrong. list-file-transfer-results cannot be used to monitor a start-remote-move and requires a transfer-id. Providing the move-id for the transfer-id argument does not work and causes the following error: "An error occurred (ResourceNotFoundException) when calling the ListFileTransferResults operation: Unknown TransferId"

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.