- Newest
- Most votes
- Most comments
Hi! You can design a workflow such that you use SFTP connectors to List files (StartDirectoryListing operation). Once file list is generated, connector emits an event to EventBridge, and you can use this event to trigger a lambda function that parses the file list and request each file path for transfer using the SFTP connectors' StartFileTransfer operation. Connectors also emit an event to EventBridge after eachfile is transferred, so that event can be used to trigger a custom lambda to delete files from remote server. Currently, SFTP connectors provide operations to List and to Transfer files, but not to delete files.
Consolidate your file operations into batches using a single SFTP connection for downloading and deleting multiple files at once, reducing the overhead of establishing numerous connections. This can significantly improve efficiency and reduce the overall processing time.
SFTP connectors (service offered by AWS) doesn't allow batch file operations or delete capabilities. I've reviewed the documentation available.
Relevant content
- asked 5 months ago
- asked 6 months ago
- AWS OFFICIALUpdated 9 months ago
- AWS OFFICIALUpdated 5 months ago
- AWS OFFICIALUpdated a year ago
- AWS OFFICIALUpdated 3 years ago
Exactly, the delete files functionality is key here and it is unavailable. Will wait until that is enabled on the connector service.