SFTP server's storage is the S3 bucket. Ie. "to send a file from S3 bucket to an SFTP server" you need to do nothing but to upload the object into the same bucket as you configured for the SFTP user. If you would want to copy automatically objects from another bucket to one configured for SFTP, then you want to look into triggering a lambda function when object is uploaded (or modified) in your source bucket.
Please note that your SFTP users root directory is by default
For more complex directory mappings, logical directories can be helpful
In regards to copying S3 bucket data to SFTP server, these are few options to do this -
- Use S3 Event notifications. S3 event notification is a mechanism to trigger a notification to different AWS destinations when a certain event happens in a s3 bucket. For example, you can invoke a Lambda function when there is an upload of new object to s3. With this mechanism, you can trigger a Lambda function upon upload of new object in s3 bucket, this Lambda function will have to take care of authenticating On-premise SFTP server and copy files to this server. This function can assume s3 permissions from IAM role associated to it. You can refer below document to know more about s3 event notification.
Note - In order to upload and Download from s3, you need to have s3:PutObject and s3:GetObject permissions. But, you may need to add more permission depending upon the actions you are doing while trying to access s3. For example, if you are trying to do a GetObject on previous version, you need to use s3:GetObjectVersion. For s3 actions, you can refer below document -
- Using shell script: You can setup an automation on an EC2 Instance (or an On-premise server) and copy files from s3 bucket to SFTP server using regular shell commands. The important thing to consider is to authenticate SFTP server. For s3 authentication, you can use IAM instance profile to assume the credentials.
Refer the below document and see if it helps to authenticate SFTP using a shell script. Please note that this is a third party document and AWS doesn't own it, I cannot guarantee any information in this document. I would recommend you to go through in detail before implementing this.
You could also consider executing shell script in the SFTP server itself. Typically, SFTP server files are stored on local disks and can be accessed directly from the OS itself. If you can automate a script, you can use aws cli s3 command to copy files directly to File system instead of authenticating SFTP server using shell script.
- Using AWS Transfer SFTP server that exposes objects in s3 bucket using SFTP protocol. You can authenticate using ssh key pair using supported SFTP client and access files from s3 bucket.
I want to explain a little about the Transfer Family service. The AWS Transfer Family provides fully managed support for file transfers directly into and out of Amazon S3 or Amazon EFS. When you create a SFTP enabled Transfer Family server and create a user that connects to a S3 bucket, then once you connect to this Transfer server via the user from your machine, you can then view all of the current files already in your S3 bucket.
The transfer family server views what is in the S3 bucket and what you could do is download/transfer the files from the S3 bucket into your local machine or you can upload/transfer files from your local machine to your S3 bucket.
For more information on getting started with Transfer Family service, such as creating servers and users, please refer here -
Can we chmod for a "subdir" under an S3 bucket from an AWS SFTP server?Accepted Answerasked 7 months ago
AWS TRANSFER FAMILY SFTP SERVERasked 6 months ago
Using S3 bucket as a file server for the publicasked 2 months ago
From S3 bucket to SFTP serverasked 24 days ago
S3 policy - Transfer Family SFTPasked 2 months ago
Accessing a SFTP (Transfer Family) endpoint from several domainsasked 5 months ago
AWS Transfer Family -SFTP Gateway : Is SSE-C for S3 supported?asked 7 months ago
AWS Transfer Family - Private SFTP server connection closedasked 5 months ago
HostKey for SFTP Transfer FamilyAccepted Answerasked 5 months ago
How to SFTP to my AWS spaceasked 7 days ago