FTP to S3 bucket

0

What is the best way to transfer files from FTP to an S3 bucket

gefragt vor 9 Monaten2437 Aufrufe
3 Antworten
1

Hello, thank you for your question. Could you share more details about your use case so we can better assist you? You can use an AWS Transfer Family FTP-enabled server upload and download data to and from S3. More details about creating an FTP-enabled server can be found here.

AWS
EXPERTE
beantwortet vor 9 Monaten
profile picture
EXPERTE
überprüft vor 9 Monaten
0

I would like to download ~1TB to an S3 bucket from a public data depository. One option is through FTP although there is an option to use Globus

beantwortet vor 9 Monaten
0

You can put SFTP or FTPS in front of the bucket using AWS Transfer Family. Note that if the source is initiating a connection over the internet to a public endpoint (which appears to be the case here) then plain FTP cannot be used, only SFTP or FTPS https://repost.aws/knowledge-center/aws-sftp-endpoint-type

Uploading 1TB will take a long time (at a sustained 100Mbps it will take about a day).

Away from (S)FTP if you are aiming to upload a relatively small number of very large files, S3 multipart upload may be more suitable https://docs.aws.amazon.com/AmazonS3/latest/userguide/mpuoverview.html

There is a useful knowledge document about using multipart upload here https://repost.aws/knowledge-center/s3-multipart-upload-cli

profile picture
EXPERTE
Steve_M
beantwortet vor 9 Monaten
profile picture
EXPERTE
überprüft vor 9 Monaten

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen