En utilisant AWS re:Post, vous acceptez les AWS re:Post Conditions d’utilisation
AWS re:Postre:Post

What is the recommended way to copy an entire S3 bucket in Step Functions, including large files, so that size limits and timeouts won't cause problems?

0

What is the recommended way to copy an entire S3 bucket (non-versioned) in Step Functions, including large files, so that size limits and timeouts won't cause problems?

Sujets
StockageSans serveurIntégration d'applications
Balises
Simple service de stockage AmazonFonctions d’étapes AWS
Langue
English
wab
demandé il y a un an918 vues
1 réponse
2

S3 buckets can have a virtually unlimited size. This means the copy operation could take some time. To perform an S3 bucket to bucket copy asynchronously, I would look at using a Lambda function to call an S3 batch operation in your StepFunction.

S3 Batch Operations are an Amazon S3 data management feature that lets you manage billions of objects at scale with just a single API request. More on S3 Batch Operations here . . .

https://aws.amazon.com/s3/features/batch-operations/

profile pictureAWS
tedtrent
répondu il y a un an
profile pictureAWS
EXPERT
Brettski-AWS
vérifié il y a un an
  • wab
    il y a un an

    Thanks @tedtrent. Does this approach require the S3 batch operation to be created in advance (with source and destination buckets predefined)? Or can the bucket references still be dynamic, based on the input parameters of the step function?

Vous n'êtes pas connecté. Se connecter pour publier une réponse.

Une bonne réponse répond clairement à la question, contient des commentaires constructifs et encourage le développement professionnel de la personne qui pose la question.

Instructions pour répondre aux questions

Contenus pertinents