By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Can AWS Batch jobs use SageMaker FastFile mode?

0

Can a regular (non-machine learning) Python job submitted via AWS Batch make use of SageMaker FastFile mode to stream data from S3 to the container? Or is there any equivalent of SageMaker FastFile mode in other Python libraries? The key point is about whether it's possible to avoid the need to copy data from S3 to the EBS or instance store volume before processing.

Topics
Machine Learning & AICompute
Tags
Amazon SageMakerAWS Batch
Language
English
AWS
Nathaniel Ng
asked 2 years ago472 views
1 Answer
1

Don't think FastFile mode is available outside of SageMaker Training. Depending on your use-case/trade-offs, some of the options are

  1. Use SageMaker Training as your processing host
  2. Could try FSx/EFS
Anonymous
answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content