By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

Sagemaker batch transform

0

Hi, it seems that Sagemaker Batch Transform is limited to 100MB payloads I'd like to run preds against a 5GB csv file, what the recommended way to do so?

  • clouduser
    2 years ago

    @Olivier_CR - i'm curious , how are you creating this batch jobs, are you doing it via console, cli or using lambda to do it via code?

Topics
Machine Learning & AI
Tags
Amazon SageMaker
Language
English
AWS
EXPERT
Olivier_CR
asked 6 years ago900 views
1 Answer
0
Accepted Answer

SageMaker Batch Transform will automatically split your input file into whatever payload size is specified if you use "SplitType": "Line" and "BatchStrategy": "MultiRecord". There's no need to split files yourself or to use large payload sizes unless you have very large single records.

Hope that helps!

AWS-User-8377309
answered 6 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content