By using AWS re:Post, you agree to the Terms of Use

What is the cluster manager in SageMaker Spark Processing?

0

SageMaker Processing can launch multi-instance jobs. What is the underlying cluster manager? Yarn? Mesos? Something custom?

1 Answer
1
Accepted Answer

The Spark container uses YARN - for ref the bootstrap script on github: https://github.com/aws/sagemaker-spark-container/blob/master/src/smspark/bootstrapper.py and the Dockerfile with hadoop-yarn dependencies

answered 2 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions