What is the cluster manager in SageMaker Spark Processing?


SageMaker Processing can launch multi-instance jobs. What is the underlying cluster manager? Yarn? Mesos? Something custom?

asked 4 years ago490 views
1 Answer
Accepted Answer

The Spark container uses YARN - for ref the bootstrap script on github: https://github.com/aws/sagemaker-spark-container/blob/master/src/smspark/bootstrapper.py and the Dockerfile with hadoop-yarn dependencies

answered 4 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions