- Newest
- Most votes
- Most comments
Unfortunately, enabling Notebook Jobs directly in EMR Studio Jupyter notebooks (created using EMR workspaces) is not currently a built-in feature like it is in Amazon SageMaker. The Notebook Jobs functionality you're referring to is specific to SageMaker and is not natively available in EMR Studio.
However, there are some alternatives you can consider for scheduling and automating notebook execution in EMR:
-
EMR Steps: You can use EMR Steps to run Jupyter notebooks as batch jobs. This involves converting your notebook to a Python script and submitting it as a step to your EMR cluster.
-
Apache Airflow or AWS Step Functions: These can be used to orchestrate and schedule the execution of your EMR notebooks as part of a larger workflow.
-
AWS Lambda with CloudWatch Events: You can set up a Lambda function triggered by CloudWatch Events to execute your notebook on a schedule.
-
Custom scripting: You can create custom scripts that use the EMR API to submit notebook execution jobs programmatically.
It's worth noting that in older versions of EMR (5.30.1 and earlier), there was a feature to install kernels and additional Python libraries on the cluster primary node for EMR Notebooks. However, this feature was disabled by default in later versions due to potential issues.
If you specifically need the Notebook Jobs functionality as implemented in SageMaker, you might want to consider using SageMaker notebooks instead of EMR Studio, as the feature is not currently available in EMR Studio Jupyter notebooks.
For the most up-to-date information on EMR features and capabilities, it's always best to check the official AWS documentation or contact AWS support directly.
Sources
Amazon EMR archive of release notes - Amazon EMR
Create a notebook job in Studio - Amazon SageMaker AI
SageMaker Notebook Jobs - Amazon SageMaker AI
answered 2 years ago
Relevant content
asked 4 years ago
asked 5 years ago

The re:Post Agent generated answer above is correct regarding Notebook Jobs in EMR Studio Jupyter notebooks