Direkt zum Inhalt

sagemaker notebook instance lifecycle configuration - pip install

0

I am running the following pip install, every time I launch my notebook instance.

!pip install torch
!pip install transformers

How do I add it to an existing life cycle configuration? Existing lifecycle configuration looks something like this:

#!/bin/bash

set -ex

# OVERVIEW
# This script stops a SageMaker notebook once it's idle for more than 1 hour (default time)
# You can change the idle time for stop using the environment variable below.
# If you want the notebook the stop only if no browsers are open, remove the --ignore-connections flag
#
# Note that this script will fail if either condition is not met
#   1. Ensure the Notebook Instance has internet connectivity to fetch the example config
#   2. Ensure the Notebook Instance execution role permissions to SageMaker:StopNotebookInstance to stop the notebook 
#       and SageMaker:DescribeNotebookInstance to describe the notebook.
#

# PARAMETERS
IDLE_TIME=3600

echo "Fetching the autostop script"
wget https://raw.githubusercontent.com/aws-samples/amazon-sagemaker-notebook-instance-lifecycle-config-samples/master/scripts/auto-stop-idle/autostop.py


echo "Detecting Python install with boto3 install"

# Find which install has boto3 and use that to run the cron command. So will use default when available
# Redirect stderr as it is unneeded
CONDA_PYTHON_DIR=$(source /home/ec2-user/anaconda3/bin/activate /home/ec2-user/anaconda3/envs/JupyterSystemEnv && which python)
if $CONDA_PYTHON_DIR -c "import boto3" 2>/dev/null; then
    PYTHON_DIR=$CONDA_PYTHON_DIR
elif /usr/bin/python -c "import boto3" 2>/dev/null; then
    PYTHON_DIR='/usr/bin/

I found this link: https://github.com/aws-samples/amazon-sagemaker-notebook-instance-lifecycle-config-samples/blob/master/scripts/install-pip-package-all-environments/on-start.sh

however, I'd like to install multiple packages.

gefragt vor 3 Jahren668 Aufrufe

1 Antwort
0

Hi,

Yes, the script that you found is the way to go.

If you replace pip install --upgrade "$PACKAGE" with 2 lines of your own packages torch and transformers, it should be fine.

Best,

Didier

EXPERTE

beantwortet vor 3 Jahren

  • @Didier_Durand Right, I have the auto-shut lifecycle configuration in place already though. So, I am trying to add/modify the existing script. I have added the script in the op. can you please share how / where I'd add the pip install .... for multiple package installations. Do I need a pip install for all packages? Is there a way to run requirements.txt on start?

  • Hi @Didier_Durand did you have any thoughts on this? how do I add this to the existing timeout lifecycle configuration?

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.