how to create/activate a virtual env programatically in sagemaker training instance?

Question

example contrived for this question, I want to run my train.py file within a virtual environment in a sagemaker training instance ( i know, i don't have to , but i want to know if i can)  , to do this , I create a virtual environment via venv package , the virtual environment is created , (sample code below) even with the flag with_pip = True, the pip is not added/installed within virtual environment.  this setup works in my local machine while testing. so what i do is create the virtual environment , then install packages via => subprocess.call(['venv/bin/python3', '-m', 'pip',  "install", "pandas"]). this installs the packages i want under the virtual environment. How do i activate a virtual environment now?  can i write a shell script with code to activate my virtual env for this and run it from my train.py file . how would that look like?

```
import subprocess
import sys
import venv
import os

venv.create("venv", with_pip=True)

subprocess.call(['venv/bin/pip3', "install", "pandas"])
```

Answer

As you mention AWS SageMaker provides its own environment for training models, and I would say it's generally recommended to use that environment rather than creating a separate virtual environment.

However, if you did need to set up the situation above in a file I think you would pull out the code you wrote to a file and then run it via `exec` in the main file. Maybe like below, but again not sure its the best/most secure method outside of an example case.

```
#activate_this.py
import subprocess
import sys
import venv
import os
venv.create("venv", with_pip=True)
subprocess.call(['venv/bin/pip3', "install", "pandas"])
```

```
#train.py
activate_this_file = "venv/bin/activate_this.py"

with open(activate_this_file) as f:
    code = compile(f.read(), activate_this_file, 'exec')
    exec(code, dict(__file__=activate_this_file))
```

-Zac

how to create/activate a virtual env programatically in sagemaker training instance?

Relevant content