By using AWS re:Post, you agree to the Terms of Use

Questions tagged with Amazon EC2

Sort by most recent

Browse through the questions and answers listed below or filter and sort to narrow down your results.

aws command installed with awscli library inside a python venv on Windows invokes a python OUTSIDE the venv

For awscli 1.25.86, installing on a freshly minted Windows EC2 (Windows Server 2022 Dataserver), I did this: 1. selected my home directory (`cd`) 2. installed `pyenv` (e.g., via PowerShell using https://github.com/pyenv-win/pyenv-win#power-shell). This said it didn't succeed but seems to have fully installed pyenv. This is not the bug. I had to select a new PowerShell to see the effect of having installed `pyenv`) 3. told `pyenv` to install python 3.8 (`pyenv install 3.8.10`) 4. selected python 3.8 globally (`pyenv global 3.8.10`) 5. created a virtual environment (`pyenv exec python -m venv myvenv`) 6. entered the venv (`myvenv\scripts\activate`) 7. installed `awscli` (`pip install awscli`) 8. tried to invoke `awscli` (`aws --version`). This gives the message `File association not found for extension .py` which is an ignorable problem followed by an error that is the bug I'm reporting: ``` Traceback (most recent call last): File "C:\Users\Andrea\GitHub\Submit4DN\s4dn_venv\Scripts\aws.cmd", line 50, in <module> import awscli.clidriver ModuleNotFoundError: No module named 'awscli' ``` After studying this problem, I believe I know the source of this problem, and am pretty sure it's in the `awscli` library. The library installs `myvenv\scripts\aws.cmd` which implements the `aws` command inside the virtual environment, but that script sniffs around for a `python` to invoke and finds one _outside_ of the virtual environment. The problem isn't that it tries to get out of the virtual environment, it's just apparently oblivious to the presence of one, and so it isn't picky about which python it finds. it successively seeks `python.cmd`, `python.bat`, and `python.exe` (see line 7 of `myenv\scripts\aws.cmd`) but finds `python.cmd` first, and that is not inside the virtual environment. Had it checked `python.exe` first, it would have found the one in the virtual environment. If you swap the order of `(cmd bat exe)` on line 7 of `aws.cmd` so that it searches `(exe bat cmd)` it will invoke the python within the virtual env and so will find the `awscli` that was just installed within the virtual environment. That's not necessarily the right fix. It still feels fragile. But it seems to me that this proves it's the locus of the problem. Another somewhat workaround is to install `awscli` outside of the virtual environment by doing `deactivate`, then `pip install awscli`, then `myvenv/scripts/activate`, and then finally trying `aws --version` and it will work _except_ that if you change to another version of python globally via pyenv, the `aws` command within the venv will break again unless you again reinstall `awscli` in each globally selected python. I don't have a good fix to suggest because I'm not current on writing of Windows shell scripts, but imagine it involves a different way of discovering python that gives strong preference to a venv if one is active, e.g., by noticing there is a `%VIRTUAL_ENV%` in effect and just invoking `python`(since virtual envs always have a `python`) or `%VIRTUAL_ENV%\scripts\python` if you're wanting to be double-sure. Note that I was able to reproduce this problem on my professional desktop version of Windows 10 at my home as well, so it's nothing specific to the EC2 itself, that's just a way to show that this problem can be demonstrated in a clean environment. The problem seems pretty definitely in the `awscli` library. Whatever solution you pick, I hope this illustrates the issue clearly enough that you can quickly issue some sort of fix to the `awscli` library because the present situation is just plain broken and this is impacting some instructions we're trying to give some users about how to access our system remotely. I'd rather not be advising users to edit scripts they got from elsewhere, nor do I want to supply alternate scripts for them to use. Things should just work.
1
answers
0
votes
53
views
profile picture
asked 2 days ago