Docker File not building in scikit_bring_your_own example for Sagemaker due to Python version conflict
0
I'm trying to build and run the code in this example: https://github.com/aws/amazon-sagemaker-examples/tree/main/advanced_functionality/scikit_bring_your_own/container
The project includes a script, build_and_push.sh, to build the example and push it to the cloud. However, the script fails building the Docker image. Here's the error:
~/Work/dev/aws/container$ ./build_and_push.sh myimg
Login Succeeded
Logging in with your password grants your terminal complete access to your account.
For better security, log in with a limited-privilege personal access token. Learn more at https://docs.docker.com/go/access-tokens/
[+] Building 4.9s (9/11) docker:desktop-linux
=> [internal] load build definition from Dockerfile 0.0s
=> => transferring dockerfile: 1.56kB 0.0s
=> [internal] load .dockerignore 0.0s
=> => transferring context: 2B 0.0s
=> [internal] load metadata for docker.io/library/ubuntu:18.04 0.0s
=> [1/7] FROM docker.io/library/ubuntu:18.04 0.0s
=> [internal] load build context 0.0s
=> => transferring context: 250B 0.0s
=> CACHED [2/7] RUN apt-get -y update && apt-get install -y --no-install-recommends wget python3-pip python3-setuptools n 0.0s
=> CACHED [3/7] RUN ln -s /usr/bin/python3 /usr/bin/python 0.0s
=> CACHED [4/7] RUN ln -s /usr/bin/pip3 /usr/bin/pip 0.0s
=> ERROR [5/7] RUN pip --no-cache-dir install numpy==1.16.2 scipy==1.2.1 scikit-learn==0.20.2 pandas flask gunicorn 4.9s
------
> [5/7] RUN pip --no-cache-dir install numpy==1.16.2 scipy==1.2.1 scikit-learn==0.20.2 pandas flask gunicorn:
0.626 Collecting numpy==1.16.2
1.015 Downloading https://files.pythonhosted.org/packages/cf/8d/6345b4f32b37945fedc1e027e83970005fc9c699068d2f566b82826515f2/numpy-1.16.2.zip (5.1MB)
1.426 Collecting scipy==1.2.1
1.629 Downloading https://files.pythonhosted.org/packages/a9/b4/5598a706697d1e2929eaf7fe68898ef4bea76e4950b9efbe1ef396b8813a/scipy-1.2.1.tar.gz (23.1MB)
4.778 Complete output from command python setup.py egg_info:
4.778 Traceback (most recent call last):
4.778 File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 154, in save_modules
4.778 yield saved
4.778 File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 195, in setup_context
4.778 yield
4.778 File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 250, in run_setup
4.778 _execfile(setup_script, ns)
4.778 File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 45, in _execfile
4.778 exec(code, globals, locals)
4.778 File "/tmp/easy_install-_0lgtcc2/numpy-1.26.1/setup.py", line 22, in <module>
4.778 import sys
4.778 RuntimeError: Python version >= 3.9 required.
4.778
4.778 During handling of the above exception, another exception occurred:
4.778
4.778 Traceback (most recent call last):
4.778 File "<string>", line 1, in <module>
4.778 File "/tmp/pip-build-chzpendt/scipy/setup.py", line 492, in <module>
4.778 setup_package()
4.778 File "/tmp/pip-build-chzpendt/scipy/setup.py", line 488, in setup_package
4.778 setup(**metadata)
4.778 File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 128, in setup
4.778 _install_setup_requires(attrs)
4.778 File "/usr/lib/python3/dist-packages/setuptools/__init__.py", line 123, in _install_setup_requires
4.778 dist.fetch_build_eggs(dist.setup_requires)
4.778 File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 513, in fetch_build_eggs
4.778 replace_conflicting=True,
4.778 File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 774, in resolve
4.778 replace_conflicting=replace_conflicting
4.778 File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 1057, in best_match
4.778 return self.obtain(req, installer)
4.778 File "/usr/lib/python3/dist-packages/pkg_resources/__init__.py", line 1069, in obtain
4.778 return installer(requirement)
4.778 File "/usr/lib/python3/dist-packages/setuptools/dist.py", line 580, in fetch_build_egg
4.778 return cmd.easy_install(req)
4.778 File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 698, in easy_install
4.778 return self.install_item(spec, dist.location, tmpdir, deps)
4.778 File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 724, in install_item
4.778 dists = self.install_eggs(spec, download, tmpdir)
4.778 File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 909, in install_eggs
4.778 return self.build_and_install(setup_script, setup_base)
4.778 File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 1177, in build_and_install
4.778 self.run_setup(setup_script, setup_base, args)
4.778 File "/usr/lib/python3/dist-packages/setuptools/command/easy_install.py", line 1163, in run_setup
4.778 run_setup(setup_script, args)
4.778 File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 253, in run_setup
4.778 raise
4.778 File "/usr/lib/python3.6/contextlib.py", line 99, in __exit__
4.778 self.gen.throw(type, value, traceback)
4.778 File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 195, in setup_context
4.778 yield
4.778 File "/usr/lib/python3.6/contextlib.py", line 99, in __exit__
4.778 self.gen.throw(type, value, traceback)
4.778 File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 166, in save_modules
4.778 saved_exc.resume()
4.778 File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 141, in resume
4.778 six.reraise(type, exc, self._tb)
4.778 File "/usr/lib/python3/dist-packages/setuptools/_vendor/six.py", line 685, in reraise
4.778 raise value.with_traceback(tb)
4.778 File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 154, in save_modules
4.778 yield saved
4.778 File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 195, in setup_context
4.778 yield
4.778 File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 250, in run_setup
4.778 _execfile(setup_script, ns)
4.778 File "/usr/lib/python3/dist-packages/setuptools/sandbox.py", line 45, in _execfile
4.778 exec(code, globals, locals)
4.778 File "/tmp/easy_install-_0lgtcc2/numpy-1.26.1/setup.py", line 22, in <module>
4.778 import sys
4.778 RuntimeError: Python version >= 3.9 required.
4.778
4.778 ----------------------------------------
4.854 Command "python setup.py egg_info" failed with error code 1 in /tmp/pip-build-chzpendt/scipy/
------
Dockerfile:26
--------------------
24 | # a significant amount of space. These optimizations save a fair amount of space in the
25 | # image, which reduces start up time.
26 | >>> RUN pip --no-cache-dir install numpy==1.16.2 scipy==1.2.1 scikit-learn==0.20.2 pandas flask gunicorn
27 |
28 | # Set some environment variables. PYTHONUNBUFFERED keeps Python from buffering our standard
--------------------
ERROR: failed to solve: process "/bin/sh -c pip --no-cache-dir install numpy==1.16.2 scipy==1.2.1 scikit-learn==0.20.2 pandas flask gunicorn" did not complete successfully: exit code: 1
Error response from daemon: No such image: myimg:latest
The push refers to repository [453802426162.dkr.ecr.us-west-2.amazonaws.com/myimg]
An image does not exist locally with the tag: 453802426162.dkr.ecr.us-west-2.amazonaws.com/myimg
As can be seen, the failure is due to a failure building the Docker image. And that failure is due to an incorrect python version. Python 3.9 or greater is required to install the libraries:
4.778 RuntimeError: Python version >= 3.9 required.
However, the version of Python installed on the Docker image is 3.6.9:
root@684b95555e06:/usr/bin# python --version
Python 3.6.9
Here are the relevant parts of the Dockerfile:
# Build an image that can do training and inference in SageMaker
# This is a Python 3 image that uses the nginx, gunicorn, flask stack
# for serving inferences in a stable way.
FROM ubuntu:18.04
MAINTAINER Amazon AI <sage-learner@amazon.com>
RUN apt-get -y update && apt-get install -y --no-install-recommends \
wget \
python3-pip \
python3-setuptools \
nginx \
ca-certificates \
&& rm -rf /var/lib/apt/lists/*
RUN ln -s /usr/bin/python3 /usr/bin/python
RUN ln -s /usr/bin/pip3 /usr/bin/pip
# Here we get all python packages.
# There's substantial overlap between scipy and numpy that we eliminate by
# linking them together. Likewise, pip leaves the install caches populated which uses
# a significant amount of space. These optimizations save a fair amount of space in the
# image, which reduces start up time.
RUN pip --no-cache-dir install numpy==1.16.2 scipy==1.2.1 scikit-learn==0.20.2 pandas flask gunicorn
(The complete Docker file can be found here: https://github.com/aws/amazon-sagemaker-examples/blob/main/advanced_functionality/scikit_bring_your_own/container/Dockerfile.)
This problem was originally reported in the repository for the example on Feb 17, 2022 (https://github.com/aws/amazon-sagemaker-examples/issues/3181). However, it appears the problem hasn't been addressed.
It look like it may be possible to solve this problem by installing Python 3.9 or greater on the image instead of 3.6.9.
I am not an Ubuntu system admin. Can someone tell me how to install Python 3.9 (or greater) on the Ubuntu image instead of Python 3.6.9?
Sprache
English
gefragt vor 7 Monaten298 Aufrufelg...
1 Antwort
- Neueste
- Die meisten Stimmen
- Die meisten Kommentare
Sind diese Antworten hilfreich? Stimmen Sie der richtigen Antwort zu, damit die Community von Ihrem Wissen profitieren kann.
0
Akzeptierte Antwort
I've been able to build an image by modifying the original Docker file to end up with this:
FROM ubuntu:22.04
RUN apt-get -y update && apt-get install -y --no-install-recommends \
wget \
python3-pip \
python3-setuptools \
nginx \
ca-certificates \
&& rm -rf /var/lib/apt/lists/*
RUN ln -s /usr/bin/python3 /usr/bin/python
RUN pip --no-cache-dir install numpy scipy scikit-learn pandas flask gunicorn
ENV PYTHONUNBUFFERED=TRUE
ENV PYTHONDONTWRITEBYTECODE=TRUE
ENV PATH="/opt/program:${PATH}"
COPY decision_trees /opt/program
WORKDIR /opt/program
Apparently, the specific Python 3 version that gets installed is determined by the Ubuntu OS version. Using a more recent OS version leads to installing a more recent Python version using the same apt-get command. (I think you get Python 3.10 with Ubuntu 22.04.)
beantwortet vor 7 Monatenlg...
Relevanter Inhalt
AWS OFFICIALAktualisiert vor 2 Jahren