Error while uploading a flask application onto elastic beanstalk

Question

I tried uploading a project onto elastic beanstalk but i am getting an error:  
  
\[Instance: i-0f627ebd7b8852e96] Command failed on instance. Return code: 1 Output: (TRUNCATED)...) File "/usr/lib64/python2.7/subprocess.py", line 190, in check_call raise CalledProcessError(retcode, cmd) CalledProcessError: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 1. Hook /opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py failed. For more detail, check /var/log/eb-activity.log using console or EB CLI.  
  
eb-activity.log file:  
  
/var/log/eb-activity.log  
-------------------------------------  
    running build_py  
    creating build  
    creating build/lib.linux-x86_64-3.6  
    creating build/lib.linux-x86_64-3.6/implicit  
    copying implicit/recommender_base.py -> build/lib.linux-x86_64-3.6/implicit  
    copying implicit/nearest_neighbours.py -> build/lib.linux-x86_64-3.6/implicit  
    copying implicit/__init__.py -> build/lib.linux-x86_64-3.6/implicit  
    copying implicit/approximate_als.py -> build/lib.linux-x86_64-3.6/implicit  
    copying implicit/utils.py -> build/lib.linux-x86_64-3.6/implicit  
    copying implicit/als.py -> build/lib.linux-x86_64-3.6/implicit  
    creating build/lib.linux-x86_64-3.6/implicit/cuda  
    copying implicit/cuda/__init__.py -> build/lib.linux-x86_64-3.6/implicit/cuda  
    creating build/lib.linux-x86_64-3.6/implicit/datasets  
    copying implicit/datasets/million_song_dataset.py -> build/lib.linux-x86_64-3.6/implicit/datasets  
    copying implicit/datasets/__init__.py -> build/lib.linux-x86_64-3.6/implicit/datasets  
    copying implicit/datasets/lastfm.py -> build/lib.linux-x86_64-3.6/implicit/datasets  
    copying implicit/datasets/reddit.py -> build/lib.linux-x86_64-3.6/implicit/datasets  
    copying implicit/datasets/_download.py -> build/lib.linux-x86_64-3.6/implicit/datasets  
    copying implicit/datasets/movielens.py -> build/lib.linux-x86_64-3.6/implicit/datasets  
    copying implicit/datasets/sketchfab.py -> build/lib.linux-x86_64-3.6/implicit/datasets  
    running build_ext  
    building 'implicit._als' extension  
    creating build/temp.linux-x86_64-3.6  
    creating build/temp.linux-x86_64-3.6/implicit  
    gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.6m -c implicit/_als.cpp -o build/temp.linux-x86_64-3.6/implicit/_als.o -Wno-unused-function -Wno-maybe-uninitialized -O3 -ffast-math -fopenmp -std=c++11  
    gcc: error trying to exec 'cc1plus': execvp: No such file or directory  
    error: command 'gcc' failed with exit status 1  
      
    ----------------------------------------  
    Failed building wheel for implicit  
    Running setup.py clean for implicit  
    Running setup.py bdist_wheel for sklearn: started  
    Running setup.py bdist_wheel for sklearn: finished with status 'done'  
    Stored in directory: /root/.cache/pip/wheels/76/03/bb/589d421d27431bcd2c6da284d5f2286c8e3b2ea3cf1594c074  
    Running setup.py bdist_wheel for visitor: started  
    Running setup.py bdist_wheel for visitor: finished with status 'done'  
    Stored in directory: /root/.cache/pip/wheels/68/b0/a2/cc8c7cf94ca3d1088a7d2e27936c1e0da170e05f560973e8dd  
  Successfully built Flask-Bootstrap sklearn visitor  
  Failed to build implicit  
  Installing collected packages: dominate, visitor, Flask-Bootstrap, numpy, scipy, tqdm, implicit, pytz, six, python-dateutil, pandas, scikit-learn, sklearn  
    Running setup.py install for implicit: started  
      Running setup.py install for implicit: finished with status 'error'  
      Complete output from command /opt/python/run/venv/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-0vg5ora6/implicit/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-p3cgxevr-record/install-record.txt --single-version-externally-managed --compile --install-headers /opt/python/run/venv/include/site/python3.6/implicit:  
      WARNING:root:The nvcc binary could not be located in your $PATH. Either add it to your path, or set $CUDAHOME to enable CUDA extensions  
      Failed to find CUDA toolkit. Building without GPU acceleration.  
      running install  
      running build  
      running build_py  
      creating build  
      creating build/lib.linux-x86_64-3.6  
      creating build/lib.linux-x86_64-3.6/implicit  
      copying implicit/recommender_base.py -> build/lib.linux-x86_64-3.6/implicit  
      copying implicit/nearest_neighbours.py -> build/lib.linux-x86_64-3.6/implicit  
      copying implicit/__init__.py -> build/lib.linux-x86_64-3.6/implicit  
      copying implicit/approximate_als.py -> build/lib.linux-x86_64-3.6/implicit  
      copying implicit/utils.py -> build/lib.linux-x86_64-3.6/implicit  
      copying implicit/als.py -> build/lib.linux-x86_64-3.6/implicit  
      creating build/lib.linux-x86_64-3.6/implicit/cuda  
      copying implicit/cuda/__init__.py -> build/lib.linux-x86_64-3.6/implicit/cuda  
      creating build/lib.linux-x86_64-3.6/implicit/datasets  
      copying implicit/datasets/million_song_dataset.py -> build/lib.linux-x86_64-3.6/implicit/datasets  
      copying implicit/datasets/__init__.py -> build/lib.linux-x86_64-3.6/implicit/datasets  
      copying implicit/datasets/lastfm.py -> build/lib.linux-x86_64-3.6/implicit/datasets  
      copying implicit/datasets/reddit.py -> build/lib.linux-x86_64-3.6/implicit/datasets  
      copying implicit/datasets/_download.py -> build/lib.linux-x86_64-3.6/implicit/datasets  
      copying implicit/datasets/movielens.py -> build/lib.linux-x86_64-3.6/implicit/datasets  
      copying implicit/datasets/sketchfab.py -> build/lib.linux-x86_64-3.6/implicit/datasets  
      running build_ext  
      building 'implicit._als' extension  
      creating build/temp.linux-x86_64-3.6  
      creating build/temp.linux-x86_64-3.6/implicit  
      gcc -pthread -Wno-unused-result -Wsign-compare -DDYNAMIC_ANNOTATIONS_ENABLED=1 -DNDEBUG -O2 -g -pipe -Wall -Wp,-D_FORTIFY_SOURCE=2 -fexceptions -fstack-protector --param=ssp-buffer-size=4 -m64 -mtune=generic -D_GNU_SOURCE -fPIC -fwrapv -fPIC -I/usr/include/python3.6m -c implicit/_als.cpp -o build/temp.linux-x86_64-3.6/implicit/_als.o -Wno-unused-function -Wno-maybe-uninitialized -O3 -ffast-math -fopenmp -std=c++11  
      gcc: error trying to exec 'cc1plus': execvp: No such file or directory  
      error: command 'gcc' failed with exit status 1  
        
      ----------------------------------------  
  Command "/opt/python/run/venv/bin/python3.6 -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-0vg5ora6/implicit/setup.py';f=getattr(tokenize, 'open', open)(__file__);code=f.read().replace('\r\n', '\n');f.close();exec(compile(code, __file__, 'exec'))" install --record /tmp/pip-p3cgxevr-record/install-record.txt --single-version-externally-managed --compile --install-headers /opt/python/run/venv/include/site/python3.6/implicit" failed with error code 1 in /tmp/pip-build-0vg5ora6/implicit/  
  You are using pip version 9.0.1, however version 19.0.3 is available.  
  You should consider upgrading via the 'pip install --upgrade pip' command.  
  2019-04-07 20:24:08,598 ERROR    Error installing dependencies: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 1  
  Traceback (most recent call last):  
    File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 22, in main  
      install_dependencies()  
    File "/opt/elasticbeanstalk/hooks/appdeploy/pre/03deploy.py", line 18, in install_dependencies  
      check_call('%s install -r %s' % (os.path.join(APP_VIRTUAL_ENV, 'bin', 'pip'), requirements_file), shell=True)  
    File "/usr/lib64/python2.7/subprocess.py", line 190, in check_call  
      raise CalledProcessError(retcode, cmd)  
  CalledProcessError: Command '/opt/python/run/venv/bin/pip install -r /opt/python/ondeck/app/requirements.txt' returned non-zero exit status 1 (Executor::NonZeroExitStatus)

\[2019-04-07T20:24:08.607Z] INFO  \[5669]  - \[Application update myProj@5/AppDeployStage0/AppDeployPreHook/03deploy.py] : Activity failed.  
\[2019-04-07T20:24:08.607Z] INFO  \[5669]  - \[Application update myProj@5/AppDeployStage0/AppDeployPreHook] : Activity failed.  
\[2019-04-07T20:24:08.607Z] INFO  \[5669]  - \[Application update myProj@5/AppDeployStage0] : Activity failed.  
\[2019-04-07T20:24:08.608Z] INFO  \[5669]  - \[Application update myProj@5] : Completed activity. Result:  
  Application update - Command CMD-AppDeploy failed  
\[2019-04-07T20:27:07.548Z] INFO  \[5986]  - \[CMD-TailLogs] : Starting activity...  
\[2019-04-07T20:27:07.548Z] INFO  \[5986]  - \[CMD-TailLogs/AddonsBefore] : Starting activity...  
\[2019-04-07T20:27:07.548Z] INFO  \[5986]  - \[CMD-TailLogs/AddonsBefore] : Completed activity.  
\[2019-04-07T20:27:07.549Z] INFO  \[5986]  - \[CMD-TailLogs/TailLogs] : Starting activity...  
\[2019-04-07T20:27:07.549Z] INFO  \[5986]  - \[CMD-TailLogs/TailLogs/TailLogs] : Starting activity...  
  
Any solution would be appreciated  
  
Edited by: niwash on Apr 7, 2019 3:34 PM

Accepted Answer

Hi niwash,  
  
Thanks for reaching out.  
  
The core of the problem is hinted at in the failure:

```
gcc: error trying to exec 'cc1plus': execvp: No such file or directory
```

Basically, EB is failing to compile certain C++ programs because support to compile C++ is missing. I think you should be able to get around this problem by adding the following snippet in a .ebextensions config file:

```
packages:
  yum:
    gcc-c++: []
```

Additionally, going by the discussion here: https://stackoverflow.com/questions/29516084/gcc-failed-during-pandas-build-on-aws-elastic-beanstalk, it is likely that you will also need to add `python-devel.x86_64` to the above `yum` list.  
  
Documentation for setting .ebextensions is here: https://docs.aws.amazon.com/elasticbeanstalk/latest/dg/customize-containers-ec2.html#linux-packages-snippet  
  
Thanks,  
Rahul.  
  
Edited by: Rahul-AWS on Apr 8, 2019 12:29 PM

Answer

Try looking if all modules in requirements.txt versions are correct/exists by using pip freeze. I had the same error when I noticed I had modules like translate-language==0.1.0 in requirements.txt, after removing the modules that mismatched or didn't exist the deployment returned to normal

Answer

requirements.txt

Click==7.0
dominate==2.3.5
Flask==1.0.2
Flask-Bootstrap==3.3.7.1
implicit==0.3.8
itsdangerous==1.1.0
Jinja2==2.10.1
MarkupSafe==1.1.1
numpy==1.16.2
pandas==0.24.2
python-dateutil==2.8.0
pytz==2018.9
scikit-learn==0.20.3
scipy==1.2.1
six==1.12.0
sklearn==0.0
tqdm==4.31.1
visitor==0.1.3
Werkzeug==0.15.2

I figured out the error is caused by implicit package >>implicit==0.3.8<<. I tried downgrading the version and deploying but still same error. It seems like the elastic bean is unable to install the implicit package. Is there any way i can manually install the package in my application environment?

Edited by: niwash on Apr 8, 2019 5:13 AM

Error while uploading a flask application onto elastic beanstalk

/var/log/eb-activity.log

Relevanter Inhalt