Not able to install aiohttp module on MWAA 2.2.2

0

I tried installing various versions of aiohttp module on airflow using the requirements.txt but nothing worked. The error I'm receiving while trying to import the DAG is " ModuleNotFoundError: No module named 'aiohttp' ". The version of aiohttp I have in my requirements.txt : " aiohttp==3.8.3 " Note : The installation for all the other modules are working as expected.

asked a year ago368 views
1 Answer
0

Hello, I have tested myself using MWAA local runner for MWAA version 2.2.2 and aiohttp installed successfully with out any issues .aiohttp has few requirements like Python >= 3.6,async-timeout, yarl, frozenlist etc.. . if any of these requirements are not compatible with Airflow version and other installed dependencies then it will not install aiohttp provided constraint file in requirements.txt. I would suggest to test using MWAA local runner and resolve the conflicts by providing right version or removing the packages that are causing conflicts .

Requirements Installation logs

Downloading aiohttp-3.8.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 625.4 kB/s eta 0:00:00 Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.7/site-packages (from aiohttp->-r /usr/local/airflow/dags/requirements.txt (line 2)) (20.3.0) Collecting asynctest==0.13.0 Downloading asynctest-0.13.0-py3-none-any.whl (26 kB) Collecting yarl<2.0,>=1.0 Downloading yarl-1.7.2-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (271 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 271.8/271.8 kB 12.9 MB/s eta 0:00:00 Collecting multidict<7.0,>=4.5 Downloading multidict-5.2.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (160 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 160.2/160.2 kB 3.2 MB/s eta 0:00:00 Collecting frozenlist>=1.1.1 Downloading frozenlist-1.2.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (192 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 192.4/192.4 kB 11.7 MB/s eta 0:00:00 Collecting aiosignal>=1.1.2 Downloading aiosignal-1.2.0-py3-none-any.whl (8.2 kB) Requirement already satisfied: typing-extensions>=3.7.4 in /usr/local/lib/python3.7/site-packages (from aiohttp->-r /usr/local/airflow/dags/requirements.txt (line 2)) (3.10.0.2) Collecting async-timeout<5.0,>=4.0.0a3 Downloading async_timeout-4.0.0-py3-none-any.whl (6.1 kB) Requirement already satisfied: charset-normalizer<3.0,>=2.0 in /usr/local/lib/python3.7/site-packages (from aiohttp->-r /usr/local/airflow/dags/requirements.txt (line 2)) (2.0.7) Requirement already satisfied: idna>=2.0 in /usr/local/lib/python3.7/site-packages (from yarl<2.0,>=1.0->aiohttp->-r /usr/local/airflow/dags/requirements.txt (line 2)) (3.3) Installing collected packages: multidict, frozenlist, asynctest, async-timeout, yarl, aiosignal, aiohttp Successfully installed aiohttp-3.8.0 aiosignal-1.2.0 async-timeout-4.0.0 asynctest-0.13.0 frozenlist-1.2.0 multidict-5.2.0 yarl-1.7.2

AWS
answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions