Not able to install aiohttp module on MWAA 2.2.2

0

I tried installing various versions of aiohttp module on airflow using the requirements.txt but nothing worked. The error I'm receiving while trying to import the DAG is " ModuleNotFoundError: No module named 'aiohttp' ". The version of aiohttp I have in my requirements.txt : " aiohttp==3.8.3 " Note : The installation for all the other modules are working as expected.

gefragt vor einem Jahr389 Aufrufe
1 Antwort
0

Hello, I have tested myself using MWAA local runner for MWAA version 2.2.2 and aiohttp installed successfully with out any issues .aiohttp has few requirements like Python >= 3.6,async-timeout, yarl, frozenlist etc.. . if any of these requirements are not compatible with Airflow version and other installed dependencies then it will not install aiohttp provided constraint file in requirements.txt. I would suggest to test using MWAA local runner and resolve the conflicts by providing right version or removing the packages that are causing conflicts .

Requirements Installation logs

Downloading aiohttp-3.8.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (1.1 MB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 1.1/1.1 MB 625.4 kB/s eta 0:00:00 Requirement already satisfied: attrs>=17.3.0 in /usr/local/lib/python3.7/site-packages (from aiohttp->-r /usr/local/airflow/dags/requirements.txt (line 2)) (20.3.0) Collecting asynctest==0.13.0 Downloading asynctest-0.13.0-py3-none-any.whl (26 kB) Collecting yarl<2.0,>=1.0 Downloading yarl-1.7.2-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (271 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 271.8/271.8 kB 12.9 MB/s eta 0:00:00 Collecting multidict<7.0,>=4.5 Downloading multidict-5.2.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (160 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 160.2/160.2 kB 3.2 MB/s eta 0:00:00 Collecting frozenlist>=1.1.1 Downloading frozenlist-1.2.0-cp37-cp37m-manylinux_2_5_x86_64.manylinux1_x86_64.manylinux_2_12_x86_64.manylinux2010_x86_64.whl (192 kB) ━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━ 192.4/192.4 kB 11.7 MB/s eta 0:00:00 Collecting aiosignal>=1.1.2 Downloading aiosignal-1.2.0-py3-none-any.whl (8.2 kB) Requirement already satisfied: typing-extensions>=3.7.4 in /usr/local/lib/python3.7/site-packages (from aiohttp->-r /usr/local/airflow/dags/requirements.txt (line 2)) (3.10.0.2) Collecting async-timeout<5.0,>=4.0.0a3 Downloading async_timeout-4.0.0-py3-none-any.whl (6.1 kB) Requirement already satisfied: charset-normalizer<3.0,>=2.0 in /usr/local/lib/python3.7/site-packages (from aiohttp->-r /usr/local/airflow/dags/requirements.txt (line 2)) (2.0.7) Requirement already satisfied: idna>=2.0 in /usr/local/lib/python3.7/site-packages (from yarl<2.0,>=1.0->aiohttp->-r /usr/local/airflow/dags/requirements.txt (line 2)) (3.3) Installing collected packages: multidict, frozenlist, asynctest, async-timeout, yarl, aiosignal, aiohttp Successfully installed aiohttp-3.8.0 aiosignal-1.2.0 async-timeout-4.0.0 asynctest-0.13.0 frozenlist-1.2.0 multidict-5.2.0 yarl-1.7.2

AWS
beantwortet vor einem Jahr

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen