Installing GDAL in MWAA

Question

Hi all,
I am trying to install GDAL in MWAA.

library link - 
https://pypi.org/project/GDAL/

I have already tried it through requirements.txt file.

Content of requirements.txt -

```
--constraint https://raw.githubusercontent.com/apache/airflow/constraints-2.2.2/constraints-3.7.txt
apache-airflow-providers-ssh==2.3.0
GDAL==3.5.1
```

But getting some errors - 
error file link - 
https://github.com/aws/aws-mwaa-local-runner/files/9478133/error_log.txt

As I can see in the pypi.org it depends on the following - 
![dependencies](/media/postImages/original/IMntrn7tDYS8qwI64sqww07Q)

looking forward to any solutions in this

Thanks,
SKC

Answer

The GDAL package is not within scope of what Airflow was intended to do, so the package is not suggested to be installed. Airflow is meant for orchestration, ETL/Geo-spatial libraries require computation and resources that are simply not available on the worker containers.

With that, there are various reasons that it could be failing, including dependency conflicts with other core packages used by Airflow. If you review the constraints line you are using in your PIP installation,  you can see that the package is not listed at all, and on [libraries.io](https://libraries.io/conda/gdal/3.5.1), it also doesn't list its dependencies which makes troubleshooting it even harder.

It could also be that according to the GDAL [website](https://gdal.org/download.html#linux-docker-images), there is currently only a working binary for Debian based Linux distros, and MWAA containers utilize Amazon Linux (which is a RedHat variant).

Installing GDAL in MWAA

関連するコンテンツ