EMR Serverless and Pandas Failing due to Numpy C-extensions

0

I've followed the methods for adding Python libraries. Documentation here: https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/using-python-libraries.html

Boto installs and loads correctly at runtime (custom library working). However, EMR Serverless job fails when Pandas is loaded during runtime with the following error:

"Importing the numpy C-extensions failed. This error can happen for many reasons, often due to issues with your setup or how NumPy was installed."

I've followed the instructions at: https://numpy.org/devdocs/user/troubleshooting-importerror.html, but it did not help.

I'm using Python 3.7.16 and EMR Serverless 6.10.0.

1 Antwort
0

I switched to an ARM based EC2 and it worked. Specifically, I used this AMI: amzn2-ami-kernel-5.10-hvm-2.0.20230504.1-arm64-gp2.

beantwortet vor einem Jahr

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen