EMR Serverless and Pandas Failing due to Numpy C-extensions

0

I've followed the methods for adding Python libraries. Documentation here: https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/using-python-libraries.html

Boto installs and loads correctly at runtime (custom library working). However, EMR Serverless job fails when Pandas is loaded during runtime with the following error:

"Importing the numpy C-extensions failed. This error can happen for many reasons, often due to issues with your setup or how NumPy was installed."

I've followed the instructions at: https://numpy.org/devdocs/user/troubleshooting-importerror.html, but it did not help.

I'm using Python 3.7.16 and EMR Serverless 6.10.0.

1 Risposta
0

I switched to an ARM based EC2 and it worked. Specifically, I used this AMI: amzn2-ami-kernel-5.10-hvm-2.0.20230504.1-arm64-gp2.

con risposta un anno fa

Accesso non effettuato. Accedi per postare una risposta.

Una buona risposta soddisfa chiaramente la domanda, fornisce un feedback costruttivo e incoraggia la crescita professionale del richiedente.

Linee guida per rispondere alle domande