EMR Serverless and Pandas Failing due to Numpy C-extensions

0

I've followed the methods for adding Python libraries. Documentation here: https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/using-python-libraries.html

Boto installs and loads correctly at runtime (custom library working). However, EMR Serverless job fails when Pandas is loaded during runtime with the following error:

"Importing the numpy C-extensions failed. This error can happen for many reasons, often due to issues with your setup or how NumPy was installed."

I've followed the instructions at: https://numpy.org/devdocs/user/troubleshooting-importerror.html, but it did not help.

I'm using Python 3.7.16 and EMR Serverless 6.10.0.

1개 답변
0

I switched to an ARM based EC2 and it worked. Specifically, I used this AMI: amzn2-ami-kernel-5.10-hvm-2.0.20230504.1-arm64-gp2.

답변함 일 년 전

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인