EMR Serverless and Pandas Failing due to Numpy C-extensions

0

I've followed the methods for adding Python libraries. Documentation here: https://docs.aws.amazon.com/emr/latest/EMR-Serverless-UserGuide/using-python-libraries.html

Boto installs and loads correctly at runtime (custom library working). However, EMR Serverless job fails when Pandas is loaded during runtime with the following error:

"Importing the numpy C-extensions failed. This error can happen for many reasons, often due to issues with your setup or how NumPy was installed."

I've followed the instructions at: https://numpy.org/devdocs/user/troubleshooting-importerror.html, but it did not help.

I'm using Python 3.7.16 and EMR Serverless 6.10.0.

1回答
0

I switched to an ARM based EC2 and it worked. Specifically, I used this AMI: amzn2-ami-kernel-5.10-hvm-2.0.20230504.1-arm64-gp2.

回答済み 1年前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ