Sage Maker Notebook - overcommit_memory settings - permission denied

0

I am doing some NLP work using spaCy in a ml.p2.xlarge notebook instance in sagemaker.

However, I get the following error:

numpy.core._exceptions._ArrayMemoryError: Unable to allocate 3.10 GiB for an array with shape (2889884, 288) and data type float32

To fix the issue, I have tried the following commands:

!echo 1 | sudo tee /proc/sys/vm/overcommit_memory
!echo 1 > /proc/sys/vm/overcommit_memory

Neither of them work and they both return:
/bin/sh: /proc/sys/vm/overcommit_memory: Permission denied

Any suggestions? I even added all policies with "admin" in them to the notebook IAM with no luck.

gefragt vor 3 Jahren879 Aufrufe
1 Antwort
0

Okay, it seems that I needed to restart my notebook after adding the admin policies to it. It's working now and I can overcommit the memory!

beantwortet vor 3 Jahren

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen