Sage Maker Notebook - overcommit_memory settings - permission denied

0

I am doing some NLP work using spaCy in a ml.p2.xlarge notebook instance in sagemaker.

However, I get the following error:

numpy.core._exceptions._ArrayMemoryError: Unable to allocate 3.10 GiB for an array with shape (2889884, 288) and data type float32

To fix the issue, I have tried the following commands:

!echo 1 | sudo tee /proc/sys/vm/overcommit_memory
!echo 1 > /proc/sys/vm/overcommit_memory

Neither of them work and they both return:
/bin/sh: /proc/sys/vm/overcommit_memory: Permission denied

Any suggestions? I even added all policies with "admin" in them to the notebook IAM with no luck.

preguntada hace 3 años879 visualizaciones
1 Respuesta
0

Okay, it seems that I needed to restart my notebook after adding the admin policies to it. It's working now and I can overcommit the memory!

respondido hace 3 años

No has iniciado sesión. Iniciar sesión para publicar una respuesta.

Una buena respuesta responde claramente a la pregunta, proporciona comentarios constructivos y fomenta el crecimiento profesional en la persona que hace la pregunta.

Pautas para responder preguntas