bootstrap action python libraries are getting over written by Amazon EMR default libraries

Question

I am trying to provision EMR with python libraries like numpy=1.19.5 and pandas. I am doing it via bootstrap script. But some how the bootstrap values are getting over written. To avoid this i have followed this particular [https://repost.aws/knowledge-center/emr-install-python-libraries](doc). Now i am using the script as the bootstrap action file. Now the NODEPROVISIONSTATE forever in  PENDING state. 
I have gone through logs /var/log/bootstrap-actions/1/ I don't see any errors or warnings there.

But later cluster got terminated with the below error.

`On the master instance (i-****), bootstrap action 1 timed out executing`

Answer

A way to go may be to use the script as a Step in EMR instead of a bootstrap script. I had also an issue with files getting overriden  and it helped. Some clues on steps and
https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-add-steps-console.html

Answer

Hi,

Thank you for reaching to re:Post with your concerns.

I understand that you were able to resolve the issue after placing the bootstrap action file in the background  as mentioned in the documentation -   https://repost.aws/knowledge-center/emr-install-python-libraries.

Please feel free to update this post if you are still facing issue related to this.

Thank you!

bootstrap action python libraries are getting over written by Amazon EMR default libraries

관련 콘텐츠