bootstrap action python libraries are getting over written by Amazon EMR default libraries

0

I am trying to provision EMR with python libraries like numpy=1.19.5 and pandas. I am doing it via bootstrap script. But some how the bootstrap values are getting over written. To avoid this i have followed this particular https://repost.aws/knowledge-center/emr-install-python-libraries. Now i am using the script as the bootstrap action file. Now the NODEPROVISIONSTATE forever in PENDING state. I have gone through logs /var/log/bootstrap-actions/1/ I don't see any errors or warnings there.

But later cluster got terminated with the below error.

On the master instance (i-****), bootstrap action 1 timed out executing

  • My bad i am able to fix the issue, I have miss read the doc. I am not bootstrap file in Back ground hence getting above error.

asked 10 months ago275 views
2 Answers
0

Hi,

Thank you for reaching to re:Post with your concerns.

I understand that you were able to resolve the issue after placing the bootstrap action file in the background as mentioned in the documentation - https://repost.aws/knowledge-center/emr-install-python-libraries.

Please feel free to update this post if you are still facing issue related to this.

Thank you!

AWS
SUPPORT ENGINEER
answered 10 months ago
0

A way to go may be to use the script as a Step in EMR instead of a bootstrap script. I had also an issue with files getting overriden and it helped. Some clues on steps and https://docs.aws.amazon.com/emr/latest/ManagementGuide/emr-add-steps-console.html

Michal
answered 9 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions