How to install additional local python libraries in AWS EMR notebooks


I am using both pyspark and local python kernel (%%local) in a single EMR notebook. I am able to install packages successfully in pyspark kernel using EMR bootstrap but I am unable to install additional local python libraries (s3fs and other packages) using EMR bootstrap action. Could you please provide your guidance on this

asked 4 months ago615 views
1 Answer
Accepted Answer

Are you receiving any specific error during installation? Please see below documentation related to installing python libraries in EMR notebooks:

answered 4 months ago
  • Had gone through the above links. I am able to install local python packages using %%local pip install <packagename> in jupyter notebook pyspark kernel. But i had to do this action everytime for each notebook session. Whether additional local python packages can be directly installed using bootstrap action in pyspark kernels ?

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions