EMR Studio - Can you import local code into a notebook?

0

We are trying to use EMR Studio as a development environment for a medium complexity project which has the code split out into multiple files for testing and maintainability. There's simply too much code to have in one long file.

I cannot work out how to import local code into a notebook to run or test it.

Example layout

Here is a simplified example (our project is much larger):

my_notebook.ipynb
my_project/
     __init__.py
     model.py
     report.py

In the notebook we might have a cell like:

from my_project.model import DataModel
from my_project.report import Report

report = Report(DataModel(spark))

report.show()

The current result is:

An error was encountered:
No module named 'my_project'
Traceback (most recent call last):
ModuleNotFoundError: No module named 'my_project'

Is this possible?

Execution environment

It appears that the Python execution environment and the shell environment are completely separate, and the current directory is not available to the Python interpreter:

Execution environmentKeyValue
PythonUserlivy
PythonCurrent working dir/mnt/var/lib/livy
%%shUseremr-notebook
%%shCurrent working dir/home/emr-notebook/e-<HEX_STRING>

The /home/emr-notebook/... dir appears to contain our code, but the livy user which we appear to be running as doesn't permission to look at it. So even if we could guess the CWD and add it to the Python path it appears Python would not have permissions to read the code.

질문됨 2년 전99회 조회
답변 없음

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠