Glue Interactive Session S3Path/ScriptLocation change

0

I have created a glue interactive session job (jupyter notebook) and trying to save it to my own s3 bucket/location. I can't find any documentation on how to do this? I have tried the following glue magic commands but with no success for glue version 2 job -

%%configure { "s3_path": 's3://mybucket/myfolder/'}

%%configure { "s3_path": 's3://mybucket/myfolder/'}

%%configure { "scriptlocation": 's3://mybucket/myfolder/'}

%%configure { "command": "ScriptLocation=s3://mybucket/myfolder/"}

Any help on the exact steps needed to save my interactive jupyter glue notebook to my custom s3 location would be appreciated.

Thanks in advance!

SG
질문됨 2년 전291회 조회
2개 답변
0
수락된 답변

Hello SG!

There's no such magics configuration available in Glue Interactive session on Jupyter Notebook.

However, if you would like to save your notebook on a S3 location, you can achieve this by using AWS CLI on Jupyter notebook. For example, if you have AWS CLI installed in your local environment, you can run below command in the last cell of your notebook:

!aws s3 cp YOUR_NOTEBOOK.ipynb s3://YOUR_BUCKET/YOUR_PREFIX/

If you don't want to install AWS CLI, another workaround is to use interactive session on Glue Studio Notebook instead, where you can specify the script location in the Job Details section. https://docs.aws.amazon.com/glue/latest/ug/notebook-getting-started.html#save-notebook

Hope it helps!

AWS
지원 엔지니어
Jann_P
답변함 2년 전
  • The support also has said the same. Thanks for the clarification !

0

You don't have to use any magic to store the script or the notebook file (.ipynb). They are saved by default in a S3 location owner by your account. Notebook is saved under the notebook folder and script is saved in the scripts folder. The location is typically in s3://aws-glue-assets-<account-id>-<region>/scripts/ and s3://aws-glue-assets-<account-id>-<region>/Notebook/ Enter image description here

**Reference: ** Check the guide, section - saving your notebook and job script

The magic commands are used to change/assign the role, add additional python files, etc. and we this default configuration is not controlled by the magics.

profile pictureAWS
답변함 2년 전
  • The issue is saving to custom S3 location (not the default location). It is not mentioned anywhere that it is not supported.

로그인하지 않았습니다. 로그인해야 답변을 게시할 수 있습니다.

좋은 답변은 질문에 명확하게 답하고 건설적인 피드백을 제공하며 질문자의 전문적인 성장을 장려합니다.

질문 답변하기에 대한 가이드라인

관련 콘텐츠