How to shut down zeppelin notebook and restart later

0

Hi, I followed the guide here to create a zeppelin notebook: https://docs.aws.amazon.com/glue/latest/dg/dev-endpoint-notebook-server-considerations.html

I am just wondering how can I shut a zeppelin notebook down so that I can start it up again at a later date without going through all the hassle of setting it up from scratch.

I tried just stopping the EC2 instance and starting it up again later. I restarted the zeppelin daemon and logged into zeppelin notebook ok. However when it came to executing pyspark statements, I received a java connection error:

####################
java.net.ConnectException: Connection refused (Connection refused)
at java.net.PlainSocketImpl.socketConnect(Native Method)
at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)
at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)
at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)
at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)
at java.net.Socket.connect(Socket.java:589)
at org.apache.thrift.transport.TSocket.open(TSocket.java:182)
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)
at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)
at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)
at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)
at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:62)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.callRemoteFunction(RemoteInterpreterProcess.java:133)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_create(RemoteInterpreter.java:165)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:132)
at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:299)
at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:407)
at org.apache.zeppelin.scheduler.Job.run(Job.java:188)
at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:315)
at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)
at java.util.concurrent.FutureTask.run(FutureTask.java:266)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)
at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)
at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)
at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)
at java.lang.Thread.run(Thread.java:748)
####################

I could not resolve this without having to create a notebook from scratch, which is a pain. This is the second time I had to go through the process.

Thanks for any help.

Edited by: freesolo on Dec 12, 2019 8:31 PM

Edited by: freesolo on Dec 12, 2019 8:32 PM

質問済み 5年前787ビュー
3回答
0
承認された回答

In order to establish connection from Zeppelin EC2 instance to Glue dev endpoint, SSH tunnel is required.
This SSH tunnel and Zeppelin daemons are started within the "setup_notebook_server.py" script.
If you restart the EC2 instance, we would recommend you to rerun the script. Interactive shell asks a few questions but you can answer with just 'n' key.

I understand that you do not want to rerun the script. There is another option to use SageMaker notebook instead. It does not require running scripts manually.

AWS
回答済み 5年前
0

Hi,

I'm working on local zeppelin connected to an Dev EndPoint and sometimes when I set up the end point and start using zeppelin I get the same error as you. After some investigation I've seen that nothing is listening on port 9007, the solution is to create a new DevEndPoint and then almost every time it works.

Furthermore, I've got the same error when I execute a job from a workflow but this doesn't happen when I execute the job directly. The problem has appeared only three weeks before now.

MerAP
回答済み 5年前
0

Thanks for the helpful replies. Rerunning the "setup_notebook_server.py" after starting up the EC2 instance was the simple solution that works! Thanks NoritakaS-AWS!

Edited by: freesolo on Jan 6, 2020 6:09 PM

回答済み 5年前

ログインしていません。 ログイン 回答を投稿する。

優れた回答とは、質問に明確に答え、建設的なフィードバックを提供し、質問者の専門分野におけるスキルの向上を促すものです。

質問に答えるためのガイドライン

関連するコンテンツ