How to shut down zeppelin notebook and restart later

Question

Hi, I followed the guide here to create a zeppelin notebook: https://docs.aws.amazon.com/glue/latest/dg/dev-endpoint-notebook-server-considerations.html  
  
I am just wondering how can I shut a zeppelin notebook down so that I can start it up again at a later date without going through all the hassle of setting it up from scratch.  
  
I tried just stopping the EC2 instance and starting it up again later. I restarted the zeppelin daemon and logged into zeppelin notebook ok. However when it came to executing pyspark statements, I received a java connection error:  
  
####################  
java.net.ConnectException: Connection refused (Connection refused)  
	at java.net.PlainSocketImpl.socketConnect(Native Method)  
	at java.net.AbstractPlainSocketImpl.doConnect(AbstractPlainSocketImpl.java:350)  
	at java.net.AbstractPlainSocketImpl.connectToAddress(AbstractPlainSocketImpl.java:206)  
	at java.net.AbstractPlainSocketImpl.connect(AbstractPlainSocketImpl.java:188)  
	at java.net.SocksSocketImpl.connect(SocksSocketImpl.java:392)  
	at java.net.Socket.connect(Socket.java:589)  
	at org.apache.thrift.transport.TSocket.open(TSocket.java:182)  
	at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:51)  
	at org.apache.zeppelin.interpreter.remote.ClientFactory.create(ClientFactory.java:37)  
	at org.apache.commons.pool2.BasePooledObjectFactory.makeObject(BasePooledObjectFactory.java:60)  
	at org.apache.commons.pool2.impl.GenericObjectPool.create(GenericObjectPool.java:861)  
	at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:435)  
	at org.apache.commons.pool2.impl.GenericObjectPool.borrowObject(GenericObjectPool.java:363)  
	at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.getClient(RemoteInterpreterProcess.java:62)  
	at org.apache.zeppelin.interpreter.remote.RemoteInterpreterProcess.callRemoteFunction(RemoteInterpreterProcess.java:133)  
	at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.internal_create(RemoteInterpreter.java:165)  
	at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.open(RemoteInterpreter.java:132)  
	at org.apache.zeppelin.interpreter.remote.RemoteInterpreter.getFormType(RemoteInterpreter.java:299)  
	at org.apache.zeppelin.notebook.Paragraph.jobRun(Paragraph.java:407)  
	at org.apache.zeppelin.scheduler.Job.run(Job.java:188)  
	at org.apache.zeppelin.scheduler.RemoteScheduler$JobRunner.run(RemoteScheduler.java:315)  
	at java.util.concurrent.Executors$RunnableAdapter.call(Executors.java:511)  
	at java.util.concurrent.FutureTask.run(FutureTask.java:266)  
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.access$201(ScheduledThreadPoolExecutor.java:180)  
	at java.util.concurrent.ScheduledThreadPoolExecutor$ScheduledFutureTask.run(ScheduledThreadPoolExecutor.java:293)  
	at java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1149)  
	at java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:624)  
	at java.lang.Thread.run(Thread.java:748)  
####################  
  
I could not resolve this without having to create a notebook from scratch, which is a pain. This is the second time I had to go through the process.  
  
Thanks for any help.  
  
Edited by: freesolo on Dec 12, 2019 8:31 PM  
  
Edited by: freesolo on Dec 12, 2019 8:32 PM

AWS-User-6283168 · Accepted Answer

In order to establish connection from Zeppelin EC2 instance to Glue dev endpoint, SSH tunnel is required.  
This SSH tunnel and Zeppelin daemons are started within the "setup_notebook_server.py" script.  
If you restart the EC2 instance, we would recommend you to rerun the script. Interactive shell asks a few questions but you can answer with just 'n' key.  
  
I understand that you do not want to rerun the script. There is another option to use SageMaker notebook instead. It does not require running scripts manually.

freesolo · Answer

Thanks for the helpful replies. Rerunning the "setup_notebook_server.py" after starting up the EC2 instance was the simple solution that works! Thanks NoritakaS-AWS!  
  
Edited by: freesolo on Jan 6, 2020 6:09 PM

MerAP · Answer

Hi,  
  
I'm working on local zeppelin connected to an Dev EndPoint and sometimes when I set up the end point and start using zeppelin I get the same error as you. After some investigation I've seen that nothing is listening on port 9007, the solution is to create a new DevEndPoint and then almost every time it works.  
  
Furthermore, I've got the same error when I execute a job from a workflow but this doesn't happen when I execute the job directly. The problem has appeared only three weeks before now.

How to shut down zeppelin notebook and restart later

相关内容