How do I resolve the errors "session '0' not found" and "Error sending http request and maximum retry encountered" in Amazon EMR?

Short description

You get one of the following errors when you run an Apache Livy application using a Jupyter Notebook on Amazon EMR:

• '404' from xxxxxxxx with error payload: "session '0' not found"
• "The code failed because of a fatal error: Error sending http request and maximum retry encountered."

These errors usually happen when you keep the Jupyter Notebook session running until it times out. To resolve these errors, increase the value of the livy.server.session.timeout property in /etc/livy/conf/livy.conf on the master node. Then, restart livy-server.

Resolution

You can modify livy.server.session.timeout on a running Amazon EMR cluster or when launching a new cluster.

On a running cluster

1.    Open /etc/livy/conf/livy.conf on the master node, and then modify the livy.server.session.timeout value (the default value is 1h). Example:

sudo vim /etc/livy/conf/livy.conf
livy.server.session.timeout 2h

2.    Run the following commands on the master node to restart livy-server:

Suppose that you're using one of the following Amazon EMR release versions that's based on Amazon Linux 2:

• Amazon EMR release version 5.30.0 or later
• Amazon EMR 6.x series

Then, run the following commands:

sudo systemctl stop livy-server
sudo systemctl start livy-server

If you're using the Amazon EMR release version 5.29.0 or earlier, then run the following commands:

sudo stop livy-server
sudo start livy-server

When livy-server is restarting, your cluster is unavailable. To avoid downtime, configure the Livy application when you launch an Amazon EMR cluster.

On a new cluster

Add a configuration objectsimilar to the following when you launch a cluster using Amazon EMR release version 4.6.0 or later. In addition to modifying livy.server.session.timeout, you can modify the following related properties, if desired:

• livy.server.session.timeout-check: When turned on, Livy stops idle sessions that reach the timeout threshold. The default setting is true.
• livy.server.yarn.app-lookup-timeout: The duration that Livy looks for the YARN application before considering it lost. The default setting is 60s.

[
    {
        "Classification": "livy-conf",
        "Properties": {
            "livy.server.session.timeout-check": "true",
            "livy.server.session.timeout": "2h",
            "livy.server.yarn.app-lookup-timeout": "120s"
        }
    }
]

After you're done running the job, be sure to close the session in Jupyter or Zeppelin. When too many sessions are open, new jobs can't start until resources become available.

---

Related information

Apache Livy

Jupyter Notebook on Amazon EMR