Remote development on an EC2 Inferentia, Trainium, or Neuron system

Development on a Neuron system (instances using Trainium or Inferentia) is a little different because you cannot test your code locally. To be able to test and run code on the instance with the Neuron device, you can use one of the following methods.

(This article assumes that you have already deployed your EC2 instance following these instructions )

SSH connections

All these methods will require an SSH connection of some kind. These instructions were written for a Mac, but should work on a Linux system as well. A PC may require using Putty.

You should have a .pem file that you created when you deployed your instance or had from previous deployments. You can connect to your system using ssh -i "/path/to/sshkey.pem" ubuntu@instance_ip_address

If you have trouble connecting, make sure you wait a minute or two and try again (it takes a little time for instances to be deployed, but you can usually connect within the first minute). Also, make sure your security group includes access to port 22 from your external IP address. If you are working from an office or university, you may need to include a block of IP addresses. If you get a prompt that the instance isn't recognized and would you like to add the key to the local cache, select yes. You can get the IP address from the AWS console. You can also use the public IPv4 DNS.

Jupyter Notebooks

This method involves running the Jupyter notebook server on the Neuron instance, mapping a port locally, then using the browser on your desktop to access the notebook server.

1. Start by mapping a port on your local machine to the Neuron instance. From a terminal on your system, run ssh -i "/path/to/sshkey.pem" -N -f -L localhost:8888:localhost:8888 ubuntu@instance_ip_address
2. Then connect to your instance using SSH (see the section above). From the command prompt, run nohup jupyter notebook --no-browser --port=8888
3. after a few seconds, check the nohup.out file to find your server's token: more nohup.out or cat nohup.out | grep localhost
4. Copy the connection string and paste it into your browser. After a few seconds, you should eventually see the Jupyter Notebook browser. It should look like http://localhost:8888/tree?token=337fc8de2aenot_a_real_tokene952c43946e4fb57131

This process works because you have mapped the 8888 port on your local machine to the 8888 port on the Neuron instance, so when you connect to localhost:8888, you end up accessing the Jupyter server on the Neuron instance.

If you have problems, make sure the initial port mapping was successful. If you already have something running on port 8888 on your machine, this may give you an error. You can always change the port (e.g. 8885) in all the instructions if you need to.

VS Code remote server

With Visual Studio Code installed on your local machine, you can use the Remote-SSH command to edit and run files that are stored on a Neuron instance. See the VS Code article for additional details.

1. select Remote-SSH: Connect to Host... from the Command Palette (F1, ⇧⌘P)
2. Enter in the full connection string from the ssh section above: ssh -i "/path/to/sshkey.pem" ubuntu@instance_ip_address
3. VS Code should connect and automatically set up the VS Code server.
4. Eventually, you should be prompted for a base directory. You can browse to a directory on the Neuron instance.
5. You may find that some commands seem greyed out in the menus, but the keyboard commands will still work (⌘S to save or ^⇧` for terminal). You may need to restart VS Code.