parallelCluster compute node public IP addresses?

0

Just started experimenting with ParallelCluster (I'm consistently amazed by what I can do with AWS ;^)) but I'd like to understand something better. My compute nodes are created with public ip addresses, but I can't seem to get to any of them (ping, ssh, etc...). That's ok, but I don't understand why. In fact I'd be happy to create my cluster such that only the master had a public IP and all the compute nodes were only visible from the master (seems to be a sensible way to run things and be a good citizen not sucking up IPv4 space ;^)) but I don't see that option.

Thanks for the clarification.

CraigR
asked 4 years ago43 views
2 Answers
0

After poking around I think I understand. It looks like what ParallelCluster does is create/generate two new security groups. The first (called something like "parallelcluster- clustername -MasterSecurityGroup-) allows inbound ssh and inbound traffic from the 2nd group ("parallelcluster- clustername -ComputeSecurityGroup-"). This last group allow inbound only from the Master sg and the Compute sg (i.e. other nodes in the compute group i suppose).

I don't quite understand why I'd need public IPv4 addresses for the compute nodes, but *shrug*

CraigR
answered 4 years ago
0

If you use a single subnet for master and compute nodes, with an Internet Gateway, then you'll need to assign public IPs to all your nodes. An alternative setup is to use a different subnet for the compute nodes and specify it with compute_subnet_id. In the compute subnet you can disable auto assign public ips and use a NAT gateway. Then you can SSH into your compute nodes by using the master as a Proxy node. First SSH into the master and then ssh into the compute node but don't forget to enable ssh agent forwarding or to copy the ssh key needed for the compute nodes to the master node.

Please take a look at the networking setup described here: https://aws-parallelcluster.readthedocs.io/en/latest/networking.html#aws-parallelcluster-with-awsbatch-scheduler. Although this setup is described for the awsbatch scheduler it should work fine with the other schedulers and might be exactly what you are looking for.

answered 4 years ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions