By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post

spack on ParallelCluster

0

I set up spack in a new ParallelCluster instance following https://aws.amazon.com/blogs/hpc/install-optimized-software-with-spack-configs-for-aws-parallelcluster/ The installation takes many hours, but it ends successfully with tail -f /var/log/spack-postinstall.log Spack setup completed

However, when I open a shell on the Head Node, the spack command is not in the PATH, in the default login shell. If I enter a bash shell I do see the spack command (in /shared/spack/bin/spack), but there must be something wrong

  1. I appear as ssm-user, but I land in /usr/bin
  2. spack install lmod fails (permission denied)
  3. sudo spack install lmod fails (command not found)
  4. sudo -E spack install lmod fails again (command not found)
  5. sudo /shared/spack/bin/spack install lmod succeeds but

sudo /shared/spack/bin/spack env activate spack-configs/INTEL/CPU ==> Error: spack env activate requires Spack's shell support.

I believe that my trick to avoid the broken permissions and PATH now shows its limits.

How can I set up a correct environment?

Topics
Compute
Tags
High Performance ComputeAWS ParallelCluster
Language
English
Luigi Scorzato
asked 2 months ago76 views
2 Answers
1

source is a shell command that interprets the file into your running shell; sudo source [...], even if spack didn't error out, isn't going to correctly set your shell environment.

I suggest starting an interactive shell as the owner of the spack directory (probably root), then sourcing the setup-env.sh, and then running spack commands. There spack command is a misnomer, it's a shell function that correctly finds the underlying spack python entrypoint.

ssm-user# sudo /bin/su - 
root # source /path/to/spack/setup-env.sh
root # spack install [...]
hpc-in-aws-workerbee
answered a month ago
  • Luigi Scorzato
    a month ago

    Thank you! Best practices normally require the use of sudo, rather than impersonating root. Perhaps you are right, and this is the only solution. I leave the question open, still hoping for a solution that uses sudo only.

  • hpc-in-aws-workerbee
    a month ago

    As as comment to your comment, while sudo is best practice, recall that without a username (or constrained commands in /etc/sudoers, your commands executed as root.

    If you're concerned about doing work as root (which is good!), I build my clusters with a specific spack:spack UID and GID, and have /opt/spack (or wherever the installation path is) owned by spack:spack.

    I then become the spack user, interactively with sudo /bin/su - spack and source [...]setup-env.sh. All installation, modification, etc. commands are run as spack, unprivileged, so you can't accidentally change the operating system itself (as root might).

    Manually setting your PATH to the spack binary (really, a python script) and calling with sudo may succeed for some operations, as you noticed, but fails with load/activate since those are manipulating your shell environment.

-1

It seems you've encountered some issues with the Spack installation on your AWS ParallelCluster. Let's address these problems step by step to set up a correct environment:

  1. The Spack installer script runs as a background task and can take 30 minutes or longer to complete, especially on smaller instance types. It's good that you've confirmed the successful completion of the installation.

  2. The fact that Spack is not in your PATH by default is likely due to the environment not being properly set up. To fix this:

  • Add Spack to your PATH by adding this line to your ~/.bashrc file: export PATH=/shared/spack/bin:$PATH

  • Source your .bashrc file or log out and log back in for the changes to take effect.

  1. The issues with permissions and sudo access suggest that the Spack installation might not have the correct ownership or permissions. To resolve this:

  • Change the ownership of the Spack installation: sudo chown -R $USER:$USER /shared/spack

  • Ensure the Spack binary is executable: chmod +x /shared/spack/bin/spack

  1. For the Spack shell integration to work properly, you need to source the appropriate setup script. Add the following to your ~/.bashrc file:

source /shared/spack/share/spack/setup-env.sh

  1. After making these changes, log out and log back in, or source your ~/.bashrc file.

  2. Now you should be able to use Spack commands without specifying the full path, and the shell integration should work correctly.

  3. If you still encounter issues with spack env activate, make sure you're using a shell that supports Spack's shell integration (bash, zsh, fish, or tcsh).

By following these steps, you should be able to set up a correct environment for using Spack on your AWS ParallelCluster. Remember to use the appropriate user account that has the necessary permissions to manage Spack and install packages.

If you continue to face issues, you may need to review the Spack installation process or consult AWS support for ParallelCluster-specific configurations.

Sources
Install optimized software with Spack configs for AWS ParallelCluster | AWS HPC Blog
Configuring AWS ParallelCluster - AWS ParallelCluster

profile picture
re:Post Agent
answered 2 months ago
  • Luigi Scorzato
    2 months ago

    I am afraid that LLMs are not sophisticated enough to answer these questions. I know how to set a PATH and source an env file. The problem is that the sudo command does not see it (not even with the -E option). The steps above will not help.

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions

Relevant content