New Sagemaker domain creates EFS volume by default

0

Hi,

We are using Infrastructure-as-Code (IaC) to deploy our infrastructure, with strict compliance requirements.

We are looking at developing ML using Sagemaker, and realized that whenever a Sagemaker Domain is creating, it would also provision an EFS volume. This causes several popups in the security scans where the EFS policy is empty.

Is there a way where we can bring in our EFS volumes and Sagemaker does not create a new EFS volume on behalf of the domain creation?

If not, what is the recommended EFS policy that I could append to the filesystem?

2 Answers
0

Sagemaker does not support mounting custom EFS volumes when creating a domain. It will always provision a new EFS volume.

Consider the following:

  • Use the default EFS volume provisioned by Sagemaker and ensure it has appropriate IAM policies. The volume will be encrypted using an AWS managed KMS key.
  • Provision the EFS volume separately before creating the Sagemaker domain. Use the FileSystemId parameter when creating the domain to associate the existing EFS volume. You will need to ensure the volume has appropriate encryption and IAM policies.
  • Leverage lifecycle policies on the EFS volume to automatically move files to infrequent access storage classes after a period of inactivity. This can help reduce storage costs over time.
  • Consider using Sagemaker Studio instead of domains if your use case allows for per-user file storage instead of shared storage.
profile picture
EXPERT
answered a month ago
profile pictureAWS
EXPERT
reviewed a month ago
  • This is correct: SageMaker creates the EFS volume for it. But, you can reference it elsewhere in your CFN via Fn::GetAtt for HomeEfsFileSystemId. See https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/aws-resource-sagemaker-domain.html

  • Thanks Giovanni,

    Currently, the team is looking at deploying SageMaker Studio (instead of Studio Classic) to take advantage of the MLOps capabilities.

    I've tried re-creating the domain with the FileSystemId specified, however, both the custom-created and SageMaker-created EFS volumes were deployed using IaC.

    Thankfully, the KMS key is using the one associated with the domain, with one less thing to worry about.

  • There are a couple of inaccuracies with this answer - 1/You cannot provision an EFS volume and attach it when you create a domain (CreateDomain API does not allow providing a default EFS, only mounting BYO EFS for the new Studio experience). 2/ Not sure what you mean by using SageMaker Studio instead of Domains? Domains are a logical entity through which you can create users and they can access SageMaker Studio. The new Studio domains also create the EFS volumes for Studio Classic and Canvas apps.

0

SageMaker Studio automatically creates an EFS volume per domain for storage. You cannot associate a different default EFS volume for a Studio domain.

For the new Studio experience, you can bring your own EFS volume (see the CustomFileSystemConfigs parameter), but for the intents and purposes of this post, I don't think you're looking at this use case.

The EFS policy required by SageMaker is below, as Didier mentioned on the comments, you can get the HomeEfsFileSystemId via CFN and update the policy for the EFS volume -

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Principal": {
                "AWS": "*"
            },
            "Action": [
                "elasticfilesystem:ClientRootAccess",
                "elasticfilesystem:ClientWrite",
                "elasticfilesystem:ClientMount"
            ],
            "Resource": "<efs-file-system-arn>",
            "Condition": {
                "Bool": {
                    "elasticfilesystem:AccessedViaMountTarget": "true"
                }
            }
        },
        {
            "Effect": "Deny",
            "Principal": {
                "AWS": "*"
            },
            "Action": "*",
            "Resource": "<efs-file-system-arn>",
            "Condition": {
                "Bool": {
                    "aws:SecureTransport": "false"
                }
            }
        }
    ]
}
AWS
Durga_S
answered a month ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions