Skip to content

Sagemaker Unified Studio

0

Over the weekend I wanted to poke around the new Sagemaker Unified Studio. After a few days of inactivity, I realized that there are recurring charges. I deleted the domain, and then attempted to delete the CloudFormation stack. The CF deletion failed.

The likely error, according to the console, is a subnet. I attempt to delete the subnet and that fails.

My question: How can I unwind the resources? What would be blocking the subnet and what chain of resources do I need to look at in order to have the stack tear down?

5 Answers
0

Hello.

Is it possible that there are still ENIs remaining in the subnet you are trying to delete?
Subnets that still have ENIs will fail to be deleted.
https://docs.aws.amazon.com/vpc/latest/userguide/subnet-deleting.html

If you no longer need a subnet, you can delete it. You cannot delete a subnet if it contains any network interfaces. For example, you must terminate any instances in a subnet before you can delete it.

Therefore, first check the ENI screen to see if there are any ENIs remaining in the subnet you are trying to delete.
If an ENI remains, check the ENI details and delete the associated AWS resources first.

EXPERT

answered a year ago

  • Please see below. Any ideas as to how I can unwind this? I am chasing my tail and currently incurring daily charges.

0

To unwind the resources and successfully delete the CloudFormation stack associated with your SageMaker Unified Studio experiment, you'll need to follow a specific sequence of steps to remove all related resources. The subnet deletion failure suggests that there are still resources attached to it, which is preventing the stack from being deleted.

Here's a recommended approach to clean up your resources:

  1. First, ensure that you've deleted all projects within the SageMaker Unified Studio domain. Projects may have created additional resources that need to be removed.

  2. Check for any remaining SageMaker endpoints, MLflow resources, or HyperPod clusters that might have been created during your exploration. Delete these if they exist.

  3. Look for and delete any AWS Glue databases and Athena workgroups that were created by your projects.

  4. Check for and remove any connections you created in the data explorer.

  5. Delete the S3 bucket associated with your domain. It should be named something like "amazon-datazone-[AWSACCOUNTID]-[AWSREGION]-[DOMAINID]".

  6. Investigate any VPC resources that might be associated with your SageMaker setup. This includes checking for NAT gateways, which can incur charges even when not actively used.

  7. If you created a VPC specifically for this experiment (often named something like "SageMakerUnifiedStudioVPC"), attempt to delete it manually.

  8. Check the EC2 console for any lingering instances or Elastic Network Interfaces (ENIs) that might be attached to your subnets. These will need to be terminated or deleted before you can remove the subnet.

  9. Look for any Elastic File Systems (EFS) that might have been created, as SageMaker Studio often uses EFS for storage.

  10. After removing all these resources, try deleting the CloudFormation stack again.

If you're still encountering issues after following these steps, you may need to use the AWS CLI or SDK to forcefully remove resources. However, be cautious with this approach as it can lead to orphaned resources if not done correctly.

If you continue to face difficulties, it would be advisable to contact AWS Support. They can provide a detailed breakdown of any remaining resources and assist you in identifying and terminating any lingering components that are preventing the complete cleanup.

Remember, it's crucial to monitor your AWS Cost Explorer during and after this process to ensure that all charges have stopped and no unexpected resources remain active.
Sources
Being charged for unused SageMaker: What beyond Instances and Domain can I shut down in SageMaker? | AWS re:Post
Foundational blocks of Amazon SageMaker Unified Studio: An admin’s guide to implement unified access to all your data, analytics, and AI | AWS Big Data Blog
An integrated experience for all your data and AI with Amazon SageMaker Unified Studio (preview) | AWS Big Data Blog
An integrated experience for all your data and AI with Amazon SageMaker Unified Studio | AWS Big Data Blog

answered a year ago

EXPERT

reviewed a year ago

0

Thank you for lending some advice. THere is a network interface, but I can't delete that.

This is the page I see when attempting to delete or detach the ENI.

Enter image description here

I have been trying to chase things down and feel like I am going in circles.

answered a year ago

0

@dm3, are you creating "Sagemaker Unified Studio" through the AWS Console?, or is it through CDK and SDK?

answered a year ago

0

did you ever figure this out? I'm having the same problem

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.