Deleting a DataZone project when some underlying resources were already deleted

Question

Hi everyone,

I'm facing an issue trying to delete a SageMaker Unified Studio/Datazone project. Two EMR Serverless applications were created in this project through the SageMaker Unified Studio UI, which used CloudFormation to create both clusters.

The CloudFormation stacks were then deleted directly through the console, outside of Unified Studio/DataZone. This succeeded.

Now I am trying to delete the DataZone project, but I'm running into an issue that occurs when the process attempts to delete the environment linked with the first EMR Serverless application. The error message returned in the SageMaker UI is simply : 
```
 Project deletion failed because one or more resources could not be deleted. There were errors while deleting project .
```
The additional details are simply general IDs about the DataZone domain and the project ID.

Looking through CloudTrail, I see the DataZone provisioning role attempts a "DescribeStacks" action on the corresponding deleted stack, and gets a "ValidationError - Stack with id  does not exist" error in response, which appears to be the underlying cause of the error as it is the last related API call I can find.

I've tried programatically deleting both the entire project and the environment through the CLI instead of the SageMaker Unified Studio UI, but the same issue occurs and the project is stuck with a failed status. It is also still visible within the SageMaker Unified Studio UI.

How can I delete this project from the domain?

Alex · Accepted Answer

In the end I found a workaround to this issue. I simply recreated both of the stacks in CloudFormation, using the same names, templates, parameters and tags as initially, and then ran the delete again. It deleted the new stacks and was able to progress and delete the project.

re:Post Agent · Answer

To resolve your issue with deleting a DataZone project where underlying resources (EMR Serverless applications) were already deleted through CloudFormation, you'll need to follow a specific approach since the project deletion is failing due to missing resources.

When you delete a project in Amazon DataZone, it attempts to clean up all associated resources, but in your case, it's failing because it's looking for CloudFormation stacks that no longer exist. This is a common issue when resources are deleted outside of their parent service's management interface.

Here's how to address this:

1. First, check for any remaining resources associated with your project. Look for:
- Any remaining ENIs (Elastic Network Interfaces) in the VPC/subnets
- Any requester-managed VPC endpoints that might have been created
- Any other AWS resources like IAM roles, Glue databases, Athena workgroups, or CloudWatch log groups

2. For the project that's stuck in deletion:
- Try using the RetainResources parameter with CloudFormation to skip the resources that are causing the deletion to fail
- Alternatively, consider using the FORCE_DELETE_STACK option with the DeletionMode parameter

3. If you're dealing with a conformance pack or config rule issue, you may need to delete these from their respective service consoles rather than directly through CloudFormation

4. As a last resort, you might need to contact AWS Support as they can help identify any hidden dependencies and potentially help remove the project from the backend

Remember that deleting a DataZone project is irreversible and will delete all project contents including data sources, environments, assets, glossaries, and metadata forms. However, it doesn't automatically delete non-DataZone AWS resources that DataZone may have helped create - which is why manually deleting the CloudFormation stacks first has caused this issue.  
**Sources**  
[Sagemaker Unified Studio | AWS re:Post](https://repost.aws/questions/QUl28lPBQOQAaGYkdASsv1PQ/sagemaker-unified-studio)  
[Delete project - Amazon DataZone](https://docs.aws.amazon.com/datazone/latest/userguide/delete-project.html)  
[I can't delete endpoint, that was created by me previously | AWS re:Post](https://repost.aws/questions/QUNqvE9QXDSsueqzwzcS18rw/i-can-t-delete-endpoint-that-was-created-by-me-previously)  
[Troubleshooting CloudFormation - AWS CloudFormation](https://docs.aws.amazon.com/AWSCloudFormation/latest/UserGuide/troubleshooting.html)

Deleting a DataZone project when some underlying resources were already deleted

Add your answer

Relevant content