Sagemaker pipeline stopped working with error "ClientError: Failed to invoke sagemaker:CreateTrainingJob. Error Details: Instance type is not supported by algorithm sagemaker-distribution

This issue appears to be related to a sudden incompatibility between the specified instance type and the SageMaker distribution algorithm. Even though your code and configuration haven't changed, it's possible that there have been updates on the AWS side that have affected the compatibility of certain instance types with specific algorithms.

To resolve this issue, you can try the following steps:

1. Check if the instance type 'ml.c6i.12xlarge' is still supported for training jobs in your region. It's possible that AWS has made changes to the available instance types for SageMaker.

2. Verify if there have been any recent updates to the SageMaker distribution algorithm that might have changed its instance type compatibility.

3. Try using a different compatible instance type. For example, you could try using an ml.c5 instance instead of ml.c6i.

4. Ensure that your SageMaker SDK is up to date. Sometimes, updating to the latest version can resolve compatibility issues.

5. Check your IAM permissions to make sure you still have the necessary permissions to create training jobs with the specified instance type.

6. Review the SageMaker quotas for your account to ensure you haven't reached any limits that might prevent you from using certain instance types.

If the issue persists after trying these steps, it would be advisable to contact AWS support for further assistance. They can provide more specific information about any recent changes that might have affected your pipeline and help you find a suitable solution.
Sources
CreateTrainingJob - Amazon SageMaker
CreateOptimizationJob - Amazon SageMaker