SAGEMAKER_RESOURCE_LIMIT

Usually that quota for that instance type is not zero by default (in my case it is 16), but seeing your error message, it is limited to zero in your account's case. If you have permission to request quota increase, please do so from the Service Quotas on the AWS console, otherwise please consult with your admin person.

The error message you're receiving suggests that your AWS account has a service limit of 0 instances for the instance type you're trying to use for your SageMaker Processing Job. This means that your account is not currently allowed to launch any instances of that type, which is why you're encountering the error.

Here are a few suggestions to resolve this issue:

Use a Different Instance Type: If you don't want to go through the process of requesting a service limit increase, you can try using a different instance type that your account is allowed to use. AWS provides a list of available instance types and their current limits in the AWS Service Quotas console. Check IAM Permissions: Ensure that your AWS Identity and Access Management (IAM) role or user has the necessary permissions to create SageMaker Processing Jobs. You can check the permissions by reviewing the IAM policy attached to your role or user. Check AWS Region: Service limits can vary across different AWS regions. Try running your pipeline in a different region where you might have a higher service limit for the instance type you want to use. Check for Existing SCPs: If you're part of an AWS Organization, there might be an existing SCP applied to your account or OU that is restricting the service limit for the instance type you're trying to use. You can check for any applicable SCPs by reviewing the "Service control policies" section in the AWS Organizations console.