how to create a sagemaker processing job ?


i have used boto3 to create a sagemaker processing job , but i want to do the same with aws cli. I see the documentation , but find at times it is not very clear. for example, i have boto3 code to create processing job ( example contrived for this question) .

what would this look like if we used aws cli?

import boto3
sage_maker = boto3.client('sagemaker')

      "S3Input" : {
           "S3Uri": "s3://BUCKET",
           "S3DataType": "S3Prefix",
           "S3InputMode": "File"
     "ClusterConfig": {
          "InstanceCount": 1,
          "InstanceType": "ml.t3.large"
1 Answer
Accepted Answer


In the case of AWS CLI, I think you will use the "create-processing-job" command.

I think the boto3 code you provided will look like the following when converted to AWS CLI.

aws sagemaker create-processing-job --processing-job-name "jakesjob" --processing-inputs S3Input={S3Uri="s3://BUCKET",S3DataType="S3Prefix",S3InputMode="File"} --processing-resources ClusterConfig={InstanceCount=1,InstanceType="ml.t3.large"}
profile picture
answered 2 months ago
  • Thanks @riku - I will try this . Looking at documentation for cli . It states that - - processing-inputs is a list . I’m not sure if we put this inside [ ]

  • When passing parameters in JSON, it must be a list. If you pass parameters using "Shorthand Syntax" instead of JSON, I think an error will occur if you put them in a list. a

  • When setting parameters in JSON, I think it would be as follows.

    aws sagemaker create-processing-job --processing-job-name "jakesjob" --processing-inputs '[{"S3Input": {"S3Uri": "s3://BUCKET", "S3DataType": "S3Prefix", "S3InputMode": "File"}}]' --processing-resources '{"ClusterConfig": {"InstanceCount": 1, "InstanceType": "ml.t3.large"}}'
  • How to use the AWS CLI's "Shorthand Syntax" is explained in the following document.
    If you use "Shorthand Syntax" and enter the list with "[ ]", an error will occur. When representing a list, use spaces.

    # "Shorthand Syntax"
    --tags Key=My1stTag,Value=Value1 Key=My2ndTag,Value=Value2 Key=My3rdTag,Value=Value3
    # JSON
        --tags '[
            {"Key": "My1stTag", "Value": "Value1"},
            {"Key": "My2ndTag", "Value": "Value2"},
            {"Key": "My3rdTag", "Value": "Value3"}

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions