how to create a sagemaker processing job ?

0

i have used boto3 to create a sagemaker processing job , but i want to do the same with aws cli. I see the documentation , but find at times it is not very clear. for example, i have boto3 code to create processing job ( example contrived for this question) .

what would this look like if we used aws cli?

import boto3
sage_maker = boto3.client('sagemaker')

sage_maker.create_processing_job(
  ProcessingJobName="jakesjob",
  ProcessingInputs={
      "S3Input" : {
           "S3Uri": "s3://BUCKET",
           "S3DataType": "S3Prefix",
           "S3InputMode": "File"
       }
  },
  ProcessingResources={
     "ClusterConfig": {
          "InstanceCount": 1,
          "InstanceType": "ml.t3.large"
      }
  }
)
1 Answer
1
Accepted Answer

Hello.

In the case of AWS CLI, I think you will use the "create-processing-job" command.
https://awscli.amazonaws.com/v2/documentation/api/latest/reference/sagemaker/create-processing-job.html

I think the boto3 code you provided will look like the following when converted to AWS CLI.

aws sagemaker create-processing-job --processing-job-name "jakesjob" --processing-inputs S3Input={S3Uri="s3://BUCKET",S3DataType="S3Prefix",S3InputMode="File"} --processing-resources ClusterConfig={InstanceCount=1,InstanceType="ml.t3.large"}
profile picture
EXPERT
answered 2 months ago
  • Thanks @riku - I will try this . Looking at documentation for cli . It states that - - processing-inputs is a list . I’m not sure if we put this inside [ ]

  • When passing parameters in JSON, it must be a list. If you pass parameters using "Shorthand Syntax" instead of JSON, I think an error will occur if you put them in a list. a

  • When setting parameters in JSON, I think it would be as follows.

    aws sagemaker create-processing-job --processing-job-name "jakesjob" --processing-inputs '[{"S3Input": {"S3Uri": "s3://BUCKET", "S3DataType": "S3Prefix", "S3InputMode": "File"}}]' --processing-resources '{"ClusterConfig": {"InstanceCount": 1, "InstanceType": "ml.t3.large"}}'
    
  • How to use the AWS CLI's "Shorthand Syntax" is explained in the following document. https://docs.aws.amazon.com/cli/latest/userguide/cli-usage-shorthand.html
    If you use "Shorthand Syntax" and enter the list with "[ ]", an error will occur. When representing a list, use spaces.

    # "Shorthand Syntax"
    --tags Key=My1stTag,Value=Value1 Key=My2ndTag,Value=Value2 Key=My3rdTag,Value=Value3
    
    # JSON
        --tags '[
            {"Key": "My1stTag", "Value": "Value1"},
            {"Key": "My2ndTag", "Value": "Value2"},
            {"Key": "My3rdTag", "Value": "Value3"}
        ]'
    

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions