SageMaker on AWS Marketplace - autoscaling, parameters, pricing and S3

Hi,

1. In general, AWS Marketplace uses a pay-as-you-go pricing model, which means that customers are charged for the resources they consume on an hourly or per-inference basis. I'm not aware of any way to disable this pricing model when selling a model package on AWS Marketplace. However, it's worth noting that AWS Marketplace also offers private listings, which allow you to sell your model package directly to a specific customer or group of customers. Private offers are not discoverable by other customers and are not subject to the same pricing and billing terms as public listings. You may want to consider using a private listing if you want to offer a different pricing model for your model package. Reference: https://docs.aws.amazon.com/marketplace/latest/buyerguide/buyer-private-offers.html

2. Yes, it is possible to define an auto scaling policy for a hosted Amazon SageMaker endpoint that runs a model package. To define an auto scaling policy for a SageMaker endpoint, you can use the UpdateEndpoint API or the SageMaker console. When updating an endpoint, you can specify the desired number of instances and the minimum and maximum number of instances for the auto scaling policy. SageMaker will automatically scale the number of instances up or down based on the incoming traffic and the defined policy.

Here's an example of how you can use the UpdateEndpoint API to update an endpoint with an auto scaling policy:

import boto3

sm = boto3.client('sagemaker')

response = sm.update_endpoint(
    EndpointName='your-endpoint-name',
    DesiredInferenceUnits=1,
    MinInferenceUnits=1,
    MaxInferenceUnits=8
)

More details: https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_UpdateEndpoint.html

3. To make an inference call to a hosted Amazon SageMaker endpoint, you can use the invoke_endpoint method of the SageMaker runtime client. This method allows you to send an HTTP POST request to the endpoint and receive the prediction results in the response.

Here's an example of how you can use the invoke_endpoint method to make an inference request:

import boto3

sm = boto3.client('sagemaker-runtime')

response = sm.invoke_endpoint(
    EndpointName='your-endpoint-name',
    Body=b'your-request-data',
    ContentType='application/json'
)
prediction = response['Body'].read()

You can pass any parameters that your model expects in the request body. The format of the request data and the expected parameters depend on the specific model that you are using. For example, if your model expects a JSON object with a single field called "input", you can pass the input data as a JSON string in the request body.

More details: https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_runtime_InvokeEndpoint.html

4. Yes, you can use S3 to store additional dependencies for your ML model and load them into Amazon SageMaker. SageMaker allows you to specify additional code and libraries to be included in your training or inference environment by using the CodeRepository parameter of the CreateTrainingJob or CreateEndpoint API. The CodeRepository parameter should be set to the Amazon S3 URI of a Git repository that contains the code and dependencies you want to include. SageMaker will clone the repository and build the code as part of the training or inference environment.

Here's an example of how you can use the CodeRepository parameter to specify an S3-based Git repository in a CreateTrainingJob request:

import boto3

sm = boto3.client('sagemaker')

response = sm.create_training_job(
    TrainingJobName='your-training-job-name',
    HyperParameters={...},
    InputDataConfig=[{...}],
    OutputDataConfig={...},
    ResourceConfig={...},
    RoleArn='your-role-arn',
    CodeRepository='s3://your-bucket/your-repository.git'
)

More details: https://docs.aws.amazon.com/sagemaker/latest/APIReference/API_CreateTrainingJob.html