By using AWS re:Post, you agree to the Terms of Use

how to use custom_attribute in sagemaker api?

1

based on documentation provided here -> https://docs.aws.amazon.com/cli/latest/reference/sagemaker-runtime/invoke-endpoint-async.html I am passing a custom attributes parameter when calling the invoke-endpoint-async function.

  invoke-endpoint-async
--endpoint-name <value>
[--custom-attributes <value>]
[--inference-id <value>]
--input-location <value>

are there any sample/example on how can i read this in my code before/after invoking my model for inference.

I am creating a preprocessing file and have input_fn and predict_fn function, can value passed in custom_attributes during api call be read or written to the response here? also, if it can be written to the response like documentation says, where can i see it . as the async endpoint only gives the output location when invoked and later process the request, i don't see the response, i just see the dumped out file in the specified output location. how can i see the full response?

sagemaker_model = TensorFlowModel(model_data = 'model_data_path',
role = execution_role,
framework_version = '1.12',
source_dir ='src',
entry_point = 'preprocessing.py', 
...)

file: preprocessing.py

def input_fn(request_body, content_type):
    //read the custom attribute here 
1 Answer
0

Additionally to AWS CLI, you can pass CustomAttributes parameter during invocation of endpoint with boto3-client or SM SDK:

  • In case of boto3: runtime_client.invoke_endpoint(CustomAttributes=json.dumps({key1:val1, key2:val2, ...}))
  • In case of SM SDK: predictor.predict(payload, initial_args={'CustomAttributes': json.dumps({key1:val1, key2:val2, ...})})

And then you could parse the context-arg in the input-handlers of preprocessing.py, as for example:

def handler(data, context):
    processed_input = _process_input(data, context)
    custom_attrs = json.loads(context.custom_attributes)
    # logic to parse / enact custom attrs ...
    response = requests.post(context.rest_uri, data=processed_input)
    return _process_output(response, context)

See also this post: https://aws.amazon.com/blogs/machine-learning/amazon-sagemaker-runtime-now-supports-the-customattribute-header/

answered 6 months ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Guidelines for Answering Questions