how to use custom_attribute in sagemaker api?

Question

based on documentation provided here -> https://docs.aws.amazon.com/cli/latest/reference/sagemaker-runtime/invoke-endpoint-async.html
I am passing  a custom attributes parameter when calling the invoke-endpoint-async function.

```
  invoke-endpoint-async
--endpoint-name 
[--custom-attributes ]
[--inference-id ]
--input-location

```

are there any sample/example on how can i read this in my code before/after invoking my model for inference.

I am creating a preprocessing file and have input_fn and predict_fn function, can value passed in custom_attributes during api call be read or written to the response here? also, if it can be written to the response like documentation says, where can i see it . as the async endpoint only gives the output location when invoked and later process the request, i don't see the response, i just see the dumped out file in the specified output location. how can i see the full response?
```
sagemaker_model = TensorFlowModel(model_data = 'model_data_path',
role = execution_role,
framework_version = '1.12',
source_dir ='src',
entry_point = 'preprocessing.py', 
...)
```
file: preprocessing.py
```
def input_fn(request_body, content_type):
    //read the custom attribute here 
```

Answer

Additionally to AWS CLI, you can pass `CustomAttributes` parameter during invocation of endpoint with boto3-client or SM SDK:

* In case of boto3: `runtime_client.invoke_endpoint(CustomAttributes=json.dumps({key1:val1, key2:val2, ...}))`
* In case of SM SDK: `predictor.predict(payload, initial_args={'CustomAttributes': json.dumps({key1:val1, key2:val2, ...})})`

And then you could parse the `context`-arg in the [input-handlers](https://sagemaker.readthedocs.io/en/stable/frameworks/tensorflow/deploying_tensorflow_serving.html#how-to-implement-the-pre-and-or-post-processing-handler-s) of `preprocessing.py`, as for example:

```
def handler(data, context):
    processed_input = _process_input(data, context)
    custom_attrs = json.loads(context.custom_attributes)
    # logic to parse / enact custom attrs ...
    response = requests.post(context.rest_uri, data=processed_input)
    return _process_output(response, context)
```

See also this post: https://aws.amazon.com/blogs/machine-learning/amazon-sagemaker-runtime-now-supports-the-customattribute-header/

how to use custom_attribute in sagemaker api?

Relevanter Inhalt