how to inference parameters to a huggingface model hosted in sagemaker?
0
I created a model resource in sagemaker . the model is a tar file , downloaded from hugging face and fine tuned. based on the documentation provided ( sample code below) . the code sample is passing HF_TASK inference parameter and i assume this is
hugging face specific, but is it possible to pass other parameters like padding or truncation and max_length ? such as
padding : True
truncation: True
max_length = 512 ...
If you are using a Pretrained model you may not be able to tweak params such as padding. I am not sure why do you want to do that while inferencing.