2 Answers
- Newest
- Most votes
- Most comments
0
Due to the fact that this is an account specific issue, that is also related to your unique use case involving a custom inference script, to answer your question with great depth and to investigate the error you have provided, we require details that are non-public information. Please kindly open a support case with AWS using the following link and our engineers will be pleased to assist you further.
In your support case, kindly include the following details:
- BatchTransform Job ARN
- The overview you have included in your question for context.
- The inference script.
- CloudWatch BatchTransform job logs (say 10 lines before error and 5 lines after the error). These can be found in the following log stream "/aws/sagemaker/TransformJobs" , as seen the doc in [1].
[1] Log Amazon SageMaker Events with Amazon CloudWatch - https://docs.aws.amazon.com/sagemaker/latest/dg/logging-cloudwatch.html
answered a year ago
0
Agree that this sounds like it'd need some specific troubleshooting and good to raise with support.
However my initial guesses would be:
- If your model output array is 1D, might be that it's getting serialized to
1,2,3,4,...
instead of1\n2\n3\n4\n...
- in which case batch transform might be seeing the output as one record with many fields, instead of many records with a single field each. Try expanding the extra dimension / ensuring the CSV output format for a batched request is as you expect - If you're trying to accept/ignore header rows in input batches, remember you need to return the same number of output rows. Generally the model should return same number of records out as the batch had going in.
- If urgently struggling with this, you could try setting
strategy="SingleRecord"
which will ensure each request sends only a single record so less to go wrong. However note that it'll be less resource-efficient (more HTTP/request overhead vs actual payload, than the default batched approach)
Relevant content
- asked a year ago
- asked a year ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 2 years ago
- AWS OFFICIALUpdated 2 years ago