Relevancy of SageMaker Model Monitor for NLP?

Yes, You can use model monitor for data capture and scheduling in your own custom container with the relevant monitoring for NLP use case.
For example, there's a blog post for model monitor for computer vision classification prediction with defined *alert * of predict more than expected.
https://aws.amazon.com/blogs/machine-learning/automated-monitoring-of-your-machine-learning-models-with-amazon-sagemaker-model-monitor-and-sending-predictions-to-human-review-workflows-using-amazon-a2i/?nc1=b_rp