Request for Assistance: Monitoring Validation Metrics with Factorization Machine Training in SageMaker

Question

Dear AWS Support/Comunity,

I am currently working on a project that involves training a Factorization Machine model using Amazon SageMaker. Despite a thorough search, I have not been able to find documentation on how to monitor validation metrics in real-time, similar to how I can with training metrics.

The ability to observe the performance of the validation set during the training process is crucial for my project, as it would allow me to detect potential overfitting issues effectively.

Could you please provide me with guidance on how to enable and access these validation metrics? Detailed steps or references to specific sections of the documentation would be greatly appreciated.

Thank you in advance for your assistance. I look forward to your prompt response.

Best,
Felipe

Answer

The following metrics are emitted by the SageMaker Factorization Machine algorithm: https://docs.aws.amazon.com/sagemaker/latest/dg/fm-tuning.html#fm-metrics

You should see these in CloudWatch logs/metrics (if a regex is defined) 
https://docs.aws.amazon.com/sagemaker/latest/dg/training-metrics.html#define-train-metrics

Request for Assistance: Monitoring Validation Metrics with Factorization Machine Training in SageMaker

Conteúdo relevante