SWF PendingTasks metric 0 although no active SWF workflows

Question

Hi,

Our team in Amazon, uses a CloudWatch metric that tracks all active/pending/scheduled SWF workflows. The metric has been working fine since last year and was accurately able to track ongoing SWF workflows. However, since Sept 6th, the metric is always in ALARM (meaning it always says that there is a pending SWF workflow), although there is no active workflow from the SWF console.

Additionally, the 'PendingTasks' CloudWatch metric count keeps on increasing since Sept 6th till now. I couldn't find any active workflows from the SWF console to terminate. Appreciate if any support that can bring back the CloudWatch monitor to accurately track active SWF workflows.

Thanks

Answer

Hello,

CloudWatch works as push based service, and not as a poll service. By this I mean that CloudWatch itself does not pull any datapoint from the service, its the service which push the datapoints to CloudWatch - https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/cloudwatch_architecture.html

Check that particular alarm metric's datapoint and see if that alarm is working fine as per the datapoint received from SWF workflow. If your alarm is in correct state based on the datapoint it received, check on SWF service that why it is not sending correct metric information.

SWF PendingTasks metric > 0 although no active SWF workflows

Contenus pertinents