AWS Eventbridge | Step Functions monitoring tips & advice?
What is the best way to collect EB events and Step Function State Machines status, preferably in one place and on a shoestring budget?
My scenario: Working on an MVP whereby Eventbridge launches a bunch of Step Function State Machines. These grab data from DDB, do some text processing, post results back to the table then put the status of the completed job to SNS (the topic emails me although I might switch to Slack) and lastly back to EB. Of course everything is logged in Cloudwatch in (mostly) individual logs.
Clearly this is a flood of notifications but as this is my first MVP with serverless, and soon to launch, I want to stay on top of things.
Are there better solutions?
The first question? What are the Points you want tom monitor? The look if it is available in CloudWatch Metrics, if not look at CloudWatch Log Filters https://docs.aws.amazon.com/AmazonCloudWatch/latest/logs/MonitoringLogData.html. Then you can create a Dashboard in CloudWatch.
An alternative is to use OpenSearch and create the Dashboard with Kibana. Or if you are more familiar with Grafana then use the managed Grafana https://aws.amazon.com/de/grafana/features/?nc=sn&loc=2 you can also combine this with Prometheus https://aws.amazon.com/prometheus/
This will add additional Services and costs to your infrastructure.
So my recommendation is to go with CloudWatch and CloudWatch Dashboards first and have a look on the alternatives if you are missing functionality or if you already familiar with Kibana or Grafana.
Yes, I want to be able to quickly assess that each (sub-)workflow was completed correctly. I will look into your suggestions. Thank you.
Step Function Retry Metricsasked 3 months ago
Trigger Step Function with API Gateway and use Fargate within Step Function?asked a month ago
Can not create a stepfunction with create-state-machine command in docker-composeasked 4 months ago
Can I specify GET URL path parameter in step function?asked 2 months ago
State Language Specificationasked 2 months ago
AWS Eventbridge | Step Functions monitoring tips & advice?Accepted Answerasked 5 months ago
how to trigger a step function from a s3 object notification?asked a month ago
Retrieve or store AWS Step function Execution history older than 90 days.asked 2 years ago
Step Function to Send Email on Error/Successasked 4 months ago
how to version step functions for ML?Accepted AnswerEXPERTasked 3 years ago