I want consolidated application logs running on AWS ECS with microservices architecture
I want consolidated logging of all Front End, Back End and ALB/Network logs for application which is running on AWS ECS fargate with microservices architecture. Which one will be good out of Datadog or Splunk or ELK or AWS Centralized Logging or Prometheus / Grafana.
I would definitely make sure that you collect all metrics on AWS CloudWatch. For non-custom metrics, these are very cheap and integrate everywhere. Whether you visualize these in CW or in Grafana, doesn't matter. I'd recommend to stick as much as possible to a single source of truth when it comes to metrics.
As for logs, with ECS + FireLens you can send your logs to CW Logs pretty much any destination (fluentbit powa!).
But considering that the ALB logs will go to S3, I'd recommend FireHose / DataStream to log things in the same bucket / different buckets, same format. That way you can at least query the data with strong correlations with Athena for example.
With 1 FireHose per source account and a central storage location, you can also log things in multi-accounts without any issues pretty easily. We have done / are in the process of doing just that: monitoring account, aggregating / accessing all others to provide near real-time view of what's happening across all our services, with each applications teams considered to be their own tenants.
The logs aren't updating in cloudwatch log group for an ECS taskasked 13 days ago
Architecture for multi-region ECS applicationasked 2 months ago
Is log cleaner logging anything on AWS MSK?asked 2 months ago
Location of ECS CodeDeploy logs?asked 6 months ago
Centralized logging - one region, perhaps one account (S3/VPC)asked 4 months ago
Deploying application on AWS ECS Clusterasked 7 months ago
Segregate logs in Opensearch based on accountsasked 5 months ago
I want consolidated application logs running on AWS ECS with microservices architectureasked a month ago
Parse application logs via Athena and Glue catalogasked 6 months ago
Are there any best practices for sending logs from ECS on EC2, ECS on Fargate and other AWS services such as API GW, load balancers (and more AWS services) to Splunk?asked 3 months ago