Are there any best practices for sending logs from ECS on EC2, ECS on Fargate and other AWS services such as API GW, load balancers (and more AWS services) to Splunk?
A customer wants to get various flavors of logs from ECS on EC2, ECS on Fargate, API GW logs, load balancer logs, (and potentially RDS Aurora) to Splunk endpoint
Following have already been referenced to the customer Splunk white paper: https://www.splunk.com/pdfs/white-papers/getting-data-into-gdi-splunk-from-aws.pdf And a couple of these blog posts https://www.splunk.com/en_us/blog/it/splunking-aws-ecs-part-2-sending-ecs-logs-to-splunk.html https://www.splunk.com/en_us/blog/it/splunking-aws-ecs-and-fargate-part-3-sending-fargate-logs-to-splunk.html
The challenge is which approach to use as we they ECS Fargate and ECS on EC2 along with other AWS services for which they want to centralize their logs. Currently they are considering separate lambda functions for ECS, lambda for LBs etc. to pull logs from cloudwatch and push them to Splunk endpoint. Trying to seek suggestions on what could be the best practices.
For ECS, you could use the Splunk log driver for ECS as described in https://aws.amazon.com/premiumsupport/knowledge-center/ecs-task-fargate-splunk-log-driver/
Scaling ECS Fargate - graceful session drainingAccepted Answerasked a year ago
Sending UDP traffic to EC2 host from ECS instance, security group issue on EC2 hostasked 2 months ago
Websoket with API Gateway, ECS and VPC linkasked 6 days ago
Moving to ECS-Fargate from EC2asked 4 months ago
Resource Utilization for Fargate ECSAccepted AnswerMODERATORasked 2 years ago
MaxInstanceLifetime value for ECS FargateAccepted Answerasked 3 months ago
Failed to expose AWS ECS Fargate Service via ALBasked 18 days ago
ECS Fargate + Service Discovery + ALBAccepted Answerasked 4 years ago
Are there any best practices for sending logs from ECS on EC2, ECS on Fargate and other AWS services such as API GW, load balancers (and more AWS services) to Splunk?asked 2 months ago
Mounting AWS FSx for Windows File Share with ECS on EC2?asked a month ago