How to send ECS container logs to Grafana Loki using Fluent Bit on Fargate?

To send ECS container logs to Grafana Loki using Fluent Bit on Fargate with FireLens, follow these steps:

1. Set up a task definition with FireLens:

• Include a log router container with Fluent Bit
• Add your application container with the awsfirelens log driver

2. Configure the log router container:

• Use the AWS for Fluent Bit image (e.g., aws-for-fluent-bit:stable)
• Set up a FireLens configuration in the container definition

3. Create a custom Fluent Bit configuration file that includes:

• The standard service and input sections
• Filters to process your logs
• An output section configured for Loki

4. Specify the custom configuration in your FireLens setup:

• Use config-file-type set to either s3 or file
• Set config-file-value to the location of your configuration file

5. Configure your application container:

• Use the awsfirelens log driver
• Set log-driver-options to specify how logs should be processed

6. Set up proper IAM permissions:

• Ensure your task has the necessary IAM role to access any AWS services
• Add permissions for your Fluent Bit container to access Loki

7. Configure Loki output in Fluent Bit:

• Add an OUTPUT section in your Fluent Bit configuration:

[OUTPUT]
Name            loki
Match           your_application_logs_tag
Host            your-loki-server
Port            3100
Labels          job=ecs,task_definition=${ecs_task_definition_family},task_id=${ecs_task_arn}

Remember that FireLens automatically adds metadata to your logs (like cluster name, task ARN, etc.), which can be useful for filtering in Grafana. If you don't want this metadata, you can disable it by setting enable-ecs-log-metadata to false.

By default, FireLens ensures the log router container starts before your application containers and stops after them, which helps prevent log loss.

For high-throughput applications, consider tuning the buffer settings in your Fluent Bit configuration to handle the log volume efficiently.
Sources
Using custom log routing with FireLens for Amazon ECS - AWS Prescriptive Guidance
Example Amazon ECS task definition: Route logs to FireLens - Amazon Elastic Container Service
Send Amazon ECS logs to an AWS service or AWS Partner - Amazon Elastic Container Service
Amazon ECS container logs for EC2 and Fargate launch types - AWS Prescriptive Guidance