How to get cloudwatch agent working on ECS with terraform

0

I'm doing my best to follow these instructions. https://docs.aws.amazon.com/AmazonCloudWatch/latest/monitoring/CloudWatch_Embedded_Metric_Format_Generation_CloudWatch_Agent.html

I've added a container definition to my terraform

resource "aws_ecs_task_definition" "taskdef" {
    family = "my_service${var.suffix}_service"
    container_definitions = jsonencode([
        {
            assign_public_ip = true
            name = "cloudwatch-agent"
            image = "public.ecr.aws/cloudwatch-agent/cloudwatch-agent:latest"
            memory = 256
            cpu = 256
            portMappings = [
                {
                    protocol = "tcp"
                    containerPort = 25888
                },
                {
                    protocol = "udp"
                    containerPort = 25888
                }
            ]
            logConfiguration = {
                logDriver = "awslogs"
                options = {
                    awslogs-create-group = "true"
                    awslogs-group = "/aws/ecs/my-log-group"
                    awslogs-region = "us-west-2"
                    awslogs-stream-prefix = "ecs"
                }
            }
            Environment = [
                {"name": "CW_CONFIG_CONTENT", "value": "{\"logs\": { \"metrics_collected\": { \"emf\": { }}}}"}
            ]
        },
        {
            assign_public_ip = true
            name = "my-service${var.suffix}container"
            cpu = 1024
            memory = 2048
            image = "${var.image-name}@${var.image-hash}"
            portMappings = [
                {
                    name = "http"
                    containerPort = 80
                    hostPort = 80
                    protocol = "http"
                }
            ]
            logConfiguration = {
                logDriver = "awslogs"
                options = {
                    awslogs-create-group = "true"
                    awslogs-group = "/aws/ecs/my-log-group"
                    awslogs-region = "us-west-2"
                    awslogs-stream-prefix = "ecs"
                }
            }
            health_check = {

            }
            Environment = [
                {"name": "CLOUDWATCH_LOG_GROUP", "value": aws_cloudwatch_log_group.metrics.name},   
                {"name": "AWS_EMF_AGENT_ENDPOINT", "value": "tcp://127.0.0.1:25888"},
                {"name": "METRICS_NAMESPACE", "value": "athena-scanrunner${var.suffix}-metrics"}
            ]
        }
    ])
    cpu = 2048
    execution_role_arn = var.service_role
    task_role_arn = var.service_role
    memory = 4096
    network_mode = "awsvpc"
    requires_compatibilities = ["FARGATE"]
    runtime_platform {
        cpu_architecture = "ARM64"
        operating_system_family = "LINUX"
    }
}

I couldn't get the valueFrom field to work so I used JSON directly for the config.

In my code I've added the config

Amazon.CloudWatch.EMF.Config.EnvironmentConfigurationProvider.Config =
        new Amazon.CloudWatch.EMF.Config.Configuration
        {
            ServiceName = "Athena-ScanRunner",
            ServiceType = "WebApi",
            LogGroupName = Environment.GetEnvironmentVariable("CLOUDWATCH_LOG_GROUP"),
            AgentEndPoint = "tcp://127.0.0.1:25888",
            EnvironmentOverride = Amazon.CloudWatch.EMF.Environment.Environments.ECS
        };


    var builder = WebApplication.CreateBuilder(args);
    builder.Configuration.GetAWSOptions();
    builder.Services.AddControllers().AddJsonOptions(options =>
    {
        options.JsonSerializerOptions.Converters.Add(new JsonStringEnumConverter());

        options.JsonSerializerOptions.DefaultIgnoreCondition = JsonIgnoreCondition.WhenWritingNull;
    });
    builder.Services.AddHealthChecks();
    builder.Services.AddAWSService<IAmazonDynamoDB>();
    builder.Services.AddAWSService<IAmazonECS>();
    builder.Services.AddAWSService<IAmazonS3>();
    builder.Services.AddAWSService<IAmazonSQS>();
    builder.Services.AddEmf();
    builder.Services.AddSingleton<IScanTaskDb, ScanTaskDynamoDb>();
    builder.Services.AddSingleton<ITaskQueue, SqsTaskQueue>();
    builder.Services.AddSingleton<IFileStore, S3FileStore>();


    var app = builder.Build();

    app.UseHealthChecks("/");
    app.UseEmfMiddleware();

and I've added the metrics to each api endpoint.

I get no metrics flowing. How do I make this setup work?

1 Antwort
1

Unfortunately you haven’t shared the full VPC setup or task role definition. However.

Do you have the ecs task running in a public subnet?

Does the task have the correct IAM permissions to create log groups and streams?

Also you should place your CW Agent Config in an SSM parameter and let the container read it in as an environment variable.

profile picture
EXPERTE
beantwortet vor 2 Monaten
profile picture
EXPERTE
A_J
überprüft vor 2 Monaten
  • Yeah public subnet, you can also see the assigned public IP.

    Permissions look correct, I'm getting no errors and normal logs work correctly.

    Also you should place your CW Agent Config in an SSM parameter and let the container read it in as an environment variable.

    valueFrom doesn't work with terraform or entering the JSON in the task definition editor. I can't do this if the mechanism specified in the docs just silently fails.

Du bist nicht angemeldet. Anmelden um eine Antwort zu veröffentlichen.

Eine gute Antwort beantwortet die Frage klar, gibt konstruktives Feedback und fördert die berufliche Weiterentwicklung des Fragenstellers.

Richtlinien für die Beantwortung von Fragen