Skip to content
By using AWS re:Post, you agree to the AWS re:Post Terms of Use
AWS re:Postre:Post
About re:Post

Why is CloudWatch Agent not collecting JMX Metrics?

0

CloudWatch Agent is not logging JVM JMX metrics to CloudWatch.

I have configured my agent as follows:

{
  "agent": {
    "metrics_collection_interval": 60,
    "run_as_user": "root"
  },
  "metrics": {
    "aggregation_dimensions": [
      [
        "BeanstalkName"
      ]
    ],
    "append_dimensions": {
      "InstanceId": "${aws:InstanceId}",
      "InstanceType": "${aws:InstanceType}"
    },
    "metrics_collected": {
      "cpu": {
        "measurement": [
          "cpu_usage_active"
        ],
        "resources": [
          "*"
        ],
        "totalcpu": false,
        "append_dimensions": {
          "BeanstalkName": "redacted"
        }
      },
      "disk": {
        "measurement": [
          "used_percent"
        ],
        "resources": [
          "*"
        ],
        "append_dimensions": {
          "BeanstalkName": "redacted"
        }
      },
      "diskio": {
        "measurement": [
          "io_time"
        ],
        "resources": [
          "*"
        ],
        "append_dimensions": {
          "BeanstalkName": "redacted"
        }
      },
      "mem": {
        "measurement": [
          "mem_used_percent"
        ],
        "append_dimensions": {
          "BeanstalkName": "redacted"
        }
      },
      "jmx": [
        {
          "endpoint": "localhost:6745",
          "jvm": {
            "measurement": [
              "jvm.gc.collections.count",
              "jvm.gc.collections.elapsed",
              "jvm.memory.heap.used",
              "jvm.memory.heap.max",
              "jvm.memory.nonheap.used",
              "jvm.memory.nonheap.max",
              "jvm.memory.pool.used",
              "jvm.threads.count"
            ]
          },
          "append_dimensions": {
            "BeanstalkName": "redacted"
          }
        }
      ]
    }
  }
}

My application jar is launched with:

-Dcom.sun.management.jmxremote.port=6745 
-Dcom.sun.management.jmxremote.authenticate=false 
-Dcom.sun.management.jmxremote.ssl=false 
-Dcom.sun.management.jmxremote.host=localhost

This configuration passes validation, and all metrics are being uploaded to CloudWatch, except the JMX ones.

Agent version is: 1.300057.2

The logs in /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log don't mention any issues at first glance:

2025-08-28T12:47:56Z I! Starting AmazonCloudWatchAgent CWAgent/1.300057.2 (go1.23.11; linux; arm64) with log file /opt/aws/amazon-cloudwatch-agent/logs/amazon-cloudwatch-agent.log with log target lumberjack
2025-08-28T12:47:56Z I! AWS SDK log level not set
2025-08-28T12:47:56Z I! creating new logs agent
2025-08-28T12:47:56Z I! [logagent] starting
2025-08-28T12:47:56Z I! [logagent] found plugin cloudwatchlogs is a log backend
2025-08-28T12:47:56Z I! [logagent] found plugin logfile is a log collection
2025-08-28T12:47:56Z I! [logagent] start logs plugin file paths [/var/log/eb-engine.log /var/log/eb-hooks.log /var/log/nginx/access.log /var/log/nginx/error.log /var/log/web.stdout.log]
2025-08-28T12:47:56Z I! [inputs.logfile] turned on logs plugin
2025-08-28T12:47:56Z I! {"caller":"service@v0.124.0/service.go:199","msg":"Setting up own telemetry..."}
2025-08-28T12:47:56Z I! {"caller":"filterprocessor@v0.124.1/metrics.go:99","msg":"Metric filter configured","include match_type":"strict","include expressions":[],"include metric names":["jvm.gc.collections.count","jvm.gc.collections.elapsed","jvm.memory.heap.used","jvm.memory.heap.max","jvm.memory.nonheap.used","jvm.memory.nonheap.max","jvm.memory.pool.used","jvm.threads.count"],"include metrics with resource attributes":null,"exclude match_type":"","exclude expressions":[],"exclude metric names":[],"exclude metrics with resource attributes":null}
2025-08-28T12:47:56Z I! {"caller":"service@v0.124.0/service.go:244","msg":"Skipped telemetry setup."}
2025-08-28T12:47:56Z I! {"caller":"service@v0.124.0/service.go:266","msg":"Starting CWAgent...","Version":"1.300057.2","NumCPU":2}
2025-08-28T12:47:56Z I! {"caller":"extensions/extensions.go:41","msg":"Starting extensions..."}
2025-08-28T12:47:56Z I! {"caller":"extensions/extensions.go:45","msg":"Extension is starting..."}
2025-08-28T12:47:56Z I! {"caller":"extensions/extensions.go:62","msg":"Extension started."}
2025-08-28T12:47:56Z I! {"caller":"extensions/extensions.go:45","msg":"Extension is starting..."}
2025-08-28T12:47:56Z I! {"caller":"extensions/extensions.go:62","msg":"Extension started."}
2025-08-28T12:47:56Z I! cloudwatch: get unique roll up list [[BeanstalkName]]
2025-08-28T12:47:56Z I! {"caller":"ec2tagger/ec2tagger.go:468","msg":"ec2tagger: Check EC2 Metadata."}
2025-08-28T12:47:56Z I! cloudwatch: publish with ForceFlushInterval: 1m0s, Publish Jitter: 50.983322367s
2025-08-28T12:47:56Z I! {"caller":"ec2tagger/ec2tagger.go:444","msg":"ec2tagger: EC2 tagger has started, finished initial retrieval of tags and Volumes"}
2025-08-28T12:47:56Z I! {"caller":"otlpreceiver@v0.124.0/otlp.go:116","msg":"Starting GRPC server","endpoint":"0.0.0.0:41769"}
2025-08-28T12:47:56Z I! {"caller":"ec2tagger/ec2tagger.go:468","msg":"ec2tagger: Check EC2 Metadata."}
2025-08-28T12:47:56Z I! {"caller":"ec2tagger/ec2tagger.go:444","msg":"ec2tagger: EC2 tagger has started, finished initial retrieval of tags and Volumes"}
2025-08-28T12:47:56Z I! {"caller":"service@v0.124.0/service.go:289","msg":"Everything is ready. Begin running and processing data."}
2025-08-28T12:47:57Z I! Reading from offset range [0-143502] in /var/log/eb-engine.log
2025-08-28T12:47:57Z I! Reading from offset range [0-631890] in /var/log/nginx/access.log
2025-08-28T12:47:57Z I! Reading from offset range [0-12804] in /var/log/nginx/error.log
2025-08-28T12:47:57Z I! Reading from offset range [0-2992275] in /var/log/web.stdout.log
Topics
Management & GovernanceCompute
Tags
Amazon EC2Amazon CloudWatch
Language
English

asked a year ago318 views

1 Answer
-1

There are a few potential reasons why your CloudWatch Agent is not collecting JMX metrics:

  1. Configuration format issue: The jmx section in your configuration appears to be formatted incorrectly. In the CloudWatch agent configuration file, jmx should be a subsection directly under metrics_collected, not an array. Your configuration shows "jmx": [...] but it should be "jmx": {...}.

  2. JVM binding verification: While you've correctly configured the JVM with the necessary JMX remote properties, you should verify that the JVM is actually binding to port 6745 and that the port is accessible from the CloudWatch agent. You can check this by using tools like netstat or lsof to confirm the port is open and listening.

  3. Namespace consideration: By default, the CloudWatch agent uses the namespace CWAgent for metrics. If you're looking for the JMX metrics in a different namespace in the CloudWatch console, you might not see them. Check if you've specified a custom namespace in your configuration or look for the metrics in the default CWAgent namespace.

  4. Agent restart requirement: If you updated the CloudWatch agent configuration file, you need to restart the agent with the fetch-config option to apply the changes. Use the following command:

sudo /opt/aws/amazon-cloudwatch-agent/bin/amazon-cloudwatch-agent-ctl -a fetch-config -s -m ec2 -c file:/path/to/your/config.json

To fix the configuration issue, modify your jmx section to follow this format:

"jmx": {
"endpoint": "localhost:6745",
"jvm": {
"measurement": [
"jvm.gc.collections.count",
"jvm.gc.collections.elapsed",
"jvm.memory.heap.used",
"jvm.memory.heap.max",
"jvm.memory.nonheap.used",
"jvm.memory.nonheap.max",
"jvm.memory.pool.used",
"jvm.threads.count"
]
},
"append_dimensions": {
"BeanstalkName": "redacted"
}
}

After making these changes, restart the CloudWatch agent with the fetch-config option to apply the new configuration.
Sources
Collect Java Management Extensions (JMX) metrics - Amazon CloudWatch
Issue with CloudWatch agent configuration | AWS re:Post
Manually create or edit the CloudWatch agent configuration file - Amazon CloudWatch
Troubleshooting the CloudWatch agent - Amazon CloudWatch

answered a year ago

You are not logged in. Log in to post an answer.

A good answer clearly answers the question and provides constructive feedback and encourages professional growth in the question asker.

Relevant content