Why do EC2 instances of the same instance type show different CPU MHz values?

Resolution

In the /proc/cpuinfo file, don't check CPU clock rate or clock speed information that's measured in hertz (Hz) because the "cpu MHz" behavior is inconsistent. Depending on the cpufreq configuration, the "cpu MHz" is either constant or reports the last requested frequency.

To inspect CPU clock rate or clock speed information, use turbostat to read the frequency reports. For more information, see turbostat on the Red Hat Enterprise Customer Portal.

To invoke turbostat, use one of the following methods:

• Supply a command: The command is forked, and statistics are printed when the command action completes.
• Omit the command: turbostat displays statistics every 5 seconds. Use the -interval option to change the interval.

To verify that your instance reaches the maximum advertised CPU MHz, use stress-ng to increase the CPU load when you run the turbostat report. For more information about stress-ng, see Chapter 29. Stress testing real-time systems with stress-ng on the Red Hat Customer Portal.

Example

Note: You can use other purpose-built analysis tools to complete these steps.

In the following example, turbostat is used to check the CPU and simultaneously increase the load with the stress-ng tool.

The m5a.xlarge instance type processor uses the AMD EPYC processor 7571. This EPYC processer 7571 can operate at an all-core turbo clock speed of 2.5 GHz.

1.    Create two SSH sessions to the same instance.

2.    On both SSH shells, run the sudo su command to become the root user.

3.    In the first shell run the turbostat command.

4.    In the second shell, run the following command:

stress-ng -cpu 4 -cpu-method matrixprod -metrics-brief -perf -t 1

5.    Stop the turbostat report in the first shell, and then read the report.

The following example turbostat report is taken when stress-ng runs:

[root@test-server]# turbostat

Core	CPU	Avg_MHz	Busy%	Bzy_MHz	TSC_MHz	IRQ
-	-	628	24.64	2550	2200	5739
0	0	631	24.77	2549	2200	1494
0	2	632	24.78	2549	2200	1329
1	1	621	24.37	2550	2200	1438
1	3	628	24.63	2550	2200	1478
Core	CPU	Avg_MHz	Busy%	Bzy_MHz	TSC_MHz	IRQ
-	-	2550	100	2550	2200	20518
0	0	2550	100	2550	2200	5221
0	2	2550	100	2550	2200	5190
1	1	2550	100	2550	2200	5051
1	3	2550	100	2550	2200	5056
Core	CPU	Avg_MHz	Busy%	Bzy_MHz	TSC_MHz	IRQ
-	-	2550	100	2550	2200	20621
0	0	2550	100	2550	2200	5162
0	2	2550	100	2550	2200	5261
1	1	2550	100	2550	2200	5068
1	3	2550	100	2550	2200	5130
Core	CPU	Avg_MHz	Busy%	Bzy_MHz	TSC_MHz	IRQ
-	-	2015	79.03	2550	2200	17053
0	0	2012	78.93	2550	2200	4262
0	2	2021	79.28	2550	2200	4301
1	1	2014	78.99	2550	2200	4296
1	3	2012	78.91	2550	2200	4194
Core	CPU	Avg_MHz	Busy%	Bzy_MHz	TSC_MHz	IRQ
-	-	97	3.8	2548	2200	973
0	0	94	3.7	2547	2200	249
0	2	102	3.99	2548	2200	119
1	1	92	3.61	2549	2200	432
1	3	100	3.91	2549	2200	173
Core	CPU	Avg_MHz	Busy%	Bzy_MHz	TSC_MHz	IRQ
-	-	95	3.71	2548	2200	939
0	0	94	3.68	2547	2200	246
0	2	103	4.05	2548	2200	305
1	1	86	3.36	2549	2200	256
1	3	96	3.76	2549	2200	132
1	1	86	3.36	2549	2200	256
1	3	96	3.76	2549	2200	132

The preceding example report shows that the Busy% column value increases from 24.64% to 100.00% and then starts to reduce. The CPU gets busier and the Avg_MHz increases from 628MHz to 2550MHz.

The report explains why you see different values. The CPU was idling on 628MHz. When you increased the load on the CPU, the CPU ran at the full CPU capacity of 2550MHz. This confirms that you received the advertised instance performance.

The Avg_MHz column reflects the total number of cycles that were run divided by the measurement interval. If the %Busy column is 100%, then the processor ran at that speed for the entire interval.

The tests confirm that you can't use the proc or sysfs kernel interfaces to inspect the CPU MHz. Instead, use purpose-built analysis tools, such as turbostat and stress-ng.

Related information

Bug 197153 - Constant "cpu MHz" in /proc/cpuinfo on Kernel.org Bugzilla