SysAdmin Series – Explaining mpstat Command Output

Published on: May 1, 2019 by Vinayan C V

SysAdmin Series – Explaining mpstat Command Output

Scenario:

Sometimes, you might have noticed high load on the CPU even though there is no unusual process running. mpstat is a useful command to monitor the CPU, it would tell you the CPU statistics. How the workload is balanced across the processors, whether a process is single threaded or not etc.

The first processors will signed as CPU 0. The second one will be signed CPU 1 and so on.

[root@server ~]# mpstat
Linux 3.10.0-514.21.2.el7.x86_64 (server.hosting.com) 01/18/2018 _x86_64_ (8 CPU)
12T49T28 AM CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice
%idle 12T49T28 AM all 9.23 0.85 2.71 1.54 0.00 0.03 0.00 0.00 0.00 85.63

all : means All CPUs %usr : show the percentage of CPU utilization that occurred while executing at the user level (application)

%nice : show the percentage of CPU utilization that occurred while executing at the user level with nice priority

%sys : show the percentage of CPU utilization that occurred while executing at the system level (kernel)

%iowait : show the percentage of time that the CPU or CPUs were idle during which the system had an outstanding disk I/O request

%irq : show the percentage of time spent by the CPU or CPUs to service hardware interrupts

%soft : show the percentage of time spent by the CPU or CPUs to service software interrupts

%steal : show the percentage of time spent in involuntary wait by the virtual CPU or CPUs while the hypervisor was servicing another virtual processor

%guest : show the percentage of time spent by the CPU or CPUs to run a virtual processor

%idle : show the percentage of time that the CPU or CPUs were idle and the system did not have an outstanding disk I/O request

The important fields to look at are %user, %sys, %iowait and %idle. Below output lists the CPU usage per core/virtual processor.

Here is some real life examples of the mpstat command.

To view all CPU utilization of processors, use -P ALL parameter.

[root@server ~]# mpstat -P ALL

Linux 3.10.0-514.21.2.el7.x86_64 (server.hosting.com) 01/18/2018 _x86_64_(8 CPU)

12T51T53 AM CPU %usr %nice %sys %iowait %irq %soft %ste%l %guest %gnice
%idle 12T51T53 AM all 9.23 0.85 2.71 1.54 0.00 0.03 0.00 0.00 0.00 85.63
12T51T53 AM 0 5.87 0.66 1.95 0.53 0.00 0.04 0.00 0.00 0.00 90.95
12T51T53 AM 1 10.72 1.26 3.27 0.96 0.00 0.01 0.00 0.00 0.00 83.78
12T51T53 AM 2 7.80 0.97 1.96 0.92 0.00 0.00 0.00 0.00 0.00 88.35
12T51T53 AM 3 9.95 0.85 3.20 0.68 0.00 0.00 0.00 0.00 0.00 85.32
12T51T53 AM 4 14.28 1.84 3.46 5.21 0.00 0.03 0.00 0.00 0.00 75.19
12T51T53 AM 5 10.73 0.58 3.16 1.74 0.00 0.01 0.00 0.00 0.00 83.80
12T51T53 AM 6 7.49 0.42 1.96 1.83 0.00 0.14 0.00 0.00 0.00 88.16
12T51T53 AM 7 7.01 0.24 2.75 0.48 0.00 0.01 0.00 0.00 0.00 89.52

To print CPU utilization by processor, use -P parameter followed by CPU number to see specific CPU utilization.

[root@server ~]# mpstat -P 0
Linux 3.10.0-514.21.2.el7.x86_64 (server.hosting.com) 01/18/2018 _x86_64_ (8 CPU)
12T49T28 AM CPU %usr %nice %sys %iowait %irq %soft %steal %guest %gnice
%idle 12T49T28 AM 0 9.23 0.85 2.71 1.54 0.00 0.03 0.00 0.00 0.00 85.63

To view CPU utilization per intervals:

[root@server ~]# mpstat 4 3
Linux 3.2.0-57-generic (USERNB01) 12/12/2013 _x86_64_ (2 CPU)
04:27:11 PM CPU %usr %nice %sys %iowait %irq %soft %steal %guest %idle
04:27:15 PM all 0.67 0.00 0.34 0.00 0.00 0.00 0.00 0.00 98.99
04:27:19 PM all 1.17 0.00 0.33 1.33 0.00 0.00 0.00 0.00 97.17
04:27:23 PM all 0.84 0.00 0.17 0.00 0.00 0.00 0.00 0.00 98.99
Average: all 0.92 0.00 0.25 0.71 0.00 0.00 0.00 0.00 98.12

The above command is to show you 3 reports about CPU utilization with 4 seconds intervals.

Hope this will help you when next time the CPU load goes high.