Implements support for arbitrary Performance Co-Pilot
metrics with per-process instance domains to form new
htop columns. The column-to-metric mappings are setup
using configuration files which will be documented via
man pages as part of a follow-up commit.
We provide an initial set of column configurations so
as to provide new capabilities to pcp-htop: including
configs for containers, open fd counts, scheduler run
queue time, tcp/udp bytes/calls sent/recv, delay acct,
virtual machine guests, detailed virtual memory, swap.
Note there is a change to the configuration file path
resolution algorithm introduced for 'dynamic meters'.
First, look in any custom PCP_HTOP_DIR location. Then
iterate, in priority order, users home directory, then
local sysadmins files in /etc/pcp/htop, then readonly
configuration files below /usr/share/pcp/htop. This
final location becomes the preferred place for our own
shipped meter and column files.
The Settings file (htoprc) writing code is updated to
not using the numeric identifier for dynamic columns.
The same strategy used for dynamic meters is used here
where we write Dynamic(name) so the name can be setup
once more at start. Regular (static) columns writing
to htoprc - i.e. numerically indexed - is unchanged.
This commit is based on exploratory work by Sohaib Mohamed.
The end goal is two-fold - to support addition of Meters we
build via configuration files for both the PCP platform and
for scripts ( https://github.com/htop-dev/htop/issues/526 )
Here, we focus on generic code and the PCP support. A new
class DynamicMeter is introduced - it uses the special case
'param' field handling that previously was used only by the
CPUMeter, such that every runtime-configured Meter is given
a unique identifier. Unlike with the CPUMeter this is used
internally only. When reading/writing to htoprc instead of
CPU(N) - where N is an integer param (CPU number) - we use
the string name for each meter. For example, if we have a
configuration for a DynamicMeter for some Redis metrics, we
might read and write "Dynamic(redis)". This identifier is
subsequently matched (back) up to the configuration file so
we're able to re-create arbitrary user configurations.
The PCP platform configuration file format is fairly simple.
We expand configs from several directories, including the
users homedir alongside htoprc (below htop/meters/) and also
/etc/pcp/htop/meters. The format will be described via a
new pcp-htop(5) man page, but its basically ini-style and
each Meter has one or more metric expressions associated, as
well as specifications for labels, color and so on via a dot
separated notation for individual metrics within the Meter.
A few initial sample configuration files are provided below
./pcp/meters that give the general idea. The PCP "derived"
metric specification - see pmRegisterDerived(3) - is used
as the syntax for specifying metrics in PCP DynamicMeters.
Refactor the sample time code to make one call to gettimeofday
(aka the realtime clock in clock_gettime, when available) and
one to the monotonic clock. Stores each in more appropriately
named ProcessList fields for ready access when needed. Every
platform gets the opportunity to provide their own clock code,
and the existing Mac OS X specific code is moved below darwin
instead of in Compat.
A couple of leftover time(2) calls are converted to use these
ProcessList fields as well, instead of yet again sampling the
system clock.
Related to https://github.com/htop-dev/htop/pull/574
The libcap code is Linux-specific so move it all below
the linux/ platform subdirectory. As this feature has
custom command-line long options I provide a mechanism
whereby each platform can add custom long options that
augment the main htop options. We'll make use this of
this with the pcp/ platform in due course to implement
the --host and --archive options there.
Related to https://github.com/htop-dev/htop/pull/536
Code that is shared across some (but not all) platforms
is moved into a 'generic' home. Makefile.am cleanups to
match plus some minor alphabetic reordering/formatting.
As discussed in https://github.com/htop-dev/htop/pull/553
Several of our newer meters have merged coding concerns in terms
of extracting values and displaying those values. This commit
rectifies that for the SysArch and Hostname meters, allowing use
of this code with alternative front/back ends. The SysArch code
is also refined to detect whether the platform has an os-release
file at all and/or the sys/utsname.h header via configure.ac.
On Linux kernels the size of the values exported for network
device bytes and packets has used a 64 bit integer for quite
some time (2.6+ IIRC). Make the procfs value extraction use
correct types and change internal types used to rate convert
these counters (within the NetworkIO Meter) 64 bit integers,
where appropriate.
Use only one enum instead of a global and a platform specific one.
Drop Platform_numberOfFields global variable.
Set known size of Process_fields array
Move platform-specific code out of the htop.c main function
and into the platform sub-directories - primarily this is
the Linux procfs path check and sensors setup/teardown; not
needed on any other platforms. No functional changes here.
The MIN, MAX, CLAMP, MINIMUM, and MAXIMUM macros appear
throughout the codebase with many re-definitions. Make
a single copy of each in a common header file, and use
the BSD variants of MINIMUM/MAXIMUM due to conflicts in
the system <sys/param.h> headers.
Reasoning:
- implementation was unsound -- broke down when I added a fairly
basic macro definition expanding to a struct initializer in a *.c
file.
- made it way too easy (e.g. via otherwise totally innocuous git
commands) to end up with timestamps such that it always ran
MakeHeader.py but never used its output, leading to overbuild noise
when running what should be a null 'make'.
- but mostly: it's just an awkward way of dealing with C code.
If no pools are imported (ARC size == 0) or the
ZFS module is not in the kernel (/proc/spl/kstat/zfs/arcstats
does not exist), then the Meter reports "Unavailable".
The pressure stall information (PSI) metrics provide useful information
on delays caused by waiting for CPU, IO and memory. Particularly on busy
servers it can provide a quick overview of what's "slowing things down".
This feature is supported on Linux >= 4.20.
The interface is documented here:
https://git.kernel.org/pub/scm/linux/kernel/git/torvalds/linux.git/plain/Documentation/accounting/psi.txt
These links provide rationale:
https://lwn.net/Articles/759781/https://facebookmicrosites.github.io/psi/
The following metrics are added, corresponding to the currently exposed
lines (see `head /proc/pressure/*`):
- PressureStallCPUSome
- PressureStallIOSome
- PressureStallIOFull
- PressureStallMemorySome
- PressureStallMemoryFull
The color scheme is the same as that used for Load Average, however I
gave it separate entries just in case someone wants to change them
specifically.
Tested on 4.20.7-arch1-1-ARCH, on the linux platform.
Also tested that other platforms still compile (changed configure to use
the unsupported platform).
Closes#879.
Specifically, Platform_signals[] and Platform_numberOfSignals. Both are
not supposed to be mutable. Marking them 'const' puts them into rodata
sections in binary. And for Platform_numberOfSignals, this aids
optimization (aids only Link Time Optimization for now). :)
Signed-off-by: Kang-Che Sung <explorer09@gmail.com>
With the CLAMP macro replacing the combination of MIN and MAX, we will
have at least two advantages:
1. It's more obvious semantically.
2. There are no more mixes of confusing uses like MIN(MAX(a,b),c) and
MAX(MIN(a,b),c) and MIN(a,MAX(b,c)) appearing everywhere. We unify
the 'clamping' with a single macro.
Note that the behavior of this CLAMP macro is different from
the combination `MAX(low,MIN(x,high))`.
* This CLAMP macro expands to two comparisons instead of three from
MAX and MIN combination. In theory, this makes the code slightly
smaller, in case that (low) or (high) or both are computed at
runtime, so that compilers cannot optimize them. (The third
comparison will matter if (low)>(high); see below.)
* CLAMP has a side effect, that if (low)>(high) it will produce weird
results. Unlike MIN & MAX which will force either (low) or (high) to
win. No assertion of ((low)<=(high)) is done in this macro, for now.
This CLAMP macro is implemented like described in glib
<http://developer.gnome.org/glib/stable/glib-Standard-Macros.html>
and does not handle weird uses like CLAMP(a++, low++, high--) .
Implementations for Linux (tested) and FreeBSD (still untested, thanks to @etosan for providing the table).
Darwin and OpenBSD(ping @mmcco) builds should be broken now, pending their own tables.
Daniel Lange
Christian Göttsche
Hisham Muhammad
Volodymyr Vasiutyk
Nathan Scott
Sohaib Mohamed
Benny Baumann
mayurdahibhate
Murloc Knight
Zev Weiss
Ross Williams
Ran Benita
Explorer09
Michael Klein