Adrián Bíro

Performance Monitoring

Usefull commands

dmesg --human --follow --ctime 

iostat -xmdzh 1 # -p sda
# lsblk -l | awk '!(/loop/||/NAME/){print $1}'

CPU statistics and input/output statistics for devices and partitions
-x     Display extended statistics.
-m     Display statistics in megabytes per second.
-d     Display the device utilization report.
-z     Omit output for any devices for which there was no activity during the sample period.
-h     --human

Meminfo to MB

awk '{printf("%-20s %.2f MB\n", $1, ($2/1024))}' /proc/meminfo | sed 's/\.00//'

mpstat -P ALL 1

Statistics for threads associated with selected tasks

pidstat -t 1

Report statistics for Linux tasks.
-t     Also display statistics for threads associated with selected tasks.
Report I/O statistics (kernels 2.6.20 and later only)
pidstat -d 1

pmap -x # pgrep <prog>

report memory map of a process
-x, --extended
less -S /proc/stat

ps

ps -ef f
ps -eo user,sz,rss,minflt,majflt,pcpu,args

cpu

ps -eo pcpu,pid,args | sort -k 1,1 -r 

rss

ps -eo user,rss,pid,args | sort -r -n -k 2,2

threads

ps -eo nlwp,pid,args | sort -nr -k 1,1

Report a snapshot of the current processes.
-e     Select all processes.
-f     Do full-format listing.  This option can be combined with many other UNIX-style options to add additional columns.  It also causes
              the command arguments to be printed.  When used with -L, the NLWP (number of threads) and LWP (thread ID) columns will be added.
              See the c option, the format keyword args, and the format keyword comm.
f      ASCII art process hierarchy (forest). --forest
-o format
NLWP      number of lwps (threads) in the process.  (alias thcount).

sort 
-k, --key=POS1[,POS2] start a key at POS1 (origin 1), end it at POS2 (default end of line)
-n, --numeric-sort
-r, --reverse

RSS
https://www.baeldung.com/linux/resident-set-vs-virtual-memory-size
This is a measure of how much memory a process is consuming in our physical RAM, to load all of its pages after its execution.
This includes memory allocated from shared libraries, given they are still present in memory. Also, it includes all heap and stack memory.
RSS is not an accurate measure of the total memory processes are consuming, because it does not include memory consumed by libraries 
that were swapped out. On the other hand, the same shared libraries may be duplicated and counted in different processes. 
However, RSS is a reliable estimate.

sar

sar -n DEV 1

Collect, report, or save system activity information.
-n { keyword[,...] | ALL }
    Report network statistics.
    With  the DEV keyword, statistics from the network devices are reported.

Sort slabinfo to see what objects taking the most space slabinfo(5) linux OOM

awk '{printf "%-30s %5d MB\n", $1 ,$3*$4/(1024*1024)}' < /proc/slabinfo | sort -nr -k2

smem

smem -s swap -r | awk 'NR==1{print; next};int($4) > 0'

Report memory usage with shared memory divided proportionally.
-s SORT, --sort=SORT
-r, --reverse
AWK print header and non-zero swap usage

Top one program show individual threads

top -b -d 1 -p "$(pgrep "${1}" | head -1)" -H

-b  Batch-mode operation
-d  Delay-time interval between screen updates
-p  Monitor-PIDs
-H  Threads-mode operation
    Instructs  top  to  display  individual  threads. Without this command-line option a summation of all threads in each
    process is shown.  Later this can be changed with the `H' interactive command.

vmstat

vmstat  -Sm 1

Report virtual memory statistics
-S, --unit character
    Switches outputs between 1000 (k), 1024 (K), 1000000 (m), or 1048576 (M) bytes.  
    Note this does not change the block (bi/bo) fields, which are always measured in blocks.
-m, --slabs
    Displays slabinfo.

Core freq cpuinfo flags

awk '
BEGIN { FS = ":" }

$1 ~ /^processor[[:space:]]+$/ { Processor = $2 }

$1 ~ /^cpu MHz[[:space:]]+$/ { Cores[Processor] = $2 }

END { for (Processor in Cores) { printf("CORE_%d: %d\n", Processor + 1, Cores[Processor]) }
}' /proc/cpuinfo

Cpu temperature

find /sys/class/thermal/thermal_zone*/ -maxdepth 0 -print0 | while IFS= read -r -d '' dir; do
    echo "$(cat "${dir}"/type): $(sed 's/\(.\)..$/.\1°C/' "${dir}/temp")"
done