# NAME Linux::GetPidstat - Monitor each process metrics avg using each pidfile # SYNOPSIS use Linux::GetPidstat; my $stat = Linux::GetPidstat->new; $stat->run(%opt); # DESCRIPTION Run `pidstat -w -s -u -d -r` commands in parallel to monitor each process metrics avg/1min. Output to a specified file \[and|or\] `mackerel service` https://mackerel.io. ## Motivation A batch server runs many batch scripts at the same time. When this server suffers by a resource short, it's difficult to grasp which processes are heavy quickly. Running pidstat manually is not appropriate in this situation, because - the target processes are changed by starting each job. - the target processes may run child processes recursively. ## Requirements pidstat pstree ## Usage Prepare pid files in a specified directory. $ mkdir /tmp/pid_dir $ echo 1234 > /tmp/pid_dir/target_script $ echo 1235 > /tmp/pid_dir/target_script2 # In production, this file is made and removed by the batch script itself for instance. Run the script every 1 mininute. # vi /etc/cron.d/linux-get-pidstat * * * * * user carton exec -- linux-get-pidstat --dry_run=0 --pid_dir=/tmp/pid_dir --res_dir=/tmp/bstat.log Done, you can monitor the result. $ tail -f /tmp/bstat.log # start(datetime),start(epoch),pidfilename,name,value 2016-04-02T19:49:32,1459594172,target_script,cswch_per_sec,19.87 2016-04-02T19:49:32,1459594172,target_script,stk_ref,25500 2016-04-02T19:49:32,1459594172,target_script,memory_percent,34.63 2016-04-02T19:49:32,1459594172,target_script,memory_rss,10881534000 2016-04-02T19:49:32,1459594172,target_script,stk_size,128500 2016-04-02T19:49:32,1459594172,target_script,nvcswch_per_sec,30.45 2016-04-02T19:49:32,1459594172,target_script,cpu,21.2 2016-04-02T19:49:32,1459594172,target_script,disk_write_per_sec,0 2016-04-02T19:49:32,1459594172,target_script,disk_read_per_sec,0 2016-04-02T19:49:32,1459594172,target_script2,memory_rss,65289204000 2016-04-02T19:49:32,1459594172,target_script2,memory_percent,207.78 2016-04-02T19:49:32,1459594172,target_script2,stk_ref,153000 2016-04-02T19:49:32,1459594172,target_script2,cswch_per_sec,119.22 2016-04-02T19:49:32,1459594172,target_script2,nvcswch_per_sec,182.7 2016-04-02T19:49:32,1459594172,target_script2,cpu,127.2 2016-04-02T19:49:32,1459594172,target_script2,disk_read_per_sec,0 2016-04-02T19:49:32,1459594172,target_script2,disk_write_per_sec,0 2016-04-02T19:49:32,1459594172,target_script2,stk_size,771000 ### Mackerel Post the results to service metrics. $ carton exec -- linux-get-pidstat \ --dry_run=0 \ --pid_dir=/tmp/pid_dir \ --mackerel_api_key=yourkey \ --mackerel_service_name=yourservice ### Help Display how to use. $ carton exec -- linux-get-pidstat --help Usage: linux-get-pidstat - command description Usage: command [options] Options: --pid_dir A directory path for pid files --res_file A file path to be stored results --interval Interval second to be given as a pidstat argument (default:60) --dry_run Dry run mode. not run the side-effects operation (default:1) --datetime Datetime (ex. '2016-06-10 00:00:00') to be recorded --include_child Flag to be enabled to include child process metrics (default:1) --max_child_limit Number to be used for limiting pidstat multi processes (default:30) --mackerel_api_key An api key to be used for posting to mackerel --mackerel_service_name An mackerel service name Requirement Programs: pidstat and pstree commands # LICENSE Copyright (C) yoheimuta. This library is free software; you can redistribute it and/or modify it under the same terms as Perl itself. # AUTHOR yoheimuta