perfsonar-user - RE: [perfsonar-user] perfSONAR nodes are down due to Memory utilization issue

Subject: perfSONAR User Q&A and Other Discussion

List archive

RE: [perfsonar-user] perfSONAR nodes are down due to Memory utilization issue

From: Muhammad Tayyab <>
To: Antoine Delvaux <>
Cc: "" <>
Subject: RE: [perfsonar-user] perfSONAR nodes are down due to Memory utilization issue
Date: Sun, 19 May 2019 11:56:29 +0000

Hi Antoine,

Thank you for your help, after applying the recommended changes device become normal.

Now I'm deploying Maddash to centralized the tests and Maddash not synchronizing since 3 days. Please find the attached psconfig_KAUST.txt file.

cid:image001.png@01D50E51.1E1ACB70

Regards,

Tayyab

-----Original Message-----
From: Antoine Delvaux [mailto:]
Sent: Wednesday, May 15, 2019 1:43 PM
To: Muhammad Tayyab <>
Cc:
Subject: Re: [perfsonar-user] perfSONAR nodes are down due to Memory utilization issue

Hi Tayyab,

> Le 15 mai 2019 à 10:05, Muhammad Tayyab <> a écrit :

> [root@ps-pul-h4309b ~]# psconfig pscheduler-stats Agent Last Run Start

> Time: 2019/05/15 12:02:03 Agent Last Run End Time: 2019/05/15 12:22:49

> Agent Last Run Process ID (PID): 6935 Agent Last Run Log GUID:

> 1B571A4E-76F0-11E9-9DC0-AD1399FCCD0D

> Total tasks managed by agent: 72

> From include files: 72

> /etc/perfsonar/psconfig/pscheduler.d/toolkit-webui.json: 72

So this shows that no remote pSconfig file is loaded but that you still have locally defined tasks running. The toolkit-webui.json contains all the tests defined through the local webUI. If, on the other hand, the webUI at https://your.node.tld/toolkit/auth/admin/tests.cgi shows you an empty list, there is a problem.

From your email on the ASREN list it seems that the content of this file is coming from a previously defined mesh, probably from an upgraded 4.0 perfSONAR installation. This part of the file tells it:

"_meta" : {

"psconfig-translation" : {

"source-format" : "mesh-config-tasks-conf",

"time-translated" : "2019-03-21T08:47:15+00:00"

}

It might be that this interferes and somehow confuses perfSONAR.

To forecibly remove this task list, you can delete the /etc/perfsonar/psconfig/pscheduler.d/toolkit-webui.json file and wait at most 60 seconds. Look at /var/log/perfsonar/psconfig-pscheduler-agent.log and wait until you first get a message stating "Running agent..." and then, after a while and some other messages, a message stating "Agent completed running".

It can take some time (multiple minutes, even up to 20 or 30 mins if the host is heavily loaded like yours) to complete. Once the message is printed, do a `psconfig pscheduler-stats` again to see if the list of tasks managed by the agent is now empty.

The load and the running tests should then hopefully start to decrease. If that doesn't happen, maybe because of load and an unresponsive pScheduler API, wait 24h and then all tasks coming from the webUI should expire by themselves.

Let us know how it goes,

Regards,

Antoine.

ï»¿ 14:23:01(271):[root@ps-pul-h4309b ~]# ps aux
14:23:01(288):USER PID %CPU %MEM VSZ RSS TTY STAT START
TIME COMMAND
14:23:01(297):root 1 0.0 2.9 549620 227632 ? Ss May15
0:28 /usr/lib/systemd/systemd --system --deserialize 16
14:23:01(302):root 2 0.0 0.0 0 0 ? S May15
0:00 [kthreadd]
14:23:01(308):root 3 0.0 0.0 0 0 ? S May15
0:15 [ksoftirqd/0]
14:23:01(314):root 5 0.0 0.0 0 0 ? S< May15
0:00 [kworker/0:0H]
14:23:01(319):root 7 0.0 0.0 0 0 ? S May15
0:01 [migration/0]
14:23:01(323):root 8 0.0 0.0 0 0 ? S May15
0:00 [rcu_bh]
14:23:01(328):root 9 0.1 0.0 0 0 ? S May15
7:45 [rcu_sched]
14:23:01(333):root 10 0.0 0.0 0 0 ? S< May15
0:00 [lru-add-drain]
14:23:01(337):root 11 0.0 0.0 0 0 ? S May15
0:01 [watchdog/0]
14:23:01(342):root 12 0.0 0.0 0 0 ? S May15
0:01 [watchdog/1]
14:23:01(353):root 13 0.0 0.0 0 0 ? S May15
0:01 [migration/1]
14:23:01(358):root 14 0.0 0.0 0 0 ? S May15
0:10 [ksoftirqd/1]
14:23:01(362):root 16 0.0 0.0 0 0 ? S< May15
0:00 [kworker/1:0H]
14:23:01(367):root 17 0.0 0.0 0 0 ? S May15
0:01 [watchdog/2]
14:23:01(372):root 18 0.0 0.0 0 0 ? S May15
0:01 [migration/2]
14:23:01(376):root 19 0.0 0.0 0 0 ? S May15
0:36 [ksoftirqd/2]
14:23:01(381):root 21 0.0 0.0 0 0 ? S< May15
0:00 [kworker/2:0H]
14:23:01(385):root 22 0.0 0.0 0 0 ? S May15
0:01 [watchdog/3]
14:23:01(411):root 23 0.0 0.0 0 0 ? S May15
0:01 [migration/3]
14:23:01(416):root 24 0.0 0.0 0 0 ? S May15
0:16 [ksoftirqd/3]
14:23:01(421):root 26 0.0 0.0 0 0 ? S< May15
0:00 [kworker/3:0H]
14:23:01(426):root 28 0.0 0.0 0 0 ? S May15
0:00 [kdevtmpfs]
14:23:01(431):root 29 0.0 0.0 0 0 ? S< May15
0:00 [netns]
14:23:01(437):root 30 0.0 0.0 0 0 ? S May15
0:00 [khungtaskd]
14:23:01(441):root 31 0.0 0.0 0 0 ? S< May15
0:00 [writeback]
14:23:01(447):root 32 0.0 0.0 0 0 ? S< May15
0:00 [kintegrityd]
14:23:01(452):root 33 0.0 0.0 0 0 ? S< May15
0:00 [bioset]
14:23:01(457):root 34 0.0 0.0 0 0 ? S< May15
0:00 [bioset]
14:23:01(462):root 35 0.0 0.0 0 0 ? S< May15
0:00 [bioset]
14:23:01(466):root 36 0.0 0.0 0 0 ? S< May15
0:00 [kblockd]
14:23:01(471):root 37 0.0 0.0 0 0 ? S< May15
0:00 [md]
14:23:01(487):root 38 0.0 0.0 0 0 ? S< May15
0:00 [edac-poller]
14:23:01(492):root 39 0.0 0.0 0 0 ? S< May15
0:00 [watchdogd]
14:23:01(496):root 45 0.2 0.0 0 0 ? S May15
14:43 [kswapd0]
14:23:01(500):root 46 0.0 0.0 0 0 ? SN May15
0:00 [ksmd]
14:23:01(505):root 47 0.0 0.0 0 0 ? SN May15
0:00 [khugepaged]
14:23:01(509):root 48 0.0 0.0 0 0 ? S< May15
0:00 [crypto]
14:23:01(513):root 57 0.0 0.0 0 0 ? S< May15
0:00 [kthrotld]
14:23:01(518):root 60 0.0 0.0 0 0 ? S< May15
0:00 [kmpath_rdacd]
14:23:01(522):root 61 0.0 0.0 0 0 ? S< May15
0:00 [kaluad]
14:23:01(527):root 65 0.0 0.0 0 0 ? S< May15
0:00 [kpsmoused]
14:23:01(531):root 66 0.0 0.0 0 0 ? S< May15
0:00 [ipv6_addrconf]
14:23:01(536):root 79 0.0 0.0 0 0 ? S< May15
0:00 [deferwq]
14:23:01(540):root 116 0.0 0.0 0 0 ? S May15
0:00 [kauditd]
14:23:01(544):root 752 0.0 0.0 0 0 ? S< May15
0:00 [ata_sff]
14:23:01(549):root 877 0.0 0.0 0 0 ? S May15
0:00 [scsi_eh_0]
14:23:01(554):root 884 0.0 0.0 0 0 ? S< May15
0:00 [scsi_tmf_0]
14:23:01(558):root 894 0.0 0.0 0 0 ? S May15
0:00 [scsi_eh_1]
14:23:01(563):root 900 0.0 0.0 0 0 ? S< May15
0:00 [scsi_tmf_1]
14:23:01(568):root 912 0.0 0.0 0 0 ? S May15
0:00 [scsi_eh_2]
14:23:01(573):root 917 0.0 0.0 0 0 ? S< May15
0:00 [scsi_tmf_2]
14:23:01(577):root 918 0.0 0.0 0 0 ? S May15
0:00 [scsi_eh_3]
14:23:01(582):root 919 0.0 0.0 0 0 ? S< May15
0:00 [scsi_tmf_3]
14:23:01(587):root 923 0.0 0.0 0 0 ? S May15
0:00 [scsi_eh_4]
14:23:01(591):root 930 0.0 0.0 0 0 ? S< May15
0:00 [scsi_tmf_4]
14:23:01(597):root 936 0.0 0.0 0 0 ? S May15
0:00 [scsi_eh_5]
14:23:01(601):root 941 0.0 0.0 0 0 ? S< May15
0:00 [scsi_tmf_5]
14:23:01(605):root 1504 0.0 0.0 0 0 ? S< May15
0:00 [ttm_swap]
14:23:01(613):postgres 1856 0.0 1.4 366596 110728 ? Ss May17
0:01 postgres: pscheduler pscheduler [local] idle
14:23:01(620):postgres 1857 0.0 1.5 364608 123212 ? Ss May17
0:01 postgres: pscheduler pscheduler [local] idle
14:23:01(625):root 2116 0.0 0.0 0 0 ? S< May15
0:00 [kdmflush]
14:23:01(629):root 2117 0.0 0.0 0 0 ? S< May15
0:00 [bioset]
14:23:01(633):root 2132 0.0 0.0 0 0 ? S< May15
0:00 [kdmflush]
14:23:01(638):root 2135 0.0 0.0 0 0 ? S< May15
0:00 [bioset]
14:23:01(642):root 2152 0.0 0.0 0 0 ? S< May15
0:00 [bioset]
14:23:01(647):root 2155 0.0 0.0 0 0 ? S< May15
0:00 [xfsalloc]
14:23:01(651):root 2160 0.0 0.0 0 0 ? S< May15
0:00 [xfs_mru_cache]
14:23:01(656):root 2165 0.0 0.0 0 0 ? S< May15
0:00 [xfs-buf/dm-0]
14:23:01(661):root 2170 0.0 0.0 0 0 ? S< May15
0:00 [xfs-data/dm-0]
14:23:01(666):root 2173 0.0 0.0 0 0 ? S< May15
0:00 [xfs-conv/dm-0]
14:23:01(671):root 2174 0.0 0.0 0 0 ? S< May15
0:00 [xfs-cil/dm-0]
14:23:01(676):root 2175 0.0 0.0 0 0 ? S< May15
0:00 [xfs-reclaim/dm-]
14:23:01(681):root 2176 0.0 0.0 0 0 ? S< May15
0:00 [xfs-log/dm-0]
14:23:01(685):root 2177 0.0 0.0 0 0 ? S< May15
0:00 [xfs-eofblocks/d]
14:23:01(690):root 2178 0.0 0.0 0 0 ? S May15
3:40 [xfsaild/dm-0]
14:23:01(694):root 2179 0.0 0.0 0 0 ? S< May15
0:03 [kworker/0:1H]
14:23:01(699):root 2180 0.0 0.0 0 0 ? S< May15
0:18 [kworker/3:1H]
14:23:01(705):root 2251 0.0 0.1 39188 8488 ? Ss May15
4:32 /usr/lib/systemd/systemd-journald
14:23:01(709):root 3219 0.0 0.0 0 0 ? SN May15
0:00 [kipmi0]
14:23:01(714):root 3528 0.0 0.0 0 0 ? S< May15
4:10 [kworker/2:1H]
14:23:01(720):root 3864 0.0 0.0 0 0 ? S< May15
0:00 [kvm-irqfd-clean]
14:23:01(724):root 4180 0.0 0.0 0 0 ? S< May15
0:10 [kworker/1:1H]
14:23:01(729):root 4190 0.0 0.0 0 0 ? S< May15
0:00 [xfs-buf/sda1]
14:23:01(734):root 4191 0.0 0.0 0 0 ? S< May15
0:00 [xfs-data/sda1]
14:23:01(739):root 4192 0.0 0.0 0 0 ? S< May15
0:00 [xfs-conv/sda1]
14:23:01(744):root 4193 0.0 0.0 0 0 ? S< May15
0:00 [xfs-cil/sda1]
14:23:01(749):root 4194 0.0 0.0 0 0 ? S< May15
0:00 [xfs-reclaim/sda]
14:23:01(753):root 4195 0.0 0.0 0 0 ? S< May15
0:00 [xfs-log/sda1]
14:23:01(759):root 4196 0.0 0.0 0 0 ? S< May15
0:00 [xfs-eofblocks/s]
14:23:01(764):root 4197 0.0 0.0 0 0 ? S May15
0:00 [xfsaild/sda1]
14:23:01(769):root 4204 0.0 0.0 0 0 ? S< May15
0:00 [kdmflush]
14:23:01(773):root 4206 0.0 0.0 0 0 ? S< May15
0:00 [bioset]
14:23:01(778):root 4224 0.0 0.0 0 0 ? S< May15
0:00 [xfs-buf/dm-2]
14:23:01(783):root 4225 0.0 0.0 0 0 ? S< May15
0:00 [xfs-data/dm-2]
14:23:01(787):root 4226 0.0 0.0 0 0 ? S< May15
0:00 [xfs-conv/dm-2]
14:23:01(792):root 4227 0.0 0.0 0 0 ? S< May15
0:00 [xfs-cil/dm-2]
14:23:01(797):root 4228 0.0 0.0 0 0 ? S< May15
0:00 [xfs-reclaim/dm-]
14:23:01(802):root 4229 0.0 0.0 0 0 ? S< May15
0:00 [xfs-log/dm-2]
14:23:01(807):root 4230 0.0 0.0 0 0 ? S< May15
0:00 [xfs-eofblocks/d]
14:23:01(812):root 4236 0.0 0.0 0 0 ? S May15
0:00 [xfsaild/dm-2]
14:23:01(816):root 4260 0.0 0.0 55520 492 ? S<sl May15
0:00 /sbin/auditd
14:23:01(821):root 4261 0.0 0.0 0 0 ? S< May15
0:00 [rpciod]
14:23:01(825):root 4262 0.0 0.0 0 0 ? S< May15
0:00 [xprtiod]
14:23:01(830):libstor+ 4290 0.0 0.0 6664 260 ? Ss May15
0:01 /usr/bin/lsmd -d
14:23:01(836):root 4294 0.0 0.0 21532 620 ? Ss May15
0:34 /usr/sbin/irqbalance --foreground
14:23:01(843):polkitd 4295 0.0 0.0 614876 1556 ? Ssl May15
0:01 /usr/lib/polkit-1/polkitd --no-debug
14:23:01(848):root 4313 0.0 0.0 195100 304 ? Ssl May15
0:00 /usr/sbin/gssproxy -D
14:23:01(853):root 4321 0.0 0.0 228252 600 ? Ss May15
0:00 /usr/sbin/abrtd -d -s
14:23:01(871):root 4323 0.0 0.0 225724 508 ? Ss May15
0:10 /usr/bin/abrt-watch-log -F BUG: WARNING: at WARNING: CPU: INFO: possible
recursive locking detected ernel BUG at list_del corruption
14:23:01(883):dbus 4325 0.0 0.0 60428 1252 ? Ss May15
0:16 /usr/bin/dbus-daemon --system --address=systemd: --nofork --nopidfile
--systemd-activation
14:23:01(887):root 4349 0.1 0.0 90500 424 ? Ss May15
6:16 /sbin/rngd -f
14:23:01(893):root 4350 0.0 0.0 24908 1072 ? Ss May15
0:06 /usr/lib/systemd/systemd-logind
14:23:01(901):root 4355 0.0 0.0 363288 2232 ? Ssl May15
0:03 /usr/bin/python -Es /usr/sbin/firewalld --nofork --nopid
14:23:01(906):root 4389 0.0 0.0 24204 316 ? Ss May15
0:00 /usr/sbin/atd -f
14:23:01(912):ntp 4392 0.0 0.0 24456 972 ? Ss May15
0:01 /usr/sbin/ntpd -u ntp:ntp -g
14:23:01(930):root 4845 0.0 0.0 105552 888 ? Ss May15
0:00 /sbin/dhclient -1 -q -lf
/var/lib/dhclient/dhclient-5e6ff194-e0e5-453d-9763-b76a53861662-eno1.lease
-pf /var/run/dhclient-eno1.pid e
14:23:01(935):root 4911 0.0 0.0 111376 516 ? Ss May15
0:00 /usr/sbin/sshd -D
14:23:01(943):root 4915 0.0 0.0 572008 1780 ? Ssl May15
0:38 /usr/bin/python2 -Es /usr/sbin/tuned -l -P
14:23:01(948):root 4921 0.0 0.1 511320 9144 ? Ssl May15
2:03 /usr/sbin/rsyslogd -n
14:23:01(958):memcach+ 4928 0.0 0.0 343212 996 ? Ssl May15
1:58 /usr/bin/memcached -u memcached -p 11211 -m 64 -c 1024 -l 127.0.0.1,::1
14:23:01(965):twamp 4932 0.0 0.0 8272 308 ? Ss May15
0:00 /usr/bin/twampd -c /etc/twamp-server -R /var/run
14:23:01(981):root 5001 0.0 0.0 547096 4152 ? Sl May15
1:55 /usr/bin/python2 -s /usr/bin/fail2ban-server -s
/var/run/fail2ban/fail2ban.sock -p /var/run/fail2ban/fail2ban.pid -x -b
14:23:01(989):postgres 5110 0.0 0.1 356152 9292 ? S May15
5:12 /usr/pgsql-9.5/bin/postgres -D /var/lib/pgsql/9.5/data
14:23:01(995):root 5270 0.0 0.2 304484 19436 ? Ss May15
1:35 toolkit_config_daemon.pl
14:23:02(000):perfson+ 5336 0.0 0.1 321804 14032 ? Ss May15
0:02 lsregistrationdaemon.pl
14:23:02(005):perfson+ 5356 0.0 0.3 267764 23632 ? Ss May15
0:01 lscachedaemon.pl
14:23:02(011):apache 5459 0.0 0.0 257116 5360 ? S 04:01
0:00 /usr/sbin/httpd -DFOREGROUND
14:23:02(016):postgres 5460 0.0 0.0 193320 592 ? Ss May15
0:00 postgres: logger process
14:23:02(022):root 5502 0.0 0.0 88376 524 ? Ss May15
0:01 /usr/libexec/postfix/master -w
14:23:02(027):postfix 5505 0.0 0.0 88440 596 ? S May15
0:00 qmgr -l -t unix -u
14:23:02(034):postgres 5518 0.0 1.7 356484 138180 ? Ss May15
0:41 postgres: checkpointer process
14:23:02(040):postgres 5519 0.0 1.7 356152 138340 ? Ss May15
2:54 postgres: writer process
14:23:02(047):postgres 5520 0.0 0.0 356152 4708 ? Ss May15
0:49 postgres: wal writer process
14:23:02(053):postgres 5521 0.0 0.9 356852 71364 ? Ss May15
0:10 postgres: autovacuum launcher process
14:23:02(059):postgres 5522 0.0 0.0 193464 840 ? Ss May15
3:01 postgres: stats collector process
14:23:02(077):pschedu+ 5564 5.0 0.2 2347072 22100 ? Ss May15
288:56 /usr/bin/python /usr/libexec/pscheduler/daemons/scheduler --daemon
--pid-file /run/pscheduler-scheduler.pid --dsn @/etc/pscheduler/d
14:23:02(094):pschedu+ 5578 0.5 0.3 2362308 23828 ? Ssl May15
33:49 /usr/bin/python /usr/libexec/pscheduler/daemons/runner --daemon
--pid-file /run/pscheduler-runner.pid --dsn @/etc/pscheduler/databas
14:23:02(112):pschedu+ 5580 0.0 0.0 349552 2428 ? Ssl May15
0:12 /usr/bin/python /usr/libexec/pscheduler/daemons/ticker --daemon
--pid-file /run/pscheduler-ticker.pid --dsn @/etc/pscheduler/databas
14:23:02(129):pschedu+ 5590 0.3 0.1 2344148 9628 ? Ss May15
19:32 /usr/bin/python /usr/libexec/pscheduler/daemons/archiver --daemon
--pid-file /run/pscheduler-archiver.pid --dsn @/etc/pscheduler/dat
14:23:02(134):perfson+ 5637 0.0 0.8 342392 63684 ? Ss May15
1:19 psconfig_pscheduler_agent
14:23:02(142):postgres 6687 16.2 1.8 364820 145780 ? Ss May15
933:21 postgres: pscheduler pscheduler [local] idle
14:23:02(150):postgres 6689 0.0 1.7 546488 136976 ? Ssl May15
5:01 postgres: pscheduler pscheduler [local] idle
14:23:02(158):postgres 6692 6.8 1.9 552780 149220 ? Ssl May15
392:16 postgres: pscheduler pscheduler [local] idle
14:23:02(165):postgres 6693 0.0 1.7 466336 137180 ? Ss May15
1:12 postgres: pscheduler pscheduler [local] idle
14:23:02(173):postgres 6837 2.0 1.8 363216 145660 ? Ss May15
119:31 postgres: pscheduler pscheduler [local] idle
14:23:02(180):postgres 6838 0.0 0.0 357372 1972 ? Ss May15
0:00 postgres: pscheduler pscheduler [local] idle
14:23:02(187):postgres 6843 0.2 1.8 362328 145572 ? Ss May15
16:36 postgres: pscheduler pscheduler [local] idle
14:23:02(194):owamp 7064 0.1 0.0 8292 720 ? Ss May15
9:48 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(200):root 7324 0.0 0.0 108192 404 tty1 Ss+ May15
0:00 /sbin/agetty --noclear tty1 linux
14:23:02(205):root 8804 0.0 0.0 0 0 ? S 12:20
0:00 [kworker/3:0]
14:23:02(209):root 9092 0.0 0.0 157896 844 ? Rs May15
0:01 sshd: root@pts/0
14:23:02(213):root 9114 0.0 0.0 114288 1624 pts/0 Ss May15
0:01 -bash
14:23:02(219):apache 9164 0.0 0.0 257116 5360 ? S 04:25
0:00 /usr/sbin/httpd -DFOREGROUND
14:23:02(224):root 9450 0.0 0.0 0 0 ? S May17
0:08 [kworker/0:1]
14:23:02(229):root 9804 0.0 0.0 0 0 ? S 00:40
0:00 [kworker/0:0]
14:23:02(234):root 10158 0.0 0.0 0 0 ? S 12:30
0:00 [kworker/2:2]
14:23:02(239):root 12953 0.0 0.0 0 0 ? S 12:52
0:00 [kworker/u8:0]
14:23:02(244):root 14651 0.0 0.0 124824 172 ? Ss May15
0:00 /usr/sbin/lvmetad -f
14:23:02(250):nscd 14988 0.0 0.0 1401592 1380 ? Ssl May15
0:04 /usr/sbin/nscd
14:23:02(255):postfix 17466 0.0 0.0 88268 3900 ? S 13:28
0:00 pickup -l -t unix -u
14:23:02(259):root 20348 0.0 0.0 0 0 ? S May18
0:12 [kworker/3:1]
14:23:02(264):root 20831 0.0 0.0 0 0 ? S May17
0:25 [kworker/2:0]
14:23:02(269):root 22291 0.0 0.0 0 0 ? S 14:06
0:00 [kworker/1:1]
14:23:02(273):root 23591 0.0 0.0 0 0 ? S 14:16
0:00 [kworker/1:2]
14:23:02(278):root 23644 0.0 0.0 0 0 ? S 06:04
0:05 [kworker/u8:2]
14:23:02(285):owamp 23952 0.0 0.0 8292 812 ? S 14:19
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(295):owamp 23954 0.0 0.0 8292 652 ? S 14:19
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(302):owamp 23968 0.0 0.0 8292 652 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(310):owamp 23974 0.0 0.0 8292 812 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(317):owamp 23976 0.0 0.0 8292 812 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(325):owamp 23978 0.0 0.0 8292 812 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(333):owamp 23984 0.0 0.0 8292 812 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(340):owamp 23986 0.0 0.0 8292 652 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(348):owamp 23991 0.0 0.0 8292 812 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(355):owamp 23993 0.0 0.0 8292 812 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(363):owamp 23996 0.0 0.0 8292 812 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(370):owamp 23998 0.0 0.0 8292 812 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(378):owamp 24001 0.0 0.0 8292 812 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(386):owamp 24004 0.0 0.0 8292 812 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(393):owamp 24006 0.0 0.0 8292 652 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(401):owamp 24008 0.0 0.0 8292 652 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(408):owamp 24011 0.0 0.0 8292 812 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(416):owamp 24015 0.0 0.0 8292 812 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(424):owamp 24018 0.0 0.0 8292 812 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(433):owamp 24020 0.0 0.0 8292 812 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(440):owamp 24023 0.0 0.0 8292 652 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(448):owamp 24025 0.0 0.0 8292 812 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(456):owamp 24028 0.0 0.0 8292 652 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(463):owamp 24030 0.0 0.0 8292 652 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(471):owamp 24032 0.0 0.0 8292 652 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(479):owamp 24035 0.0 0.0 8292 652 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(486):owamp 24038 0.0 0.0 8292 812 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(493):owamp 24040 0.0 0.0 8292 812 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(501):owamp 24042 0.0 0.0 8292 812 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(508):owamp 24045 0.0 0.0 8292 652 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(516):owamp 24048 0.0 0.0 8292 652 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(535):owamp 24050 0.0 0.0 8292 652 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(542):owamp 24052 0.0 0.0 8292 652 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(550):owamp 24054 0.0 0.0 8292 652 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(557):owamp 24056 0.0 0.0 8292 812 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(565):owamp 24058 0.0 0.0 8292 652 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(572):owamp 24063 0.0 0.0 8292 812 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(580):owamp 24065 0.0 0.0 8292 812 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(587):owamp 24067 0.0 0.0 8292 652 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(595):owamp 24069 0.0 0.0 8292 812 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(603):owamp 24073 0.0 0.0 8292 812 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(610):owamp 24074 0.0 0.0 8292 460 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(618):owamp 24075 0.0 0.0 8292 812 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(625):owamp 24078 0.0 0.0 8292 652 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(633):owamp 24079 0.0 0.0 8292 256 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(641):owamp 24080 0.0 0.0 8292 812 ? S 14:20
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(648):owamp 24083 0.0 0.0 8292 652 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(656):owamp 24087 0.0 0.0 8292 812 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(663):owamp 24092 0.0 0.0 8292 812 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(671):owamp 24093 0.0 0.0 8292 460 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(678):owamp 24094 0.0 0.0 8292 812 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(686):owamp 24095 0.0 0.0 8292 460 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(693):owamp 24096 0.0 0.0 8292 652 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(701):owamp 24097 0.0 0.0 8292 256 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(709):owamp 24099 0.0 0.0 8292 812 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(716):owamp 24100 0.0 0.0 8292 460 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(724):owamp 24102 0.0 0.0 8292 812 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(732):owamp 24103 0.0 0.0 8292 460 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(739):owamp 24104 0.0 0.0 8292 652 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(747):owamp 24105 0.0 0.0 8292 256 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(755):owamp 24107 0.0 0.0 8292 812 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(762):owamp 24109 0.0 0.0 8292 460 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(770):owamp 24111 0.0 0.0 8292 812 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(778):owamp 24114 0.0 0.0 8292 812 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(785):owamp 24115 0.0 0.0 8292 460 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(793):owamp 24116 0.0 0.0 8292 812 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(801):owamp 24117 0.0 0.0 8292 460 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(808):owamp 24119 0.0 0.0 8292 812 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(816):owamp 24120 0.0 0.0 8292 460 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(824):owamp 24122 0.0 0.0 8292 812 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(831):owamp 24123 0.0 0.0 8292 460 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(839):owamp 24125 0.0 0.0 8292 652 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(847):owamp 24127 0.0 0.0 8292 812 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(854):owamp 24128 0.0 0.0 8292 460 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(862):owamp 24131 0.0 0.0 8292 652 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(870):owamp 24132 0.0 0.0 8292 256 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(877):owamp 24133 0.0 0.0 8292 812 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(885):owamp 24134 0.0 0.0 8292 460 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(892):owamp 24135 0.0 0.0 8292 652 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(901):owamp 24136 0.0 0.0 8292 256 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(909):owamp 24139 0.0 0.0 8292 812 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(916):owamp 24140 0.0 0.0 8292 460 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(924):owamp 24141 0.0 0.0 8292 812 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(932):owamp 24142 0.0 0.0 8292 460 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(939):owamp 24143 0.0 0.0 8292 652 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(947):owamp 24144 0.0 0.0 8292 256 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(955):owamp 24145 0.0 0.0 8292 652 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(962):owamp 24146 0.0 0.0 8292 256 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(970):owamp 24147 0.0 0.0 8292 652 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(978):owamp 24148 0.0 0.0 8292 652 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(985):owamp 24149 0.0 0.0 8292 256 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:02(992):owamp 24150 0.0 0.0 8292 256 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(001):owamp 24151 0.0 0.0 8292 652 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(008):owamp 24152 0.0 0.0 8292 256 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(016):owamp 24154 0.0 0.0 8292 652 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(024):owamp 24155 0.0 0.0 8292 256 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(032):owamp 24157 0.0 0.0 8292 812 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(039):owamp 24158 0.0 0.0 8292 460 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(047):owamp 24159 0.0 0.0 8292 812 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(054):owamp 24160 0.0 0.0 8292 460 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(062):owamp 24161 0.0 0.0 8292 652 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(069):owamp 24162 0.0 0.0 8292 256 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(077):owamp 24164 0.0 0.0 8292 652 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(085):owamp 24165 0.0 0.0 8292 256 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(093):owamp 24167 0.0 0.0 8292 652 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(100):owamp 24168 0.0 0.0 8292 256 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(108):owamp 24169 0.0 0.0 8292 812 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(116):owamp 24170 0.0 0.0 8292 460 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(123):owamp 24172 0.0 0.0 8292 812 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(131):owamp 24173 0.0 0.0 8292 460 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(138):owamp 24175 0.0 0.0 8292 812 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(146):owamp 24176 0.0 0.0 8292 460 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(153):owamp 24177 0.0 0.0 8292 788 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(161):owamp 24178 0.0 0.0 8292 652 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(168):owamp 24179 0.0 0.0 8292 256 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(175):owamp 24180 0.0 0.0 8292 652 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(183):owamp 24181 0.0 0.0 8292 652 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(191):owamp 24182 0.0 0.0 8292 256 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(199):owamp 24183 0.0 0.0 8292 256 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(206):owamp 24185 0.0 0.0 8292 812 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(214):owamp 24186 0.0 0.0 8292 460 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(221):owamp 24187 0.0 0.0 8292 788 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(229):owamp 24188 0.0 0.0 8292 812 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(236):owamp 24189 0.0 0.0 8292 460 ? S 14:21
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(244):owamp 24192 0.0 0.0 8292 812 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(252):owamp 24193 0.0 0.0 8292 460 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(259):owamp 24196 0.0 0.0 8292 812 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(267):owamp 24197 0.0 0.0 8292 460 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(275):owamp 24198 0.0 0.0 8292 812 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(282):owamp 24199 0.0 0.0 8292 460 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(289):owamp 24200 0.0 0.0 8292 788 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(297):owamp 24203 0.0 0.0 8292 652 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(304):owamp 24204 0.0 0.0 8292 256 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(311):owamp 24206 0.0 0.0 8292 812 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(319):owamp 24207 0.0 0.0 8292 460 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(327):owamp 24208 0.0 0.0 8292 652 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(335):owamp 24209 0.0 0.0 8292 256 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(342):owamp 24210 0.0 0.0 8292 812 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(350):owamp 24211 0.0 0.0 8292 460 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(358):owamp 24212 0.0 0.0 8292 812 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(365):owamp 24213 0.0 0.0 8292 460 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(373):owamp 24214 0.0 0.0 8292 788 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(380):owamp 24220 0.0 0.0 8292 812 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(388):owamp 24221 0.0 0.0 8292 460 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(395):owamp 24222 0.0 0.0 8292 652 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(403):owamp 24223 0.0 0.0 8292 256 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(410):owamp 24224 0.0 0.0 8292 652 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(418):owamp 24225 0.0 0.0 8292 256 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(426):owamp 24226 0.0 0.0 8292 812 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(434):owamp 24227 0.0 0.0 8292 460 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(441):owamp 24228 0.0 0.0 8292 788 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(449):owamp 24229 0.0 0.0 8292 812 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(457):owamp 24230 0.0 0.0 8292 460 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(464):owamp 24231 0.0 0.0 8292 788 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(472):owamp 24233 0.0 0.0 8292 812 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(479):owamp 24234 0.0 0.0 8292 460 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(487):owamp 24237 0.0 0.0 8292 812 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(495):owamp 24238 0.0 0.0 8292 460 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(503):owamp 24239 0.0 0.0 8292 812 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(510):owamp 24240 0.0 0.0 8292 460 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(518):owamp 24241 0.0 0.0 8292 812 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(526):owamp 24242 0.0 0.0 8292 460 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(534):owamp 24243 0.0 0.0 8292 788 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(541):owamp 24246 0.0 0.0 8292 812 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(549):owamp 24247 0.0 0.0 8292 460 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(557):owamp 24248 0.0 0.0 8292 812 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(564):owamp 24249 0.0 0.0 8292 460 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(569):root 24251 0.0 0.0 0 0 ? S 14:22
0:00 [kworker/1:0]
14:23:03(577):owamp 24252 0.0 0.0 8292 812 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(584):owamp 24253 0.0 0.0 8292 460 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(592):owamp 24254 0.0 0.0 8292 652 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(599):owamp 24255 0.0 0.0 8292 256 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(607):owamp 24256 0.0 0.0 8292 652 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(614):owamp 24257 0.0 0.0 8292 256 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(622):owamp 24261 0.0 0.0 8292 812 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(630):owamp 24262 0.0 0.0 8292 460 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(638):owamp 24264 0.0 0.0 8292 812 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(645):owamp 24265 0.0 0.0 8292 460 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(653):owamp 24266 0.0 0.0 8292 812 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(660):owamp 24267 0.0 0.0 8292 460 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(668):owamp 24268 0.0 0.0 8292 652 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(675):owamp 24269 0.0 0.0 8292 256 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(683):owamp 24270 0.0 0.0 8292 652 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(691):owamp 24271 0.0 0.0 8292 256 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(699):owamp 24272 0.0 0.0 8292 652 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(706):owamp 24273 0.0 0.0 8292 256 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(714):owamp 24274 0.0 0.0 8292 652 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(722):owamp 24275 0.0 0.0 8292 256 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(729):owamp 24277 0.0 0.0 8292 812 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(738):owamp 24278 0.0 0.0 8292 460 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(745):owamp 24280 0.0 0.0 8292 812 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(753):owamp 24281 0.0 0.0 8292 460 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(760):owamp 24282 0.0 0.0 8292 788 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(768):owamp 24283 0.0 0.0 8292 652 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(775):owamp 24284 0.0 0.0 8292 256 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(783):owamp 24285 0.0 0.0 8292 812 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(791):owamp 24286 0.0 0.0 8292 460 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(798):owamp 24287 0.0 0.0 8292 652 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(806):owamp 24288 0.0 0.0 8292 256 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(813):owamp 24289 0.0 0.0 8292 652 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(821):owamp 24290 0.0 0.0 8292 256 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(828):owamp 24293 0.0 0.0 8292 652 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(836):owamp 24294 0.0 0.0 8292 256 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(843):owamp 24295 0.0 0.0 8292 652 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(851):owamp 24296 0.0 0.0 8292 256 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(859):owamp 24297 0.0 0.0 8292 812 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(866):owamp 24299 0.0 0.0 8292 460 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(874):owamp 24300 0.0 0.0 8292 652 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(882):owamp 24301 0.0 0.0 8292 256 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(889):owamp 24302 0.0 0.0 8292 812 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(897):owamp 24303 0.0 0.0 8292 460 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(904):owamp 24304 0.0 0.0 8292 652 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(911):owamp 24305 0.0 0.0 8292 256 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(919):owamp 24307 0.0 0.0 8292 652 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(927):owamp 24308 0.0 0.0 8292 256 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(934):owamp 24309 0.0 0.0 8292 812 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(942):owamp 24310 0.0 0.0 8292 460 ? S 14:22
0:00 /usr/bin/owampd -c /etc/owamp-server -R /var/run
14:23:03(946):root 24311 0.0 0.0 153448 1816 pts/0 R+ 14:23
0:00 ps aux
14:23:03(951):root 25019 0.0 0.0 124372 604 ? Ss May15
0:00 /usr/sbin/crond -n
14:23:03(957):root 25460 0.0 0.0 42588 408 ? Ss May15
0:00 /usr/lib/systemd/systemd-udevd
14:23:03(962):root 26549 0.0 0.0 256980 7724 ? Ss May15
0:49 /usr/sbin/httpd -DFOREGROUND
14:23:03(968):apache 30267 0.0 0.0 382072 6516 ? Sl 03:14
0:02 /usr/sbin/httpd -DFOREGROUND
14:23:03(974):apache 30268 0.0 0.0 447608 6500 ? Sl 03:14
0:02 /usr/sbin/httpd -DFOREGROUND
14:23:03(979):apache 30269 0.0 0.0 382072 6500 ? Sl 03:14
0:02 /usr/sbin/httpd -DFOREGROUND
14:23:03(985):apache 30270 0.0 0.0 382072 6500 ? Sl 03:14
0:02 /usr/sbin/httpd -DFOREGROUND
14:23:03(991):apache 30271 0.0 0.0 382072 6500 ? Sl 03:14
0:02 /usr/sbin/httpd -DFOREGROUND
14:23:03(997):apache 30272 0.0 0.0 382072 6500 ? Sl 03:14
0:02 /usr/sbin/httpd -DFOREGROUND
14:23:04(003):apache 30273 0.0 0.0 382072 6500 ? Sl 03:14
0:02 /usr/sbin/httpd -DFOREGROUND
14:23:04(008):apache 30274 0.0 0.0 382072 6500 ? Sl 03:14
0:02 /usr/sbin/httpd -DFOREGROUND
14:23:04(014):apache 30275 0.0 0.0 382072 6500 ? Sl 03:14
0:02 /usr/sbin/httpd -DFOREGROUND
14:23:04(020):apache 30276 0.0 0.0 382072 6500 ? Sl 03:14
0:02 /usr/sbin/httpd -DFOREGROUND
14:23:04(026):pschedu+ 30277 0.0 0.4 790364 33448 ? Sl 03:14
0:03 /usr/sbin/httpd -DFOREGROUND
14:23:04(033):pschedu+ 30278 0.0 0.4 790876 35916 ? Sl 03:14
0:03 /usr/sbin/httpd -DFOREGROUND
14:23:04(039):pschedu+ 30279 0.0 0.4 857436 37908 ? Sl 03:14
0:03 /usr/sbin/httpd -DFOREGROUND
14:23:04(045):pschedu+ 30280 0.0 0.4 856156 37780 ? Sl 03:14
0:03 /usr/sbin/httpd -DFOREGROUND
14:23:04(053):pschedu+ 30281 0.0 0.4 856924 37096 ? Sl 03:14
0:03 /usr/sbin/httpd -DFOREGROUND
14:23:04(059):pschedu+ 30282 0.0 0.4 856624 37060 ? Sl 03:14
0:03 /usr/sbin/httpd -DFOREGROUND
14:23:04(065):pschedu+ 30283 0.0 0.4 790364 34028 ? Sl 03:14
0:02 /usr/sbin/httpd -DFOREGROUND
14:23:04(070):pschedu+ 30284 0.0 0.4 855900 33280 ? Sl 03:14
0:02 /usr/sbin/httpd -DFOREGROUND
14:23:04(076):pschedu+ 30285 0.0 0.4 790364 33804 ? Sl 03:14
0:03 /usr/sbin/httpd -DFOREGROUND
14:23:04(082):pschedu+ 30286 0.0 0.4 856668 37064 ? Sl 03:14
0:03 /usr/sbin/httpd -DFOREGROUND
14:23:04(087):apache 30287 0.0 0.0 257116 5360 ? S 03:14
0:00 /usr/sbin/httpd -DFOREGROUND
14:23:04(093):apache 30288 0.0 0.0 257116 5280 ? S 03:14
0:00 /usr/sbin/httpd -DFOREGROUND
14:23:04(099):apache 30289 0.0 0.0 257116 5292 ? S 03:14
0:00 /usr/sbin/httpd -DFOREGROUND
14:23:04(105):apache 30290 0.0 0.0 257116 5360 ? S 03:14
0:00 /usr/sbin/httpd -DFOREGROUND
14:23:04(110):apache 30305 0.0 0.0 257116 5360 ? S 03:14
0:00 /usr/sbin/httpd -DFOREGROUND
14:23:04(117):apache 31021 0.0 0.0 257116 5360 ? S 03:17
0:00 /usr/sbin/httpd -DFOREGROUND
14:23:04(136):cassand+ 31555 2.9 18.2 3826820 1427348 ? Sl May15
170:14 java -ea -javaagent:/usr/share/cassandra/lib/jamm-0.2.5.jar
-XX:+CMSClassUnloadingEnabled -XX:+UseThreadPriorities -XX:ThreadPriorit
14:23:05(359):[root@ps-pul-h4309b ~]#
14:23:12(086):[root@ps-pul-h4309b ~]# free -m
14:23:12(098): total used free shared
buff/cache available
14:23:12(103):Mem: 7646 2489 693 156
4462 4652
14:23:12(111):Swap: 7807 1540 6267
14:23:13(537):[root@ps-pul-h4309b ~]# top
14:23:13(804):top - 14:23:13 up 3 days, 23:58, 1 user, load average: 0.16,
0.21, 0.22
14:23:13(831):Tasks: 413 total, 1 running, 412 sleeping, 0 stopped, 0
zombie
14:23:13(875):%Cpu(s): 2.7 us, 5.3 sy, 0.0 ni, 92.0 id, 0.0 wa, 0.0 hi,
0.0 si, 0.0 st
14:23:13(899):KiB Mem : 7829620 total, 709688 free, 2550488 used,
4569444 buff/cache
14:23:13(922):KiB Swap: 7995388 total, 6417636 free, 1577752 used.
4763560 avail Mem
14:23:13(922):
14:23:13(931): PID USER PR NI VIRT RES SHR S %CPU %MEM
TIME+ COMMAND

14:23:13(939):24330 root 20 0 160392 2576 1516 R 12.5 0.0
0:00.03 top

14:23:13(948): 1 root 20 0 549620 227652 1444 S 0.0 2.9
0:28.63 systemd

14:23:13(958): 2 root 20 0 0 0 0 S 0.0 0.0
0:00.43 kthreadd

14:23:13(968): 3 root 20 0 0 0 0 S 0.0 0.0
0:15.84 ksoftirqd/0

14:23:13(978): 5 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 kworker/0:0H

14:23:13(988): 7 root rt 0 0 0 0 S 0.0 0.0
0:01.08 migration/0

14:23:13(997): 8 root 20 0 0 0 0 S 0.0 0.0
0:00.00 rcu_bh

14:23:14(006): 9 root 20 0 0 0 0 S 0.0 0.0
7:45.98 rcu_sched

14:23:14(016): 10 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 lru-add-drain

14:23:14(025): 11 root rt 0 0 0 0 S 0.0 0.0
0:01.41 watchdog/0

14:23:14(034): 12 root rt 0 0 0 0 S 0.0 0.0
0:01.33 watchdog/1

14:23:14(045): 13 root rt 0 0 0 0 S 0.0 0.0
0:01.08 migration/1

14:23:14(055): 14 root 20 0 0 0 0 S 0.0 0.0
0:10.20 ksoftirqd/1

14:23:14(065): 16 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 kworker/1:0H

14:23:14(075): 17 root rt 0 0 0 0 S 0.0 0.0
0:01.23 watchdog/2

14:23:14(084): 18 root rt 0 0 0 0 S 0.0 0.0
0:01.35 migration/2

14:23:14(094): 19 root 20 0 0 0 0 S 0.0 0.0
0:36.38 ksoftirqd/2

14:23:14(104): 21 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 kworker/2:0H

14:23:14(114): 22 root rt 0 0 0 0 S 0.0 0.0
0:01.20 watchdog/3

14:23:14(123): 23 root rt 0 0 0 0 S 0.0 0.0
0:01.12 migration/3

14:23:14(133): 24 root 20 0 0 0 0 S 0.0 0.0
0:16.47 ksoftirqd/3

14:23:14(143): 26 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 kworker/3:0H

14:23:14(152): 28 root 20 0 0 0 0 S 0.0 0.0
0:00.00 kdevtmpfs

14:23:14(162): 29 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 netns

14:23:14(171): 30 root 20 0 0 0 0 S 0.0 0.0
0:00.54 khungtaskd

14:23:14(181): 31 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 writeback

14:23:14(191): 32 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 kintegrityd

14:23:14(200): 33 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 bioset

14:23:14(209): 34 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 bioset

14:23:14(218): 35 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 bioset

14:23:14(228): 36 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 kblockd

14:23:14(236): 37 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 md

14:23:14(246): 38 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 edac-poller

14:23:14(255): 39 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 watchdogd

14:23:14(265): 45 root 20 0 0 0 0 S 0.0 0.0
14:43.29 kswapd0

14:23:14(273): 46 root 25 5 0 0 0 S 0.0 0.0
0:00.00 ksmd

14:23:14(283): 47 root 39 19 0 0 0 S 0.0 0.0
0:00.90 khugepaged

14:23:14(292): 48 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 crypto

14:23:14(301): 57 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 kthrotld

14:23:14(311): 60 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 kmpath_rdacd

14:23:14(319): 61 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 kaluad

14:23:14(329): 65 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 kpsmoused

14:23:14(339): 66 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 ipv6_addrconf

14:23:14(348): 79 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 deferwq

14:23:14(357): 116 root 20 0 0 0 0 S 0.0 0.0
0:00.10 kauditd

14:23:14(367): 752 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 ata_sff

14:23:14(376): 877 root 20 0 0 0 0 S 0.0 0.0
0:00.00 scsi_eh_0

14:23:14(386): 884 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 scsi_tmf_0

14:23:14(395): 894 root 20 0 0 0 0 S 0.0 0.0
0:00.00 scsi_eh_1

14:23:14(405): 900 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 scsi_tmf_1

14:23:14(415): 912 root 20 0 0 0 0 S 0.0 0.0
0:00.00 scsi_eh_2

14:23:14(425): 917 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 scsi_tmf_2

14:23:14(434): 918 root 20 0 0 0 0 S 0.0 0.0
0:00.00 scsi_eh_3

14:23:14(444): 919 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 scsi_tmf_3

14:23:14(453): 923 root 20 0 0 0 0 S 0.0 0.0
0:00.00 scsi_eh_4

14:23:14(463): 930 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 scsi_tmf_4

14:23:14(473): 936 root 20 0 0 0 0 S 0.0 0.0
0:00.00 scsi_eh_5

14:23:14(483): 941 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 scsi_tmf_5

14:23:14(493): 1504 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 ttm_swap

14:23:14(502): 1856 postgres 20 0 366596 110728 105088 S 0.0 1.4
0:01.63 postgres

14:23:14(512): 1857 postgres 20 0 364608 123212 117188 S 0.0 1.6
0:01.88 postgres

14:23:14(521): 2116 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 kdmflush

14:23:14(531): 2117 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 bioset

14:23:14(539): 2132 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 kdmflush

14:23:14(549): 2135 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 bioset

14:23:14(558): 2152 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 bioset

14:23:14(567): 2155 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 xfsalloc

14:23:14(578): 2160 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 xfs_mru_cache

14:23:14(587): 2165 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 xfs-buf/dm-0

14:23:14(597): 2170 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 xfs-data/dm-0

14:23:14(607): 2173 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 xfs-conv/dm-0

14:23:16(763):top - 14:23:16 up 3 days, 23:58, 1 user, load average: 0.15,
0.21, 0.22
14:23:16(801):Tasks: 413 total, 2 running, 411 sleeping, 0 stopped, 0
zombie
14:23:16(848):%Cpu(s): 1.3 us, 2.8 sy, 0.1 ni, 95.6 id, 0.0 wa, 0.0 hi,
0.1 si, 0.0 st
14:23:16(874):KiB Mem : 7829620 total, 709972 free, 2550184 used,
4569464 buff/cache
14:23:16(899):KiB Swap: 7995388 total, 6417636 free, 1577752 used.
4763864 avail Mem
14:23:16(900):
14:23:16(900): PID USER PR NI VIRT RES SHR S %CPU %MEM
TIME+ COMMAND

14:23:16(909): 6692 postgres 20 0 552780 149220 143932 R 10.9 1.9
392:18.91 postgres

14:23:16(920):31555 cassand+ 20 0 3826820 1.4g 4616 S 1.0 18.2
170:14.61 java

14:23:16(929): 9 root 20 0 0 0 0 S 0.3 0.0
7:45.99 rcu_sched

14:23:16(940): 4325 dbus 20 0 60428 1252 924 S 0.3 0.0
0:16.42 dbus-daemon

14:23:16(950): 5519 postgres 20 0 356152 138340 138048 S 0.3 1.8
2:54.89 postgres

14:23:16(959): 7064 owamp 20 0 8292 720 488 S 0.3 0.0
9:48.79 owampd

14:23:16(968): 9092 root 20 0 157896 844 708 S 0.3 0.0
0:01.02 sshd

14:23:16(977):24136 owamp 20 0 8292 256 0 S 0.3 0.0
0:00.04 owampd

14:23:16(986):24142 owamp 20 0 8292 460 204 S 0.3 0.0
0:00.04 owampd

14:23:16(995):24149 owamp 20 0 8292 256 0 S 0.3 0.0
0:00.04 owampd

14:23:17(004):24152 owamp 20 0 8292 256 0 S 0.3 0.0
0:00.03 owampd

14:23:17(013):24155 owamp 20 0 8292 256 0 S 0.3 0.0
0:00.03 owampd

14:23:17(022):24158 owamp 20 0 8292 460 204 S 0.3 0.0
0:00.03 owampd

14:23:17(031):24165 owamp 20 0 8292 256 0 S 0.3 0.0
0:00.03 owampd

14:23:17(039):24183 owamp 20 0 8292 256 0 S 0.3 0.0
0:00.02 owampd

14:23:17(048):24193 owamp 20 0 8292 460 204 S 0.3 0.0
0:00.02 owampd

14:23:17(063):24199 owamp 20 0 8292 460 204 S 0.3 0.0
0:00.03 owampd

14:23:17(071):24213 owamp 20 0 8292 460 204 S 0.3 0.0
0:00.04 owampd

14:23:17(081):24221 owamp 20 0 8292 460 204 S 0.3 0.0
0:00.01 owampd

14:23:17(089):24223 owamp 20 0 8292 256 0 S 0.3 0.0
0:00.01 owampd

14:23:17(098):24227 owamp 20 0 8292 460 204 S 0.3 0.0
0:00.03 owampd

14:23:17(107):24330 root 20 0 160392 2636 1556 R 0.3 0.0
0:00.04 top

14:23:17(116): 1 root 20 0 549752 227652 1444 S 0.0 2.9
0:28.63 systemd

14:23:17(125): 2 root 20 0 0 0 0 S 0.0 0.0
0:00.43 kthreadd

14:23:17(134): 3 root 20 0 0 0 0 S 0.0 0.0
0:15.84 ksoftirqd/0

14:23:17(144): 5 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 kworker/0:0H

14:23:17(154): 7 root rt 0 0 0 0 S 0.0 0.0
0:01.08 migration/0

14:23:17(163): 8 root 20 0 0 0 0 S 0.0 0.0
0:00.00 rcu_bh

14:23:17(173): 10 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 lru-add-drain

14:23:17(182): 11 root rt 0 0 0 0 S 0.0 0.0
0:01.41 watchdog/0

14:23:17(192): 12 root rt 0 0 0 0 S 0.0 0.0
0:01.33 watchdog/1

14:23:17(201): 13 root rt 0 0 0 0 S 0.0 0.0
0:01.08 migration/1

14:23:17(212): 14 root 20 0 0 0 0 S 0.0 0.0
0:10.20 ksoftirqd/1

14:23:17(221): 16 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 kworker/1:0H

14:23:17(231): 17 root rt 0 0 0 0 S 0.0 0.0
0:01.23 watchdog/2

14:23:17(240): 18 root rt 0 0 0 0 S 0.0 0.0
0:01.35 migration/2

14:23:17(250): 19 root 20 0 0 0 0 S 0.0 0.0
0:36.38 ksoftirqd/2

14:23:17(260): 21 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 kworker/2:0H

14:23:17(269): 22 root rt 0 0 0 0 S 0.0 0.0
0:01.20 watchdog/3

14:23:17(279): 23 root rt 0 0 0 0 S 0.0 0.0
0:01.12 migration/3

14:23:17(288): 24 root 20 0 0 0 0 S 0.0 0.0
0:16.47 ksoftirqd/3

14:23:17(298): 26 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 kworker/3:0H

14:23:17(307): 28 root 20 0 0 0 0 S 0.0 0.0
0:00.00 kdevtmpfs

14:23:17(316): 29 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 netns

14:23:17(325): 30 root 20 0 0 0 0 S 0.0 0.0
0:00.54 khungtaskd

14:23:17(335): 31 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 writeback

14:23:17(345): 32 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 kintegrityd

14:23:17(353): 33 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 bioset

14:23:17(362): 34 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 bioset

14:23:17(371): 35 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 bioset

14:23:17(380): 36 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 kblockd

14:23:17(389): 37 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 md

14:23:17(399): 38 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 edac-poller

14:23:17(408): 39 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 watchdogd

14:23:17(417): 45 root 20 0 0 0 0 S 0.0 0.0
14:43.29 kswapd0

14:23:17(425): 46 root 25 5 0 0 0 S 0.0 0.0
0:00.00 ksmd

14:23:17(435): 47 root 39 19 0 0 0 S 0.0 0.0
0:00.90 khugepaged

14:23:17(444): 48 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 crypto

14:23:17(453): 57 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 kthrotld

14:23:17(463): 60 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 kmpath_rdacd

14:23:17(471): 61 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 kaluad

14:23:17(480): 65 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 kpsmoused

14:23:17(490): 66 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 ipv6_addrconf

14:23:17(500): 79 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 deferwq

14:23:17(508): 116 root 20 0 0 0 0 S 0.0 0.0
0:00.10 kauditd

14:23:17(518): 752 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 ata_sff

14:23:17(527): 877 root 20 0 0 0 0 S 0.0 0.0
0:00.00 scsi_eh_0

14:23:17(537): 884 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 scsi_tmf_0

14:23:17(546): 894 root 20 0 0 0 0 S 0.0 0.0
0:00.00 scsi_eh_1

14:23:17(556): 900 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 scsi_tmf_1

14:23:17(565): 912 root 20 0 0 0 0 S 0.0 0.0
0:00.00 scsi_eh_2

14:23:17(578): 917 root 0 -20 0 0 0 S 0.0 0.0
0:00.00 scsi_tmf_2

14:23:17(580):[1]+ Stopped top
14:23:17(957):[root@ps-pul-h4309b ~]# ^C
14:23:18(807):[root@ps-pul-h4309b ~]#
{
"_meta": {
"display-name": "KAUST Maddash Dashboard"
},

"addresses": {

"ps-rac1-g1755.kaust.edu.sa": {
"address": "ps-rac1-g1755.kaust.edu.sa",
"host": "ps-rac1-g1755.kaust.edu.sa",
"_meta": {
"display-name": "Racquet Club 01",
"organization": "KAUST"
}
},
"ps-isc2-g2263.kaust.edu.sa": {
"address": "ps-isc2-g2263.kaust.edu.sa",
"host": "ps-isc2-g2263.kaust.edu.sa",
"_meta": {
"display-name": "International School Center
02",
"organization": "KAUST"
}
},
"ps-rac2-g1631.kaust.edu.sa": {
"address": "ps-rac2-g1631.kaust.edu.sa",
"host": "ps-rac2-g1631.kaust.edu.sa",
"_meta": {
"display-name": "Racquet Club 02",
"organization": "KAUST"
}
},
"ps-isc1-g1133.kaust.edu.sa": {
"address": "ps-isc1-g1133.kaust.edu.sa",
"host": "ps-isc1-g1133.kaust.edu.sa",
"_meta": {
"display-name": "International School Center
01",
"organization": "KAUST"
}
},
"ps-ec3-g3622.kaust.edu.sa": {
"address": "ps-ec3-g3622.kaust.edu.sa",
"host": "ps-ec3-g3622.kaust.edu.sa",
"_meta": {
"display-name": "Early Childhood Center 03",
"organization": "KAUST"
}
}

},

"groups": {

"infrastructure": {
"type": "mesh",
"addresses": [
{ "name": "ps-rac1-g1755.kaust.edu.sa" },
{ "name": "ps-isc2-g2263.kaust.edu.sa" },
{ "name": "ps-rac2-g1631.kaust.edu.sa" },
{ "name": "ps-isc1-g1133.kaust.edu.sa" },
{ "name": "ps-ec3-g3622.kaust.edu.sa" }
]
},
"1ge-throughput": {
"type": "disjoint",
"a-addresses": [
{ "name": "ps-rac1-g1755.kaust.edu.sa" },
{ "name": "ps-isc2-g2263.kaust.edu.sa" }
],
"b-addresses": [
{ "name": "ps-rac2-g1631.kaust.edu.sa" },
{ "name": "ps-isc1-g1133.kaust.edu.sa" },
{ "name": "ps-ec3-g3622.kaust.edu.sa" }
]
}
},

"tests": {

"throughput_iperf3_tcp_test_v4": {
"type": "throughput",
"spec": {
"source": "{% address[0] %}",
"dest": "{% address[1] %}",
"duration": "PT20S",
"ip-version": 4
}
},
"latencybg_test_v4": {
"type": "latencybg",
"spec": {
"source": "{% address[0] %}",
"dest": "{% address[1] %}",
"packet-timeout": 10,
"packet-interval": 0.1,
"packet-count": 600,
"packet-padding": 0,
"bucket-width": 0.001,
"ip-version": 4
}
},
"trace_test_v4": {
"type": "trace",
"spec": {
"source": "{% address[0] %}",
"dest": "{% address[1] %}",
"ip-version": 4
}
},
"rtt_test_v4": {
"type": "rtt",
"spec": {
"source": "{% address[0] %}",
"dest": "{% address[1] %}",
"count": 10,
"interval": "PT20S",
"length": 1000,
"ip-version": 4
}
}
},

"schedules": {
"frequently": {
"repeat": "PT10M",
"slip": "PT5M",
"sliprand": false
},

"sparsely": {
"repeat": "PT30M",
"slip": "PT15M",
"sliprand": false
},

"hourly": {
"repeat": "PT1H",
"slip": "PT30M",
"sliprand": false
},

"bigly": {
"repeat": "PT4H",
"slip": "PT2H",
"sliprand": false
}
},

"tasks": {
"infrastructure-rtt": {
"group": "infrastructure",
"test": "rtt_test_v4",
"schedule": "frequently",
"archives": ["10.126.74.203"],
"_meta": {
"display-name": "Infrastructure - rtt"
}
},
"latency": {
"group": "infrastructure",
"test": "latencybg_test_v4",
"archives": ["10.126.74.203"],
"_meta": {
"display-name": "Infrastructure - Latency"
}
},
"1ge-throughput": {
"group": "1ge-throughput",
"test": "throughput_iperf3_tcp_test_v4",
"schedule": "bigly",
"archives": ["10.126.74.203"],
"_meta": {
"display-name": "1GE Throughput - iperf3"
}
}

},

"archives": {
"10.126.74.203": {
"archiver": "esmond",
"data": {
"measurement-agent": "{% scheduled_by_address %}",
"url":
"http://10.126.74.203/esmond/perfsonar/archive/";
}
}
}

}

RE: [perfsonar-user] perfSONAR nodes are down due to Memory utilization issue, (continued)

List archive

RE: [perfsonar-user] perfSONAR nodes are down due to Memory utilization issue