Skip to Content.
Sympa Menu

perfsonar-user - [perfsonar-user] perfsonar bwctl and owamp unidirectional test failure

Subject: perfSONAR User Q&A and Other Discussion

List archive

[perfsonar-user] perfsonar bwctl and owamp unidirectional test failure


Chronological Thread 
  • From: "Uhl, George D. (GSFC-423.0)[SGT INC]" <>
  • To: Aaron Brown <>
  • Cc: perfsonar-user <>
  • Subject: [perfsonar-user] perfsonar bwctl and owamp unidirectional test failure
  • Date: Wed, 17 Sep 2014 16:49:08 +0000
  • Accept-language: en-US

Aaron,

The files are attached. FYI, this system is running toolkit 3.3.2 and
runs locally scheduled tests as needed and also participates in a mesh.

Thanks,
George

On 9/16/14 9:05 PM, "Aaron Brown"
<>
wrote:

>Hey George,
>
>That¹s just the perfsonarbuoy owamp testing service restarting. Can you
>send the output of ³ps auxw² on the host, along with owmesh.conf, and
>owamp_bwctl.log?
>
>Cheers,
>Aaron
>
>On Sep 15, 2014, at 3:15 PM, Uhl, George D. (GSFC-423.0)[SGT INC]
><>
> wrote:
>
>>
>> I'm getting unidirectional test failures in my mesh for bwctl/iperf and
>> owamp. I'm seeing the following entries in my owamp_bwctl.log. Are
>>these
>> error messages indicating a cause of failure?
>>
>> Sep 15 14:31:56 larc-pth bwmaster.pl[1679]: Socket closed to server! at
>> /opt/perfsonar_ps/perfsonarbuoy_ma/bin/bwmaster.pl line 874.#012
>> Sep 15 14:31:56 larc-pth bwmaster.pl[1679]: Server socket unwritable: :
>> Closing at /opt/perfsonar_ps/perfsonarbuoy_ma/bin/bwmaster.pl line
>>739.#012
>> Sep 15 14:46:58 larc-pth powmaster.pl[9428]: Server socket unwritable:
>>:
>> Closing at /opt/perfsonar_ps/perfsonarbuoy_ma/bin/powmaster.pl line
>> 764.#012
>> Sep 15 14:46:58 larc-pth powmaster.pl[9428]: txfr() failed to transfer
>> information to archive.eos.nasa.gov at
>> /opt/perfsonar_ps/perfsonarbuoy_ma/bin/powmaster.pl line 1091.#012
>> Sep 15 14:47:58 larc-pth powmaster.pl[1694]: send_data died, restarting!
>> at /opt/perfsonar_ps/perfsonarbuoy_ma/bin/powmaster.pl line 707.#012
>> Sep 15 14:47:58 larc-pth powmaster.pl[1694]: Handling SIGHUP... Stop
>> processing...#012
>> Sep 15 14:47:59 larc-pth powstream[9429]: SIGTERM/SIGINT Caught:
>>Exiting.
>> Sep 15 14:47:59 larc-pth powmaster.pl[1694]: Restarting...:
>> /opt/perfsonar_ps/perfsonarbuoy_ma/bin/powmaster.pl -c
>> /opt/perfsonar_ps/perfsonarbuoy_ma/etc -Z at
>> /opt/perfsonar_ps/perfsonarbuoy_ma/bin/powmaster.pl line 730.#012
>> Sep 15 14:47:59 larc-pth powmaster.pl[1694]: Opened syslog
>>
>>
>>
>> Thanks,
>> George
>>
>

Attachment: owamp_bwctl.log
Description: owamp_bwctl.log

Attachment: owmesh.conf
Description: owmesh.conf

#
USER PID %CPU %MEM VSZ RSS TTY STAT START TIME COMMAND
root 1 0.0 0.0 19364 1548 ? Ss 09:43 0:00 /sbin/init
root 2 0.0 0.0 0 0 ? S 09:43 0:00 [kthreadd]
root 3 0.0 0.0 0 0 ? S 09:43 0:00 [migration/0]
root 4 0.0 0.0 0 0 ? S 09:43 0:00 [ksoftirqd/0]
root 5 0.0 0.0 0 0 ? S 09:43 0:00 [migration/0]
root 6 0.0 0.0 0 0 ? S 09:43 0:00 [watchdog/0]
root 7 0.0 0.0 0 0 ? S 09:43 0:00 [migration/1]
root 8 0.0 0.0 0 0 ? S 09:43 0:00 [migration/1]
root 9 0.0 0.0 0 0 ? S 09:43 0:00 [ksoftirqd/1]
root 10 0.0 0.0 0 0 ? S 09:43 0:00 [watchdog/1]
root 11 0.0 0.0 0 0 ? S 09:43 0:00 [migration/2]
root 12 0.0 0.0 0 0 ? S 09:43 0:00 [migration/2]
root 13 0.0 0.0 0 0 ? S 09:43 0:00 [ksoftirqd/2]
root 14 0.0 0.0 0 0 ? S 09:43 0:00 [watchdog/2]
root 15 0.0 0.0 0 0 ? S 09:43 0:00 [migration/3]
root 16 0.0 0.0 0 0 ? S 09:43 0:00 [migration/3]
root 17 0.0 0.0 0 0 ? S 09:43 0:00 [ksoftirqd/3]
root 18 0.0 0.0 0 0 ? S 09:43 0:00 [watchdog/3]
root 19 0.0 0.0 0 0 ? S 09:43 0:00 [events/0]
root 20 0.0 0.0 0 0 ? S 09:43 0:00 [events/1]
root 21 0.0 0.0 0 0 ? S 09:43 0:00 [events/2]
root 22 0.0 0.0 0 0 ? S 09:43 0:00 [events/3]
root 23 0.0 0.0 0 0 ? S 09:43 0:00 [cgroup]
root 24 0.0 0.0 0 0 ? S 09:43 0:00 [khelper]
root 25 0.0 0.0 0 0 ? S 09:43 0:00 [netns]
root 26 0.0 0.0 0 0 ? S 09:43 0:00 [async/mgr]
root 27 0.0 0.0 0 0 ? S 09:43 0:00 [pm]
root 28 0.0 0.0 0 0 ? S 09:43 0:00 [sync_supers]
root 29 0.0 0.0 0 0 ? S 09:43 0:00 [bdi-default]
root 30 0.0 0.0 0 0 ? S 09:43 0:00
[kintegrityd/0]
root 31 0.0 0.0 0 0 ? S 09:43 0:00
[kintegrityd/1]
root 32 0.0 0.0 0 0 ? S 09:43 0:00
[kintegrityd/2]
root 33 0.0 0.0 0 0 ? S 09:43 0:00
[kintegrityd/3]
root 34 0.0 0.0 0 0 ? S 09:43 0:00 [kblockd/0]
root 35 0.0 0.0 0 0 ? S 09:43 0:00 [kblockd/1]
root 36 0.0 0.0 0 0 ? S 09:43 0:00 [kblockd/2]
root 37 0.0 0.0 0 0 ? S 09:43 0:00 [kblockd/3]
root 38 0.0 0.0 0 0 ? S 09:43 0:00 [kacpid]
root 39 0.0 0.0 0 0 ? S 09:43 0:00
[kacpi_notify]
root 40 0.0 0.0 0 0 ? S 09:43 0:00
[kacpi_hotplug]
root 41 0.0 0.0 0 0 ? S 09:43 0:00 [ata_aux]
root 42 0.0 0.0 0 0 ? S 09:43 0:00 [ata_sff/0]
root 43 0.0 0.0 0 0 ? S 09:43 0:00 [ata_sff/1]
root 44 0.0 0.0 0 0 ? S 09:43 0:00 [ata_sff/2]
root 45 0.0 0.0 0 0 ? S 09:43 0:00 [ata_sff/3]
root 46 0.0 0.0 0 0 ? S 09:43 0:00
[ksuspend_usbd]
root 47 0.0 0.0 0 0 ? S 09:43 0:00 [khubd]
root 48 0.0 0.0 0 0 ? S 09:43 0:00 [kseriod]
root 49 0.0 0.0 0 0 ? S 09:43 0:00 [md/0]
root 50 0.0 0.0 0 0 ? S 09:43 0:00 [md/1]
root 51 0.0 0.0 0 0 ? S 09:43 0:00 [md/2]
root 52 0.0 0.0 0 0 ? S 09:43 0:00 [md/3]
root 53 0.0 0.0 0 0 ? S 09:43 0:00 [md_misc/0]
root 54 0.0 0.0 0 0 ? S 09:43 0:00 [md_misc/1]
root 55 0.0 0.0 0 0 ? S 09:43 0:00 [md_misc/2]
root 56 0.0 0.0 0 0 ? S 09:43 0:00 [md_misc/3]
root 57 0.0 0.0 0 0 ? S 09:43 0:00 [linkwatch]
root 58 0.0 0.0 0 0 ? S 09:43 0:00 [khungtaskd]
root 59 0.0 0.0 0 0 ? S 09:43 0:00 [kswapd0]
root 60 0.0 0.0 0 0 ? SN 09:43 0:00 [ksmd]
root 61 0.0 0.0 0 0 ? SN 09:43 0:00 [khugepaged]
root 62 0.0 0.0 0 0 ? S 09:43 0:00 [aio/0]
root 63 0.0 0.0 0 0 ? S 09:43 0:00 [aio/1]
root 64 0.0 0.0 0 0 ? S 09:43 0:00 [aio/2]
root 65 0.0 0.0 0 0 ? S 09:43 0:00 [aio/3]
root 66 0.0 0.0 0 0 ? S 09:43 0:00 [crypto/0]
root 67 0.0 0.0 0 0 ? S 09:43 0:00 [crypto/1]
root 68 0.0 0.0 0 0 ? S 09:43 0:00 [crypto/2]
root 69 0.0 0.0 0 0 ? S 09:43 0:00 [crypto/3]
root 74 0.0 0.0 0 0 ? S 09:43 0:00 [kthrotld/0]
root 75 0.0 0.0 0 0 ? S 09:43 0:00 [kthrotld/1]
root 76 0.0 0.0 0 0 ? S 09:43 0:00 [kthrotld/2]
root 77 0.0 0.0 0 0 ? S 09:43 0:00 [kthrotld/3]
root 78 0.0 0.0 0 0 ? S 09:43 0:00 [pciehpd]
root 80 0.0 0.0 0 0 ? S 09:43 0:00 [kpsmoused]
root 81 0.0 0.0 0 0 ? S 09:43 0:00
[usbhid_resumer]
root 112 0.0 0.0 0 0 ? S 09:43 0:00 [kstriped]
root 193 0.0 0.0 0 0 ? S 09:43 0:00 [scsi_eh_0]
root 194 0.0 0.0 0 0 ? S 09:43 0:00 [scsi_eh_1]
root 206 0.0 0.0 0 0 ? S 09:43 0:00 [scsi_eh_2]
root 207 0.0 0.0 0 0 ? S 09:43 0:00
[vmw_pvscsi_wq_2]
root 337 0.0 0.0 0 0 ? S 09:43 0:00 [kdmflush]
root 339 0.0 0.0 0 0 ? S 09:43 0:00 [kdmflush]
root 356 0.0 0.0 0 0 ? S 09:43 0:00 [jbd2/dm-0-8]
root 357 0.0 0.0 0 0 ? S 09:43 0:00
[ext4-dio-unwrit]
root 446 0.0 0.0 10868 968 ? S<s 09:43 0:00 /sbin/udevd
-d
root 664 0.0 0.0 0 0 ? S 09:43 0:00 [vmmemctl]
root 800 0.0 0.0 10656 792 ? S< 09:43 0:00 /sbin/udevd
-d
root 801 0.0 0.0 10652 776 ? S< 09:43 0:00 /sbin/udevd
-d
root 846 0.0 0.0 0 0 ? S 09:43 0:00 [jbd2/sda1-8]
root 847 0.0 0.0 0 0 ? S 09:43 0:00
[ext4-dio-unwrit]
root 946 0.0 0.0 0 0 ? S 09:43 0:00 [kauditd]
root 999 0.0 0.0 0 0 ? S 09:43 0:00 [flush-253:0]
root 1234 0.0 0.0 27640 828 ? S<sl 09:43 0:00 auditd
root 1249 0.0 0.3 203524 31292 ? Ss 09:43 0:01
toolkit_config_daemon.pl
root 1259 0.0 0.0 6160 584 ? Ss 09:43 0:00
/sbin/portreserve
root 1266 0.0 0.0 249152 1660 ? Sl 09:43 0:00
/sbin/rsyslogd -i /var/run/syslogd.pid -c 5
rpc 1284 0.0 0.0 18976 732 ? Ss 09:43 0:00 rpcbind
dbus 1386 0.0 0.0 21404 956 ? Ss 09:43 0:00 dbus-daemon
--system
nscd 1413 0.0 0.0 898472 1528 ? Ssl 09:43 0:00
/usr/sbin/nscd
root 1435 0.0 0.0 64724 1216 ? Ss 09:43 0:00
/usr/sbin/sshd
ntp 1443 0.0 0.0 25252 1832 ? Ss 09:43 0:00 ntpd -u
ntp:ntp -p /var/run/ntpd.pid -g
bwctl 1460 0.0 0.0 9568 728 ? Ss 09:43 0:00
/usr/bin/bwctld -c /etc/bwctld -R /var/run
owamp 1486 0.0 0.0 7256 568 ? Ss 09:43 0:00
/usr/bin/owampd -c /etc/owampd -R /var/run
499 1492 0.0 0.3 197928 27552 ? Ss 09:43 0:00
traceroute_master.pl
499 1498 0.0 0.3 203500 30412 ? Ss 09:43 0:00
perfsonar-daemon.pl (1498)
499 1499 0.0 0.3 203500 30288 ? S 09:43 0:00
perfsonar-daemon.pl (1498) - Listener (8087)
499 1505 0.0 0.3 203788 29828 ? Ss 09:43 0:00
traceroute_scheduler.pl
root 1539 0.0 0.0 106312 1376 ? S 09:43 0:00 /bin/sh
/usr/bin/mysqld_safe --datadir=/var/lib/mysql
--socket=/var/lib/mysql/mysql.sock --pid-file=/var/run/mysqld/mysqld.pid
--basedir=/usr --user=mysql
mysql 1647 0.0 0.3 442084 31228 ? Sl 09:43 0:02
/usr/libexec/mysqld --basedir=/usr --datadir=/var/lib/mysql --user=mysql
--log-error=/var/log/mysqld.log --pid-file=/var/run/mysqld/mysqld.pid
--socket=/var
499 1671 0.0 0.1 164144 14140 ? Ss 09:43 0:00
/opt/perfsonar_ps/perfsonarbuoy_ma/bin/bwcollector.pl:master
499 1678 0.0 0.1 154260 12248 ? Ss 09:43 0:00
/opt/perfsonar_ps/perfsonarbuoy_ma/bin/bwmaster.pl:master
499 1679 0.0 0.1 154604 12600 ? S 09:43 0:00
/opt/perfsonar_ps/perfsonarbuoy_ma/bin/bwmaster.pl:send_data
499 1680 0.0 0.0 9556 1108 ? S 09:43 0:00
/usr/bin/bwctl -T iperf -e local5 -p -B larc-pth.eosdis.nasa.gov -q -I 7200
-t 30 -d
EOSDIS_NETWORK_MONITORING_MESH_BW1_1/EOSDIS_NETWORK_MONITORING_MESH_LAR
499 1687 0.0 0.1 164408 14368 ? Ss 09:43 0:00
/opt/perfsonar_ps/perfsonarbuoy_ma/bin/powcollector.pl:master
499 1694 0.0 0.1 154392 12404 ? Ss 09:43 0:00
/opt/perfsonar_ps/perfsonarbuoy_ma/bin/powmaster.pl:master
499 1695 0.0 0.1 155176 13100 ? S 09:43 0:04
/opt/perfsonar_ps/perfsonarbuoy_ma/bin/powmaster.pl:send_data
499 1696 0.0 0.0 7200 1288 ? S 09:43 0:00
/usr/bin/powstream -e local5 -p -S larc-pth.eosdis.nasa.gov -i 0.1 -c 108000
-d 9537001.IPV4/KNOPPIX/KNOPPIX -N 600 -L 10 -b 0.0001
larc-pth.eosdis.nasa.gov
499 1698 0.0 0.0 7200 1280 ? S 09:43 0:00
/usr/bin/powstream -e local5 -p -S larc-pth.eosdis.nasa.gov -i 0.1 -c 108000
-d 9537001.IPV4/KNOPPIX/PERFSONAR_EOS_NASA_GOV -N 600 -L 10 -b 0.0001
[perfsona
499 1700 0.0 0.0 7272 1380 ? S 09:43 0:01
/usr/bin/powstream -e local5 -p -S larc-pth.eosdis.nasa.gov -i 0.1 -c 108000
-t -d 9537001.IPV4/PERFSONAR_EOS_NASA_GOV/KNOPPIX -N 600 -L 10 -b 0.0001
[perfs
499 1702 0.0 0.0 7316 1436 ? S 09:43 0:00
/usr/bin/powstream -e local5 -p -S larc-pth.eosdis.nasa.gov -i 0.1 -c 18000
-d
EOSDIS_NETWORK_MONITORING_MESH_OWP1_1/EOSDIS_NETWORK_MONITORING_MESH_LARC_PTH
owamp 1705 0.0 0.0 7468 644 ? S 09:43 0:00
/usr/bin/owampd -c /etc/owampd -R /var/run
owamp 1713 0.0 0.0 7640 476 ? S 09:43 0:02
/usr/bin/owampd -c /etc/owampd -R /var/run
499 1714 0.0 0.0 7052 380 ? Ss 09:43 0:02
/usr/bin/powstream -e local5 -p -S larc-pth.eosdis.nasa.gov -i 0.1 -c 108000
-d 9537001.IPV4/KNOPPIX/KNOPPIX -N 600 -L 10 -b 0.0001
larc-pth.eosdis.nasa.gov
owamp 1716 0.0 0.0 7468 648 ? S 09:43 0:00
/usr/bin/owampd -c /etc/owampd -R /var/run
owamp 1743 0.0 0.0 7640 468 ? S 09:43 0:00
/usr/bin/owampd -c /etc/owampd -R /var/run
499 1744 0.0 0.0 7056 384 ? Ss 09:43 0:00
/usr/bin/powstream -e local5 -p -S larc-pth.eosdis.nasa.gov -i 0.1 -c 108000
-d 9537001.IPV4/KNOPPIX/KNOPPIX -N 600 -L 10 -b 0.0001
larc-pth.eosdis.nasa.gov
499 1745 0.0 0.0 7224 456 ? Ss 09:43 0:02
/usr/bin/powstream -e local5 -p -S larc-pth.eosdis.nasa.gov -i 0.1 -c 108000
-t -d 9537001.IPV4/PERFSONAR_EOS_NASA_GOV/KNOPPIX -N 600 -L 10 -b 0.0001
[perfs
499 1747 0.0 0.0 7052 372 ? Ss 09:43 0:00
/usr/bin/powstream -e local5 -p -S larc-pth.eosdis.nasa.gov -i 0.1 -c 108000
-d 9537001.IPV4/KNOPPIX/PERFSONAR_EOS_NASA_GOV -N 600 -L 10 -b 0.0001
[perfsona
499 1777 0.0 0.0 7264 464 ? Ss 09:43 0:00
/usr/bin/powstream -e local5 -p -S larc-pth.eosdis.nasa.gov -i 0.1 -c 108000
-t -d 9537001.IPV4/PERFSONAR_EOS_NASA_GOV/KNOPPIX -N 600 -L 10 -b 0.0001
[perfs
root 1786 0.0 0.0 80040 3328 ? Ss 09:43 0:00
/usr/libexec/postfix/master
499 1794 0.0 0.0 7056 384 ? Ss 09:43 0:00
/usr/bin/powstream -e local5 -p -S larc-pth.eosdis.nasa.gov -i 0.1 -c 108000
-d 9537001.IPV4/KNOPPIX/PERFSONAR_EOS_NASA_GOV -N 600 -L 10 -b 0.0001
[perfsona
postfix 1796 0.0 0.0 80092 3492 ? S 09:43 0:00 qmgr -l -t
fifo -u
root 1813 0.0 0.0 108428 664 ? Ss 09:43 0:00
/usr/sbin/abrtd
root 1823 0.0 0.1 303252 12720 ? Ss 09:43 0:00
/usr/sbin/httpd
root 1831 0.0 0.0 115408 1028 ? Ss 09:43 0:00 crond
apache 1838 0.0 0.0 303252 6988 ? S 09:43 0:00
/usr/sbin/httpd
apache 1839 0.0 0.0 303252 6984 ? S 09:43 0:00
/usr/sbin/httpd
apache 1840 0.0 0.0 303252 6984 ? S 09:43 0:00
/usr/sbin/httpd
apache 1841 0.0 0.0 303252 6984 ? S 09:43 0:00
/usr/sbin/httpd
apache 1842 0.0 0.0 303252 6984 ? S 09:43 0:00
/usr/sbin/httpd
apache 1843 0.0 0.0 303252 6984 ? S 09:43 0:00
/usr/sbin/httpd
apache 1844 0.0 0.0 303252 6984 ? S 09:43 0:00
/usr/sbin/httpd
apache 1845 0.0 0.0 303252 6984 ? S 09:43 0:00
/usr/sbin/httpd
root 1848 0.0 0.1 425296 11248 ? Sl 09:43 0:00
/usr/bin/python /usr/bin/fail2ban-server -b -s
/var/run/fail2ban/fail2ban.sock -p /var/run/fail2ban/fail2ban.pid -x
root 1876 0.0 0.0 19860 472 ? Ss 09:43 0:00 /usr/sbin/atd
root 1912 0.0 0.0 4064 552 tty1 Ss+ 09:43 0:00
/sbin/mingetty /dev/tty1
root 1914 0.0 0.0 4064 552 tty2 Ss+ 09:43 0:00
/sbin/mingetty /dev/tty2
root 1916 0.0 0.0 4064 552 tty3 Ss+ 09:43 0:00
/sbin/mingetty /dev/tty3
root 1918 0.0 0.0 4064 552 tty4 Ss+ 09:43 0:00
/sbin/mingetty /dev/tty4
root 1920 0.0 0.0 4064 552 tty5 Ss+ 09:43 0:00
/sbin/mingetty /dev/tty5
root 1922 0.0 0.0 4064 548 tty6 Ss+ 09:43 0:00
/sbin/mingetty /dev/tty6
root 1923 0.0 0.0 98924 3848 ? Ss 09:43 0:00 sshd: uhl
[priv]
uhl 1932 0.0 0.0 99068 1908 ? S 09:44 0:00 sshd:
uhl@pts/0

uhl 1933 0.0 0.0 106448 1692 pts/0 Ss 09:44 0:00 -bash
root 1960 0.0 0.0 164088 2752 pts/0 S 09:44 0:00 su -
root 1963 0.0 0.0 106580 1824 pts/0 S+ 09:44 0:00 -bash
499 2429 0.0 0.4 116448 34456 ? Ss 10:00 0:00
SimpleLSBootStrapClientDaemon.pl
agermain 2576 0.0 0.0 4060 196 ? S 10:00 0:00 bin/datasink
-w10000000
agermain 2578 0.0 0.0 4400 220 ? Ss 10:00 0:00
/usr/bin/nuttcp -S
postfix 5645 0.0 0.0 79908 3304 ? S 11:23 0:00 pickup -l -t
fifo -u
499 6820 0.0 0.0 7320 728 ? Ss 12:00 0:00
/usr/bin/powstream -e local5 -p -S larc-pth.eosdis.nasa.gov -i 0.1 -c 18000
-d
EOSDIS_NETWORK_MONITORING_MESH_OWP1_1/EOSDIS_NETWORK_MONITORING_MESH_LARC_PTH
499 7907 0.0 0.0 7316 712 ? Ss 12:30 0:00
/usr/bin/powstream -e local5 -p -S larc-pth.eosdis.nasa.gov -i 0.1 -c 18000
-d
EOSDIS_NETWORK_MONITORING_MESH_OWP1_1/EOSDIS_NETWORK_MONITORING_MESH_LARC_PTH
root 7967 0.0 0.0 98932 3844 ? Ss 12:32 0:00 sshd: uhl
[priv]
uhl 7980 0.0 0.0 98932 1756 ? S 12:33 0:00 sshd:
uhl@pts/1

uhl 7981 0.0 0.0 106448 1688 pts/1 Ss 12:33 0:00 -bash
root 8011 0.0 0.0 164088 2748 pts/1 S 12:33 0:00 su -
root 8014 0.0 0.0 106580 1776 pts/1 S 12:33 0:00 -bash
499 8116 0.0 0.0 103736 812 ? S 12:37 0:00 traceroute
-n -s larc-pth.eosdis.nasa.gov -w 10 -m 30 esdis-ps.eosdis.nasa.gov 40
root 8117 0.0 0.0 108352 1052 pts/1 R+ 12:37 0:00 ps auwx
#



Archive powered by MHonArc 2.6.16.

Top of Page