perfSONAR User Q&A and Other Discussion

[Tao Zhang <>]

Hi Mark,

After reinstalling the whole thing on the box (198.165.161.174), the situation is slightly better than before.

 

The pscheduler looks fine now.

# pscheduler ping 198.165.161.174

198.165.161.174: pScheduler is alive

 

#pscheduler ping localhost

localhost: pScheduler is alive

 

# pscheduler troubleshoot

Performing basic troubleshooting of localhost.

Checking for pScheduler on localhost... OK.

Idle test on localhost.... 9 seconds.... OK.

pScheduler appears to be functioning normally.

 

The thoughput result also looks fine, however, I need to wait about 3 minutes for the result

 

bwctl -T iperf3 -s 198.165.161.174 -c 205.189.32.28 -t 30 -i 1 -f m -O 3

bwctl: Using tool: iperf3

bwctl: 50 seconds until test results available

 

SENDER START

Connecting to host 205.189.32.28, port 5399

[ 16] local 198.165.161.174 port 52167 connected to 205.189.32.28 port 5399

[ ID] Interval           Transfer     Bitrate         Retr  Cwnd

[ 16]   0.00-1.00   sec   166 MBytes  1392 Mbits/sec    0   8.01 MBytes       (omitted)

[ 16]   1.00-2.00   sec   112 MBytes   944 Mbits/sec    0   8.03 MBytes       (omitted)

[ 16]   2.00-3.00   sec   111 MBytes   933 Mbits/sec    0   8.07 MBytes       (omitted)

[ 16]   0.00-1.00   sec   112 MBytes   944 Mbits/sec    0   8.08 MBytes

[ 16]   1.00-2.00   sec   112 MBytes   944 Mbits/sec    0   8.11 MBytes

[ 16]   2.00-3.00   sec   112 MBytes   944 Mbits/sec    0   8.15 MBytes

[ 16]   3.00-4.00   sec   111 MBytes   933 Mbits/sec    0   8.22 MBytes

[ 16]   4.00-5.00   sec   112 MBytes   944 Mbits/sec    0   11.1 MBytes

[ 16]   5.00-6.00   sec   112 MBytes   944 Mbits/sec    0   11.1 MBytes

[ 16]   6.00-7.00   sec   112 MBytes   944 Mbits/sec    0   11.1 MBytes

[ 16]   7.00-8.00   sec   111 MBytes   933 Mbits/sec    0   11.1 MBytes

[ 16]   8.00-9.00   sec   112 MBytes   944 Mbits/sec    0   11.1 MBytes

[ 16]   9.00-10.00  sec   112 MBytes   944 Mbits/sec    0   11.1 MBytes

[ 16]  10.00-11.00  sec   112 MBytes   944 Mbits/sec    0   11.1 MBytes

[ 16]  11.00-12.00  sec   111 MBytes   933 Mbits/sec    0   11.1 MBytes

[ 16]  12.00-13.00  sec   112 MBytes   944 Mbits/sec    0   11.1 MBytes

[ 16]  13.00-14.00  sec   112 MBytes   944 Mbits/sec    0   11.1 MBytes

[ 16]  14.00-15.00  sec   112 MBytes   944 Mbits/sec    0   11.1 MBytes

[ 16]  15.00-16.00  sec   111 MBytes   933 Mbits/sec    0   11.1 MBytes

[ 16]  16.00-17.00  sec   112 MBytes   944 Mbits/sec    0   11.1 MBytes

[ 16]  17.00-18.00  sec   112 MBytes   944 Mbits/sec    0   11.1 MBytes

[ 16]  18.00-19.00  sec   112 MBytes   944 Mbits/sec    0   11.1 MBytes

[ 16]  19.00-20.00  sec   112 MBytes   944 Mbits/sec    0   11.1 MBytes

[ 16]  20.00-21.00  sec   111 MBytes   933 Mbits/sec    0   11.1 MBytes

[ 16]  21.00-22.00  sec   112 MBytes   944 Mbits/sec    0   11.1 MBytes

[ 16]  22.00-23.00  sec   112 MBytes   944 Mbits/sec    0   11.1 MBytes

[ 16]  23.00-24.00  sec   112 MBytes   944 Mbits/sec    0   11.1 MBytes

[ 16]  24.00-25.00  sec   111 MBytes   933 Mbits/sec    0   11.1 MBytes

[ 16]  25.00-26.00  sec   112 MBytes   944 Mbits/sec    0   11.1 MBytes

[ 16]  26.00-27.00  sec   112 MBytes   944 Mbits/sec    0   11.1 MBytes

[ 16]  27.00-28.00  sec   112 MBytes   944 Mbits/sec    0   11.1 MBytes

[ 16]  28.00-29.00  sec   111 MBytes   933 Mbits/sec    0   11.1 MBytes

[ 16]  29.00-30.00  sec   112 MBytes   944 Mbits/sec    0   11.1 MBytes

- - - - - - - - - - - - - - - - - - - - - - - - -

[ ID] Interval           Transfer     Bitrate         Retr

[ 16]   0.00-30.00  sec  3.29 GBytes   941 Mbits/sec    0             sender

[ 16]   0.00-30.04  sec  3.29 GBytes   941 Mbits/sec                  receiver

iperf Done.

 

the owping test is more problematic.

1)       owping the box itself

 

owping 198.165.161.174

Approximately -232.8 seconds until results available

 

--- owping statistics from [pS-mun-stj.acorn-nl.ca]:9060 to [198.165.161.174]:8766 ---

SID:        c6a5a1aede6572f532ee99d18c5d5059

first:       2018-03-27T23:16:57.571

last:        2018-03-27T23:17:08.069

0 sent, 0 lost (0.000%), 0 duplicates

one-way delay min/median/max = nan/nan/nan ms, (err=0 ms)

one-way jitter = nan ms (P95-P50)

TTL not reported

no reordering

 

--- owping statistics from [198.165.161.174]:9175 to [pS-mun-stj.acorn-nl.ca]:8940 ---

SID:        c6a5a1aede6573403f4f930326c7be2a

first:       2018-03-27T23:16:57.512

last:        2018-03-27T23:17:06.940

0 sent, 0 lost (0.000%), 0 duplicates

one-way delay min/median/max = nan/nan/nan ms, (err=0 ms)

one-way jitter = nan ms (P95-P50)

TTL not reported

no reordering

 

2)       owping other box

owping 205.189.32.28

owping: FILE=protocol.c, LINE=1758, _OWPReadStartAck: Unable to read from socket.

owping: FILE=owping.c, LINE=200, Session Failed!

 

 

Tao

 

From: Tao Zhang <>
Date: Monday, March 26, 2018 at 12:06 PM
To: Mark Feit <>, Szymon Trocha <>
Cc: "" <>
Subject: Re: [perfsonar-user] pScheduler ping localhost failed

 

Hi Mark and Szymon,

Both “pscheduler troubleshoot” and “pscheduler ping localhost” commands timed out on the host.

pscheduler troubleshoot

Performing basic troubleshooting of localhost.

 

Checking for pScheduler on localhost... Failed.

  Request timed out

 

Tao

From: Mark Feit <>
Date: Monday, March 26, 2018 at 11:30 AM
To: Tao Zhang <>, David Szydloski <>
Cc: "" <>
Subject: Re: [perfsonar-user] pScheduler ping localhost failed

 

Tao Zhang writes:

 

2) Are you able to run "sudo pscheduler ping <host ip>" sucessfully? 

sudo pscheduler ping 198.165.161.174

198.165.161.174: Request timed out

 

I had a quick look at this from a host here at Internet2 and am seeing some interesting behavior.  The address pings successfully and the web server answers queries for the toolkit and endpoints that don’t exist (e.g., https://198.165.161.174/this-does-not-exist).  The connection times out when retrieving the URL that pscheduler ping does (https://198.165.161.174/pscheduler/), and the code behind that doesn’t produce anything more than a greeting.  Some remote poking around shows that many of the other endpoints involving GETs work and those involving POSTs do not.

 

If you can log onto that host, run “pscheduler troubleshoot”.   If it completes successfully, pScheduler on the host is fine and the problem is either in the host firewall or somewhere else in the network.

 

By the way, there’s no use sudo; the pScheduler CLI just makes HTTP requests and will work just fine when run as an ordinary user.

 

--Mark