perfSONAR User Q&A and Other Discussion

["Garnizov, Ivan" <>]

Hello Michael,

IMO, also, your case appears similar to the issue discussed in August (the 
link you refer to)
Please tell about the versions of pS on both sides, or at least make sure 
both run 4.4.2 and above. 

Your comment about the killing the rogue iperf3 process does not hold water. 
Since iperf3 tool is started and stopped with each pscheduler throughput run.
If on your side iperf3 operation is different, then we need to investigate 
this.

Of course killing iperf3 during the test also won't help.

Regards,
Ivan Garnizov

GEANT WP6T3: pS development team
GEANT WP7T1: pS deployments GN Operations
GEANT WP9T2: Software governance in GEANT




-----Original Message-----
From:  
[] On Behalf Of Michael Thomas
Sent: Monday, March 7, 2022 10:40 PM
To: 
Subject: [perfsonar-user] throughput test timeouts

I recently set up a perfsonar server and configured it to run throughput 
tests with two other sites in our group.  The iperf3 tests from my site to 
the remote locations works fine, but all incoming tests fail, with the 
following results in my pscheduler logs:

Mar  7 12:19:48 perfsonar.ligo-la.caltech.edu journal: runner INFO
12832: Running
https://perfsonar-gc.ligo.caltech.edu/pscheduler/tasks/274d650f-166f-40bd-babe-c5189e16fda9/runs/a68ec259-b011-48a1-93a6-3be2879293cf
Mar  7 12:19:48 perfsonar.ligo-la.caltech.edu journal: runner INFO
12832: With iperf3: throughput --source perfsonar-gc.ligo.caltech.edu 
--source-node perfsonar-gc.ligo.caltech.edu --dest 208.69.128.92 --dest-node 
208.69.128.92 --duration PT20S --ip-version 4 Mar  7 12:20:21 
perfsonar.ligo-la.caltech.edu journal: runner INFO
12832: Run failed 2: Process took too long to run.

Testing from the CLI shows an authorization failure:

Error from perfsonar.ligo-wa.caltech.edu:
   iperf3 output has neither 'sum_sent' nor 'sum' field, and one of them is 
required

Diagnostics from perfsonar.ligo-wa.caltech.edu:
   /usr/bin/iperf3 -p 5201 -B 198.129.208.245 -c 208.69.128.92 -t 10 --json 
--rsa -public-key-path 
/var/run/pscheduler-server/runner/tmp/tmp_j91baqu/tmpumiughjb/public-key
--username <username redacted>

   {
         "start":        {
                 "connected":    [],
                 "version":      "iperf 3.11",
                 "system_info":  "Linux perfsonar 4.9.0-17-amd64 #1 SMP 
Debian 4.
9.290-1 (2021-12-12) x86_64",
                 "timestamp":    {
                         "time": "Mon, 07 Mar 2022 21:01:06 GMT",
                         "timesecs":     1646686866
                 },
                 "connecting_to":        {
                         "host": "208.69.128.92",
                         "port": 5201
                 },
                 "cookie":       "<cookie redacted>",
                 "tcp_mss_default":      8948,
                 "target_bitrate":       0
         },
         "intervals":    [],
         "end":  {
         },
         "error":        "test authorization failed"
   }

How can I further debug this authorization failure?

I will also note that this looks suspiciously like the same issue described 
in a previous post, but killing the rogue iperf3 process does not allow 
subsequent tests to work.

https://lists.internet2.edu/sympa/arc/perfsonar-user/2021-08/msg00011.html

--Mike