perfSONAR User Q&A and Other Discussion

[Mark Feit <>]

Bhat, Preeti P. writes:

I have encountered an issue with one of the sites and would appreciate your assistance in resolving it. The BW node perfsonar02.cmsaf.mit.edu, running the PS Toolkit on bare metal, is currently not executing any throughput tests. As a result, I do not see throughput results on the Grafana dashboard and on central dashboard.

That system is executing throughput tests, but its clock appears to be unsynchronized and off by about 30 seconds:

$ pscheduler clock perfsonar02.cmsaf.mit.edu

clock      2025-08-15T22:31:47.602099+00:00 synchronized   localhost

clock      2025-08-15T18:32:17.138767-04:00 unsynchronized perfsonar02.cmsaf.mit.edu

difference PT29.536668S unsafe

The last throughput test I saw on the MIT host was to  losa-ps-tp.lhcone.es.net.  The run diagnostics I saw for that show that the test did start but the iperf3 client did not find a server waiting at the other end, probably because of the clock being so far off.   pScheduler will tolerate a clock difference of up to one second for multiple-participant tests like throughput.   The clock on ESnet's system is synced and showing the correct time.

How I went about troubleshooting this:

• Use the pscheduler schedule command to look back through the schedule and find a throughput process.
• Copy the run URL from the bottom of a throughput entry.
• Use pscheduler result --full RUN-URL to  dump out the result and any diagnostic information.
• In the case of the run I looked at, the iperf3 client complained about not being able to connect to the ESnet server with the connection having been refused.  This is usually because there's no iperf3 server running on the other end.
• The next thing I did was to check the time on the MIT host using the clock command shown above.   This showed the time difference.
• Usually, if it's a firewall problem, the connection will time out or the complaint will be no route to host.

Hope that helps.

--Mark