perfSONAR User Q&A and Other Discussion

[Raul Lopes <>]

Hi Mark,

The 443 ports between those hosts are all open. For example, I do:

 - stop all perfsonar services;

 - stop httpd;

 - start iperf on port 443.

It shows that ps01-ban is open and I get 7+Gbps from ps03-ban, when I connect to ps01-ban on 443.

[root@ps03 perfsonar]# iperf -B ps03-ban -c ps01-ban -p 443                                                                                                                

------------------------------------------------------------

Client connecting to ps01-ban, TCP port 443

Binding to local address ps03-ban

TCP window size:  170 KByte (default)

------------------------------------------------------------

[  3] local 130.246.152.210 port 33572 connected with 130.246.152.194 port 443

[ ID] Interval       Transfer     Bandwidth

[  3]  0.0-10.0 sec  7.93 GBytes  6.81 Gbits/sec

iperf from ps01-ban tp ps03-ban on port 443.

 

[root@ps01 perfsonar]# iperf -B ps01-ban -c ps03-ban -p 443

------------------------------------------------------------

Client connecting to ps03-ban, TCP port 443

Binding to local address ps01-ban

TCP window size: 2.56 MByte (default)

------------------------------------------------------------

[  3] local 130.246.152.194 port 37502 connected with 130.246.152.210 port 443

[ ID] Interval       Transfer     Bandwidth

[  3]  0.0-10.0 sec  9.65 GBytes  8.29 Gbits/sec

Same for hosts Perfsonar hosts I am using. My reading is that Perfsonar troubleshooting is not enough.

I have also been running tests with lead-bind and posting results to this list and to Szymon. Tests requested by Szymon. Szymon's conclusion seemed to be that 443 is closed. It isn't.

Important: MadDash is showing perfect results for latency and trace tests. Only badwidth fails.

If I kill http and Perfsonaer, iperf/iperf3 work nicely over their default ports and over 443.

I believe that either another port is closed  or we have configuration problem, maybe in psconfig. My problem is "how to find".

Which  ports should I  probe that the throughput task is using?

Which configuration files are MadDash and the Perfsoar hosts using that might have configuration errors only for throughput tests?

Regards, Raul

---

From: Mark Feit <>
Sent: 27 February 2020 03:17
To: Raul Lopes <>; Szymon Trocha <>
Cc: <>; Tim Chown <>; Ivan Garnizov <>
Subject: Re: [perfsonar-user] Problem with Persfonar firewall ports

 

Raul Lopes writes:

 

I'd like an option like

  pscheduler task --rawtool iperf3 ...

 

That’s not in the cards.  pScheduler was designed to not be a proxy for running individual tools directly.

 

The task command has a --tool switch that allows specifying the tool(s) to use and the throughput test has a --single-ended switch which, for iperf2 and iperf3, will run a test without trying to schedule it with the far end.  That only works in the pScheduler-to-iperf3 direction.  If you need to test iperf3-to-pScheduler, you can still run a single-ended task on the pScheduler node and use the throughput test’s --reverse switch.

 

If you need to reserve some time on pScheduler’s timeline where it won’t run any non-background tasks, there’s a test called idleex that will do nothing and grab exclusive use of the system while it’s running.  (For example, pscheduler task idleex --duration PT5M will give you five minutes.)

 

 

As for what appears to be the problem, specifically:

[PSREMOTE@ps03 ~]$ pscheduler task throughput --source ps03-ban --dest ps01-ban

Submitting task...

Failed to post task: Error getting tools from ps01-ban: Connection timed out after 10001 milliseconds

 

This is part of the pScheduler-to-pScheduler negotiation that takes place long before iperf3 is ever involved.  In this case, pScheduler is trying to connect from ps03 or ps03-ban to the HTTPS port on ps01-ban and failing.    Without access to ps03, my guess would be that ps01-ban or the network between them is dropping traffic to the HTTPS port rather than the usual refusal or no-route reaction.

 

If that can’t be corrected, there are some ways to control how pScheduler handles is communication with other pSchedulers.  The first is lead binding, which is described at https://github.com/perfsonar/pscheduler/wiki/Binding-Requests.  The second is that tests with source, dest or host parameters have companion source-node, dest-node or host-node parameters that specify what address should be connected to so pScheduler can negotiate.  Putting all of this together for your task might look like this:

 

pscheduler task --lead-bind ps03-ban throughput --source ps03-ban --dest ps01-ban --dest-node ps01

 

 

Hope that helps get you a little closer to getting your tasks to run.

 

--Mark

 

Jisc is a registered charity (number 1149740) and a company limited by guarantee which is registered in England under company number. 05747339, VAT number GB 197 0632 86. Jisc’s registered office is: 4 Portwall Lane, Bristol, BS1 6NB. T 0203 697 5800.


Jisc Services Limited is a wholly owned Jisc subsidiary and a company limited by guarantee which is registered in England under company number 02881024, VAT number GB 197 0632 86. The registered office is: 4 Portwall Lane, Bristol, BS1 6NB. T 0203 697 5800.


For more details on how Jisc handles your data see our privacy notice here: https://www.jisc.ac.uk/website/privacy-notice