perfSONAR User Q&A and Other Discussion

["Uhl, George D. (GSFC-423.0)[SGT INC]" <>]

Andy,

 

I tried the –lead-bind option and I’m getting a Type error - “invalid arguments to setopt”.  I tried this on another multihomed server and received the same error.

 

Thanks,

George

 

# pscheduler task --lead-bind xxx.xxx.8.206 throughput --source xxx.xxx.8.206 --dest xxx.xxx.197.28

Submitting task...

Failed to post task: Internal problem; see system logs.

 

The 'pscheduler troubleshoot' command may be of use in problem

diagnosis. 'pscheduler troubleshoot --help' for more information.

 

 

Aug 29 14:47:14 [localhost] journal: pscheduler-api ERROR    Exception: TypeError: invalid arguments to setopt

Aug 29 14:47:14 [localhost] journal: pscheduler-api ERROR    Traceback (most recent call last):

Aug 29 14:47:14 [localhost] journal: pscheduler-api ERROR      File "/usr/lib/python2.7/site-packages/flask/app.py", line 1475, in full_dispatch_request

Aug 29 14:47:14 [localhost] journal: pscheduler-api ERROR        rv = self.dispatch_request()

Aug 29 14:47:14 [localhost] journal: pscheduler-api ERROR      File "/usr/lib/python2.7/site-packages/flask/app.py", line 1461, in dispatch_request

Aug 29 14:47:14 [localhost] journal: pscheduler-api ERROR        return self.view_functions[rule.endpoint](**req.view_args)

Aug 29 14:47:14 [localhost] journal: pscheduler-api ERROR      File "./pschedulerapiserver/tasks.py", line 387, in tasks

Aug 29 14:47:14 [localhost] journal: pscheduler-api ERROR      File "/usr/lib/python2.7/site-packages/pscheduler/psurl.py", line 102, in url_get

Aug 29 14:47:14 [localhost] journal: pscheduler-api ERROR        curl = PycURLRunner(url, params, bind, timeout, allow_redirects, headers, verify_keys)

Aug 29 14:47:14 [localhost] journal: pscheduler-api ERROR      File "/usr/lib/python2.7/site-packages/pscheduler/psurl.py", line 39, in __init__

Aug 29 14:47:14 [localhost] journal: pscheduler-api ERROR        self.curl.setopt(pycurl.INTERFACE, bind)

Aug 29 14:47:14 [localhost] journal: pscheduler-api ERROR    TypeError: invalid arguments to setopt

Aug 29 14:47:14 [localhost] journal: pscheduler-api ERROR    Internal error 158.39.8.206 POST https://158.39.8.206/pscheduler/tasks: Internal problem; see system logs.

 

 

From: Andrew Lake <>
Date: Thursday, August 29, 2019 at 9:49 AM
To: "Uhl, George D. (GSFC-423.0)[SGT INC]" <>, "George.D.Uhl" <>, "" <>
Subject: [EXTERNAL] Re: [perfsonar-user] pscheduler throughput task failing on multihomed test host.

 

Hi George,

 

I think you just got the wrong bind option set because there are a couple similar-looking ones that do slightly different things. I think what you want is the following:

 

 pscheduler task --lead-bind xxx.xxx.8.206 throughput --source xxx.xxx.8.206 --dest xxx.xxx.197.28 

 

It looks like its not binding to the right interface when the source pScheduler server tries to talk to the destination server to figure out what tools it supports. The --lead-bind option between “task” and “throughput” tells the lead (the source in this case) to bind to the supplied address when talking to the dest to pass around pscheduler messages. The -B option you gave after “throughput” tells iperf where to bind, but the task you shared is not even getting to that point. I don’t think you need the -B, iperf should bind to the source automatically.

 

Thanks,

Andy

 

 

On August 28, 2019 at 11:24:53 AM, Uhl, George D. (GSFC-423.0)[SGT INC] () wrote:

Hi,

 

I’m having problems sourcing throughput tests on a test host with two interfaces.  I’ve configured the host to operate using multiple interfaces including running the mod_interface_route script for both interfaces.  Sourcing traffic over the correct interface works EXCEPT when I run pscheduler throughput task.  In this case the following error:

 

# pscheduler task throughput --source xxx.xxx.8.206 --dest xxx.xxx.197.28 -B xxx.xxx.8.206

Submitting task...

Failed to post task: Error getting tools from xxx.xxx.197.28: Failed connect to xxx.xxx.197.28:443; Connection refused

 

The 'pscheduler troubleshoot' command may be of use in problem

diagnosis. 'pscheduler troubleshoot --help' for more information.

 

When I run a trace task from the same source/dest pair it is successful.  Traceroutes reveal that the wrong interface is used for throughput tasks despite binding the test to the desired interface’s IP address.  Trace tasks use the proper interface indicating that the proper ip rule is getting invoked and using the local interface route.  When I run other applications and bind them to one of the source IP, they are successful and they use the correct interface.  The pscheduler throughput task is successful only after I put a route for the destination in the Linux default routing table.

 

# route add -host xxx.xxx.197.28 gw xxx.xxx.8.193

[root@epsosgs-pt1g-01 ~]# pscheduler task throughput --source xxx.xxx.8.206 --dest xxx.xxx.197.28

Submitting task...

Task URL:

https:// xxx.xxx.8.206/pscheduler/tasks/8255b181-c58c-42bf-9ba8-439ace11157d

Running with tool 'iperf3'

Fetching first run...

 

Next scheduled run:

https:// xxx.xxx.8.206/pscheduler/tasks/8255b181-c58c-42bf-9ba8-439ace11157d/runs/6ef62abd-8364-4af0-b6e3-1269164d62c5

Starts 2019-08-28T14:58:25Z (~19 seconds)

Ends   2019-08-28T14:58:44Z (~18 seconds)

Waiting for result...

 

* Stream ID 5

Interval       Throughput     Retransmits    Current Window 

0.0 - 1.0      18.32 Mbps     0              430.06 KBytes  

1.0 - 2.0      197.08 Mbps    183            1.89 MBytes    

2.0 - 3.0      94.36 Mbps     7              946.99 KBytes  

3.0 - 4.0      52.44 Mbps     0              957.13 KBytes  

4.0 - 5.0      52.43 Mbps     0              1.05 MBytes    

5.0 - 6.0      52.42 Mbps     0              1.26 MBytes    

6.0 - 7.0      83.89 Mbps     0              1.62 MBytes    

7.0 - 8.0      94.37 Mbps     0              2.10 MBytes    

8.0 - 9.0      125.83 Mbps    0              2.72 MBytes    

9.0 - 10.0     157.28 Mbps    0              3.53 MBytes    

 

Summary

Interval       Throughput     Retransmits    

0.0 - 10.0     92.84 Mbps     190

 

No further runs scheduled.

 

IP address binding for throughput tasks was working in the 4.1.2 perfsonar release.

 

Thanks,

George

--
To unsubscribe from this list: https://lists.internet2.edu/sympa/signoff/perfsonar-user