perfSONAR User Q&A and Other Discussion

[Bill Owens <>]

Bumping this to see if anybody has looked into what seems like a v6 problem. 
Some more info at the bottom...

On Thu, Oct 06, 2016 at 05:59:29PM -0400, Bill Owens wrote:
> [owens@bss532 perfsonar]$ tail owamp_bwctl.log
> Oct  6 16:21:56 localhost bwctld[28837]: FILE=endpoint.c, LINE=1310, 
> PeerAgent: Peer cancelled test before expected
> Oct  6 16:22:53 localhost bwctld[30779]: FILE=endpoint.c, LINE=1310, 
> PeerAgent: Peer cancelled test before expected
> Oct  6 16:25:42 localhost owampd[5137]: FILE=sapi.c, LINE=303, Connection 
> to ([bss532.nysernet.org]:861) from ([bss532.nysernet.org]:38122)
> Oct  6 16:30:41 localhost bwctld[15524]: FILE=endpoint.c, LINE=1310, 
> PeerAgent: Peer cancelled test before expected
> Oct  6 16:31:44 localhost bwctld[17706]: FILE=endpoint.c, LINE=1310, 
> PeerAgent: Peer cancelled test before expected
> Oct  6 16:32:02 localhost bwctld[18569]: FILE=endpoint.c, LINE=1310, 
> PeerAgent: Peer cancelled test before expected
> Oct  6 16:39:48 localhost bwctld[22236]: FILE=endpoint.c, LINE=1310, 
> PeerAgent: Peer cancelled test before expected
> Oct  6 16:42:08 localhost bwctld[22564]: FILE=endpoint.c, LINE=1310, 
> PeerAgent: Peer cancelled test before expected
> Oct  6 16:42:09 localhost bwctld[22568]: FILE=endpoint.c, LINE=1310, 
> PeerAgent: Peer cancelled test before expected
> Oct  6 16:49:36 localhost bwctld[23525]: FILE=endpoint.c, LINE=1310, 
> PeerAgent: Peer cancelled test before expected
> 
> 
> Looking at the other log files, nothing jumps out at me except for this in 
> meshconfig-agent.log:
> 
> 2016/10/06 16:21:32 (2915) WARN> perfsonar_meshconfig_agent:381 main:: - 
> Problem adding test, continuing with rest of config: 403 FO
> RBIDDEN: Task forbidden by limits:
> Identified as everybody
> Classified as everybody, default
> Application: Defaults
>     Group 1: Limit 'innocuous-tests' failed: Test type not in list
>     Group 1: Limit 'rtt-default' failed: Test is not 'rtt'
>     Group 1: Limit 'throughput-default' failed: Test is not 'throughput'
>     Group 1: Want any, 0/3 passed, 3/3 failed: FAIL
>     Group 1: Failed; stopping here.
>     Application FAILS
> Proposal does not meet limits
> 2016/10/06 16:21:32 (2915) WARN> perfsonar_meshconfig_agent:381 main:: - 
> Problem adding test, continuing with rest of config: 500 Ca
> n't connect to 2620:f:0:914::3: Can't connect to 2620:f:0:914::3
> 
> Name or service not known at /usr/share/perl5/LWP/Protocol/http.pm line 51.
> 2016/10/06 16:21:32 (2915) WARN> perfsonar_meshconfig_agent:381 main:: - 
> Problem adding test, continuing with rest of config: 500 Ca
> n't connect to 2620:f:0:914::3: Can't connect to 2620:f:0:914::3
> 
> Name or service not known at /usr/share/perl5/LWP/Protocol/http.pm line 51.
> 2016/10/06 16:21:32 (2915) WARN> perfsonar_meshconfig_agent:381 main:: - 
> Problem adding test, continuing with rest of config: 500 Ca
> n't connect to 2620:f:0:914::3: Can't connect to 2620:f:0:914::3
> 
> Name or service not known at /usr/share/perl5/LWP/Protocol/http.pm line 51.
> 2016/10/06 16:21:32 (2915) WARN> perfsonar_meshconfig_agent:381 main:: - 
> Problem adding test, continuing with rest of config: 500 Ca
> n't connect to 2620:f:0:914::3: Can't connect to 2620:f:0:914::3
> 
> Name or service not known at /usr/share/perl5/LWP/Protocol/http.pm line 51.
> 2016/10/06 16:21:32 (2915) WARN> perfsonar_meshconfig_agent:381 main:: - 
> Problem determining which pscheduler to submit test to for
> creation, skipping test: 400 BAD REQUEST: At /source: u'2620:f:0:914::3' is 
> not a 'host-name'
> 
> 2016/10/06 16:21:32 (2915) WARN> perfsonar_meshconfig_agent:381 main:: - 
> Problem determining which pscheduler to submit test to for
> creation, skipping test: 400 BAD REQUEST: At /source: u'2620:f:0:914::3' is 
> not a 'host-name'
> 
> 2016/10/06 16:21:32 (2915) INFO> perfsonar_meshconfig_agent:389 main:: - 
> Added 8 new tasks, and deleted 11 old tasks
> [owens@bss532 perfsonar]$ tail meshconfig-agent.log
> 2016/10/06 17:14:32 (2915) WARN> perfsonar_meshconfig_agent:381 main:: - 
> Problem adding test, continuing with rest of config: 500 Can't connect to 
> 2620:f:0:914::3: Can't connect to 2620:f:0:914::3
> 
> Name or service not known at /usr/share/perl5/LWP/Protocol/http.pm line 51.
> 2016/10/06 17:14:32 (2915) WARN> perfsonar_meshconfig_agent:381 main:: - 
> Problem adding test, continuing with rest of config: 500 Can't connect to 
> 2620:f:0:914::3: Can't connect to 2620:f:0:914::3
> 
> Name or service not known at /usr/share/perl5/LWP/Protocol/http.pm line 51.
> 2016/10/06 17:14:32 (2915) WARN> perfsonar_meshconfig_agent:381 main:: - 
> Problem determining which pscheduler to submit test to for creation, 
> skipping test: 400 BAD REQUEST: At /source: u'2620:f:0:914::3' is not a 
> 'host-name'
> 
> 2016/10/06 17:14:32 (2915) WARN> perfsonar_meshconfig_agent:381 main:: - 
> Problem determining which pscheduler to submit test to for creation, 
> skipping test: 400 BAD REQUEST: At /source: u'2620:f:0:914::3' is not a 
> 'host-name'
> 
> Maybe a Perl vs. IPv6 issue, but I'm not sure?
> 
> Bill.


So this error still shows up in the logs even with the current build, which 
makes me think that nothing has changed in that particular module. 

However, I discovered that some of my test destinations would actually work, 
but others were still failing. I eventually hit upon the fact that the 
working ones have only A records (IPv4 addresses) while the failing ones have 
both A and AAAA (v4 and v6). Note that I wasn't configuring v6 tests on any 
of them - I figured I'd try to make v4 work first - but even with that, the 
tests would not work.

I've now checked this theory by creating a local DNS entry for one of the 
remote pS hosts, and instead of including both of its addresses I just have 
IPv4. Sure enough, configuring that as a test works fine, while using the 
original name still fails.

And as a further twist, when I reconfigured my test machine to remove its 
IPv6 address and hopefully fix the meshconfig log errors, something bad 
happened to the display of the previously recorded test results. My three 
working tests still show up, with bandwidth numbers, but the graphs contain 
only 'failure' points where there had been a couple days worth of test 
results. I'm going to let the box run for a while in this mode and see 
whether it starts showing good data, if not I'll put the v6 address back on 
and see what that does.

Bill.