perfSONAR User Q&A and Other Discussion

[Jason Zurawski <>]

Hey Trey;

Your mail is timely, we are trying to re-write some documentation on this.  
This will be sort of scattered, but may be a enough to get you started:

1) Take a look at the hosts you have configured vs. the ones that are 
succeeding.  See if you can do a test by hand to one of the failures (e.g. on 
the cmd line: bwctl -T iperf3 -t 20 -i 1 -f m -x -vv -c HOST and bwctl -T 
iperf3 -t 20 -i 1 -f m -x -vv -S HOST).  If the host is dead, doesn’t support 
the requested test type, or denies your test in some way it is most likely a 
problem on the other end.  Maybe send an email to that admin to see what is 
up.

2) If the test from the first part succeeded, its time to look in the 
database.  In your case, open your favorite browser (preferably one with a 
JSON prettifier in place) to your esmond database:

http://psonar-bwctl.brazos.tamu.edu/esmond/perfsonar/archive/?format=json

Each test should have a corresponding record.  The ones that succeed, and the 
ones that fail.  Picking one that is failing for you (to 
ps1-akard-dlls.tx-learn.net - the LEARN Dallas node), we can pull up the 
failure event type:

http://psonar-bwctl.brazos.tamu.edu/esmond/perfsonar/archive/7f8d5be4115b4927b7ce6a5dfbfa8e44/failures/base?format=json

In this case we get a (useful?) error message:

> Problem parsing output: malformed JSON string, neither array, object, 
> number, string or atom, at character offset 0 (before "(end of string)") at 
> /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Parsers/Iperf3.pm
>  line 54. 


This is a known problem right now between certain versions of BWCTL and 
iperf3 that the developers are working on.  Another common error:

> interrupt - the client has terminated


Typically means there was either a firewall or NTP issue.  

3) Check your server side logs (/var/log/perfsonar) to see if anything is 
showing up in the bwctl_owamp log.  Common errors could be permission denied 
due to limits violations, the lack of being able to get a testing slot 
(common for busy perfSONAR nodes) or the aforementioned firewall issues.  

In general if your node is working for ‘some’ things, it may be on the 
healthy side.  Things that you are trying to test against may just not be 
reachable, or could be in need of an upgrade (another common issue we are 
seeing - until we can get a majority of the instances to 3.4.x).  

Hope this helps;

-jason

On Jan 20, 2015, at 10:54 AM, Trey Dockendorf 
<>
 wrote:

> I have about 14 test members added for throughput tests and only 3 of them 
> show up in the Throughput/Latency Graphs page.  I am unsure how to begin 
> debugging this to find the cause of the missing data.
> 
> The host is http://psonar-bwctl.brazos.tamu.edu/.  I have most of these 
> sites also added on a separate perfsonar instance that does latency tests 
> and that host is not missing data.
> 
> Thanks,
> - Trey
> 
> =============================
> 
> Trey Dockendorf 
> Systems Analyst I 
> Texas A&M University 
> Academy for Advanced Telecommunications and Learning Technologies 
> Phone: (979)458-2396 
> Email: 
> 
>  
> Jabber: 
>