perfSONAR User Q&A and Other Discussion

[Brian Candler <>]

I now have a couple of VMs running (one is perfsonar 3.4.1, one is 
Ubuntu Trusty running bwctl), and have compiled bwctld from source and 
with --enable-debug.

On my first set of tests I was able to show the problem:

On the Ubuntu side:  bwctl -c perfsonar1 -T iperf3
Result: empty between SENDER START and SENDER END.

Logs at perfsonar side:

Dec 19 19:42:34 perfsonar1 bwctld[6386]: FILE=sapi.c, LINE=510, 
ControlSession([
perfsonar1.ws.nsrc.org]:6139) accepted from 
userid(nil):([192.168.56.13]:40778)
Dec 19 19:42:40 perfsonar1 bwctld[6386]: FILE=endpoint.c, LINE=1314, 
PeerAgent:
Peer cancelled test before expected

Logs at ubuntu side:

Dec 19 19:42:34 trusty bwctld[12137]: FILE=sapi.c, LINE=353, Connection 
to (trus
ty.ws.nsrc.org:4823) from (trusty.ws.nsrc.org:39273)
Dec 19 19:42:34 trusty bwctld[12137]: FILE=sapi.c, LINE=510, 
ControlSession([tru
sty.ws.nsrc.org]:4823) accepted from 
userid(nil):([trusty.ws.nsrc.org]:39273)
Dec 19 19:42:34 trusty bwctl: Using tool: iperf3
Dec 19 19:42:34 trusty bwctld[12137]: FILE=policy.c, LINE=1419, ResReq 
ALLOWED:
regular:request:duration = 10 (result = 0, limit = 30)
Dec 19 19:42:34 trusty bwctld[12137]: FILE=policy.c, LINE=1419, ResReq 
ALLOWED:
root:request:duration = 10 (result = 0, limit = 30)
Dec 19 19:42:34 trusty bwctld[12137]: FILE=policy.c, LINE=1419, ResReq 
ALLOWED:
regular:request:duration = 10 (result = 0, limit = 30)
Dec 19 19:42:34 trusty bwctld[12137]: FILE=policy.c, LINE=1419, ResReq 
ALLOWED:
root:request:duration = 10 (result = 0, limit = 30)
Dec 19 19:42:34 trusty bwctld[12137]: FILE=policy.c, LINE=1419, ResReq 
ALLOWED:
regular:request:max_time_error = 2 (result = 0, limit = 20)
Dec 19 19:42:34 trusty bwctld[12137]: FILE=policy.c, LINE=1419, ResReq 
ALLOWED:
root:request:max_time_error = 2 (result = 0, limit = 20)
Dec 19 19:42:34 trusty bwctld[12137]: FILE=policy.c, LINE=1419, ResReq 
ALLOWED:
regular:request:allow_tcp = 1 (result = 0, limit = 1)
Dec 19 19:42:34 trusty bwctld[12137]: FILE=policy.c, LINE=1419, ResReq 
ALLOWED:
root:request:allow_tcp = 1 (result = 0, limit = 1)
Dec 19 19:42:34 trusty bwctld[11925]: FILE=policy.c, LINE=1419, ResReq 
ALLOWED:
regular:request:pending = 1 (result = 0, limit = 0)
Dec 19 19:42:34 trusty bwctld[11925]: FILE=policy.c, LINE=1419, ResReq 
ALLOWED:
root:request:pending = 1 (result = 0, limit = 0)
Dec 19 19:42:34 trusty bwctld[11925]: FILE=bwctld.c, LINE=795, Test 
Reservation
Information: Current Time: 1419018154, Fuzz: 1.736328, Reservation 
Start: 141901
8156, Reservation End: 1419018171, Test Start Time: 1419018158
Dec 19 19:42:34 trusty bwctld[11925]: FILE=bwctld.c, LINE=446, 
Reservation Statu
s: time=1419018154 action=new sender=192.168.56.13 receiver=192.168.56.6 
tool=ip
erf3 res_start=1419018156 res_end=1419018170 test_start=1419018158
Dec 19 19:42:34 trusty bwctld[11925]: FILE=bwctld.c, LINE=826, Time Slots
Dec 19 19:42:34 trusty bwctld[11925]: FILE=bwctld.c, LINE=831, Time Slot 
1: 1419
018156 to 1419018171: 1 reservations
Dec 19 19:42:34 trusty bwctl: 15 seconds until test results available
Dec 19 19:42:40 trusty bwctld[11925]: FILE=policy.c, LINE=1419, ResReq 
ALLOWED:
regular:release:pending = 1 (result = 0, limit = 0)
Dec 19 19:42:40 trusty bwctld[11925]: FILE=policy.c, LINE=1419, ResReq 
ALLOWED:
root:release:pending = 1 (result = 0, limit = 0)
Dec 19 19:42:40 trusty bwctld[11925]: FILE=bwctld.c, LINE=446, 
Reservation Statu
s: time=1419018160 action=remove sender=192.168.56.13 
receiver=192.168.56.6 tool
=iperf3 res_start=1419018156 res_end=1419018170 test_start=1419018158

However after stopping the daemons, and re-running them in the 
foreground (with -Z), the problem went away :-(

Something is *very* borderline here. /me digs further.