perfSONAR User Q&A and Other Discussion

[Aaron Brown <>]

On Jan 6, 2010, at 11:06 AM, Niederberger, Ralph wrote:

> Dear all,
> 
> I am just experimenting with bwctl.
> 
> 
> Having three servers SA,SB and SC where SA is the control server executing
> command sequences like 
> 
> 1.  bwctl -s SB -c SC
> 2.  bwctl -s SC -c SB
> 
> the second connection fails with:
> 
> "bind failed: Address already in use
> ......
> Server listening on TCP port 5001
> .....
> bwctl: remote peer cancelled test
> bwctl: stop_exec: 3471777541.009142"
> 
> 
> When using the command sequence:
> 
> 1.   bwctl -s SB -c SC
> 2.   bwctl -s SC -c SB
> 3.   bwctl -s SC -c SB   
> 4.   bwctl -s SB -c SC
> 5.   bwctl -s SB -c SC
> 6.   bwctl -s SC -c SB
> 7.   bwctl -s SC -c SB
> 
> Connections 2, 4, and 6 fail whereas 1, 3, 5, and 7 will work.
> 
> It seems that starting a failing session will reset the port,
> so that it can be used again.
> 
> The problem seems to be that the iperf or bwctl program does not 
> close the socket correctly, so that the server is in a TIME_WAIT.
> 
> Any hints what the real problem is and how it can be solved?
> 
> At the end I would like to start "any-to-any"-measurements and store them in
> a database.
> This seems not to be possible unless every bwctl(iperf) call has correctly
> closed its 
> sockets and new connections can be started.

It's a problem with the iperf socket staying open longer than it should. A 
work around for this is to either 

The problem is that the iperf socket stays open even iperf has closed, and 
bwctl has no way of discerning when the socket is available again. The hope 
is that we'll be able to handle this case better with the release of iperf3, 
but for the current iteration, it's still a problem.

bwctl 1.3 should lessen the problem as long as you have a range of ports 
specified in the iperf_port variable (e.g. iperf_port   5001-5010). Another 
option is to decrease the amount of time a socket will stay in time_wait. You 
can do this by running: "echo 20 > /proc/sys/net/ipv4/tcp_fin_timeout"  which 
will make the timeout 20 seconds instead of the default 60. You'll need to 
add that to the rc.local script to get that survive a reboot.

Cheers,
Aaron

Winter 2010 ESCC/Internet2 Joint Techs
Hosted by the University of Utah - Salt Lake City, UT
January 31 - February 4, 2010
http://events.internet2.edu/2010/jt-slc/