perfSONAR User Q&A and Other Discussion

[Aaron Brown <>]

Hey Ben,

What are the iperf port settings in bwctld.conf? Also, what version of bwctld 
is it running?

Cheers,
Aaron

On Jun 11, 2014, at 3:30 PM, Ben Nelson 
<>
 wrote:

> Hi All,
> 
> We've seen an issue pop up a few times where consecutive bwctl tests on
> the same host manage to use the same port or iperf testing.  If the
> tests are close enough, it seems like the port doesn't get freed up
> quickly enough for the second test to run, and ends up failing.  Here
> are some log snippets of what we see:
> 
> May 24 17:01:53 pas.bldc.grnoc.iu.edu bwcollector.pl[45056]: BEGIN
> IGNORED: empty AL3S-BW4_CHIC_LOSA file =
> /var/lib/perfsonar/perfsonarbuoy_ma/bwctl/upload/K5pQCMJl0S at
> /opt/perfsonar_ps/perfsonarbuoy_m
> a/bin/bwcollector.pl line 891, <SESS> line 17.#012
> May 24 17:01:53 pas.bldc.grnoc.iu.edu bwcollector.pl[45056]:
> /var/lib/perfsonar/perfsonarbuoy_ma/bwctl/upload/K5pQCMJl0S: bwctl:
> start_endpoint: 3609939653.158082#012#012
> May 24 17:01:53 pas.bldc.grnoc.iu.edu bwcollector.pl[45056]:
> /var/lib/perfsonar/perfsonarbuoy_ma/bwctl/upload/K5pQCMJl0S: bwctl:
> run_endpoint: receiver: 198.71.46.56#012#012
> May 24 17:01:53 pas.bldc.grnoc.iu.edu bwcollector.pl[45056]:
> /var/lib/perfsonar/perfsonarbuoy_ma/bwctl/upload/K5pQCMJl0S: bwctl:
> run_endpoint: sender: 198.71.46.50#012#012
> May 24 17:01:53 pas.bldc.grnoc.iu.edu bwcollector.pl[45056]:
> /var/lib/perfsonar/perfsonarbuoy_ma/bwctl/upload/K5pQCMJl0S: bwctl:
> exec_line: iperf -B em-2.400.pas-tst.chic.net.internet2.edu -s -f b -l 4487
> 2 -m -p 5001 -u -w 100663296 -t 20 -i 1.000000#012#012
> May 24 17:01:53 pas.bldc.grnoc.iu.edu bwcollector.pl[45056]:
> /var/lib/perfsonar/perfsonarbuoy_ma/bwctl/upload/K5pQCMJl0S: bwctl:
> run_tool: tester: iperf#012#012
> May 24 17:01:53 pas.bldc.grnoc.iu.edu bwcollector.pl[45056]:
> /var/lib/perfsonar/perfsonarbuoy_ma/bwctl/upload/K5pQCMJl0S: bwctl:
> run_tool: receiver: 198.71.46.56#012#012
> May 24 17:01:53 pas.bldc.grnoc.iu.edu bwcollector.pl[45056]:
> /var/lib/perfsonar/perfsonarbuoy_ma/bwctl/upload/K5pQCMJl0S: bwctl:
> run_tool: sender: 198.71.46.50#012#012
> May 24 17:01:53 pas.bldc.grnoc.iu.edu bwcollector.pl[45056]:
> /var/lib/perfsonar/perfsonarbuoy_ma/bwctl/upload/K5pQCMJl0S: bwctl:
> start_tool: 3609939684.650671#012#012
> May 24 17:01:53 pas.bldc.grnoc.iu.edu bwcollector.pl[45056]:
> /var/lib/perfsonar/perfsonarbuoy_ma/bwctl/upload/K5pQCMJl0S: bind
> failed: Address already in use#012#012
> May 24 17:01:53 pas.bldc.grnoc.iu.edu bwcollector.pl[45056]:
> /var/lib/perfsonar/perfsonarbuoy_ma/bwctl/upload/K5pQCMJl0S:
> ------------------------------------------------------------#012#012
> May 24 17:01:53 pas.bldc.grnoc.iu.edu bwcollector.pl[45056]:
> /var/lib/perfsonar/perfsonarbuoy_ma/bwctl/upload/K5pQCMJl0S: Server
> listening on UDP port 5001#012#012
> May 24 17:01:53 pas.bldc.grnoc.iu.edu bwcollector.pl[45056]:
> /var/lib/perfsonar/perfsonarbuoy_ma/bwctl/upload/K5pQCMJl0S: Binding to
> local address em-2.400.pas-tst.chic.net.internet2.edu#012#012
> May 24 17:01:53 pas.bldc.grnoc.iu.edu bwcollector.pl[45056]:
> /var/lib/perfsonar/perfsonarbuoy_ma/bwctl/upload/K5pQCMJl0S: Receiving
> 44872 byte datagrams#012#012
> May 24 17:01:53 pas.bldc.grnoc.iu.edu bwcollector.pl[45056]:
> /var/lib/perfsonar/perfsonarbuoy_ma/bwctl/upload/K5pQCMJl0S: UDP buffer
> size: 201326592 Byte (WARNING: requested 100663296 Byte)#012#012
> May 24 17:01:53 pas.bldc.grnoc.iu.edu bwcollector.pl[45056]:
> /var/lib/perfsonar/perfsonarbuoy_ma/bwctl/upload/K5pQCMJl0S:
> ------------------------------------------------------------#012#012
> May 24 17:01:53 pas.bldc.grnoc.iu.edu bwcollector.pl[45056]:
> /var/lib/perfsonar/perfsonarbuoy_ma/bwctl/upload/K5pQCMJl0S: bwctl:
> stop_tool: 3609939713.725873#012#012
> May 24 17:01:53 pas.bldc.grnoc.iu.edu bwcollector.pl[45056]:
> /var/lib/perfsonar/perfsonarbuoy_ma/bwctl/upload/K5pQCMJl0S: bwctl:
> stop_endpoint: 3609939713.726364#012#012
> 
> Can anything be done to unbind the port quicker, so the second test is
> able to finish?  Or possible a change in bwctl testing to choose a port
> that hasn't been used recently?
> 
> Thanks,
> Ben
> 
> 
> -- 
> Ben Nelson, Systems Engineer
> Indiana University GlobalNOC
> <>
>