perfsonar-user - Re: [perfsonar-user] Tests only work unidirectionally with some endpoint
Subject: perfSONAR User Q&A and Other Discussion
List archive
- From: John-Paul Robinson <>
- To: <>, <>
- Subject: Re: [perfsonar-user] Tests only work unidirectionally with some endpoint
- Date: Tue, 11 Nov 2014 13:38:46 -0600
It may also be possible to change the perfSONAR parsers to request and
process the "local" side of the bwctl result since it runs the newer
tools that will produce the complete header.
All automated tests defined via the web appear to have localhost as one
end of the connection. If the bwctl command that negotiates the
scheduled tests had the -x option added it would record data from both
the sending and receiving bwcltd it contacts. The localhost bwctld will
produce the correct header. If the Bwctl/iperf parser code were to read
the appropriate output file it would be ensured data in a format that
can be parsed.
The other option would be to do something fancier for iperf result
parsing to support older end points.
~jpr
On 11/11/2014 11:05 AM,
wrote:
> Hi all,
>
> Is it possible to trap this problem and flag it in a more visible way?
>
>
> ________________________________________
> From: Jason Zurawski
> []
> Sent: 11 November 2014 17:00
> To: John-Paul Robinson
> Cc:
>
> Subject: Re: [perfsonar-user] Tests only work unidirectionally with some
> endpoint
>
> Howdy John-Paul;
>
> So I believe there are two issues here. I can offer the technical
> solution, which may be superseded by Layer8/9 issues that others (from
> Internet2 or the Global NOC) will need to speak to:
>
> - The BWCTL server software on machine in question
> (bwctl.atla.net.internet2.edu) is old, and not behaving in a way consistent
> with what the new BWCTL on version 3.4 of perfSONAR would expect. I can’t
> say for certain, but I would guess its 1.4.x vs 1.5.y.
>
> - The BWCTL server is not a part of the new list related to the Internet2
> Performance Service that support ad-hoc testing
> (http://www.internet2.edu/products-services/performance-analytics/performance-assurance-service/#features-adhoc).
> This appears to be an orphaned server that needs some TLC, or to be put
> out of its misery.
>
> I would suggest maybe trying one of the other ad-hoc servers in the
> meantime, or you can follow the procedure to be granted a golden ticket to
> test to the other resources that are not public.
>
> Hope this helps;
>
> Thanks;
>
> -jason
>
>> On Nov 11, 2014, at 10:33 AM, John-Paul Robinson
>> <>
>> wrote:
>>
>> I've traced this down to a problem resulting from the bwctld on
>> bwctl.atla.net.internet2.edu not producing the correct metadata in its
>> output to clients.
>>
>> The output header should look something like this:
>>
>> bwctl: start_endpoint: 3624691387.449529
>> bwctl: run_endpoint: receiver: 64.57.16.66
>> bwctl: run_endpoint: sender: 138.26.220.65
>> bwctl: exec_line: iperf -c 64.57.16.66 -B 138.26.220.65 -f b -P 1 -m -p
>> 5014 -t 20 -i 1.000000
>> bwctl: run_tool: tester: iperf
>> bwctl: run_tool: receiver: 64.57.16.66
>> bwctl: run_tool: sender: 138.26.220.65
>> bwctl: start_tool: 3624691432.432794
>>
>> But the problematic node is only producing a metadata header that contains:
>>
>> bwctl: exec_line: /usr/bin/iperf -B 64.57.16.66 -s -f b -P 1 -m -p 5014 -t
>> 20 -i 1
>> bwctl: start_tool: 3624691430.401291
>>
>> This causes the perfSONAR scripts to miss-parse the results, failing on
>> the $tool assignment because it doesn't match "bwctl: exec_line: iperf".
>> I changed that pattern to accept full paths and then the basic result
>> parse succeeds but, due to missing source data, cannot be recorded in the
>> tests database.
>>
>> I'm not sure why this node is behaving this way.
>>
>> You can reproduce this behavior by running the following tests. Both the
>> send and recv files should include the complete header, however, when
>> bwctl.atla.net.internet2.edu returns data it produces an incomplete header
>> which then causes the failed parsing by perfSONAR.
>>
>> Test with bwctl.atla as receiver produces the borked header:
>>
>> /usr/bin/bwctl -s perfsonar-10g-scidmz.uabgrid.uab.edu -c
>> bwctl.atla.net.internet2.edu -i 1 --parsable -4 -T iperf -p -d ./ -v -P 1
>> -t 20 -x
>>
>> $ head 15567942049171924331.recv.bw
>> bwctl: exec_line: /usr/bin/iperf -B 64.57.16.66 -s -f b -P 1 -m -p 5015 -t
>> 20 -i 1
>> bwctl: start_tool: 3624693965.158426
>> ------------------------------------------------------------
>> Server listening on TCP port 5015
>> Binding to local address 64.57.16.66
>> TCP window size: 87380 Byte (default)
>> ------------------------------------------------------------
>> [ 15] local 64.57.16.66 port 5015 connected with 138.26.220.65 port 49750
>> [ ID] Interval Transfer Bandwidth
>> [ 15] 0.0- 1.0 sec 62630432 Bytes 501043456 bits/sec
>>
>>
>> Tests with working node as receiver produces a legit header:
>>
>> /usr/bin/bwctl -s perfsonar-10g-scidmz.uabgrid.uab.edu -c 56m-ps.sox.net
>> -i 1 --parsable -4 -T iperf -p -d ./ -v -P 1 -t 20 -x
>>
>> $ head 15567942589368762368.recv.bw
>> bwctl: start_endpoint: 3624694082.873576
>> bwctl: run_endpoint: receiver: 143.215.194.125
>> bwctl: run_endpoint: sender: 138.26.220.65
>> bwctl: exec_line: iperf -B 143.215.194.125 -s -f b -P 1 -m -p 5128 -t 20
>> -i 1.000000
>> bwctl: run_tool: tester: iperf
>> bwctl: run_tool: receiver: 143.215.194.125
>> bwctl: run_tool: sender: 138.26.220.65
>> bwctl: start_tool: 3624694091.000167
>> ------------------------------------------------------------
>> Server listening on TCP port 5128
>>
>> I've contacted the maintainers of the failed node but thought the debug
>> info might help others determine if their problematic one-way target nodes
>> are behaving similarly.
>>
>> ~jpr
>>
>> On 11/10/2014 04:58 PM, John-Paul Robinson wrote:
>>> While the cause of the problem does seem to be a failure to parse the
>>> iperf output, the source of the problem doesn't appear to be Bwctl.pm but
>>> rather the bwctl process itself.
>>>
>>> When I run bwctl against a known working end point:
>>>
>>> /usr/bin/bwctl -c 56m-ps.sox.net -i 1 --parsable -4 -T iperf -p -d
>>> ./soxtest/ -v -P 1 -t 20 -I 300
>>>
>>> The results file created contains the "locally" gather bwctl results
>>> which includes the correctly parsable header that describes the
>>> metadata for the test:
>>>
>>> $ cat soxtest/15567672641991999488.recv.bw
>>> bwctl: start_endpoint: 3624631187.210311
>>> bwctl: run_endpoint: receiver: 143.215.194.125
>>> bwctl: run_endpoint: sender: 138.26.220.65
>>> bwctl: exec_line: iperf -B 143.215.194.125 -s -f b -P 1 -m -p 5045 -t 20
>>> -i 1.000000
>>> bwctl: run_tool: tester: iperf
>>> bwctl: run_tool: receiver: 143.215.194.125
>>> bwctl: run_tool: sender: 138.26.220.65
>>> bwctl: start_tool: 3624631239.004069
>>>
>>> However, when I run an identical test against the problematic endpoint:
>>>
>>> /usr/bin/bwctl -c 64.57.16.66 -T iperf -4 -I 900 -p -d i2atla-iperfonly
>>> -v -P 1 -t 20 -i 1 --parsable
>>>
>>> The bwctl result file contains the "remotely" gather results which has
>>> insufficient metadata for correct parsing:
>>>
>>> $ cat i2atla-iperfonly/15567745226308775992.recv.bw
>>> bwctl: exec_line: /usr/bin/iperf -B 64.57.16.66 -s -f b -P 1 -m -p 5009
>>> -t 20 -i 1
>>> bwctl: start_tool: 3624648139.253983
>>> ------------------------------------------------------------
>>> Server listening on TCP port 5009
>>> Binding to local address 64.57.16.66
>>> TCP window size: 87380 Byte (default)
>>> ------------------------------------------------------------
>>>
>>> It's actually the first line with the /usr/bin/iperf path that causes the
>>> unparsable errors reported earlier in Bwctl.pm (below). I changed the
>>> pattern match thinking this was the bug but although the data is parsed
>>> it then failes to be added to the cassandra data base because of a
>>> missing source address (since there isn't one on the iperf server side).
>>> stracing the bwctl process shows that the parsable local data is recieved
>>> but is not written to the test output file.
>>>
>>> It's not clear why bwctl would have this odd behavior for just this one
>>> host. It seems almost as if the internal stdout/stderr channels are
>>> somehow crossed. If bwctl wrote the correct, locally generated output it
>>> would have the necessary metadata header and would parse with the
>>> existing Bwctl.pm code.
>>>
>>> Any thoughts on what's going on here?
>>>
>>> ~jpr
>>>
>>> On 11/10/2014 01:31 PM, John-Paul Robinson wrote:
>>>> Hi Folks,
>>>>
>>>> From what I can tell the problem my tests are running into has to do
>>>> with having iperf results instead of iperf3 results. The host that is
>>>> failing negotiates tests with iperf. All my other test endpoints
>>>> negotiate tests with iperf3.
>>>>
>>>> It appears there is some sort of parsing error in the
>>>> parse_bwctl_output() routine. The error in the regular_testing.log
>>>> relates to the $tool variable not being set which eventually bubbles up
>>>> to full parsing error (see the log below in my previous email).
>>>>
>>>> The code segment where $tool would be set looks like this:
>>>>
>>>> elsif ($line =~ /bwctl: exec_line: iperf/) {
>>>> $results{tool} = "iperf";
>>>> }
>>>> elsif ($line =~ /bwctl: exec_line: nuttcp/) {
>>>> $results{tool} = "nuttcp";
>>>> }
>>>> elsif ($line =~ /bwctl: run_tool: tester: (.*)/) {
>>>> $results{tool} = $1;
>>>>
>>>> It appears when the test is an "iperf" test it should parse the line
>>>> "bwctl: exec_line: iperf". For whatever reason the value for tool
>>>> doesn't get set and that triggers the subsequent warnings and eventual
>>>> parse failure.
>>>>
>>>> I suspect this is why the src-to-dst test is failing and why I'm not
>>>> seeing the results for these tests in my graphs.
>>>>
>>>> ~jpr
>>>>
>>>> On 11/07/2014 10:22 AM, John-Paul Robinson wrote:
>>>>> I'm seeing something similar with one of my tests. It works
>>>>> bi-directionally on the command line but the automated web test is no
>>>>> longer working in the local-to-remote direction.
>>>>>
>>>>> I've poked around the regular testing log some and suspect there may be
>>>>> some empty data value borking the test results by causing perl to barf
>>>>> on the test results.
>>>>>
>>>>> Here is the log for the failed local-to-remote part of the tests. Note
>>>>> that the test seems to complete given that the gap in the log entry
>>>>> time matches the reported wait time:
>>>>>
>>>>> 2014/11/07 00:00:21 (13317) DEBUG> Cmd.pm:130
>>>>> perfSONAR_PS::RegularTesting::Utils::CmdRunner::Cmd::readlines - Lines:
>>>>> $VAR1 = [
>>>>> 'bwctl: Using 138.26.220.65 as the address for local sender',
>>>>> 'bwctl: Using 64.57.16.66 as the address for remote receiver',
>>>>> 'bwctl: Available in-common: iperf nuttcp ',
>>>>> 'bwctl: Using tool: iperf'
>>>>> ];
>>>>> 2014/11/07 00:00:21 (13317) DEBUG> Cmd.pm:130
>>>>> perfSONAR_PS::RegularTesting::Utils::CmdRunner::Cmd::readlines - Lines:
>>>>> $VAR1 = [
>>>>> 'bwctl: Server \'64.57.16.66\' accepted test request at time
>>>>> 1415340026.515434'
>>>>> ];
>>>>> 2014/11/07 00:00:21 (13317) DEBUG> Cmd.pm:130
>>>>> perfSONAR_PS::RegularTesting::Utils::CmdRunner::Cmd::readlines - Lines:
>>>>> $VAR1 = [
>>>>> 'bwctl: Client \'localhost\' accepted test request at time
>>>>> 1415340026.515434'
>>>>> ];
>>>>> 2014/11/07 00:00:21 (13317) DEBUG> Cmd.pm:130
>>>>> perfSONAR_PS::RegularTesting::Utils::CmdRunner::Cmd::readlines - Lines:
>>>>> $VAR1 = [
>>>>> 'bwctl: 26 seconds until test results available'
>>>>> ];
>>>>> 2014/11/07 00:00:47 (13317) DEBUG> Cmd.pm:130
>>>>> perfSONAR_PS::RegularTesting::Utils::CmdRunner::Cmd::readlines - Lines:
>>>>> $VAR1 = [
>>>>>
>>>>> '/var/lib/perfsonar/regular_testing/bwctl_Du5FB/15566373779833847474.recv.bw'
>>>>> ];
>>>>> 2014/11/07 00:00:47 (13317) WARN> daemon:103 main::__ANON__ - Warned:
>>>>> Use of uninitialized value $tool in string eq at
>>>>> /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Parsers/Bwctl.pm
>>>>> line 117.
>>>>> 2014/11/07 00:00:47 (13317) WARN> daemon:103 main::__ANON__ - Warned:
>>>>> Use of uninitialized value $tool in string eq at
>>>>> /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Parsers/Bwctl.pm
>>>>> line 120.
>>>>> 2014/11/07 00:00:47 (13317) WARN> daemon:103 main::__ANON__ - Warned:
>>>>> Use of uninitialized value $tool in string eq at
>>>>> /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Parsers/Bwctl.pm
>>>>> line 123.
>>>>> 2014/11/07 00:00:47 (13317) WARN> daemon:103 main::__ANON__ - Warned:
>>>>> Use of uninitialized value $tool in string eq at
>>>>> /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Parsers/Bwctl.pm
>>>>> line 126.
>>>>> 2014/11/07 00:00:47 (13317) WARN> daemon:103 main::__ANON__ - Warned:
>>>>> Use of uninitialized value $tool in string eq at
>>>>> /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Parsers/Bwctl.pm
>>>>> line 129.
>>>>> 2014/11/07 00:00:47 (13317) WARN> daemon:103 main::__ANON__ - Warned:
>>>>> Use of uninitialized value $tool in string eq at
>>>>> /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Parsers/Bwctl.pm
>>>>> line 132.
>>>>> 2014/11/07 00:00:47 (13317) WARN> daemon:103 main::__ANON__ - Warned:
>>>>> Use of uninitialized value $tool in concatenation (.) or string at
>>>>> /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Parsers/Bwctl.pm
>>>>> line 136.
>>>>> 2014/11/07 00:00:47 (13317) ERROR> CmdRunner.pm:110
>>>>> perfSONAR_PS::RegularTesting::Utils::CmdRunner::run - Problem with
>>>>> results callback: Attribute (tool) does not pass the type constraint
>>>>> because: Validation failed for 'Str' with value undef at
>>>>> /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Tests/Bwctl.pm
>>>>> line 119
>>>>>
>>>>> perfSONAR_PS::RegularTesting::Tests::Bwctl::__ANON__('perfSONAR_PS::RegularTesting::Tests::Bwctl=HASH(0x46501d8)',
>>>>> 'HASH(0x4656c38)') called at
>>>>> /usr/lib64/perl5/vendor_perl/Moose/Meta/Method/Overridden.pm line 36
>>>>>
>>>>> perfSONAR_PS::RegularTesting::Tests::Bwctl::build_results('perfSONAR_PS::RegularTesting::Tests::Bwctl=HASH(0x46501d8)',
>>>>> 'HASH(0x4656c38)') called at
>>>>> /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Tests/BwctlBase.pm
>>>>> line 237
>>>>>
>>>>> perfSONAR_PS::RegularTesting::Tests::BwctlBase::handle_output('perfSONAR_PS::RegularTesting::Tests::Bwctl=HASH(0x46501d8)',
>>>>> 'HASH(0x4880ad0)') called at
>>>>> /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Tests/BwctlBase.pm
>>>>> line 180
>>>>>
>>>>> perfSONAR_PS::RegularTesting::Tests::BwctlBase::__ANON__('perfSONAR_PS::RegularTesting::Utils::CmdRunner::Cmd=HASH(0x47...',
>>>>> 'HASH(0x46a2850)') called at
>>>>> /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Utils/CmdRunner.pm
>>>>> line 108
>>>>> eval {...} called at
>>>>> /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Utils/CmdRunner.pm
>>>>> line 107
>>>>>
>>>>> perfSONAR_PS::RegularTesting::Utils::CmdRunner::run('perfSONAR_PS::RegularTesting::Utils::CmdRunner=HASH(0x478afc0)')
>>>>> called at
>>>>> /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Tests/BwctlBase.pm
>>>>> line 188
>>>>>
>>>>> perfSONAR_PS::RegularTesting::Tests::BwctlBase::__ANON__('perfSONAR_PS::RegularTesting::Tests::Bwctl=HASH(0x46501d8)',
>>>>> 'HASH(0x4755cd8)') called at
>>>>> /usr/lib64/perl5/vendor_perl/Moose/Meta/Method/Overridden.pm line 36
>>>>>
>>>>> perfSONAR_PS::RegularTesting::Tests::BwctlBase::run_test('perfSONAR_PS::RegularTesting::Tests::Bwctl=HASH(0x46501d8)',
>>>>> 'HASH(0x4755cd8)') called at
>>>>> /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Test.pm
>>>>> line 75
>>>>>
>>>>> perfSONAR_PS::RegularTesting::Test::run_test('perfSONAR_PS::RegularTesting::Test=HASH(0x465ed40)',
>>>>> 'handle_results', 'CODE(0x1415c78)') called at
>>>>> /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Master/SelfScheduledTestChild.pm
>>>>> line 50
>>>>> eval {...} called at
>>>>> /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Master/SelfScheduledTestChild.pm
>>>>> line 40
>>>>>
>>>>> perfSONAR_PS::RegularTesting::Master::SelfScheduledTestChild::__ANON__('perfSONAR_PS::RegularTesting::Master::SelfScheduledTestChild=...')
>>>>> called at /usr/lib64/perl5/vendor_perl/Moose/Meta/Method/Overridden.pm
>>>>> line 36
>>>>>
>>>>> perfSONAR_PS::RegularTesting::Master::SelfScheduledTestChild::child_main_loop('perfSONAR_PS::RegularTesting::Master::SelfScheduledTestChild=...')
>>>>> called at
>>>>> /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Master/BaseChild.pm
>>>>> line 32
>>>>>
>>>>> perfSONAR_PS::RegularTesting::Master::BaseChild::run('perfSONAR_PS::RegularTesting::Master::SelfScheduledTestChild=...')
>>>>> called at
>>>>> /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Master.pm
>>>>> line 182
>>>>>
>>>>> perfSONAR_PS::RegularTesting::Master::run('perfSONAR_PS::RegularTesting::Master=HASH(0x4671bb8)')
>>>>> called at /opt/perfsonar_ps/regular_testing/bin/daemon line 122
>>>>> 2014/11/07 00:00:47 (13317) DEBUG> Cmd.pm:130
>>>>> perfSONAR_PS::RegularTesting::Utils::CmdRunner::Cmd::readlines - Lines:
>>>>> $VAR1 = [
>>>>> 'bwctl: 913 seconds until next testing period'
>>>>> ];
>>>>>
>>>>> Here is the corresponding part of the successful test for the
>>>>> remote-to-local part of the test. It runs the test, gets the results,
>>>>> parses them and sticks them in the database (given that I see them in
>>>>> the graphs):
>>>>>
>>>>> 2014/11/07 00:01:27 (13317) DEBUG> Cmd.pm:130
>>>>> perfSONAR_PS::RegularTesting::Utils::CmdRunner::Cmd::readlines - Lines:
>>>>> $VAR1 = [
>>>>> 'bwctl: Using bwctl.atla.net.internet2.edu as the address for
>>>>> remote sender',
>>>>> 'bwctl: Using 138.26.220.65 as the address for local
>>>>> receiver',
>>>>> 'bwctl: Available in-common: iperf nuttcp ',
>>>>> 'bwctl: Using tool: iperf'
>>>>> ];
>>>>> 2014/11/07 00:01:27 (13317) DEBUG> Cmd.pm:130
>>>>> perfSONAR_PS::RegularTesting::Utils::CmdRunner::Cmd::readlines - Lines:
>>>>> $VAR1 = [
>>>>> 'bwctl: Server \'localhost\' accepted test request at time
>>>>> 1415340091.510944'
>>>>> ];
>>>>> 2014/11/07 00:01:27 (13317) DEBUG> Cmd.pm:130
>>>>> perfSONAR_PS::RegularTesting::Utils::CmdRunner::Cmd::readlines - Lines:
>>>>> $VAR1 = [
>>>>> 'bwctl: Client \'bwctl.atla.net.internet2.edu\' accepted test
>>>>> request at time 1415340091.510944'
>>>>> ];
>>>>> 2014/11/07 00:01:27 (13317) DEBUG> Cmd.pm:130
>>>>> perfSONAR_PS::RegularTesting::Utils::CmdRunner::Cmd::readlines - Lines:
>>>>> $VAR1 = [
>>>>> 'bwctl: 25 seconds until test results available'
>>>>> ];
>>>>> 2014/11/07 00:01:52 (13317) DEBUG> Cmd.pm:130
>>>>> perfSONAR_PS::RegularTesting::Utils::CmdRunner::Cmd::readlines - Lines:
>>>>> $VAR1 = [
>>>>>
>>>>> '/var/lib/perfsonar/regular_testing/bwctl_BkMZ9/15566374058987434689.recv.bw'
>>>>> ];
>>>>> 2014/11/07 00:01:52 (13317) DEBUG> Bwctl.pm:138
>>>>> perfSONAR_PS::RegularTesting::Tests::Bwctl::__ANON__ - Build Results:
>>>>> $VAR1 = {
>>>>> 'summary_results' => {
>>>>> 'summary_results' => {
>>>>> 'throughput' =>
>>>>> '4480866964'
>>>>> },
>>>>> 'streams' => [
>>>>> {
>>>>> 'throughput' =>
>>>>> '4480866964',
>>>>> 'stream_id' => '15'
>>>>> }
>>>>> ]
>>>>> },
>>>>> 'protocol' => 'tcp',
>>>>> 'source' => {
>>>>> 'protocol' => 'tcp',
>>>>> 'address' => '64.57.16.66',
>>>>> 'hostname' => 'bwctl.atla.net.internet2.edu'
>>>>> },
>>>>> 'errors' => [],
>>>>> 'destination' => {
>>>>> 'protocol' => 'tcp',
>>>>> 'address' => '138.26.220.65'
>>>>> },
>>>>> 'raw_results' => 'bwctl: start_endpoint: 3624328887.156787
>>>>> bwctl: run_endpoint: receiver: 138.26.220.65
>>>>> bwctl: run_endpoint: sender: 64.57.16.66
>>>>> bwctl: exec_line: iperf -B 138.26.220.65 -s -f b -P 1 -m -p 5182 -t 20
>>>>> -i 1.000000
>>>>> bwctl: run_tool: tester: iperf
>>>>> bwctl: run_tool: receiver: 138.26.220.65
>>>>> bwctl: run_tool: sender: 64.57.16.66
>>>>> bwctl: start_tool: 3624328889.822644
>>>>> ------------------------------------------------------------
>>>>> Server listening on TCP port 5182
>>>>> Binding to local address 138.26.220.65
>>>>> TCP window size: 87380 Byte (default)
>>>>> ------------------------------------------------------------
>>>>> [ 15] local 138.26.220.65 port 5182 connected with 64.57.16.66 port 5182
>>>>> [ ID] Interval Transfer Bandwidth
>>>>> [ 15] 0.0- 1.0 sec 278847688 Bytes 2230781504 bits/sec
>>>>> [ 15] 1.0- 2.0 sec 574872044 Bytes 4598976352 bits/sec
>>>>> [ 15] 2.0- 3.0 sec 574443704 Bytes 4595549632 bits/sec
>>>>> [ 15] 3.0- 4.0 sec 575490620 Bytes 4603924960 bits/sec
>>>>> [ 15] 4.0- 5.0 sec 573504164 Bytes 4588033312 bits/sec
>>>>> [ 15] 5.0- 6.0 sec 576805976 Bytes 4614447808 bits/sec
>>>>> [ 15] 6.0- 7.0 sec 575192188 Bytes 4601537504 bits/sec
>>>>> [ 15] 7.0- 8.0 sec 575457480 Bytes 4603659840 bits/sec
>>>>> [ 15] 8.0- 9.0 sec 574730536 Bytes 4597844288 bits/sec
>>>>> [ 15] 9.0-10.0 sec 573548904 Bytes 4588391232 bits/sec
>>>>> [ 15] 10.0-11.0 sec 574434756 Bytes 4595478048 bits/sec
>>>>> [ 15] 11.0-12.0 sec 575257476 Bytes 4602059808 bits/sec
>>>>> [ 15] 12.0-13.0 sec 575052664 Bytes 4600421312 bits/sec
>>>>> [ 15] 13.0-14.0 sec 575186388 Bytes 4601491104 bits/sec
>>>>> [ 15] 14.0-15.0 sec 575061116 Bytes 4600488928 bits/sec
>>>>> [ 15] 15.0-16.0 sec 575683832 Bytes 4605470656 bits/sec
>>>>> [ 15] 16.0-17.0 sec 575449524 Bytes 4603596192 bits/sec
>>>>> [ 15] 17.0-18.0 sec 574237900 Bytes 4593903200 bits/sec
>>>>> [ 15] 18.0-19.0 sec 574121576 Bytes 4592972608 bits/sec
>>>>> [ 15] 19.0-20.0 sec 574765336 Bytes 4598122688 bits/sec
>>>>> [ 15] 0.0-20.0 sec 11209015296 Bytes 4480866964 bits/sec
>>>>> [ 15] MSS size 8948 bytes (MTU 8988 bytes, unknown interface)
>>>>> bwctl: stop_tool: 3624328911.581018
>>>>> bwctl: stop_endpoint: 3624328912.581453
>>>>> ',
>>>>> 'start_time' => '2014-11-07T06:01:29',
>>>>> 'end_time' => '2014-11-07T06:01:51',
>>>>> 'tool' => 'iperf',
>>>>> 'streams' => 1,
>>>>> 'type' => 'throughput',
>>>>> 'intervals' => [
>>>>> ... and so forth
>>>>>
>>>>> This gives me some confidence that there is a problem with the
>>>>> local-to-remote.
>>>>>
>>>>> ~jpr
>>>>>
>>>>> On 11/04/2014 08:53 AM, Jarvis Zhang wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I set several tests on my host, but some tests only work
>>>>>> unidirectionally. For throughput
>>>>>>
>>>>>> This problem is a little complicate. Here is my address:
>>>>>> http://115.25.138.244/serviceTest/psGraph.cgi
>>>>>>
>>>>>> The throughput test works on command line but fails on web interface.
>>>>>> The one-way delay test doesn’t work even on command line; I think
>>>>>> maybe the problem has something to do with the port, but I have no
>>>>>> idea where to start with.
>>>>>> The ping delay test never work.
>>>>>>
>>>>>> Attachment is regular_testing.log. Here is more details:
>>>>>>
>>>>>> 1. For throughput test:
>>>>>> a) 115.25.138.244(my host) to 128.104.100.67(komatsu) stops working
>>>>>> since I update from v3.3.2 to 3.4 yesterday. but the command line
>>>>>> still works.
>>>>>>> [zwzhang@jsi-inspur02
>>>>>>> perfsonar]$ bwctl -c 128.104.100.67
>>>>>>> bwctl: Using tool: iperf
>>>>>>> bwctl: 27 seconds until test results available
>>>>>>>
>>>>>>> RECEIVER START
>>>>>>> bwctl: exec_line: iperf -B komatsu.chtc.wisc.edu -s -f b -m -p 5019
>>>>>>> -t 10
>>>>>>> bwctl: start_tool: 3624097458.082252
>>>>>>> ------------------------------------------------------------
>>>>>>> Server listening on TCP port 5019
>>>>>>> Binding to local address komatsu.chtc.wisc.edu
>>>>>>> TCP window size: 87380 Byte (default)
>>>>>>> ------------------------------------------------------------
>>>>>>> [ 14] local 128.104.100.67 port 5019 connected with 115.25.138.244
>>>>>>> port 52456
>>>>>>> [ ID] Interval Transfer Bandwidth
>>>>>>> [ 14] 0.0-11.5 sec 36438016 Bytes 25453583 bits/sec
>>>>>>> [ 14] MSS size 1436 bytes (MTU 1500 bytes, ethernet)
>>>>>>> bwctl: stop_exec: 3624097473.816542
>>>>>>>
>>>>>>> RECEIVER END
>>>>>> b) 2001:da8:203:d406:16da:e9ff:fef9:b68f(my host) to
>>>>>> 2607:f388:107c:501:202:c9ff:fee8:85a0(komatsu) never works, but the
>>>>>> command line still works, as above.
>>>>>>
>>>>>> c) 159.226.15.235(cnic) to 115.25.138.244(my host) stops working since
>>>>>> I update from v3.3.2 to 3.4 yesterday. so does the command line.
>>>>>>> [zwzhang@jsi-inspur02
>>>>>>> perfsonar]$ bwctl -s 159.226.15.235
>>>>>>> bwctl: Using tool: iperf
>>>>>>> bwctl: 20 seconds until test results available
>>>>>>>
>>>>>>> RECEIVER START
>>>>>>>
>>>>>>> bwctl: remote peer cancelled test
>>>>>>>
>>>>>>> RECEIVER END
>>>>>> d) 2400:dd01:1011:1:92b1:1cff:fe0c:7c0a(cnic) to
>>>>>> 2001:da8:203:d406:16da:e9ff:fef9:b68f(my host) stops working since I
>>>>>> update from v3.3.2 to 3.4 yesterday. so does the command line, as
>>>>>> above.
>>>>>>
>>>>>> e) 115.25.138.244 to 137.110.119.113 works well bidirectionally.
>>>>>>
>>>>>> 2. For one-way latency
>>>>>>
>>>>>> a) 128.104.100.67(komatsu) to 115.25.138.244(my host) doesn’t work, so
>>>>>> does the command line.
>>>>>>> --- owping statistics from [128.104.100.67]:8944 to
>>>>>>> [jsi-inspur02]:8841 ---
>>>>>>> SID: 73198af4d8035c80a36f5244c1e63df9
>>>>>>> first: 2014-11-04T22:09:06.866
>>>>>>> last: 2014-11-04T22:09:17.026
>>>>>>> 100 sent, 100 lost (100.000%), 0 duplicates
>>>>>>> one-way delay min/median/max = nan/nan/nan ms, (err=0.104 ms)
>>>>>>> one-way jitter = nan ms (P95-P50)
>>>>>>> TTL not reported
>>>>>>> no reordering
>>>>>> b)159.226.15.235(cnic) to 115.25.138.244(my host) doesn’t work , so
>>>>>> does the command line.
>>>>>>
>>>>>> c) 137.110.119.113(murpa) doesn’t work bidirectionally.
>>>>>>
>>>>>> d) Other ipv6 addresses work well.
>>>>>>
>>>>>> 3. All ping latency doesn’t work.
>>>>>>
>>>>>> Any insights appreciated.
>>>>>>
>>>>>> Thanks,
>>>>>> Jarvis
- [perfsonar-user] Tests only work unidirectionally with some endpoint, Jarvis Zhang, 11/04/2014
- Re: [perfsonar-user] Tests only work unidirectionally with some endpoint, John-Paul Robinson, 11/07/2014
- Re: [perfsonar-user] Tests only work unidirectionally with some endpoint, John-Paul Robinson, 11/10/2014
- Re: [perfsonar-user] Tests only work unidirectionally with some endpoint, John-Paul Robinson, 11/10/2014
- Re: [perfsonar-user] Tests only work unidirectionally with some endpoint, John-Paul Robinson, 11/11/2014
- Re: [perfsonar-user] Tests only work unidirectionally with some endpoint, Jason Zurawski, 11/11/2014
- RE: [perfsonar-user] Tests only work unidirectionally with some endpoint, alex.white, 11/11/2014
- Re: [perfsonar-user] Tests only work unidirectionally with some endpoint, John-Paul Robinson, 11/11/2014
- Re: [perfsonar-user] Tests only work unidirectionally with some endpoint, Aaron Brown, 11/13/2014
- RE: [perfsonar-user] Tests only work unidirectionally with some endpoint, alex.white, 11/11/2014
- Re: [perfsonar-user] Tests only work unidirectionally with some endpoint, Jason Zurawski, 11/11/2014
- Re: [perfsonar-user] Tests only work unidirectionally with some endpoint, John-Paul Robinson, 11/11/2014
- Re: [perfsonar-user] Tests only work unidirectionally with some endpoint, John-Paul Robinson, 11/10/2014
- Re: [perfsonar-user] Tests only work unidirectionally with some endpoint, John-Paul Robinson, 11/10/2014
- Re: [perfsonar-user] Tests only work unidirectionally with some endpoint, John-Paul Robinson, 11/07/2014
Archive powered by MHonArc 2.6.16.