hey jpr,
I am seeing the same issue on a BW node that I maintain, but it
affects multiple tests. The error condition hangs around for several
days, clears itself up for a day or two, and then reverts back to
the error condition with the cycle continuing. 3.4.2 didn't solve
this for me either, and my duty cycle is very low, (3%). If anyone
has a recommendation, it would be greatly appreciated.
Below is an example:
.
.
.
[ 15] 16.0-17.0 sec 204602400 Bytes 1636819200 bits/sec
[ 15] 17.0-18.0 sec 228125160 Bytes 1825001280 bits/sec
[ 15] 18.0-19.0 sec 251061480 Bytes 2008491840 bits/sec
[ 15] 19.0-20.0 sec 287225280 Bytes 2297802240 bits/sec
[ 15] 0.0-20.1 sec 5173280768 Bytes 2063613329 bits/sec
[ 15] MSS size 1448 bytes (MTU 1500 bytes, ethernet)
bwctl: stop_exec: 3634702618.013936
2015/04/03 13:47:36 (23455) ERROR>
MeasurementArchiveChild.pm:125
perfSONAR_PS::RegularTesting::Master::MeasurementArchiveChild::__ANON__
- Problem handling test results: Problem storing results: Error
writing metadata: Error running test to clpk-ps.maxgigapop.net
with output bwctl: exec_line: iperf -B clpk-ps.maxgigapop.net -s
-f b -P 1 -m -p 5101 -t 20 -i 1
This results in missing data (red dots) in the various graphs with
labels on the order of:
'Error from bwctl/iperf3: error: unable to connect to server -
Connection refused.
Thanks, --Paul W.
---
Scientific Linux release 6.4 (Carbon)
Linux pfs3.icecube.wisc.edu
2.6.32-504.12.2.el6.aufs.web100.x86_64 #1 SMP Fri Mar 13 16:14:56
PDT 2015 x86_64 x86_64 x86_64 GNU/Linux
Dell r710
On 3/25/15 10:37 AM, John-Paul Robinson wrote:
Hi Folks,
I had hoped the 3.4.2 release would include a fix for the parsing bug
that affects the capture of results from tests with Internet2 and other
bwctl nodes running older iperf releases. This affects presentation of
bi-directional throughput in the the Throughput/Latency Graphs:
https://code.google.com/p/perfsonar-ps/issues/detail?id=1021
The regular_testing.log messages suggest the code now recognizes the
error but the outcome is still that the collected data doesn't get
inserted into the database:
2015/03/25 10:29:57 (2320) ERROR> MeasurementArchiveChild.pm:125
perfSONAR_PS::RegularTesting::Master::MeasurementArchiveChild::__ANON__
- Pro
blem handling test results: Problem storing results: Error writing
metadata: Error running test to bwctl.atla.net.internet2.edu with output b
wctl: exec_line: /usr/bin/iperf -B 64.57.16.66 -s -f b -P 1 -m -p 5017
-t 20 -i 1
Any thoughts on how best to resolve this?
Thanks,
~jpr
On 11/13/2014 12:47 PM, Aaron Brown wrote:
Hey John-Paul,
The “local” side, unfortunately, does not provide the data we need, because the sending side of iperf can be nearly worthless on a lossy network, with a large window size. I’m guessing there are few bwctl 1.4 instances out there, but it’d probably be best to modify the RegularTesting parsing to handle the old bwctl instances.
Cheers,
Aaron
On Nov 11, 2014, at 2:38 PM, John-Paul Robinson wrote:
It may also be possible to change the perfSONAR parsers to request and
process the "local" side of the bwctl result since it runs the newer
tools that will produce the complete header.
All automated tests defined via the web appear to have localhost as one
end of the connection. If the bwctl command that negotiates the
scheduled tests had the -x option added it would record data from both
the sending and receiving bwcltd it contacts. The localhost bwctld will
produce the correct header. If the Bwctl/iperf parser code were to read
the appropriate output file it would be ensured data in a format that
can be parsed.
The other option would be to do something fancier for iperf result
parsing to support older end points.
~jpr
On 11/11/2014 11:05 AM, wrote:
Hi all,
Is it possible to trap this problem and flag it in a more visible way?
________________________________________
From: Jason Zurawski []
Sent: 11 November 2014 17:00
To: John-Paul Robinson
Cc:
Subject: Re: [perfsonar-user] Tests only work unidirectionally with some endpoint
Howdy John-Paul;
So I believe there are two issues here. I can offer the technical solution, which may be superseded by Layer8/9 issues that others (from Internet2 or the Global NOC) will need to speak to:
- The BWCTL server software on machine in question (bwctl.atla.net.internet2.edu) is old, and not behaving in a way consistent with what the new BWCTL on version 3.4 of perfSONAR would expect. I can’t say for certain, but I would guess its 1.4.x vs 1.5.y.
- The BWCTL server is not a part of the new list related to the Internet2 Performance Service that support ad-hoc testing (http://www.internet2.edu/products-services/performance-analytics/performance-assurance-service/#features-adhoc). This appears to be an orphaned server that needs some TLC, or to be put out of its misery.
I would suggest maybe trying one of the other ad-hoc servers in the meantime, or you can follow the procedure to be granted a golden ticket to test to the other resources that are not public.
Hope this helps;
Thanks;
-jason
On Nov 11, 2014, at 10:33 AM, John-Paul Robinson wrote:
I've traced this down to a problem resulting from the bwctld on bwctl.atla.net.internet2.edu not producing the correct metadata in its output to clients.
The output header should look something like this:
bwctl: start_endpoint: 3624691387.449529
bwctl: run_endpoint: receiver: 64.57.16.66
bwctl: run_endpoint: sender: 138.26.220.65
bwctl: exec_line: iperf -c 64.57.16.66 -B 138.26.220.65 -f b -P 1 -m -p 5014 -t 20 -i 1.000000
bwctl: run_tool: tester: iperf
bwctl: run_tool: receiver: 64.57.16.66
bwctl: run_tool: sender: 138.26.220.65
bwctl: start_tool: 3624691432.432794
But the problematic node is only producing a metadata header that contains:
bwctl: exec_line: /usr/bin/iperf -B 64.57.16.66 -s -f b -P 1 -m -p 5014 -t 20 -i 1
bwctl: start_tool: 3624691430.401291
This causes the perfSONAR scripts to miss-parse the results, failing on the $tool assignment because it doesn't match "bwctl: exec_line: iperf". I changed that pattern to accept full paths and then the basic result parse succeeds but, due to missing source data, cannot be recorded in the tests database.
I'm not sure why this node is behaving this way.
You can reproduce this behavior by running the following tests. Both the send and recv files should include the complete header, however, when bwctl.atla.net.internet2.edu returns data it produces an incomplete header which then causes the failed parsing by perfSONAR.
Test with bwctl.atla as receiver produces the borked header:
/usr/bin/bwctl -s perfsonar-10g-scidmz.uabgrid.uab.edu -c bwctl.atla.net.internet2.edu -i 1 --parsable -4 -T iperf -p -d ./ -v -P 1 -t 20 -x
$ head 15567942049171924331.recv.bw
bwctl: exec_line: /usr/bin/iperf -B 64.57.16.66 -s -f b -P 1 -m -p 5015 -t 20 -i 1
bwctl: start_tool: 3624693965.158426
------------------------------------------------------------
Server listening on TCP port 5015
Binding to local address 64.57.16.66
TCP window size: 87380 Byte (default)
------------------------------------------------------------
[ 15] local 64.57.16.66 port 5015 connected with 138.26.220.65 port 49750
[ ID] Interval Transfer Bandwidth
[ 15] 0.0- 1.0 sec 62630432 Bytes 501043456 bits/sec
Tests with working node as receiver produces a legit header:
/usr/bin/bwctl -s perfsonar-10g-scidmz.uabgrid.uab.edu -c 56m-ps.sox.net -i 1 --parsable -4 -T iperf -p -d ./ -v -P 1 -t 20 -x
$ head 15567942589368762368.recv.bw
bwctl: start_endpoint: 3624694082.873576
bwctl: run_endpoint: receiver: 143.215.194.125
bwctl: run_endpoint: sender: 138.26.220.65
bwctl: exec_line: iperf -B 143.215.194.125 -s -f b -P 1 -m -p 5128 -t 20 -i 1.000000
bwctl: run_tool: tester: iperf
bwctl: run_tool: receiver: 143.215.194.125
bwctl: run_tool: sender: 138.26.220.65
bwctl: start_tool: 3624694091.000167
------------------------------------------------------------
Server listening on TCP port 5128
I've contacted the maintainers of the failed node but thought the debug info might help others determine if their problematic one-way target nodes are behaving similarly.
~jpr
On 11/10/2014 04:58 PM, John-Paul Robinson wrote:
While the cause of the problem does seem to be a failure to parse the iperf output, the source of the problem doesn't appear to be Bwctl.pm but rather the bwctl process itself.
When I run bwctl against a known working end point:
/usr/bin/bwctl -c 56m-ps.sox.net -i 1 --parsable -4 -T iperf -p -d ./soxtest/ -v -P 1 -t 20 -I 300
The results file created contains the "locally" gather bwctl results which includes the correctly parsable header that describes the metadata for the test:
$ cat soxtest/15567672641991999488.recv.bw
bwctl: start_endpoint: 3624631187.210311
bwctl: run_endpoint: receiver: 143.215.194.125
bwctl: run_endpoint: sender: 138.26.220.65
bwctl: exec_line: iperf -B 143.215.194.125 -s -f b -P 1 -m -p 5045 -t 20 -i 1.000000
bwctl: run_tool: tester: iperf
bwctl: run_tool: receiver: 143.215.194.125
bwctl: run_tool: sender: 138.26.220.65
bwctl: start_tool: 3624631239.004069
However, when I run an identical test against the problematic endpoint:
/usr/bin/bwctl -c 64.57.16.66 -T iperf -4 -I 900 -p -d i2atla-iperfonly -v -P 1 -t 20 -i 1 --parsable
The bwctl result file contains the "remotely" gather results which has insufficient metadata for correct parsing:
$ cat i2atla-iperfonly/15567745226308775992.recv.bw
bwctl: exec_line: /usr/bin/iperf -B 64.57.16.66 -s -f b -P 1 -m -p 5009 -t 20 -i 1
bwctl: start_tool: 3624648139.253983
------------------------------------------------------------
Server listening on TCP port 5009
Binding to local address 64.57.16.66
TCP window size: 87380 Byte (default)
------------------------------------------------------------
It's actually the first line with the /usr/bin/iperf path that causes the unparsable errors reported earlier in Bwctl.pm (below). I changed the pattern match thinking this was the bug but although the data is parsed it then failes to be added to the cassandra data base because of a missing source address (since there isn't one on the iperf server side). stracing the bwctl process shows that the parsable local data is recieved but is not written to the test output file.
It's not clear why bwctl would have this odd behavior for just this one host. It seems almost as if the internal stdout/stderr channels are somehow crossed. If bwctl wrote the correct, locally generated output it would have the necessary metadata header and would parse with the existing Bwctl.pm code.
Any thoughts on what's going on here?
~jpr
On 11/10/2014 01:31 PM, John-Paul Robinson wrote:
Hi Folks,
From what I can tell the problem my tests are running into has to do with having iperf results instead of iperf3 results. The host that is failing negotiates tests with iperf. All my other test endpoints negotiate tests with iperf3.
It appears there is some sort of parsing error in the parse_bwctl_output() routine. The error in the regular_testing.log relates to the $tool variable not being set which eventually bubbles up to full parsing error (see the log below in my previous email).
The code segment where $tool would be set looks like this:
elsif ($line =~ /bwctl: exec_line: iperf/) {
$results{tool} = "iperf";
}
elsif ($line =~ /bwctl: exec_line: nuttcp/) {
$results{tool} = "nuttcp";
}
elsif ($line =~ /bwctl: run_tool: tester: (.*)/) {
$results{tool} = $1;
It appears when the test is an "iperf" test it should parse the line "bwctl: exec_line: iperf". For whatever reason the value for tool doesn't get set and that triggers the subsequent warnings and eventual parse failure.
I suspect this is why the src-to-dst test is failing and why I'm not seeing the results for these tests in my graphs.
~jpr
On 11/07/2014 10:22 AM, John-Paul Robinson wrote:
I'm seeing something similar with one of my tests. It works bi-directionally on the command line but the automated web test is no longer working in the local-to-remote direction.
I've poked around the regular testing log some and suspect there may be some empty data value borking the test results by causing perl to barf on the test results.
Here is the log for the failed local-to-remote part of the tests. Note that the test seems to complete given that the gap in the log entry time matches the reported wait time:
2014/11/07 00:00:21 (13317) DEBUG> Cmd.pm:130 perfSONAR_PS::RegularTesting::Utils::CmdRunner::Cmd::readlines - Lines: $VAR1 = [
'bwctl: Using 138.26.220.65 as the address for local sender',
'bwctl: Using 64.57.16.66 as the address for remote receiver',
'bwctl: Available in-common: iperf nuttcp ',
'bwctl: Using tool: iperf'
];
2014/11/07 00:00:21 (13317) DEBUG> Cmd.pm:130 perfSONAR_PS::RegularTesting::Utils::CmdRunner::Cmd::readlines - Lines: $VAR1 = [
'bwctl: Server \'64.57.16.66\' accepted test request at time 1415340026.515434'
];
2014/11/07 00:00:21 (13317) DEBUG> Cmd.pm:130 perfSONAR_PS::RegularTesting::Utils::CmdRunner::Cmd::readlines - Lines: $VAR1 = [
'bwctl: Client \'localhost\' accepted test request at time 1415340026.515434'
];
2014/11/07 00:00:21 (13317) DEBUG> Cmd.pm:130 perfSONAR_PS::RegularTesting::Utils::CmdRunner::Cmd::readlines - Lines: $VAR1 = [
'bwctl: 26 seconds until test results available'
];
2014/11/07 00:00:47 (13317) DEBUG> Cmd.pm:130 perfSONAR_PS::RegularTesting::Utils::CmdRunner::Cmd::readlines - Lines: $VAR1 = [
'/var/lib/perfsonar/regular_testing/bwctl_Du5FB/15566373779833847474.recv.bw'
];
2014/11/07 00:00:47 (13317) WARN> daemon:103 main::__ANON__ - Warned: Use of uninitialized value $tool in string eq at /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Parsers/Bwctl.pm line 117.
2014/11/07 00:00:47 (13317) WARN> daemon:103 main::__ANON__ - Warned: Use of uninitialized value $tool in string eq at /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Parsers/Bwctl.pm line 120.
2014/11/07 00:00:47 (13317) WARN> daemon:103 main::__ANON__ - Warned: Use of uninitialized value $tool in string eq at /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Parsers/Bwctl.pm line 123.
2014/11/07 00:00:47 (13317) WARN> daemon:103 main::__ANON__ - Warned: Use of uninitialized value $tool in string eq at /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Parsers/Bwctl.pm line 126.
2014/11/07 00:00:47 (13317) WARN> daemon:103 main::__ANON__ - Warned: Use of uninitialized value $tool in string eq at /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Parsers/Bwctl.pm line 129.
2014/11/07 00:00:47 (13317) WARN> daemon:103 main::__ANON__ - Warned: Use of uninitialized value $tool in string eq at /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Parsers/Bwctl.pm line 132.
2014/11/07 00:00:47 (13317) WARN> daemon:103 main::__ANON__ - Warned: Use of uninitialized value $tool in concatenation (.) or string at /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Parsers/Bwctl.pm line 136.
2014/11/07 00:00:47 (13317) ERROR> CmdRunner.pm:110 perfSONAR_PS::RegularTesting::Utils::CmdRunner::run - Problem with results callback: Attribute (tool) does not pass the type constraint because: Validation failed for 'Str' with value undef at /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Tests/Bwctl.pm line 119
perfSONAR_PS::RegularTesting::Tests::Bwctl::__ANON__('perfSONAR_PS::RegularTesting::Tests::Bwctl=HASH(0x46501d8)', 'HASH(0x4656c38)') called at /usr/lib64/perl5/vendor_perl/Moose/Meta/Method/Overridden.pm line 36
perfSONAR_PS::RegularTesting::Tests::Bwctl::build_results('perfSONAR_PS::RegularTesting::Tests::Bwctl=HASH(0x46501d8)', 'HASH(0x4656c38)') called at /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Tests/BwctlBase.pm line 237
perfSONAR_PS::RegularTesting::Tests::BwctlBase::handle_output('perfSONAR_PS::RegularTesting::Tests::Bwctl=HASH(0x46501d8)', 'HASH(0x4880ad0)') called at /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Tests/BwctlBase.pm line 180
perfSONAR_PS::RegularTesting::Tests::BwctlBase::__ANON__('perfSONAR_PS::RegularTesting::Utils::CmdRunner::Cmd=HASH(0x47...', 'HASH(0x46a2850)') called at /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Utils/CmdRunner.pm line 108
eval {...} called at /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Utils/CmdRunner.pm line 107
perfSONAR_PS::RegularTesting::Utils::CmdRunner::run('perfSONAR_PS::RegularTesting::Utils::CmdRunner=HASH(0x478afc0)') called at /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Tests/BwctlBase.pm line 188
perfSONAR_PS::RegularTesting::Tests::BwctlBase::__ANON__('perfSONAR_PS::RegularTesting::Tests::Bwctl=HASH(0x46501d8)', 'HASH(0x4755cd8)') called at /usr/lib64/perl5/vendor_perl/Moose/Meta/Method/Overridden.pm line 36
perfSONAR_PS::RegularTesting::Tests::BwctlBase::run_test('perfSONAR_PS::RegularTesting::Tests::Bwctl=HASH(0x46501d8)', 'HASH(0x4755cd8)') called at /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Test.pm line 75
perfSONAR_PS::RegularTesting::Test::run_test('perfSONAR_PS::RegularTesting::Test=HASH(0x465ed40)', 'handle_results', 'CODE(0x1415c78)') called at /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Master/SelfScheduledTestChild.pm line 50
eval {...} called at /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Master/SelfScheduledTestChild.pm line 40
perfSONAR_PS::RegularTesting::Master::SelfScheduledTestChild::__ANON__('perfSONAR_PS::RegularTesting::Master::SelfScheduledTestChild=...') called at /usr/lib64/perl5/vendor_perl/Moose/Meta/Method/Overridden.pm line 36
perfSONAR_PS::RegularTesting::Master::SelfScheduledTestChild::child_main_loop('perfSONAR_PS::RegularTesting::Master::SelfScheduledTestChild=...') called at /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Master/BaseChild.pm line 32
perfSONAR_PS::RegularTesting::Master::BaseChild::run('perfSONAR_PS::RegularTesting::Master::SelfScheduledTestChild=...') called at /opt/perfsonar_ps/regular_testing/bin/../lib/perfSONAR_PS/RegularTesting/Master.pm line 182
perfSONAR_PS::RegularTesting::Master::run('perfSONAR_PS::RegularTesting::Master=HASH(0x4671bb8)') called at /opt/perfsonar_ps/regular_testing/bin/daemon line 122
2014/11/07 00:00:47 (13317) DEBUG> Cmd.pm:130 perfSONAR_PS::RegularTesting::Utils::CmdRunner::Cmd::readlines - Lines: $VAR1 = [
'bwctl: 913 seconds until next testing period'
];
Here is the corresponding part of the successful test for the remote-to-local part of the test. It runs the test, gets the results, parses them and sticks them in the database (given that I see them in the graphs):
2014/11/07 00:01:27 (13317) DEBUG> Cmd.pm:130 perfSONAR_PS::RegularTesting::Utils::CmdRunner::Cmd::readlines - Lines: $VAR1 = [
'bwctl: Using bwctl.atla.net.internet2.edu as the address for remote sender',
'bwctl: Using 138.26.220.65 as the address for local receiver',
'bwctl: Available in-common: iperf nuttcp ',
'bwctl: Using tool: iperf'
];
2014/11/07 00:01:27 (13317) DEBUG> Cmd.pm:130 perfSONAR_PS::RegularTesting::Utils::CmdRunner::Cmd::readlines - Lines: $VAR1 = [
'bwctl: Server \'localhost\' accepted test request at time 1415340091.510944'
];
2014/11/07 00:01:27 (13317) DEBUG> Cmd.pm:130 perfSONAR_PS::RegularTesting::Utils::CmdRunner::Cmd::readlines - Lines: $VAR1 = [
'bwctl: Client \'bwctl.atla.net.internet2.edu\' accepted test request at time 1415340091.510944'
];
2014/11/07 00:01:27 (13317) DEBUG> Cmd.pm:130 perfSONAR_PS::RegularTesting::Utils::CmdRunner::Cmd::readlines - Lines: $VAR1 = [
'bwctl: 25 seconds until test results available'
];
2014/11/07 00:01:52 (13317) DEBUG> Cmd.pm:130 perfSONAR_PS::RegularTesting::Utils::CmdRunner::Cmd::readlines - Lines: $VAR1 = [
'/var/lib/perfsonar/regular_testing/bwctl_BkMZ9/15566374058987434689.recv.bw'
];
2014/11/07 00:01:52 (13317) DEBUG> Bwctl.pm:138 perfSONAR_PS::RegularTesting::Tests::Bwctl::__ANON__ - Build Results: $VAR1 = {
'summary_results' => {
'summary_results' => {
'throughput' => '4480866964'
},
'streams' => [
{
'throughput' => '4480866964',
'stream_id' => '15'
}
]
},
'protocol' => 'tcp',
'source' => {
'protocol' => 'tcp',
'address' => '64.57.16.66',
'hostname' => 'bwctl.atla.net.internet2.edu'
},
'errors' => [],
'destination' => {
'protocol' => 'tcp',
'address' => '138.26.220.65'
},
'raw_results' => 'bwctl: start_endpoint: 3624328887.156787
bwctl: run_endpoint: receiver: 138.26.220.65
bwctl: run_endpoint: sender: 64.57.16.66
bwctl: exec_line: iperf -B 138.26.220.65 -s -f b -P 1 -m -p 5182 -t 20 -i 1.000000
bwctl: run_tool: tester: iperf
bwctl: run_tool: receiver: 138.26.220.65
bwctl: run_tool: sender: 64.57.16.66
bwctl: start_tool: 3624328889.822644
------------------------------------------------------------
Server listening on TCP port 5182
Binding to local address 138.26.220.65
TCP window size: 87380 Byte (default)
------------------------------------------------------------
[ 15] local 138.26.220.65 port 5182 connected with 64.57.16.66 port 5182
[ ID] Interval Transfer Bandwidth
[ 15] 0.0- 1.0 sec 278847688 Bytes 2230781504 bits/sec
[ 15] 1.0- 2.0 sec 574872044 Bytes 4598976352 bits/sec
[ 15] 2.0- 3.0 sec 574443704 Bytes 4595549632 bits/sec
[ 15] 3.0- 4.0 sec 575490620 Bytes 4603924960 bits/sec
[ 15] 4.0- 5.0 sec 573504164 Bytes 4588033312 bits/sec
[ 15] 5.0- 6.0 sec 576805976 Bytes 4614447808 bits/sec
[ 15] 6.0- 7.0 sec 575192188 Bytes 4601537504 bits/sec
[ 15] 7.0- 8.0 sec 575457480 Bytes 4603659840 bits/sec
[ 15] 8.0- 9.0 sec 574730536 Bytes 4597844288 bits/sec
[ 15] 9.0-10.0 sec 573548904 Bytes 4588391232 bits/sec
[ 15] 10.0-11.0 sec 574434756 Bytes 4595478048 bits/sec
[ 15] 11.0-12.0 sec 575257476 Bytes 4602059808 bits/sec
[ 15] 12.0-13.0 sec 575052664 Bytes 4600421312 bits/sec
[ 15] 13.0-14.0 sec 575186388 Bytes 4601491104 bits/sec
[ 15] 14.0-15.0 sec 575061116 Bytes 4600488928 bits/sec
[ 15] 15.0-16.0 sec 575683832 Bytes 4605470656 bits/sec
[ 15] 16.0-17.0 sec 575449524 Bytes 4603596192 bits/sec
[ 15] 17.0-18.0 sec 574237900 Bytes 4593903200 bits/sec
[ 15] 18.0-19.0 sec 574121576 Bytes 4592972608 bits/sec
[ 15] 19.0-20.0 sec 574765336 Bytes 4598122688 bits/sec
[ 15] 0.0-20.0 sec 11209015296 Bytes 4480866964 bits/sec
[ 15] MSS size 8948 bytes (MTU 8988 bytes, unknown interface)
bwctl: stop_tool: 3624328911.581018
bwctl: stop_endpoint: 3624328912.581453
',
'start_time' => '2014-11-07T06:01:29',
'end_time' => '2014-11-07T06:01:51',
'tool' => 'iperf',
'streams' => 1,
'type' => 'throughput',
'intervals' => [
... and so forth
This gives me some confidence that there is a problem with the local-to-remote.
~jpr
On 11/04/2014 08:53 AM, Jarvis Zhang wrote:
Hi,
I set several tests on my host, but some tests only work unidirectionally. For throughput
This problem is a little complicate. Here is my address: http://115.25.138.244/serviceTest/psGraph.cgi
The throughput test works on command line but fails on web interface.
The one-way delay test doesn’t work even on command line; I think maybe the problem has something to do with the port, but I have no idea where to start with.
The ping delay test never work.
Attachment is regular_testing.log. Here is more details:
1. For throughput test:
a) 115.25.138.244(my host) to 128.104.100.67(komatsu) stops working since I update from v3.3.2 to 3.4 yesterday. but the command line still works.
[zwzhang@jsi-inspur02 perfsonar]$ bwctl -c 128.104.100.67
bwctl: Using tool: iperf
bwctl: 27 seconds until test results available
RECEIVER START
bwctl: exec_line: iperf -B komatsu.chtc.wisc.edu -s -f b -m -p 5019 -t 10
bwctl: start_tool: 3624097458.082252
------------------------------------------------------------
Server listening on TCP port 5019
Binding to local address komatsu.chtc.wisc.edu
TCP window size: 87380 Byte (default)
------------------------------------------------------------
[ 14] local 128.104.100.67 port 5019 connected with 115.25.138.244 port 52456
[ ID] Interval Transfer Bandwidth
[ 14] 0.0-11.5 sec 36438016 Bytes 25453583 bits/sec
[ 14] MSS size 1436 bytes (MTU 1500 bytes, ethernet)
bwctl: stop_exec: 3624097473.816542
RECEIVER END
b) 2001:da8:203:d406:16da:e9ff:fef9:b68f(my host) to 2607:f388:107c:501:202:c9ff:fee8:85a0(komatsu) never works, but the command line still works, as above.
c) 159.226.15.235(cnic) to 115.25.138.244(my host) stops working since I update from v3.3.2 to 3.4 yesterday. so does the command line.
[zwzhang@jsi-inspur02 perfsonar]$ bwctl -s 159.226.15.235
bwctl: Using tool: iperf
bwctl: 20 seconds until test results available
RECEIVER START
bwctl: remote peer cancelled test
RECEIVER END
d) 2400:dd01:1011:1:92b1:1cff:fe0c:7c0a(cnic) to 2001:da8:203:d406:16da:e9ff:fef9:b68f(my host) stops working since I update from v3.3.2 to 3.4 yesterday. so does the command line, as above.
e) 115.25.138.244 to 137.110.119.113 works well bidirectionally.
2. For one-way latency
a) 128.104.100.67(komatsu) to 115.25.138.244(my host) doesn’t work, so does the command line.
--- owping statistics from [128.104.100.67]:8944 to [jsi-inspur02]:8841 ---
SID: 73198af4d8035c80a36f5244c1e63df9
first: 2014-11-04T22:09:06.866
last: 2014-11-04T22:09:17.026
100 sent, 100 lost (100.000%), 0 duplicates
one-way delay min/median/max = nan/nan/nan ms, (err=0.104 ms)
one-way jitter = nan ms (P95-P50)
TTL not reported
no reordering
b)159.226.15.235(cnic) to 115.25.138.244(my host) doesn’t work , so does the command line.
c) 137.110.119.113(murpa) doesn’t work bidirectionally.
d) Other ipv6 addresses work well.
3. All ping latency doesn’t work.
Any insights appreciated.
Thanks,
Jarvis
|
