Skip to Content.
Sympa Menu

perfsonar-user - [perfsonar-user] Different results from iperf3 vs. bwctl -T iperf3

Subject: perfSONAR User Q&A and Other Discussion

List archive

[perfsonar-user] Different results from iperf3 vs. bwctl -T iperf3


Chronological Thread 
  • From: Brian Candler <>
  • To: "" <>
  • Subject: [perfsonar-user] Different results from iperf3 vs. bwctl -T iperf3
  • Date: Thu, 04 Dec 2014 15:07:37 +0000
  • Domainkey-signature: a=rsa-sha1; c=nofws; d=pobox.com; h=message-id:date :from:mime-version:to:subject:content-type :content-transfer-encoding; q=dns; s=sasl; b=GzoS4STrWYWspAIiR8R sgM9u1MkwWBBG61ucmZewCK2iOSrScd+myTTGjH39DqUVUs0ZrSAklHcD7yQjuNk 5c29n9JLSKrjtaZi2uv2Q9WpvVMMCm7KMdgWMBhmbPCpNzHZ0In/J+Sc6GDa1klc NI3VT5cPJbC17/ujQuxQo5EE=

I have two perfsonar 3.4.1 boxes set up next to each other. They are HP Proliant 726042-425 with quad-core E3-1220 v3 @ 3.1GHz, 8GB RAM, 10G Intel X520-DA2 NIC connected back-to-back with an SFP+ direct attach cable.

For reference below, the hostnames are "cov-perf-1" and "pb-perf-1".

If I run iperf3 directly from the command line, I get good and consistent results:

[on cov-perf-1] iperf3 -s

[on pb-perf-1] $ iperf3 -c cov-perf-1
Connecting to host cov-perf-1, port 5201
[ 4] local 192.0.2.125 port 52124 connected to 192.0.2.254 port 5201
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 4] 0.00-1.00 sec 787 MBytes 6.60 Gbits/sec 0 587 KBytes
[ 4] 1.00-2.00 sec 1.01 GBytes 8.64 Gbits/sec 0 675 KBytes
[ 4] 2.00-3.00 sec 1.01 GBytes 8.66 Gbits/sec 0 741 KBytes
[ 4] 3.00-4.00 sec 1.01 GBytes 8.65 Gbits/sec 0 776 KBytes
[ 4] 4.00-5.00 sec 1.01 GBytes 8.65 Gbits/sec 0 827 KBytes
[ 4] 5.00-6.00 sec 1.01 GBytes 8.65 Gbits/sec 0 865 KBytes
[ 4] 6.00-7.00 sec 1.01 GBytes 8.65 Gbits/sec 0 904 KBytes
[ 4] 7.00-8.00 sec 1.01 GBytes 8.66 Gbits/sec 0 928 KBytes
[ 4] 8.00-9.00 sec 1.01 GBytes 8.66 Gbits/sec 0 970 KBytes
[ 4] 9.00-10.00 sec 1.01 GBytes 8.65 Gbits/sec 0 980 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 4] 0.00-10.00 sec 9.83 GBytes 8.45 Gbits/sec 0 sender
[ 4] 0.00-10.00 sec 9.83 GBytes 8.44 Gbits/sec receiver

iperf Done.

However if I run apparently the "same" test with bwctl I get lower and more variable results:

[pb-perf-1] $ bwctl -s cov-perf-1 -T iperf3
bwctl: Using tool: iperf3
bwctl: 19 seconds until test results available

SENDER START
Connecting to host 192.0.2.125, port 5126
[ 16] local 192.0.2.254 port 56340 connected to 192.0.2.125 port 5126
[ ID] Interval Transfer Bandwidth Retr Cwnd
[ 16] 0.00-1.00 sec 694 MBytes 5.82 Gbits/sec 0 532 KBytes
[ 16] 1.00-2.00 sec 821 MBytes 6.89 Gbits/sec 0 576 KBytes
[ 16] 2.00-3.00 sec 759 MBytes 6.36 Gbits/sec 0 614 KBytes
[ 16] 3.00-4.00 sec 830 MBytes 6.96 Gbits/sec 0 655 KBytes
[ 16] 4.00-5.00 sec 710 MBytes 5.96 Gbits/sec 0 721 KBytes
[ 16] 5.00-6.00 sec 811 MBytes 6.80 Gbits/sec 0 754 KBytes
[ 16] 6.00-7.00 sec 955 MBytes 8.00 Gbits/sec 0 799 KBytes
[ 16] 7.00-8.00 sec 827 MBytes 6.94 Gbits/sec 0 813 KBytes
[ 16] 8.00-9.00 sec 712 MBytes 5.97 Gbits/sec 0 837 KBytes
[ 16] 9.00-10.00 sec 811 MBytes 6.81 Gbits/sec 0 846 KBytes
- - - - - - - - - - - - - - - - - - - - - - - - -
[ ID] Interval Transfer Bandwidth Retr
[ 16] 0.00-10.00 sec 7.74 GBytes 6.65 Gbits/sec 0 sender
[ 16] 0.00-10.00 sec 7.74 GBytes 6.65 Gbits/sec receiver

iperf Done.

SENDER END

Furthermore, if I try to run it in the opposite direction, I get no results at all:

[pb-perf-1] $ bwctl -c cov-perf-1 -T iperf3
bwctl: Using tool: iperf3
bwctl: 19 seconds until test results available

SENDER START

SENDER END

$ bwctl -s pb-perf-1 -c cov-perf-1 -T iperf3
bwctl: Using tool: iperf3
bwctl: 19 seconds until test results available

SENDER START

SENDER END

ps shows that three new bwctld processes/threads appear during the test (at both sides), but no iperf/iperf3 processes.

If I use strace -f -p <pid-of-bwctld> I do not see any fork or exec, but two instances of clone(), so maybe these are threads running directly within the bwctld daemon?

"top" shows the process running at nice 0, so that's not the problem.

Can anyone explain why these figures are worse with bwctl than native iperf3, and why the tests don't work in both directions?

Thanks,

Brian Candler.

P.S. I have the default unmodified limits file in place, which has "bandwidth=900m". Even assuming this means 900 Megabits per second rather than 900 millibits per second, clearly I'm achieving more than that (or is this supposed to be a long-term average from a particular IP address, rather than an instantaneous peak?)

The file also contains this comment:

# This example configuration allows Abilene measurement hosts to run
# tests to your server, but will not allow you to run tests to your own
# hosts. This is intentional so you will have to look at this file and
# modifiy it.

But I can see nothing specific to Abilene.



Archive powered by MHonArc 2.6.16.

Top of Page