Subject: NDT-DEV email list created
[ndt-dev] Issue 83 in ndt: NDT reports erroneous results with 10G interface
- Subject: [ndt-dev] Issue 83 in ndt: NDT reports erroneous results with 10G interface
- Date: Mon, 17 Jun 2013 17:51:10 +0000
- Authentication-results: sfpop-ironport02.merit.edu; dkim=neutral (message not signed) header.i=none
Labels: Type-Defect Severity-Medium
New issue 83 by : NDT reports erroneous results with 10G interface
This may already be fixed in trunk. Basically boils own to an issue in test_s2c_clt.c line 32 where a uint32_t should be uint64_t. See below for full details:
To: Nat Stoddard
Subject: Re: Using NDT with 10 gigabit interfaces
This is good. As I thought, the CPU isn't the bottleneck.
The connection is sender limited, meaning its' not network or receiver
limited. The send/receive buffers don't limit the throughput.
The pkt-pair timing say OC-48 to 10 GE so that's good.
So lets look at the raw numbers.
CurMSS = 1448 (normal Ethernet frame)
Data Pkts Out = 7873701
Data Bytes Out = 2,147,483,647
Timesec = 10.00 sec
This is suspicious.
Bytes sent should be: 7,873,701 x 1448 = 11,401,119,048
Speed calculates to 9.1 Gbps So this is what the server acheived, but
not what is logged by the client.
But the server report 2.2 GB which converts to 1.7 Gbps. which is what
the spd: variable says.
I wonder if this is just a variable overflow error. The DataBytesOut
variable is a 32 bit counter. There is an HCDataOctetsOut variable (a
64 bit counter)
So, this is a problem that may impact some of the analysis messages.
line 1082 in web100srv.c sets the variable spd based on the DataBytesOut
value. Either the code should grab the HCDataOctetsOut value or a test
should be made and if s2cspd is greater that something like 5 Gbps, use
the HC value otherwise use the existing value.
I'm looking at the code and it show the server is able to right log
messages that might help here. Restart the server with 1 or 2 debug
flags (-d or -dd) and post the output here.
Got it !!!!!! While the DataByteOut vs HCDataOctetOut issue noted above
is a potential bug, it is not causing this bug.
The file test_s2c_clt.c is the culprit. The data read subroutine
declares the byte counter as an uint32_t. This is too small for the
amount of data we are getting at 10 Gbps, and it is overflowing.
Changing that to a uint64_t variable should fix this problem. The
server uses double, so it doesn't overflow.
So change line 32 of test_s2c_clt.c s/uint32_t/uint64_t
and recompile. See if that fixes this bug.
Same with the Java client, line 1253 of the Tcpbw100.java file declares
bytes as an int, this should be a double to prevent counter overflows at
On 05/16/2013 01:30 PM, Nat Stoddard wrote:
Thank you for taking notice of this.
1. I notice this behavior with both the web and the command line
clients. I only have the command line client to refer to when I am
testing between my two NDT servers. I get numbers very close to the
same in both directions:
server1 to server2 outbound 8517.57 Mb/s, inbound 2164.96 Mb/s
server2 to server1 outbound 9101.33 Mb/s, inbound 2245.74 Mb/s
2. Top reports the CPU load on the server as high as 49%. The client
goes up to 45%.
3. I have pasted the web100clt -ll output below:
$ web100clt -n lblnet-test.lbl.gov -ll
Testing network path for configuration and performance problems --
Using IPv4 address
Checking for Middleboxes . . . . . . . . . . . . . . . . . . Done
checking for firewalls . . . . . . . . . . . . . . . . . . . Done
running 10s outbound test (client to server) . . . . . 9101.33 Mb/s
running 10s inbound test (server to client) . . . . . . 2245.74 Mb/s
The slowest link in the end-to-end path is a 2.4 Gbps OC-48 subnet
Information [S2C]: Packet queuing detected: 75.37% (remote buffers)
Server 'lblnet-test.lbl.gov' is not behind a firewall. [Connection to
the ephemeral port was successful]
Client is not behind a firewall. [Connection to the ephemeral port was
------ Web100 Detailed Analysis ------
Web100 reports the Round trip time = 10.54 msec;the Packet size = 1448
No packet loss was observed.
This connection is receiver limited 1.67% of the time.
This connection is sender limited 98.30% of the time.
Web100 reports TCP negotiated the optional Performance Settings to:
RFC 2018 Selective Acknowledgment: ON
RFC 896 Nagle Algorithm: ON
RFC 3168 Explicit Congestion Notification: OFF
RFC 1323 Time Stamping: ON
RFC 1323 Window Scaling: ON; Scaling Factors - Server=10, Client=10
The theoretical network limit is 104855.13 Mbps
The NDT server has a 16384 KByte buffer which limits the throughput to
Your PC/Workstation has a 12269 KByte buffer which limits the throughput
to 9097.53 Mbps
The network based flow control limits the throughput to 13645.63 Mbps
Client Data reports link is ' 8', Client Acks report link is ' 9'
Server Data reports link is ' 9', Server Acks report link is ' 9'
Packet size is preserved End-to-End
Server IP addresses are preserved End-to-End
Client IP addresses are preserved End-to-End
You received this message because this project is configured to send all issue notifications to this address.
You may adjust your notification preferences at:
- [ndt-dev] Issue 83 in ndt: NDT reports erroneous results with 10G interface, ndt, 06/17/2013
Archive powered by MHonArc 2.6.16.