ndt-users list created

["Hao, Justin C" <>]

Howdy Rich,

Here are the results with the -ll option (web100clt -4 -ll -n x.x.x.x) for 
both directions.  Thanks for looking into this! Please let me know if there's 
anything else you need.  FYI, I am planning on reimaging Server 2 to match 
Server 1 (instead of running perfsonar+ndt 3.4.4a) today.  Should i install 
ndt 3.6.4 on both servers or is 3.6.3 ok?

###Server 1 to Server 2

[jhao@udcc-ndt-1
 ~]$ web100clt -4 -ll -n 10.0.0.2
Testing network path for configuration and performance problems  --  Using 
IPv4 address
Checking for Middleboxes . . . . . . . . . . . . . . . . . .  Done
checking for firewalls . . . . . . . . . . . . . . . . . . .  Done
running 10s outbound test (client to server) . . . . .  9893.83 Mb/s
running 10s inbound test (server to client) . . . . . . 3037.69 Mb/s
Server unable to determine bottleneck link type.
Information [S2C]: Packet queuing detected: 69.36% (local buffers)
Server '10.0.0.2' is not behind a firewall. [Connection to the ephemeral port 
was successful]
Client is not behind a firewall. [Connection to the ephemeral port was 
successful]

        ------  Web100 Detailed Analysis  ------

Web100 reports the Round trip time = 2.84 msec;the Packet size = 8948 Bytes; 
and
There were 5 packets retransmitted, 1128 duplicate acks received, and 3012 
SACK blocks received
Packets arrived out-of-order 0.66% of the time.
This connection is sender limited 99.01% of the time.
This connection is network limited 0.97% of the time.

    Web100 reports TCP negotiated the optional Performance Settings to:
RFC 2018 Selective Acknowledgment: ON
RFC 896 Nagle Algorithm: ON
RFC 3168 Explicit Congestion Notification: OFF
RFC 1323 Time Stamping: ON
RFC 1323 Window Scaling: ON; Scaling Factors - Server=10, Client=13
The theoretical network limit is 12651.09 Mbps
The NDT server has a 32768 KByte buffer which limits the throughput to 
90204.37 Mbps
Your PC/Workstation has a 6144 KByte buffer which limits the throughput to 
16913.32 Mbps
The network based flow control limits the throughput to 10199.30 Mbps

Client Data reports link is ' -1', Client Acks report link is ' -1'
Server Data reports link is ' -1', Server Acks report link is ' -1'
Packet size is preserved End-to-End
Server IP addresses are preserved End-to-End
Client IP addresses are preserved End-to-End
CurMSS: 8948
X_Rcvbuf: 33554432
X_Sndbuf: 33554432
AckPktsIn: 171920
AckPktsOut: 0
BytesRetrans: 44740
CongAvoid: 0
CongestionOverCount: 83
CongestionSignals: 5
CountRTT: 170186
CurCwnd: 3355500
CurRTO: 202
CurRwinRcvd: 6154240
CurRwinSent: 24576
CurSsthresh: 3033372
DSACKDups: 3
DataBytesIn: 0
DataBytesOut: 2147483647
DataPktsIn: 0
DataPktsOut: 1383372
DupAcksIn: 1128
ECNEnabled: 0
FastRetran: 5
MaxCwnd: 3793952
MaxMSS: 8948
MaxRTO: 204
MaxRTT: 4
MaxRwinRcvd: 6291456
MaxRwinSent: 24576
MaxSsthresh: 3033372
MinMSS: 8948
MinRTO: 201
MinRTT: 0
MinRwinRcvd: 4096
MinRwinSent: 17896
NagleEnabled: 1
OtherReductions: 1443
PktsIn: 171920
PktsOut: 1383372
PktsRetrans: 5
RcvWinScale: 13
SACKEnabled: 3
SACKsRcvd: 3012
SendStall: 0
SlowStart: 0
SampleRTT: 2
SmoothedRTT: 2
SndWinScale: 10
SndLimTimeRwin: 2015
SndLimTimeCwnd: 98261
SndLimTimeSender: 10006987
SndLimTransRwin: 2
SndLimTransCwnd: 1260
SndLimTransSender: 1260
SndLimBytesRwin: 175456
SndLimBytesCwnd: 78485880
SndLimBytesSender: 2147483647
SubsequentTimeouts: 0
SumRTT: 483055
Timeouts: 0
TimestampsEnabled: 1
WinScaleRcvd: 10
WinScaleSent: 13
DupAcksOut: 0
StartTimeUsec: 241272
Duration: 10107366
c2sData: -1
c2sAck: -1
s2cData: -1
s2cAck: -1
half_duplex: 0
link: 100
congestion: 0
bad_cable: 0
mismatch: 0
spd: 1699.75
bw: 12651.09
loss: 0.000003614
avgrtt: 2.84
waitsec: 0.00
timesec: 10.00
order: 0.0066
rwintime: 0.0002
sendtime: 0.9901
cwndtime: 0.0097
rwin: 48.0000
swin: 256.0000
cwin: 28.9456
rttsec: 0.002838
Sndbuf: 33554432
aspd: 0.00000
CWND-Limited: 2106.00
minCWNDpeak: 322128
maxCWNDpeak: 3785004
CWNDpeaks: 351

###Server 2 to Server 1

[jhao@udcc-ndt-2
 ~]$ web100clt -4 -ll -n 10.0.0.1
Testing network path for configuration and performance problems  --  Using 
IPv4 address
Checking for Middleboxes . . . . . . . . . . . . . . . . . .  Done
checking for firewalls . . . . . . . . . . . . . . . . . . .  Done
running 10s outbound test (client to server) . . . . .  9898.25 Mb/s
running 10s inbound test (server to client) . . . . . . 2937.24 Mb/s
Server unable to determine bottleneck link type.
Information: Other network traffic is congesting the link
Information [S2C]: Packet queuing detected: 70.06% (remote buffers)
Server '10.0.0.1' is not behind a firewall. [Connection to the ephemeral port 
was successful]
Client is not behind a firewall. [Connection to the ephemeral port was 
successful]

        ------  Web100 Detailed Analysis  ------

Web100 reports the Round trip time = 1.15 msec;the Packet size = 8948 Bytes; 
and
There were 202 packets retransmitted, 4311 duplicate acks received, and 17285 
SACK blocks received
Packets arrived out-of-order 2.48% of the time.
This connection is receiver limited 2.22% of the time.
This connection is sender limited 84.10% of the time.
This connection is network limited 13.68% of the time.

    Web100 reports TCP negotiated the optional Performance Settings to:
RFC 2018 Selective Acknowledgment: ON
RFC 896 Nagle Algorithm: ON
RFC 3168 Explicit Congestion Notification: OFF
RFC 1323 Time Stamping: ON
RFC 1323 Window Scaling: ON; Scaling Factors - Server=10, Client=13
The theoretical network limit is 20977.76 Mbps
The NDT server has a 4362 KByte buffer which limits the throughput to 
29661.10 Mbps
Your PC/Workstation has a 5000 KByte buffer which limits the throughput to 
33996.95 Mbps
The network based flow control limits the throughput to 14021.93 Mbps

Client Data reports link is ' -1', Client Acks report link is ' -1'
Server Data reports link is ' -1', Server Acks report link is ' -1'
Packet size is preserved End-to-End
Server IP addresses are preserved End-to-End
Client IP addresses are preserved End-to-End
CurMSS: 8948
X_Rcvbuf: 87380
X_Sndbuf: 4467008
AckPktsIn: 173792
AckPktsOut: 0
BytesRetrans: 1807496
CongAvoid: 0
CongestionOverCount: 472
CongestionSignals: 11
CountRTT: 168263
CurCwnd: 1798548
CurRTO: 201
CurRwinRcvd: 344064
CurRwinSent: 18432
CurSsthresh: 1583796
DSACKDups: 0
DataBytesIn: 0
DataBytesOut: -612239176
DataPktsIn: 0
DataPktsOut: 1370815
DupAcksIn: 4311
ECNEnabled: 0
FastRetran: 11
MaxCwnd: 2111728
MaxMSS: 8948
MaxRTO: 203
MaxRTT: 4
MaxRwinRcvd: 5120000
MaxRwinSent: 18432
MaxSsthresh: 1583796
MinMSS: 8948
MinRTO: 201
MinRTT: 0
MinRwinRcvd: 24576
MinRwinSent: 17896
NagleEnabled: 1
OtherReductions: 72
PktsIn: 173792
PktsOut: 1370815
PktsRetrans: 202
RcvWinScale: 10
SACKEnabled: 3
SACKsRcvd: 17285
SendStall: 0
SlowStart: 0
SampleRTT: 1
SmoothedRTT: 1
SndWinScale: 13
SndLimTimeRwin: 222848
SndLimTimeCwnd: 1370543
SndLimTimeSender: 8426525
SndLimTransRwin: 3113
SndLimTransCwnd: 16685
SndLimTransSender: 19788
SndLimBytesRwin: 358936868
SndLimBytesCwnd: 2105595904
SndLimBytesSender: 1218195348
SubsequentTimeouts: 0
SumRTT: 193303
Timeouts: 0
TimestampsEnabled: 1
WinScaleRcvd: 13
WinScaleSent: 10
DupAcksOut: 0
StartTimeUsec: 358222
Duration: 10019954
c2sData: -1
c2sAck: -1
s2cData: -1
s2cAck: -1
half_duplex: 0
link: 100
congestion: 1
bad_cable: 0
mismatch: 0
spd: -488.82
bw: 20977.76
loss: 0.000008024
avgrtt: 1.15
waitsec: 0.00
timesec: 10.00
order: 0.0248
rwintime: 0.0222
sendtime: 0.8410
cwndtime: 0.1368
rwin: 39.0625
swin: 34.0806
cwin: 16.1112
rttsec: 0.001149
Sndbuf: 4467008
aspd: 0.00000
CWND-Limited: 666.00
minCWNDpeak: 438452
maxCWNDpeak: 2111728
CWNDpeaks: 14


-----
Justin Hao
CCNA
Network Engineer, ITS Networking
The University of Texas at Austin

-----

On Sep 30, 2010, at 9:23 AM, Rich Carlson wrote:

> Hi Justin;
>
> Please run the tests with the -ll option and post the results to this
> list or just send them to me.  Off the top of my head I don't know why
> the results would be different based on the direction.
>
> Rich
>
> On 9/29/2010 2:59 PM, Hao, Justin C wrote:
>> Howdy Rich,
>>
>> I've got the boxes connected to each other, Server 1 is running ndt 3.6.3 
>> on CentOS 5.5 (2.6.35-web100 kernel) and Server 2 is running Perfsonar 
>> 3.2rc1 (looks like ndt 3.4.4a)
>>
>> I'm still seeing the asynch C2S/S2C values and was wondering if you could 
>> shed any light/point me in the right direction.   I've included snapshots 
>> of the web100clt results for each server, please let me know if you need 
>> additional information. I've configured both servers identically in terms 
>> of sysctl.conf tcp settings as well as txqueuelen of 10000 and MTU of 9000
>>
>> Server 1(10.0.0.1) to Server 2(10.0.0.2)
>> [root@dhcp-135-164
>>  etc]# web100clt -4 -n 10.0.0.2
>> Testing network path for configuration and performance problems  --  Using 
>> IPv4 address
>> Checking for Middleboxes . . . . . . . . . . . . . . . . . .  Done
>> checking for firewalls . . . . . . . . . . . . . . . . . . .  Done
>> running 10s outbound test (client to server) . . . . .  9876.02 Mb/s
>> running 10s inbound test (server to client) . . . . . . 3041.14 Mb/s
>> Server unable to determine bottleneck link type.
>> Information [S2C]: Packet queuing detected: 69.33% (local buffers)
>> Server '10.0.0.2' is not behind a firewall. [Connection to the ephemeral 
>> port was successful]
>> Client is not behind a firewall. [Connection to the ephemeral port was 
>> successful]
>> Packet size is preserved End-to-End
>> Server IP addresses are preserved End-to-End
>> Client IP addresses are preserved End-to-End
>> [root@dhcp-135-164
>>  etc]#
>>
>> Server 2(10.0.0.2) to Server 1(10.0.0.1)
>> [root@localhost
>>  etc]# web100clt -4 -n 10.0.0.1
>> Testing network path for configuration and performance problems  --  Using 
>> IPv4 address
>> Checking for Middleboxes . . . . . . . . . . . . . . . . . .  Done
>> checking for firewalls . . . . . . . . . . . . . . . . . . .  Done
>> running 10s outbound test (client to server) . . . . .  9898.48 Mb/s
>> running 10s inbound test (server to client) . . . . . . 2622.73 Mb/s
>> Server unable to determine bottleneck link type.
>> Information: Other network traffic is congesting the link
>> Information [S2C]: Packet queuing detected: 72.39% (remote buffers)
>> Server '10.0.0.1' is not behind a firewall. [Connection to the ephemeral 
>> port was successful]
>> Client is not behind a firewall. [Connection to the ephemeral port was 
>> successful]
>> Packet size is preserved End-to-End
>> Server IP addresses are preserved End-to-End
>> Client IP addresses are preserved End-to-End
>> [root@localhost
>>  etc]#
>>
>>
>> -----
>> Justin Hao
>> CCNA
>> Network Engineer, ITS Networking
>> The University of Texas at Austin
>> 
>> -----
>>
>> On Sep 29, 2010, at 9:01 AM, Hao, Justin C wrote:
>>
>>> that was step two, i'm going to connect the two servers to each other and 
>>> see what i can see
>>>
>>> -----
>>> Justin Hao
>>> CCNA
>>> Network Engineer, ITS Networking
>>> The University of Texas at Austin
>>> 
>>> -----
>>>
>>> On Sep 29, 2010, at 9:00 AM, Rich Carlson wrote:
>>>
>>>> Justin;
>>>>
>>>> Use the latest JRE, but look at the console service.  On Win based
>>>> machines I've found the upgrade process doesn't remove old version of
>>>> the java console.  You need to manually do this through the control
>>>> panel.  I don't have a linux box handy to see what it does.
>>>>
>>>> In any case, I'd say you aren't really testing the NDT tool on a
>>>> loopback interface, so running the server and client on a single machine
>>>> isn't really going to tell you much.  Bring up a server and use the
>>>> applet to test a number of different clients to see what is going on.
>>>>
>>>> Rich
>>>>
>>>> On 9/29/2010 9:37 AM, Hao, Justin C wrote:
>>>>> Howdy Rich,
>>>>>
>>>>> I've tried with both the command line and the web client, only the web 
>>>>> client has returned the negative values, but the command line client 
>>>>> has also returned a wide range of results. what is the recommended java 
>>>>> RE to use?
>>>>>
>>>>> -----
>>>>> Justin Hao
>>>>> CCNA
>>>>> Network Engineer, ITS Networking
>>>>> The University of Texas at Austin
>>>>> 
>>>>> -----
>>>>>
>>>>> On Sep 29, 2010, at 8:30 AM, Rich Carlson wrote:
>>>>>
>>>>>> Justin;
>>>>>>
>>>>>> The Web100 system uses 32bit counters, so I suspect the negative speeds
>>>>>> come from the use of signed int vars instead of unsigned.
>>>>>>
>>>>>> When testing on the local box, the traffic goes through the loopback
>>>>>> interface (lo0) instead of the NIC.  This means you are testing the OS
>>>>>> and it's memory management system more that anything else.
>>>>>> Changing/setting variables on the ethx interface will have no effect.
>>>>>>
>>>>>> You should not set the tcp_mem value to 16 M.  The value for this
>>>>>> variable is in pages, NOT bytes.  See
>>>>>> Documentation/networking/ip-sysctl.txt doc in the kernel source tree 
>>>>>> for
>>>>>> more details.
>>>>>>
>>>>>> tcp_mem - vector of 3 INTEGERs: min, pressure, max
>>>>>>        min: below this number of pages TCP is not bothered about its
>>>>>>        memory appetite.
>>>>>>
>>>>>>        pressure: when amount of memory allocated by TCP exceeds this
>>>>>> number
>>>>>>        of pages, TCP moderates its memory consumption and enters memory
>>>>>>        pressure mode, which is exited when memory consumption falls
>>>>>>        under "min".
>>>>>>
>>>>>>        max: number of pages allowed for queueing by all TCP sockets.
>>>>>>
>>>>>>        Defaults are calculated at boot time from amount of available
>>>>>>        memory.
>>>>>>
>>>>>>
>>>>>> Are you using the command line client (web100clt) or the Java Applet
>>>>>> (via the browser) to run these tests?  It was recently reported that a
>>>>>> site was getting high variability in the S2C tests with the Java 
>>>>>> client.
>>>>>> It turned out that they had 2 Java consoles installed.  Removing the
>>>>>> older console cleared up the problem.
>>>>>>
>>>>>> Regards;
>>>>>> Rich
>>>>>>
>>>>>> On 9/27/2010 6:49 PM, Hao, Justin C wrote:
>>>>>>> So i'm setting up a pair of ndt test servers for our new datacenter 
>>>>>>> and running into some hurdles.
>>>>>>>
>>>>>>> First off, i haven't connected them to each other (or to a 10Gig 
>>>>>>> network), so all my testing has been on a single box running loopback 
>>>>>>> tests to itself (i have no other 10GigE hosts available to me at the 
>>>>>>> moment).
>>>>>>>
>>>>>>> I'm running CentOS 5.5 and patched kernel 2.6.35 with the proper 
>>>>>>> web100 version etc.
>>>>>>>
>>>>>>> It's currently hooked up via 1GigE and i've run several different 
>>>>>>> clients against it and I get good results for 1Gig performance
>>>>>>>
>>>>>>> I'm running loopback tests to tweak TCP settings while I wait for our 
>>>>>>> 10Gig environment to be made ready. I'm getting 15-18Gig/s for C2S 
>>>>>>> but for S2C i'm getting numbers all over the place. from 3.5Gig/s to 
>>>>>>> 500Mb/s.  and most particularly odd, i'm seeing negative numbers in 
>>>>>>> some of the test output.
>>>>>>>
>>>>>>> I welcome any comments and suggestions for tuning this server (it's a 
>>>>>>> dell r610 w/ an intel 10GigE adapter)
>>>>>>>
>>>>>>> note: I've also configured the ethernet interface to use 9000 MTU and 
>>>>>>> a txqueuelen of 10000
>>>>>>>
>>>>>>> Server TCP settings:
>>>>>>>
>>>>>>> # increase TCP max buffer size setable using setsockopt()
>>>>>>>
>>>>>>> net.core.rmem_max = 16777216
>>>>>>> net.core.wmem_max = 16777216
>>>>>>>
>>>>>>> # increase Linux autotuning TCP buffer limits
>>>>>>> # min, default, and max number of bytes to use
>>>>>>> # set max to 16MB for 1GE, and 32M or 54M for 10GE
>>>>>>>
>>>>>>> net.ipv4.tcp_mem = 16777216 16777216 16777216
>>>>>>> net.ipv4.tcp_rmem = 10240 87380 16777216
>>>>>>> net.ipv4.tcp_wmem = 10240 65536 16777216
>>>>>>>
>>>>>>> net.ipv4.tcp_window_scaling = 1
>>>>>>>
>>>>>>> # don't cache ssthresh from previous connection
>>>>>>> net.ipv4.tcp_no_metrics_save = 1
>>>>>>>
>>>>>>> # recommended to increase this for 10G NICS
>>>>>>> net.core.netdev_max_backlog = 262144
>>>>>>>
>>>>>>> NDT Output:
>>>>>>>
>>>>>>> TCP/Web100 Network Diagnostic Tool v3.6.3
>>>>>>> Click START to start the test
>>>>>>>
>>>>>>> ** Starting test 1 of 1 **
>>>>>>> Connecting to '127.0.0.1' [/127.0.0.1] to run test
>>>>>>> Connected to: 127.0.0.1--  Using IPv4 address
>>>>>>> Checking for Middleboxes  . . . . . . . . . . . . . . . . . .  Done.
>>>>>>> Checking for firewalls . . . . . . . . . . . . . . . . . . .  Done.
>>>>>>> running 10s outbound test (client-to-server [C2S]) . . . . . 
>>>>>>> 13744.22Mb/s
>>>>>>> running 10s inbound test (server-to-client [S2C]) . . . . . . 
>>>>>>> -83702.57kb/s
>>>>>>> Server unable to determine bottleneck link type.
>>>>>>> [S2C]: Packet queueing detected
>>>>>>>
>>>>>>> Click START to re-test
>>>>>>>
>>>>>>> ** Starting test 1 of 1 **
>>>>>>> Connecting to '127.0.0.1' [/127.0.0.1] to run test
>>>>>>> Connected to: 127.0.0.1--  Using IPv4 address
>>>>>>> Checking for Middleboxes  . . . . . . . . . . . . . . . . . .  Done.
>>>>>>> Checking for firewalls . . . . . . . . . . . . . . . . . . .  Done.
>>>>>>> running 10s outbound test (client-to-server [C2S]) . . . . . 
>>>>>>> 12876.05Mb/s
>>>>>>> running 10s inbound test (server-to-client [S2C]) . . . . . . 
>>>>>>> 1006.86Mb/s
>>>>>>> Server unable to determine bottleneck link type.
>>>>>>> [S2C]: Packet queueing detected
>>>>>>>
>>>>>>> Click START to re-test
>>>>>>>
>>>>>>> ** Starting test 1 of 1 **
>>>>>>> Connecting to '127.0.0.1' [/127.0.0.1] to run test
>>>>>>> Connected to: 127.0.0.1--  Using IPv4 address
>>>>>>> Checking for Middleboxes  . . . . . . . . . . . . . . . . . .  Done.
>>>>>>> Checking for firewalls . . . . . . . . . . . . . . . . . . .  Done.
>>>>>>> running 10s outbound test (client-to-server [C2S]) . . . . . 
>>>>>>> 18466.0Mb/s
>>>>>>> running 10s inbound test (server-to-client [S2C]) . . . . . . 
>>>>>>> -1710004.63kb/s
>>>>>>> Server unable to determine bottleneck link type.
>>>>>>> [S2C]: Packet queueing detected
>>>>>>>
>>>>>>> Click START to re-test
>>>>>>>
>>>>>>>
>>>>>>> -----
>>>>>>> Justin Hao
>>>>>>> CCNA
>>>>>>> Network Engineer, ITS Networking
>>>>>>> The University of Texas at Austin
>>>>>>> 
>>>>>>> -----
>>>>>>>
>>>>>>>
>>>>>
>>>>>
>>>
>>
>>