ndt-users list created

[Clayton Keller <>]

I believe this is what we have done, and have changed the buffers from
2M to 1M, and seen varying changes and this does seem to be a workaround
that has worked for us.

Unfortunately, this is a bit outside of my realm working with the queues
at that level but am glad to provide any further assistance that I can.

Clay

Richard Carlson wrote:
> Hi Dale;
> 
> I don't recall if I replied to this earlier but here it is again.
> 
> I'm seeing this problem more and more and I am currently trying to find
> a real fix for this problem.  Here's what's happening.
> 
> The NDT server starts sending data to the remote client.  It enters a
> simple send loop and pumps data into the network for 10 seconds.  It
> then exits the loop and sends the final results over to the client.
> 
> The problem is, that when in this loop it is possible for the OS to
> transfer data to the TCP stack, faster than the TCP stack pump it out
> into the network.  This results in a large standing queue, visible with
> the netstat -nat command).  So at the end of 10 seconds the code stops
> pumping more data but the client keeps reading until the queue empties. 
> Note the Web100 Duration variable has a value of 34,893,925
> microseconds, or almost 35 seconds.
> 
> One temporary step is to limit the max buffer space (tcp_wmax) to
> something in the 512 KB to 1MB range.  This will keep the queue from
> building up too much, but it's really just a band-aid until I can figure
> out how to monitor the queue length to prevent such large queues in the
> first place.
> 
> If anyone has any suggestions on how to do this, please let me know.
> 
> Rich
> 
> At 10:52 AM 2/10/2006, Dale Blount wrote:
>> Hello again,
>>
>> I've just upgraded my old web100/ndt server to new hardware with onboard
>> gigE (instead of a card) and a xeon instead of a p2.  The new server is
>> also running 2.6.15 (the old one was running 2.6.11 or so).  It seems to
>> run fine on the lan (my workstation only has 100mbps, and it clocks
>> around 94mbps both ways).  However, any distant connection -
>> DSL/Cable/dialup seems to test horribly in the downstream direction.
>>
>> I've tweaked the sysctl params the same they were on the other server,
>> but they don't seem to help much.
>>
>> Downloading a file from the server appears to travel at normal speed.
>>
>> Below are the logs from a 5mb/5mb cable modem connection, 3 hops from
>> the ndt server.
>>
>> Thanks for any pointers and/or tuning advice.
>>
>> Dale
>>
>>
>>
>>
>>
>>
>>
>> Checking for Middleboxes . . . . . . . . . . . . . . . . . .  Done
>> running 10s outbound test (client to server) . . . . . 3.73Mb/s
>> running 10s inbound test (server to client) . . . . . . 72.66kb/s
>> The slowest link in the end-to-end path is a 10 Mbps Ethernet subnet
>> Information: The receive buffer should be 73.99 Kbytes to maximize
>> throughput
>>
>> ********************************************************************
>>
>> WEB100 Enabled Statistics:
>> Checking for Middleboxes . . . . . . . . . . . . . . . . . .  Done
>> running 10s outbound test (client to server) . . . . . 3.73Mb/s
>> running 10s inbound test (server to client) . . . . . . 72.66kb/s
>>
>>         ------  Client System Details  ------
>> OS data: Name = Windows 2003, Architecture = x86, Version = 5.2
>> Java data: Vendor = Sun Microsystems Inc., Version = 1.5.0_04
>>
>>         ------  Web100 Detailed Analysis  ------
>> 10 Mbps Ethernet link found.
>> Link set to Full Duplex mode
>> No network congestion discovered.
>> Good network cable(s) found
>> Normal duplex operation found.
>>
>> Web100 reports the Round trip time = 60.62 msec; the Packet size = 1460
>> Bytes; and
>> There were 201 packets retransmitted, 18 duplicate acks received, and 20
>> SACK blocks received
>> The connection stalled 1 times due to packet loss
>> The connection was idle 0.26 seconds (0.76%) of the time
>> This connection is network limited 99.64% of the time.
>>
>> Web100 reports TCP negotiated the optional Performance Settings to:
>> RFC 2018 Selective Acknowledgment: ON
>> RFC 896 Nagle Algorithm: ON
>> RFC 3168 Explicit Congestion Notification: OFF
>> RFC 1323 Time Stamping: OFF
>> RFC 1323 Window Scaling: OFF
>> Packet size is preserved End-to-End
>> Server IP addresses are preserved End-to-End
>> Information: Network Address Translation (NAT) box is modifying the
>> Client's IP address
>>         Server says [W.X.Y.Z] but Client says [Z.Y.X.W]
>>
>>
>> *******************************************************************
>>
>> WEB100 Kernel Variables:
>> Client: localhost/127.0.0.1
>> AckPktsIn: 231
>> AckPktsOut: 0
>> BytesRetrans: 293460
>> CongAvoid: 0
>> CongestionOverCount: 0
>> CongestionSignals: 2
>> CountRTT: 13
>> CurCwnd: 1460
>> CurMSS: 1460
>> CurRTO: 264
>> CurRwinRcvd: 17520
>> CurRwinSent: 5840
>> CurSsthresh: 16060
>> DSACKDups: 0
>> DataBytesIn: 0
>> DataBytesOut: 3403472
>> DataPktsIn: 0
>> DataPktsOut: 2248
>> DupAcksIn: 18
>> ECNEnabled: 0
>> FastRetran: 1
>> MaxCwnd: 21900
>> MaxMSS: 1460
>> MaxRTO: 264
>> MaxRTT: 100
>> MaxRwinRcvd: 17520
>> MaxRwinSent: 5840
>> MaxSsthresh: 146000
>> MinMSS: 1460
>> MinRTO: 224
>> MinRTT: 24
>> MinRwinRcvd: 17520
>> MinRwinSent: 5840
>> NagleEnabled: 1
>> OtherReductions: 0
>> PktsIn: 231
>> PktsOut: 2248
>> PktsRetrans: 201
>> X_Rcvbuf: 4194304
>> RcvWinScale: 2147483647
>> SACKEnabled: 3
>> SACKsRcvd: 20
>> SendStall: 0
>> SlowStart: 0
>> SampleRTT: 100
>> SmoothedRTT: 64
>> X_Sndbuf: 4194304
>> SndWinScale: 2147483647
>> SndLimTimeRwin: 0
>> SndLimTimeCwnd: 34764344
>> SndLimTimeSender: 126942
>> SndLimTransRwin: 0
>> SndLimTransCwnd: 1
>> SndLimTransSender: 1
>> SndLimBytesRwin: 0
>> SndLimBytesCwnd: 3403472
>> SndLimBytesSender: 0
>> SubsequentTimeouts: 0
>> SumRTT: 788
>> Timeouts: 1
>> TimestampsEnabled: 0
>> WinScaleRcvd: 2147483647
>> WinScaleSent: 2147483647
>> DupAcksOut: 0
>> StartTimeUsec: 464790
>> Duration: 34893925
>> c2sData: 3
>> c2sAck: 3
>> s2cData: 1
>> s2cAck: 1
>> half_duplex: 0
>> link: 100
>> congestion: 0
>> bad_cable: 0
>> mismatch: 0
>> spd: -1.78
>> bw: 6.16
>> loss: 0.000889680
>> avgrtt: 60.62
>> waitsec: 0.26
>> timesec: 34.00
>> order: 0.0779
>> rwintime: 0.0000
>> sendtime: 0.0036
>> cwndtime: 0.9964
>> rwin: 0.1337
>> swin: 32.0000
>> cwin: 0.1671
>> rttsec: 0.060615
>> Sndbuf: 4194304
>> aspd: 0.19621
>>
>> Checking for mismatch on uplink
>>         (speed > 50 [-1.78>50], (xmitspeed < 5) [3.73<5]
>>         (rwintime > .9) [0>.9], (loss < .01) [8.89<.01]
>> Checking for excessive errors condition
>>         (loss/sec > .15) [2.61>.15], (cwndtime > .6) [0.99>.6],
>>         (loss < .01) [8.89<.01], (MaxSsthresh > 0) [146000>0]
>> Checking for 10 Mbps link
>>         (speed < 9.5) [-1.78<9.5], (speed > 3.0) [-1.78>3.0]
>>         (xmitspeed < 9.5) [3.73<9.5] (loss < .01) [8.89<.01], (mylink
>> > 0)
>> [10.0>0]
>> Checking for Wireless link
>>         (sendtime = 0) [0.00=0], (speed < 5) [-1.78<5]
>>         (Estimate > 50 [6.16>50], (Rwintime > 90) [0>.90]
>>         (RwinTrans/CwndTrans = 1) [0/1=1], (mylink > 0) [10.0>0]
>> Checking for DSL/Cable Modem link
>>         (speed < 2) [-1.78<2], (SndLimTransSender = 0) [1=0]
>>         (SendTime = 0) [0.0036=0], (mylink > 0) [10.0>0]
>> Checking for half-duplex condition
>>         (rwintime > .95) [0>.95], (RwinTrans/sec > 30) [0>30],
>>         (SenderTrans/sec > 30) [0.02>30], OR (mylink <= 10) [10.0<=10]
>> Checking for congestion
>>         (cwndtime > .02) [0.99>.02], (mismatch = 0) [0=0]
>>         (MaxSsthresh > 0) [146000>0]
>>
>> estimate = 6.16 based on packet size = 11Kbits, RTT = 60.62msec, and
>> loss = 8.8968E-4
>> The theoretical network limit is 6.16 Mbps
>> The NDT server has a 4096.0 KByte buffer which limits the throughput to
>> 527.92 Mbps
>> Your PC/Workstation has a 17.0 KByte buffer which limits the throughput
>> to 2.20 Mbps
>> The network based flow control limits the throughput to 2.75 Mbps
>>
>> Client Data reports link is 'Ethernet', Client Acks report link is
>> 'Ethernet'
>> Server Data reports link is 'Dial-up', Server Acks report link is
>> 'Dial-up'
> 
> ------------------------------------
> 
> 
> 
> Richard A. Carlson                              e-mail:
> 
> Network Engineer                                phone:  (734) 352-7043
> Internet2                                       fax:    (734) 913-4255
> 1000 Oakbrook Dr; Suite 300
> Ann Arbor,  MI  48104
>