Skip to Content.
Sympa Menu

ndt-users - Re: NDT cannot allocated larger than 32k TCP buffer on a Linux client

Subject: ndt-users list created

List archive

Re: NDT cannot allocated larger than 32k TCP buffer on a Linux client


Chronological Thread 
  • From: Chris Hawkinson <>
  • To:
  • Subject: Re: NDT cannot allocated larger than 32k TCP buffer on a Linux client
  • Date: Fri, 08 Oct 2010 10:40:32 -0700

After further investigation, I found that "TCP Window Scaling" is being negotiated to "OFF", which is obviously the reason for the smaller TCP buffer. Now we're off to try to determine why it is being turned "OFF" between this particular client and NDT server.

Thanks,
Chris

On 10/8/10 8:58 AM, Chris Hawkinson wrote:
Hi Everyone,

I'm trying to help someone with an NDT issue.

Their client machine is 32bit Linux 2.6.18 (Here's what uname says ... "2.6.18-194.17.1.el5PAE #1 SMP Wed Sep 29 13:31:51 EDT 2010 i686 i686 i386 GNU/Linux" ).

They tuned there sysctl values, as shown below...

cat /proc/sys/net/ipv4/tcp_moderate_rcvbuf
1

cat /proc/sys/net/core/rmem_max
8388608

cat /proc/sys/net/core/wmem_max
8388608

cat /proc/sys/net/ipv4/tcp_rmem
8192 87380 8388608

cat /proc/sys/net/ipv4/tcp_wmem
8192 65536 8388608

The problem is, everytime they run the NDT Java client to a remote NDT server, the output shows that NDT can only get a 32k buffer (full test results are below). We've rebooted the Linux machine several times, but no luck.

Has anyone else run into this? What are we doing wrong?

Thanks,
Chris

Chris Hawkinson
Network Performance Engineer
CENIC

Full test results....

TCP/Web100 Network Diagnostic Tool v5.5.4a
** Starting test 1 of 1 ** Connected to: ndt2.xxxxxx.edu --
Using IPv4 address Another client is currently being served,
your test will begin within 540 seconds Checking for
Middleboxes . . . . . . . . . . . . . . . . . . Done checking
for firewalls . . . . . . . . . . . . . . . . . . . Done
running 10s outbound test (client-to-server [C2S]) . . . . .
3.27Mb/s running 10s inbound test (server-to-client [S2C]) .
. . . . . 3.08Mb/s The slowest link in the end-to-end path is
a 100 Mbps Full duplex Fast Ethernet subnet [S2C]: Packet
queuing detected

Statistics: ===========

WEB100 Enabled Statistics: Checking for Middleboxes . . . . .
. . . . . . . . . . . . . Done checking for firewalls . . . .
. . . . . . . . . . . . . . . Done running 10s outbound test
(client-to-server [C2S]) . . . . . 3.27Mb/s running 10s
inbound test (server-to-client [S2C]) . . . . . . 3.08Mb/s

------ Client System Details ------ OS data: Name = Windows
XP, Architecture = x86, Version = 5.1 Java data: Vendor = Sun
Microsystems Inc., Version = 1.6.0_21

------ Web100 Detailed Analysis ------ 100 Mbps FastEthernet
link found. Link set to Full Duplex mode No network
congestion discovered. Good network cable(s) found Normal
duplex operation found.

Web100 reports the Round trip time = 74.78 msec; the Packet
size = 1448 Bytes; and No packet loss - but packets arrived
out-of-order 71.84% of the time S2C throughput test: Packet
queuing detected: 87.01% This connection is receiver limited
54.82% of the time. Increasing the the client's receive
buffer (33.0 KB) will improve performance This connection is
sender limited 37.76% of the time. This connection is network
limited 7.42% of the time.

Web100 reports TCP negotiated the optional Performance
Settings to: RFC 2018 Selective Acknowledgment: ON RFC 896
Nagle Algorithm: ON RFC 3168 Explicit Congestion
Notification: OFF RFC 1323 Time Stamping: ON RFC 1323 Window
Scaling: OFF

Server 'ndt2.xxxxx.edu' is probably behind a firewall.
[Connection to the ephemeral port failed] Client is probably
behind a firewall. [Connection to the ephemeral port failed]
Information: Network Middlebox is modifying MSS variable
Server IP addresses are preserved End-to-End Client IP
addresses are preserved End-to-End


More details: ============

WEB100 Kernel Variables: Client: localhost/127.0.0.1 CurMSS:
1448 X_Rcvbuf: 33554432 X_Sndbuf: 33554432 AckPktsIn: 1275
AckPktsOut: 0 BytesRetrans: 0 CongAvoid: 0
CongestionOverCount: 0 CongestionSignals: 0 CountRTT: 360
CurCwnd: 56472 CurRTO: 274 CurRwinRcvd: 31856 CurRwinSent:
6144 CurSsthresh: 144800 DSACKDups: 0 DataBytesIn: 0
DataBytesOut: 4261812 DataPktsIn: 0 DataPktsOut: 3188
DupAcksIn: 916 ECNEnabled: 0 FastRetran: 0 MaxCwnd: 56472
MaxMSS: 1448 MaxRTO: 277 MaxRTT: 81 MaxRwinRcvd: 34020
MaxRwinSent: 6144 MaxSsthresh: 144800 MinMSS: 1448 MinRTO:
273 MinRTT: 72 MinRwinRcvd: 4328 MinRwinSent: 5792
NagleEnabled: 1 OtherReductions: 0 PktsIn: 1275 PktsOut:
3188 PktsRetrans: 0 RcvWinScale: 10 SACKEnabled: 3 SACKsRcvd:
0 SendStall: 0 SlowStart: 0 SampleRTT: 72 SmoothedRTT: 74
SndWinScale: 0 SndLimTimeRwin: 6068059 SndLimTimeCwnd:
821177 SndLimTimeSender: 4179761 SndLimTransRwin: 280
SndLimTransCwnd: 4 SndLimTransSender: 284 SndLimBytesRwin:
3124388 SndLimBytesCwnd: 143960 SndLimBytesSender: 993464
SubsequentTimeouts: 0 SumRTT: 26922 Timeouts: 0
TimestampsEnabled: 1 WinScaleRcvd: 0 WinScaleSent: 10
DupAcksOut: 0 StartTimeUsec: 424323 Duration: 11072093
c2sData: 5 c2sAck: 5 s2cData: 8 s2cAck: 3 half_duplex: 0
link: 100 congestion: 0 bad_cable: 0 mismatch: 0 spd: 3.08
bw: 147.72 loss: 0.000001000 avgrtt: 74.78 waitsec: 0.00
timesec: 11.00 order: 0.7184 rwintime: 0.5482 sendtime:
0.3776 cwndtime: 0.0742 rwin: 0.2596 swin: 256.0000 cwin:
0.4308 rttsec: 0.074783 Sndbuf: 33554432 aspd: 0.00000
CWND-Limited: 134.23 minCWNDpeak: -1 maxCWNDpeak: -1
CWNDpeaks: -1

The theoretical network limit is 147.72 Mbps The NDT server
has a 16384.0 KByte buffer which limits the throughput to
3423.23 Mbps Your PC/Workstation has a 33.0 KByte buffer
which limits the throughput to 3.47 Mbps The network based
flow control limits the throughput to 5.76 Mbps

Client Data reports link is 'FastE', Client Acks report link
is 'FastE' Server Data reports link is 'OC-48', Server Acks
report link is 'Ethernet'




Archive powered by MHonArc 2.6.16.

Top of Page