ndt-users list created

[Dale Blount <>]

On Thu, 2006-02-16 at 17:02 -0500, Richard Carlson wrote:
> Hi Dale;
> 
> At 02:07 PM 2/16/2006, Dale Blount wrote:
> >On Wed, 2006-02-15 at 10:40 -0500, Richard Carlson wrote:
> > > Hi Dale;
> > >
> > > I don't recall if I replied to this earlier but here it is again.
> > >
> > > I'm seeing this problem more and more and I am currently trying to find 
> > > a
> > > real fix for this problem.  Here's what's happening.
> > >
> > > The NDT server starts sending data to the remote client.  It enters a
> > > simple send loop and pumps data into the network for 10 seconds.  It 
> > > then
> > > exits the loop and sends the final results over to the client.
> > >
> > > The problem is, that when in this loop it is possible for the OS to
> > > transfer data to the TCP stack, faster than the TCP stack pump it out 
> > > into
> > > the network.  This results in a large standing queue, visible with the
> > > netstat -nat command).  So at the end of 10 seconds the code stops 
> > > pumping
> > > more data but the client keeps reading until the queue empties.  Note 
> > > the
> > > Web100 Duration variable has a value of 34,893,925 microseconds, or 
> > > almost
> > > 35 seconds.
> > >
> > > One temporary step is to limit the max buffer space (tcp_wmax) to 
> > something
> > > in the 512 KB to 1MB range.  This will keep the queue from building up 
> > > too
> > > much, but it's really just a band-aid until I can figure out how to 
> > monitor
> > > the queue length to prevent such large queues in the first place.
> > >
> >
> >Rich,
> >
> >I can't find a tcp_wmax setting, but here is what I have set:
> 
> Sorry that was a typo on my part.  It should have been tcp_wmem not _wmax..
> 
> >net.core.wmem_max = 131072
> >net.core.rmem_max = 131072
> >net.ipv4.tcp_wmem = 4096 16384 131072
> >net.ipv4.tcp_rmem = 4096 16384 131072
> >


so my "net.ipv4.tcp_wmem = 4096 16384 131072" is ok to limit the queue
length?

> >Upload always works OK, but on anything but the lan, download is right
> >around 75k.  It doesn't really matter if it's set to 128kb/512kb/2Mb,
> >it's always 70-80kb (both on a 5Mbps upload cable modem and a 768kbps
> >upload dsl link, both 3 hops from the ndt server).
> >
> >The old server that this is replacing is still around, and speedtests to
> >it work just fine.  Could the newer hardware alone be causing this whole
> >problem?  I've tried the sysctl settings from the old server with the
> >same results.
> 
> It could be a hardware issue or an OS issue.  Did you change/upgrade the OS 
> level too?  I noticed a problem with my server when I went from Linux 
> 2.6.12 to 2.6 13.  

I went from 2.6.12.2-web100 to 2.6.15.3-web100.  Distro is the same
version.

> I finally tracked it down to a change in the Intel 
> FastEthernet (e100) NIC driver that came with the new OS.  I replaced the 
> new e100.c file with the one from the .12 kernel and everything started 
> working again.  I also have a report of a problem with a built in NIC, and 
> the problem was resolved when a PCI bus based NIC was installed.  Perhaps 
> this is a bigger problem than I realize.
> 

I also moved from a Dlink PCI card to an onboard TG3 chipset.


> > > If anyone has any suggestions on how to do this, please let me know.
> > >
> >
> >Couldn't the client be adjusted to stop reading after 10 seconds?  It
> >could then report the data transferred so far.
> 
> There is a timer that runs in the client to clean things up, but I either 
> there's a bug in my code or something else is wrong and the timer isn't 
> working.  I am currently testing a patch on my server at 
> http://web100.internet2.edu:7123  It would help if you could try this 
> server and let me know if the tests run long or what happens.
> 

LAN: Duration: 12510326
        running 10s outbound test (client to server) . . . 8.30Mb/s
        running 10s inbound test (server to client) . . . 16.56Mb/s


CABLE: Duration: 17116116
        running 10s outbound test (client to server) . . . . . 1.40Mb/s
        running 10s inbound test (server to client) . . . . . . 1.80Mb/s


Dale