Skip to Content.
Sympa Menu

ndt-users - Re: gigabit slower than fast-e

Subject: ndt-users list created

List archive

Re: gigabit slower than fast-e


Chronological Thread 
  • From: Bill Abbott <>
  • To: Jim Cotton <>
  • Cc: Matthew J Zekauskas <>,
  • Subject: Re: gigabit slower than fast-e
  • Date: Fri, 07 Mar 2008 15:07:14 -0500

I don't think it's this; the same two servers could transfer at much higher speeds several months ago, when I did the original tuning.

Jim Cotton wrote:
-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA1


I have seen several Dell servers on campus that exhibited such a problem
with their on-board chipset.

There is a problem with the Broadcom chip set below a certain
revision... If you get the right driver it will work around it.

Jim Cotton

Bill Abbott wrote:
I've tried tcp send and receive (max) buffers from 64k up to 8 MB, no
improvement.

Smaller buffers are more consistent, larger buffers tend to go up then
hit zero, then climb again.

The current settings are:

$ cat /proc/sys/net/ipv4/tcp_rmem
4096 87380 8174760
$ cat /proc/sys/net/ipv4/tcp_wmem
4096 16384 8131072

I've also modified a bunch of other stuff, but it didn't make a
difference so I set it back. ECN, /proc/sys/net/core/rmem_max and
wmem_max, netdev_max_backlog, etc.




Matthew J Zekauskas wrote:
To follow up to myself...

* The Internet2 servers are all gigE, and all 9000-byte MTU capable.

* Another thing you could do is tune for the bottleneck instead of
gigabit..

150 Mbps = 150*1000*1000 bits * 0.07 ms RTT = 10500000 bits
/8 = 1312500 bytes /(1024*1024) = 1.26 megabytes.

or

100 Mbps is 850K

and see if that improves things. It may be that something can't
handle even that much back-to-back when they are sent at 1000Mbps.

There are also bwctl (<http://e2epi.internet2.edu/bwctl/>) servers at
Internet2 router nodes, for a different way to split the problem
(essentially run iperf to the core).

--Matt



-----BEGIN PGP SIGNATURE-----
Version: GnuPG v2.0.4-svn0 (GNU/Linux)
Comment: Using GnuPG with SUSE - http://enigmail.mozdev.org

iD8DBQFH0OaH2MvRXu0NtoARAtTrAJ0Z+dDKVvI9i3SYAwIBPeUhu/KqZACgqk2H
RltkpeLYhKoJYkh0BlWF7Ow=
=V7Ah
-----END PGP SIGNATURE-----



Archive powered by MHonArc 2.6.16.

Top of Page