perfSONAR User Q&A and Other Discussion

["Azher Mughal" <>]

I have used LAG (using round robin) in Linux over multiple 40G NICs back to back among two servers. However out of the order packets was an issue.

Cheers
-Azher

On 12/30/2014 1:34 PM, Philip Papadopoulos wrote:

Let me see if I can clarify.

4x10G LAG -- any particular TCP "conversation" (IP:port  (host 1) <--> IP:port (host 2)) will always go over the same

leg of a LAG (unless the link physically goes down). This is to help preserve in-order delivery of packets. In general, TCP doesn't perform well when packets arrive out of order (the TCP connection will be correct if the packets arrive out of order).   Hence the statement that "you will go only as fast as the slowest link").   Between your 40G host and 

your 4x10G host, any single TCP conversation will be rate limited to one-leg of the LAG.  

 

You can get more than 10G throughput between your 40G host and your 4x10G host, but it requires multiple simultaneous TCP conversations to be going at the same time. Hence the comment of needing at least 4 streams. 

 

 

 

 

On Tue, Dec 30, 2014 at 1:08 PM, Christoph Galuschka <> wrote:

Hi,

Am 30.12.2014 um 19:21 schrieb Cas D'Angelo:

Does anyone have experience with testing between a Mellanox 40G card in
one PS host and a 4x10G (or 6x 10G) card (or 2x 2x 10G cards) in another
PS host?  Any tips on configuring the LAG would be appreciated.

No real advice, sorry. But you do know throughput is limited to the port speed of the slowest card (i.e. 10G).

all the best

--
Christoph Galuschka
CentOS-QA-Team member | IRC: tigalch

 

--

Philip Papadopoulos, PhD
University of California, San Diego
858-822-3628 (Ofc)
619-331-2990 (Fax)

--
This message has been scanned for viruses and
dangerous content by MailScanner, and is
believed to be clean.