perfSONAR User Q&A and Other Discussion

[Mark Feit <>]

Hagen, Skye () writes:

 

I have a dual 10G Intel NIC (X520-DA2). Can I use both interfaces on the NIC, or should I be looking at two separate NIC cards? It is PCI Express 2.0, 5.0 GT/s lane x 8 lane. Is there are way to tell if both interfaces are capable of simultaneous 10G throughput, without moving the contention problem to the PCI bus?

 

The controller Intel uses on most of those cards these days is designed to go full-speed on two interfaces at the same time.  Your PCIe slot runs 5 GT/s raw transfer per lane in one direction.  Chop out 20% for bus overhead (4 Gb/s per lane), multiply by 8 lanes and you have 32 Gb/s, more than enough for both interfaces.

 

One thing worth watching for:  some PC manufacturers install slots physically large enough to hold a card with more lanes than there are contacts to support (e.g., x8 physically but x4 electrically).  You can check this by running ‘lspci –vv’ as root and looking at the information for the card.  The Width value in the LnkCap section will show the number of lanes the slot can handle and the same under LinkSta will show how many lanes were negotiated.  Latency measurement is a low-enough-bandwidth operation that you could still run throughput at full speed on one port and have more than enough bus left over for it on the other, even in a x4 slot.

 

 

Can I run bwctl on a 10G NIC, and owamp on a 1G NIC? Are the results comparable? Or, is the recommendation to keep all testing interfaces at the same speed?

 

Sure, that’s not a problem.  There would be a difference in the amount of time it takes to transmit each packet that will have a very small impact on the result, but the limitations of the timekeeping on both ends will muddy the measurement enough that you’re not going to notice it.

 

--Mark