perfSONAR User Q&A and Other Discussion

[Mark Feit <>]

Dom Ashworth writes:

 

Am I right in in thinking that I should connect the perfSonar servers in the
same way so as to closely replicate the deployment of our server
infrastructure or could I get away using a single homed connection to each ToR
(as inter-cabinet patches are limited)?  Would a single homed connection allow
performance monitoring of the same accuracy and equivalence as a NIC bonded
deployment.

 

Generally, you want to make your measurement systems to have the same “user experience” as the systems providing your services.   Like everything else, there are trade-offs.

 

A non-bonded connection will run at half the rate of a bonded one.  A performance degradation that doesn’t make the available bandwidth drop below what the perfSONAR node can measure would go unnoticed.  There’s also the risk that someone will look at a usually-10 Gb/s measurement that drops to 9, conclude that that it’s still running at 90% of capacity and is therefore not an emergency while all of your 20 Gb/s systems are getting 45%.

 

Upgraded server kernel or switch firmware on the switches harboring a performance-impacting bug that only affects bonded connections would show no change on a singly-connected perfSONAR node.  Even if your servers are instrumented for other things (e.g., you see a drop in their transaction rate), the time to diagnose and repair will be longer because you don’t know if it’s the application or the network.

 

A good compromise might be to run a bonded connection to two ports on the same switch.  That would provide the same amount of available bandwidth and covers the case of a bonding-related bug in the kernel.  Additionally, it would help pinpoint network failures because a measurement point that lives only in rack 1 showing a problem has half the potential failure surface of one connected to racks 1 and 2.  For example, if the rack 1 switch is delaying packets, you could see the problem intermittently depending on where the traffic goes.

 

--Mark