NTAC Peering Working Group

[Chris Robb <>]

On Jun 7, 2013, at 5:04 PM, Brad Fleming <> wrote:

I've been thinking through the discussion topics from the call today and figured I'd give some thoughts and opinions.

+ IP to AL2S Interconnects

I like the "Backbone Split/Member Access Split" (the last one in the section) approach. It spreads the risk rather than pooling all services into a single risk domain. I don't know what failures we're most likely to see but as a Brocade NetIron shop we've seen the most likely hardware problem be switch fabrics which are linecard agonistic in their failure and cause all kinds of fun messes. The kinds of software failures I expect are difficult to describe and not well informed so they shouldn't drive a design decision.

I tend to agree. It spreads the risk around and leaves a backup methodology in place so nothing should be completely down. 

+ Upcoming Topology and Architecture

I think general consensus was to start with the "Mirror Legacy Network" approach with a mind toward adding L2 paths between IP nodes as needed at a later date. I support that idea. If there's any concern about a specific AL2S path causing problems the NOC can simply move traffic back to the direct, WDM links and see if issues are magically resolved. Provides very clean fallbacks in the event something goes wrong. That ability should help troubleshooting while everyone gets their feet wet with this tiered connectivity approach.

It's probably worth putting thought now into how the need for a new bypass link will be discovered and "justified". Then how the information will be disseminated to groups and with how much notice. Example: crazy amount of traffic going LA<>Chi but is looping through Salt Lake, how quickly will I2 move to put a new LA<>Chi path in place, what groups (tech, C*O, everyone?) will be notified, and with how much notice? Does there need to be input from the NTAC before the new path is "approved"? I don't have any awesome ideas in this regard but these questions need answering before moving to production mainly to set expectation for network participants.

Agree that we need to document a plan to evaluate. If we go down this path, we'll need to set up some data monitors that provide us with some trends. We can layer policy onto it that allows for a discussion to begin at a certain trigger point. 

+ Possible Link Restoration Methods

The problem is really two-fold: 1) detecting a failure in the path and 2) rerouting traffic around the failure. To my understanding Loop-Free Alternatives will help speed along part #2 but still requires something discover the failure. Better IS-IS timers will help but a 3-9 second outage before /starting/ the process of selecting new paths, installing them, and forwarding traffic is a really long time in the VoIP world.

For this reason I think (unfortunately) we shouldn't move IP-atop-AL2S until BFD and LFA can be included in the design. I know that makes the rollout timeline longer but any other move would result in a significant step backward for dependability of services which depend on the IP network; enough so it's an unacceptable risk IMHO.

I agree this is something we need to be careful about, but I don't see any way to get around it in the near term. With the IP network topology divorced from the physical topology, we need to look elsewhere. The IGP timers have a lot of room to be tuned today, so we may end up improving the restoration in certain scenarios. 

-Chris

-- 
Chris Robb, Internet2 Director of Operations and Engineering
O: 812.855.8604  C: 812.345.3188
****************
Visit our website: www.internet2.edu
Follow us on Twitter: www.twitter.com/internet2
Become a Fan on Facebook: www.internet2.edu/facebook