Regional Technical Contacts

[Chris Robb <>]

Dave and Dave:

You both raise excellent points and it’s something we did consider. We gave it some more thought this morning and will be doing a bit of additional outreach to the owners of all the OESS circuits through the Connectors. It will inform them of the maintenance window and invite them to join the bridge. If they aren’t able to join the bridge, we’ll be also sending an “all clear” message when we believe the maintenance has successfully completed so that they may verify their side as soon as they are able. Given our testing and experience, we don’t believe it will be a common failure scenario where we’re able to recover some VLANs but not others. So, if we verify the traffic we can observe more easily and can verify traffic levels look like they’re returning on the L2 circuits, we have a high amount of confidence the change was successful. 

-Chris

-- 
Chris Robb, Director Internet2 Operations and Engineering
O: 812.855.8604  C: 812.345.3188

On Jun 4, 2021, at 4:37 PM, Dave Diller <> wrote:

I’m on the same page….   I’m actually not sure what the value-add is for a _connector_ joining the bridge in terms of “verifying” L2 services.   I have the exact same visibility for L2 across MAX as you have across I2.  Comes in one port, goes out another.  If, say, there’s a BGP session between a campus router and AWS, I can’t see that either.

I can look at MAC addresses to be sure I see the same ones before/after (you could do the same); I can look at traffic levels to verify it appears similar (which you are already doing); and that’s really about it.

Unless you are expecting every end entity with an OESS VLAN to have a representative on the call signing off on the move, I’m not sure what you mean by “alarm and provide feedback” - anything that was not working at the time of the move most likely would not be caught until normal business hours, if it was subtle enough that checking MACs and traffic levels from the midpoint did not catch it.

-dd

On Jun 4, 2021, at 3:16 PM, David Farmer (via ngi-community-packet Mailing List) <> wrote:

Linda,

Thanks for the update, and the request makes sense to me.  However, I’d like to suggest a slight modification to the request.  I think your request should be, to have an engineer on the call capable of verifying each layer 2 service passing through the connection is operational following the cut. In some cases, maybe even many cases, an engineer from the connector will be sufficient. 

However in other cases, an engineer from a campus when the layer2 service is passed through the connector to the campus, or even possible an engineer from a research group, if the layer2 service passes through the connector and the campus all the way to a research group, will be needed to actually verify some layer 2 services in these cases. 

So to reiterate, I think you want an engineer capable of verifying each layer2 service and not necessarily an engineer from the connector.

Thanks 

On Fri, Jun 4, 2021 at 13:41 Linda Roos <> wrote:

Hi,

Internet2 successfully completed our first shim over the night of June 1-2. Thank you to KyRON for working with us in Louisville, KY.

One of the things we learned is that the shim maintenance window will require an engineer from the Connector organization to be on the bridge with the Internet2 implementation team. Having a local engineer engaged will allow us to get immediate verification that any Layer2 services passing through the Internet2 network are working as anticipated. Layer2 services are harder for Internet2 staff to verify since we don’t have any direct operational relationship that would alarm and provide feedback if the change doesn’t happen as expected. Having one of the engineers from your connector organization join our bridge during the maintenance will allow us to verify that everything on the connector side has restored before we end the maintenance call. This will be a real opportunity for all of us to learn about the NGI implementation together.

What can you expect when your site comes up to be shimmed? Before the shim, the Internet2 team prepares for the shim by running fibers and educating the field tech on the process. During the shim maintenance, the work consists of swapping cables, ensuring that the Layer3 services re-establish on our side and all OESS VLANs have the same amount of traffic on them after the shim as before.  

We created a short video of a post-install meeting held by the NGI engineering team after the successful Louisville SHIM effort on June 1-2. You can drop in on the huddle to learn more about how we prepared, the role NSO played in the automated implementation, and hear what the implementation engineer experienced. Speakers include:

Camille Davis-Alfs - Network Engineering Manager - GlobalNOC 

Karl Newell Network Software Engineer - Internet2 

Curtis Schongalla - Network Engineer- GlobalNOC 

Hosted by Chris Robb - Senior Manager Internet2 Network Engineering - GlobalNOC

https://internet2.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=3b7f26d1-f663-4053-a1b8-ad3d00f3f9d8

The engineers will reach out to you to work out the details of the shim in your location. We really appreciate your identifying the appropriate engineer from your organization to join the shim maintenance. Please let me know what questions you have. Thanks.

Linda

—

Linda Roos
Sr Director, State and Regional Networks
Internet2

812-558-0615

--

===============================================
David Farmer              
Networking & Telecommunication Services
Office of Information Technology
University of Minnesota  
2218 University Ave SE        Phone: 612-626-0815
Minneapolis, MN 55414-3029   Cell: 612-812-9952
===============================================