Regional Technical Contacts

[Matt Valenzisi <>]

Hello Chris:

I hope you are doing well this afternoon.

Have you given out the notice to MCNC/NCREN Constituents yet?

If not, I'd like to ask you to hold off on that.

As with all maintenance, our downstream constituents always get a
maintenance notice from MCNC.

We'd prefer that this communication come directly from MCNC.

We'd also prefer to set up our own bridge that our customers can call
if they're interested and we'll have an engineer from our NOC be on
the Internet2 bridge to answer any questions that I2 might have about
the effect of the maintenance on their L2 connections.

We'd have an engineer available anyway because all of our peers are on VLANs.

Let me know if you have any questions.

Are our maintenance dates set yet?

Thanks in advance,
Matt

On Mon, Jun 7, 2021 at 3:47 PM Chris Robb <> wrote:
>
> Dave and Dave:
>
> You both raise excellent points and it’s something we did consider. We gave 
> it some more thought this morning and will be doing a bit of additional 
> outreach to the owners of all the OESS circuits through the Connectors. It 
> will inform them of the maintenance window and invite them to join the 
> bridge. If they aren’t able to join the bridge, we’ll be also sending an 
> “all clear” message when we believe the maintenance has successfully 
> completed so that they may verify their side as soon as they are able. 
> Given our testing and experience, we don’t believe it will be a common 
> failure scenario where we’re able to recover some VLANs but not others. So, 
> if we verify the traffic we can observe more easily and can verify traffic 
> levels look like they’re returning on the L2 circuits, we have a high 
> amount of confidence the change was successful.
>
> -Chris
>
> --
> Chris Robb, Director Internet2 Operations and Engineering
> O: 812.855.8604  C: 812.345.3188
>
> On Jun 4, 2021, at 4:37 PM, Dave Diller <> wrote:
>
> I’m on the same page….   I’m actually not sure what the value-add is for a 
> _connector_ joining the bridge in terms of “verifying” L2 services.   I 
> have the exact same visibility for L2 across MAX as you have across I2.  
> Comes in one port, goes out another.  If, say, there’s a BGP session 
> between a campus router and AWS, I can’t see that either.
>
> I can look at MAC addresses to be sure I see the same ones before/after 
> (you could do the same); I can look at traffic levels to verify it appears 
> similar (which you are already doing); and that’s really about it.
>
> Unless you are expecting every end entity with an OESS VLAN to have a 
> representative on the call signing off on the move, I’m not sure what you 
> mean by “alarm and provide feedback” - anything that was not working at the 
> time of the move most likely would not be caught until normal business 
> hours, if it was subtle enough that checking MACs and traffic levels from 
> the midpoint did not catch it.
>
> -dd
>
>
> On Jun 4, 2021, at 3:16 PM, David Farmer (via ngi-community-packet Mailing 
> List) <> wrote:
>
> Linda,
>
> Thanks for the update, and the request makes sense to me.  However, I’d 
> like to suggest a slight modification to the request.  I think your request 
> should be, to have an engineer on the call capable of verifying each layer 
> 2 service passing through the connection is operational following the cut. 
> In some cases, maybe even many cases, an engineer from the connector will 
> be sufficient.
>
> However in other cases, an engineer from a campus when the layer2 service 
> is passed through the connector to the campus, or even possible an engineer 
> from a research group, if the layer2 service passes through the connector 
> and the campus all the way to a research group, will be needed to actually 
> verify some layer 2 services in these cases.
>
> So to reiterate, I think you want an engineer capable of verifying each 
> layer2 service and not necessarily an engineer from the connector.
>
> Thanks
>
> On Fri, Jun 4, 2021 at 13:41 Linda Roos <> wrote:
>>
>> Hi,
>>
>> Internet2 successfully completed our first shim over the night of June 
>> 1-2. Thank you to KyRON for working with us in Louisville, KY.
>>
>> One of the things we learned is that the shim maintenance window will 
>> require an engineer from the Connector organization to be on the bridge 
>> with the Internet2 implementation team. Having a local engineer engaged 
>> will allow us to get immediate verification that any Layer2 services 
>> passing through the Internet2 network are working as anticipated. Layer2 
>> services are harder for Internet2 staff to verify since we don’t have any 
>> direct operational relationship that would alarm and provide feedback if 
>> the change doesn’t happen as expected. Having one of the engineers from 
>> your connector organization join our bridge during the maintenance will 
>> allow us to verify that everything on the connector side has restored 
>> before we end the maintenance call. This will be a real opportunity for 
>> all of us to learn about the NGI implementation together.
>>
>> What can you expect when your site comes up to be shimmed? Before the 
>> shim, the Internet2 team prepares for the shim by running fibers and 
>> educating the field tech on the process. During the shim maintenance, the 
>> work consists of swapping cables, ensuring that the Layer3 services 
>> re-establish on our side and all OESS VLANs have the same amount of 
>> traffic on them after the shim as before.
>>
>> We created a short video of a post-install meeting held by the NGI 
>> engineering team after the successful Louisville SHIM effort on June 1-2. 
>> You can drop in on the huddle to learn more about how we prepared, the 
>> role NSO played in the automated implementation, and hear what the 
>> implementation engineer experienced. Speakers include:
>>
>> Camille Davis-Alfs - Network Engineering Manager - GlobalNOC
>> Karl Newell Network Software Engineer - Internet2
>> Curtis Schongalla - Network Engineer- GlobalNOC
>> Hosted by Chris Robb - Senior Manager Internet2 Network Engineering - 
>> GlobalNOC
>>
>> https://internet2.hosted.panopto.com/Panopto/Pages/Viewer.aspx?id=3b7f26d1-f663-4053-a1b8-ad3d00f3f9d8
>>
>> The engineers will reach out to you to work out the details of the shim in 
>> your location. We really appreciate your identifying the appropriate 
>> engineer from your organization to join the shim maintenance. Please let 
>> me know what questions you have. Thanks.
>> Linda
>> —
>> Linda Roos
>> Sr Director, State and Regional Networks
>> Internet2
>> 
>> 812-558-0615
>>
>>
> --
> ===============================================
> David Farmer               Email:
> Networking & Telecommunication Services
> Office of Information Technology
> University of Minnesota
> 2218 University Ave SE        Phone: 612-626-0815
> Minneapolis, MN 55414-3029   Cell: 612-812-9952
> ===============================================
>
>
>