All things related to multicast

[Hugh LaMaster <>]

On Sun, 10 Jun 2001, Bill Nickless wrote:

> I am involved in a project ( http://www.startap.net/starlight/ ) which I 
> hope to include, as one of its features, an inter-Autonomous System IPv4 
> multicast exchange point.  Following is a survey of the various options for 
> the layer-2/3 exchange point architecture I have considered, with pros and 
> cons of each.  I would be grateful for comments, corrections or suggestions 
> to flesh out these options and considerations--or any alternatives that I 
> have not considered.
> 
> SHARED MEDIUM
> =============
> This is the traditional multicast exchange point architecture, which 
> consists of a shared medium for all participants.  (e.g. FDDI, unbridged 
> Ethernet, Token Ring.)

This is the architecture of both MIXes at the Ames Research Center 
gateway.  We started with FDDI and moved to Ethernet/Gigabit Ethernet
with Jumbo Frames (MTU 4470 - ~ FDDI).


> Benefits:  - trivial to implement
               ^^^^^^^
  In retrospect, nothing is trivial.  But, there are few technical 
  challenges these days, if that is what you mean ;-)  But, this
  solution requires some active participation and coordination.

> Drawbacks: - all participants see all the multicast traffic, whether
>               or not they care to see it.
>             - May cause packet duplication.  Consider four peers, A, B, C,
>               and D.  A sends an (S,G) join to B.  C sends an identical
>               (S,G) join to D.   B and D each send the (S,G) traffic to
>               the exchange point.  A and C receive duplicate (S,G)
>               traffic, one copy each from B and D.
>             - Total exchange point bandwidth limited to that available
>               from the shared medium.

AFAIK, PIM Asserts stop the duplicate traffic.  The way that PIM
Asserts are supposed to work in this context is that all routers
MBGP peer with all others and have a common protocol preference 
(distance) that favors MBGP over, e.g., OSPF or IS-IS.

This usually works pretty well, although we have occasionally had
PIM Assert problems.

There is not much of a disadvantage in having all ports see all traffic
so far -- but, we are coming up against the limits of the slower routers
on the MIX right now, and, I would expect to see more of a problem 
in the future, because traffic has been building up over time.  

At this stage, I would require Jumbo Frame GigE on all ports on all 
routers as the price of admission for the exchange.  Router interfaces
must be able to see all of, and ignore most of, a high-speed stream 
of traffic -- this isn't a job for a router that can't handle the better
part of wire-speed traffic.  

Folks should also realize that we are beginning to see 20 Mbits/sec
streams now -- so FastE isn't going to be good enough.


> IGMPv2 SNOOPING HARDWARE
> ========================

Doesn't add anything useful, as you say, in a router-router configuration.


> SEPARATE VLAN FOR EACH PAIR OF PEERS
> ====================================
> One could isolate multicast traffic to interested peers by defining 
> separate VLANs for each pair of peers to use.  This would be logically 
> equivalent to the current ATM peering model in use at the Chicago NAP 
> (STARTAP/NGIX/MREN/etc) today.
> 
> Benefits:  - Isolates traffic to those peers interested in it.
>             - Allows peers to choose RPF individually for any (S,G).
> 
> Drawbacks: - Requires N^2 VLAN definitions (N=number of peers) in the
>               exchange point switching infrastructure
>             - Requires peers to support a trunking protocol (ISL, 802.1q)
>             - Requires peers to do packet replication for each
>               interested peer, duplicating traffic outbound on the
>               link towards the exchange point.
> 
> (Alternatively, s/a trunking protocol/MPLS/g and s/VLAN/MPLS tunnel/g)

NREN effectively has this in Chicago via ATM, where NREN peers with 
a large number of folks over ATM.  The good side is that there are no
PIM Assert problems to worry about, and policy discussions are pairwise,
simplifying administration considerably.  The bad side is that there is
a large amount of "duplicate" (perhaps redundant would be a better
word) traffic on the egress side, making this less efficient for both
router and switch.


> EXCHANGE POINT MULTICAST ROUTER
> ===============================
> Many Gigabit/10-Gigabit Ethernet switches include the capability of running 
> M-BGP, MSDP, and PIM-SM.  One could set up the exchange point switch in its 
> own Autonomous System and run M-BGP, MSDP, and PIM-SM with each peer.
> 
> Benefits:  - Isolates traffic to those peers interested
>             - Requires only N VLAN definitions (N=number of peers)
>               which is likely to be a tractable number
>             - Each peer only sends one copy of outgoing packets to
>               the exchange point; the exchange point replicates packets
> 
> Drawbacks: - Requires the exchange point operator to operate an
>               Autonomous System with everything that entails (a routing
>               policy, M-BGP, MSDP, etc.)  All peers would have to agree
>               on the exchange point routing policy, as the exchange
>               point would be choosing the RPF for any (S,G) requested
>               by any peer.
>             - Requires peers to support a trunking protocol (ISL, 802.1q)
>               or to have two exchange point connections (unicast, mcast).
>             - Complicates peer BGP routing policy, because the exchange
>               point AS would have to be preferred by the peers over
>               direct peerings, in spite of the shorter AS path that may
>               be available elsewhere.

I think this is a great solution when people are doing a "Gigapop" style
exchange point unicast solution.  I wouldn't mess with it if the unicast
exchange peerings are direct -- you're going to get an extra AS in the
path that you don't see for unicast.


> RGMP
> ====
> "RGMP constrains multicast traffic that exits the [Ethernet] switch through 
> ports to which only disinterested multicast routers are 
> connected."  Supported on Cisco switches (see 
> http://www.cisco.com/univercd/cc/td/doc/product/lan/cat6000/sw_6_2/confg_gd/multi.htm#xtocid2897139
>  
> for more information.)
> 
> Benefits:  - Peers don't get traffic for groups they don't care about.
> 
> Drawbacks: - Requires each peer (who wants the benefit) to use RGMP.
>             - May cause packet duplication.  Consider four peers, A, B, C,
>               and D.  A sends an (S,G) join to B.  C sends an identical
>               (S,G) join to D.   B and D each send the (S,G) traffic to
>               the exchange point.  RGMP operates on a (*,G) basis, so A
>               and C receive duplicate (S,G) traffic, one copy each from
>               B and D.
> 
> Fact:      - Would require the use of Cisco hardware in the exchange
>               point and by each peer, because RGMP is apparently a
>               Cisco-proprietary protocol.


RGMP is a possible enhancement to the "SHARED MEDIUM" solution above.
I see it as more important in a Catalyst-based campus LAN environment
where you might need to keep a lot of traffic from flooding out some
relatively slow (e.g. FastE) links.



> IGMPv3 SNOOPING
> ===============

I'm not sure how this would work router-router.


> PIM-SM SNOOPING/SPOOFING ETHERNET SWITCH
> ========================================
> Consider a Gigabit/10-Gigabit Ethernet switch that is PIM aware.  It could 
> set up appropriate hardware forwarding based on the PIM Joins sent between 

> Benefits:  - Peers get traffic only for (S,G)s they care about
>             - Peers send only one copy of traffic for (S,G)s and
>               the exchange point does the packet replication.
>             - Peers get to choose (by their own policy) the best
>               RPF for a given (S,G).
> 
> Drawbacks: - Not yet available in the marketplace (to my knowledge).

As others have stated, Foundry may have this working already, although
I don't have personal experience with it.  It certainly should work well
router-router, unlike the other snooping solutions.

I think it is a good solution, although, I tend to be nervous about any
snooping solution unless I know it is implemented extremely robustly --
solutions that end up running in software on the supervisor processor 
of the router/switch are potentially extremely vulnerable to any kind of
deliberate, or (more often) accidental, DoS attack.  For example, in this
case, what if some host scans all of 224/8.  IMHO, that should not cause the
local switch/router to interrupt the supervisor engine on each packet,
causing 100% CPU utilization.

> At this time I'm leaning towards setting up an exchange point router living 
> in its own autonomous system, because it's the most tractable option--it's 
> harder to configure but it *can* be configured.

I think the strongest solution is the shared-medium solution --
specifically a (jumbo-frame) GigE switch.


--
 Hugh LaMaster, M/S 233-21,    Email: 

 NASA Ames Research Center     Or:    

 Moffett Field, CA 94035-1000  Or:    

 Phone: 650/604-1056           Disc:  Unofficial, personal *opinion*.