Internet2 Network Security SIG

[Paul Howell <>]

Hi Everyone,

Thank you for your time and interesest in network security for our
community.  It was a pleasure to meet you and I¹m looking forward to
seeing you in person at the upcoming Global Summit.

Here are the notes that we took.   If you happen to have corrections,
please send them to Eric or myself.

Michael: Attack traffic across I2 (sourced by or directed at a member)
echo sentiments of everyone else, pipe dream: mechanism where attacks
are detected automatically, notify connector, with their permission,
block on ingress into I2 networks (including transpacific peerings),
recent botnet started by high school student, block things before they
hit US network

Nick: Sourcing DDOS traffic is more important that receiving DDOS
traffic, order of magnitude more sourcing than receiving; set of stages
that would be useful (things he would have wanted when running regional,
planned for ESnet): agreed upon set of BGP communities that allow for
black hole routing, automated based on sFlow data, allows for
sledgehammer, failure trivial to implement, ability for a connector to
announce a bad actor, want it blocked, I2 pushes it upstream to transit
providers who allow for this as well (hairier); flowspec builds on top
of that (more granular, resource heavy, fine tuned blocking), has done
work on flowspec for other things and it works very well; couple of very
good NANOG talks (black hole routing, filtering, DDOS attacks, SDN
through flowspec), e.g. block regional attacks (anything more than 1,000
miles from my region); maybe a couple education sessions for those who
don't know how to deal with this

Jeff: How do we protect ourselves from high velocity attack (internal
smarts to shut down an attack), but a 10G stream into us is hard to handle

Ken: trying to roll out mitigation at border to handle sFlow data, last
week 40,000 souces / 25 minutes / filled pipes coming in, would like to
push it upstream further, make tools more stable, operational than a
testbed, try to push it out one hop higher, last attack was 2.5 gig
commodity traffic, 5or 6 DDos attacks last week, some smaller, out of 5,
2 filled commodity pipe for 10 minutes or longer, mostly hammered on
commodity traffic, usually someone hammering an Xbox in a residence hall,

Dave: Service provider ... manual process to work with those who are
hammered, would like automation / be more proactive / regional perspective

Will: Interested in seeing what can be done upstream, mitigated attacks
with ACLs on campus, 2 weeks ago big DDOS attack on campus, had to work
with regional provider to block it, they were throwing away 11.5 GIG
inbound, would have saturated 10G pipe

Paul: Echo ISP comments, null route using BGP, in DDOS scenarios, they
have 10-12 ports into network, only way to react in a meaningful, quick
way; in some ways attack succeeds by sparing network at expense of
target, having a way that moves it out farther closer to sources would
be good for OARNET, signal upstream, using BGP, in his scenario, not
able to get enough of a grip (depends on attack profile) to knock some
of it down, been experimenting with throttling attack, but not cutting
it off, using policers; recent attack using UDP 443 against state of
Ohio, throttled it

Tony: not a lot to add, there was a presentation by GEANT at SMM where
they discussed how they were dealing with it, wondering if that could
benefit US (Paul H is talking with Wayne Routely regularly), pretty
impressive automation, dealing with larger DDOS attacks and protecting
customers

Roy: echo what others have said, stop it further upstream, so
institutiaonl connection isn't compromised, but also keep pipes open for
science, interest in seeing if I2 can work with regionals for a standard
mechanism to do this, looking at this for something more fine-grained
(flowspec?) than blackholing source IP

Paul H: Scrubbing Services?

If you detect an attack, you reroute attack traffic to scrubbing
service, scrubbing service removes malicious traffic, but allows
non-malicious traffic through. Better than null routing using BGP. Don't
take out target. Squelch actual attack. Scrubbing services are a
reflection of not being able to block attack closest to source.
Scrubbing service that works on traffic through I2, and from commodity
traffic (redirected to scrubbing service).

Ken: It's similar to what they are doing. Hybrid OF. Focusing on
destination or person being attacked and doing scrubbing of known attacks.

Roy: Is scrubbing done on exiting I2 or entering I2? We would do it
where there were the least consequences after receiving approval from
connector.

Paul H: Scrubbing would not just be for attack traffic traversing I2. If
we went in on a common scrubbing service, allow folks to send commodity
traffic to scrubbing service and receive a cleaner stream.

Roy: If TR-CPS? Is scrubber needed? Or do you need to block it?

Paul H: Do scrubbing services have value?

Nick: Scrubbing services and flowspec are apples and oranges. Interested
to build R&E modular scrubbing service. Could also research traffic that
goes through it. Not as many privacy issues as shipping data beyond I2
(or vetting a trusted source). If implementation of scrubbing service
works well, there could be a lot of value for application-specific
attacks. Wondering if SDN bits in I2 would lend themselves to making it
easier than for a typical enterprise.

Ken: Local institution could load rules up to I2 to be implemented by
SDN capabilities. Presentation on this. Box folder has it. Detecit it
yourself. Be able to publish attack to other schools.

Tony: Could only block traffic destined towards the connector. Could add
or ignore.

Paul H: Is everyone using SDN? For some definition of using, but not in
production. SDN solutions are interesting, worth pursuing, wondering if
there's enough schools that can do it.

Eric: Idea would be that SDN in core, not in regional or campus. Campus
could push their decisions upstream into I2 to mitigate the attack.
Other campuses could adopt if necessary.

Paul H: Balance of commercial vs. open solutions. Any preference?

Michael, Tony: Many of us are too budget constrained for many commercial
products. Hard to justify until after time when you need it.

Ken: Looking at build vs. buy. Lots of work with Brocade, Inmon. Trying
to beat up on Juniper to get sFlow stuff in there as well. At SC,
figured out Arista and Alcatel are supporting sFlow. Some of our tools
can look at other vendors.

Paul H: One option would be a Net+ offering that brought together
scrubbing, detection capabilities, under a Net+ service. Is that of
interest? Budget standpoint might be problematic.

Michael: In principle that would be a good approach, but perhaps not the
only approach. The stuff Ken is talking about (portal with sufficient
AAA to upload block requests and have them installed as required), is
another approach.

Paul H: That would help with TR-CPS and R&E attacks, but not commodity
attacks.

__: Maybe R&E community could set a precedent for internet exchanges.
Show value dollars / sense.

Michael: At some level, preferable to have attacks come in over TR-CPS
allows avoidance of bursting charges by commodity internet providers for
individual connectors.

Paul H: These are great ideas. Are some or all of you interested in
forming a WG?

Tony: F2F meeting this spring. +1 here.

Either way ...


Roy: What's the scope of this WG? Network Security WG? Security WG? On
the agenda, talk about how to rope in a member of security group from
campuses. Overlap with security activity.

Paul H: Security-focused WG. Initial topic is around DDOS, doesn't limit
it.

Roy: If it's a security WG, membership larger than just network engineers.

Nick: Also a very good opportunity to engage those two teams even inside
insitution, opportunity to tightly couple them, something like this
would require that. Would be a valuable product.

__: Agree. Based on CC-IIE grant. Brings together networking, HPC,
security, advanced cyberinfrastructure. Security group is really looking
at using SDN and router capabilties.