Transport protocols and bulk file transfer

[Steven Low <>]

Hi Stas

I was travelling Fri and missed the call,
but your minutes are very helpful, thanks!
We'd like to continue to be involved,
contribute whatever that makes sense
(ideas, codes, hosting machines etc),
and in particular, an experimental
infrastructure - WAN in Lab - and
a theoretical infrastructure for
design, analysis, and testing of your
protocols.

Steven


stanislav shalunov wrote:

>         Transport Project Discussion Conference Call
>         2004-10-29
>         3:00-4:00PM US/Eastern time
>
> Participants:
>
>                 Bob Grossman - University of Illinois-Chicago
>                 Xinwei Hong - University of Illinois-Chicago
>                 David Hanley - University of Illinois-Chicago
>                 David Lapsley - MIT Haystack Observatory
>                 Shawn McKee - University of Michigan
>                 Steven Senger - University of Wisconsin at La Crosse
>                 Injong Rhee - University of North Carolina
>                 Martin Swany - University of Delaware
>                 Stanislav Shalunov - Internet2
>
> Scribe:         Terri Saarinen - Internet2
>
> Agenda:
>
>                 Introductions - 5minutes
>                 Agenda Bashing - 5 minutes
>                 Tentative Work Plan - 5 minutes
>                 Prospective Applications - 15 minutes
>                 Mode of Participation - 15 minutes
>                 Problem Factorization - 15 minutes
>                 Action Item Summary - 2 minutes
>
> (Discussion prior to start of call and clarification that this is an
> open source effort.)
>
> Introductions:
>
> Stanislav Shalunov: Internet2; trying to work with community to turn
> the excellent research in transport protocols into a user tool to
> solve actual problems.
>
> David Lapsley: Haystack; user of tool for eVLBI data transport; easy
> to use, easy to install and reliable.
>
> Bob Grossman: University of Illinois at Chicago; interested in working
> together.  UDT source forge just released by Yunhong Gu.  Heading off
> to teach a class, but David Hanley and Xinwei Hong will remain on the
> call.  Want to participate in this effort.
>
> Steven Senger: University of Wisconsin at La Crosse; collaborating
> with Parvati Dev and has written a couple of UDP-based protocols and
> would like to both cooperate and learn from this project.
>
> Shawn McKee: University of Michigan; physicist, interested in moving
> large amounts of data, would like to come up with workable solutions.
> Stas asked what tools Shawn uses for large scale transfer?  Shawn:
> GridFTP and Iperf for testing.  Storage resource brokers and managers,
> Grid FTP good, people open multiple streams, also Grid NFS funded by
> NMI.
>
> (Procedural note: Stas asked participants to identify themselves
> before speaking.)
>
> Martin Swany: University of Delaware; interested, eventually thinking
> about moving large data sets, has some ideas and wants to think about
> and hear what others are doing.  Interest is more on the researcher
> side than user side.
>
> Injong Rhee: North Carolina State University; worked on BIC-TCP
> protocol, interested in design for high speed, long distance network.
> He wants to contribute ideas and code.
>
> Agenda Comments?  (none)
>
> Prospective Applications:
>
> Killer application moving file from one point to another.
>
> Important to understand application before hand so APIs are designed
> right.
>
> What are the high level applications other than file transfer that
> bulk transfer would support?
>
> David: intensive data rates, say 4096 megabits/sec in near future.
> Real-time streams, timing, synchronization, the application can
> tolerate 5% loss and still produce meaningful results.
>
> Injong asked if David was seeing losses of 5-10%?  Sometimes.
>
> David L. mentioned fiber links at 1gig/sec.
>
> Reliability and not high bandwidth?
>
> David L. said reliability or high bandwidth depends on the
> application.  It it▓s a real time experiment, then there is a concern
> for high bandwidth flow and still get good results.  He gave further
> explanation about telescopes generating data at a high rate.  Aime for
> real time application is to keep correlator fed at constant rate.  Is
> link shared?  Currently a shared IP - Abilene to get access to
> telescope.  May have dedicated links in the future.
>
> Stas said that the tool could contain congestion control algorithm,
> descendent rate, correlator consumption, etc, what should happen?
>
> David L. talked about two data streams 512 megabit/sec into the
> correlator.  If 90% available bandwidth, then congestion control
> algorithm knows what is available and discards 10% of data stream.
> Filler frames to indicate loss, but still able to process the request.
>
> Stas said there needs to be a way to tell tool to not try to deliver
> all data but take data at a fixed rate and deliver what can be
> delivered.  Tool would be given minimum requirements, and if falls
> below, then transmission would not be useful and would yield link to
> others.
>
> David L. agreed.  Or specify nominal rate, and tolerate 10% loss =
> threshold, and only take action if 10% loss.  10% loss within a fair
> rate.  If in common rate is greater than fair rate, then 10% of data
> could be discarded.  What about bandwidth condition control?  If less
> than 90%, then you have to take other action.  Shut down, save til
> later, or store.  TFRC (TCP Friendly Rate Control) or TCP
> (Transmission Control Protocol)?  TFRC would be nice.  David L.:
> application can buffer up to about 5 seconds worth of data.
>
> Stas noted a 10Gb/s TCP connection with RTT=100ms and MTU=1500B would
> have oscillation period of 90 minutes.  To completely smooth that out,
> quite a bit more than a 5 sec buffer is needed.
>
> Other Applications?
>
> Steve: highly interaction, variable data rates, implemented his own
> transport to make it work.  Can tolerate modest loss sometimes, but
> other times he needs to recover it all.  Need to maintain activity
> between client rate and server.  He doesn▓t care about underlying
> protocol.  What if rate is lower than you want to send?  Steve would
> want information out of API and would change the amount of demand
> (codec, or resolution) at user lever.  Stas asked if data transmits
> and returns current desired rate it wants to be fed?  Yes, Steve could
> deal with that.  Interactive applications, spikey traffic, but not
> bulk transfer.  He wants to provide feedback to the user; what they
> can expect out of the application.
>
> Shawn: Mostly disk-to-disk bulk file transfer.  Open files across
> network of the future.  Depends on network infrastructure, virtual
> circuits, need to utilize infrastructure for access to data source.
> Video conferences.  Monitoring applications agent-based.  Bulk moved
> is in file storage to storage over or to CPU to do application. Disk
> to processors.  Depends if pre-staging data.  Output streams, but not
> where the processors are.
>
> Other requirements for API?
>
> Steve: a couple of apps have multiple individual streams transported.
> Congestion control is important.  Do streams sum to more than the size
> of the pipe?  Steve said that if end-point pipe size is unknown, then
> fighting self is a possibility.
>
> Stas sees work proceeding by first working on a prototype tool by
> first quarter or March 2005, testing ideas in congestion control
> algorithm to prove ideas can work or if changes are necessary.  And
> then re-doing what▓s necessary and releasing as a production tool.
> Agreement that this is the right track.
>
> Mode of Participation: What would you contribute?
>
> Shawn: testing role, some node access, some places to store large
> transfer at end of test.
>
> David L.: testing resources, large data nodes, ideas and advice.
> Code? Some existing, free to look it over, but few new coding
> resources available.
>
> Stas: wants outcome to be open source and usable by anyone interested.
> Internet2 likes BSD license.  Agreement that this is the appropriate
> way to go.  No one is saying ``why not GPL?''
>
> Injong: can contribute ideas and coding - some done by himself, some
> done by grad students.
>
> Injong asked about the availability of grant funding for this effort.
> Stas explained that so far, the intent is to get this project moving
> forward.  Once credible and a work in progress, then grant funding
> will be sought as tool applies to complete community of high-end
> network users.  **Let Stas know if your support needs external
> funding.  Injong agrees with approach to work unfunded at the
> beginning, then apply for funding to continue and finish effort.
>
> Martin: can contribute self and a part of a grad student for
> short-term, applying for funding to continue and finish effort.
>
> David H.: has code base that does a large amount of what▓s needed and
> he▓ll give it to the project.  Open source GPL (GNU public license) -
> can probably release it.
>
> Stas said GPL may be difficult since Internet2 is a consortium with
> commercial companies as partners.  Internet2 intellectual property
> framework says BSD.  **David H. will investigate.  
> ()
> **Stas will add David to the mail group, others interested may also
> self subscribe.
>
> Steve: would like to contribute however he can.  Testing, machine
> access at Stanford and Michigan, contributing code if no licensing
> problems.
>
> Problem Factorization:  Different dimensions of design space
> independent of each other.
>
> 1.      Congestion Control
> 2.      Kernel space vs. User Space
> 3.      Protocol and Framing
> 4.      Window based vs. Rate based
> 5.      Explicit Congestion Feedback vs. Implementation
> 6.      Reno-friendly vs. better proforming
> 7.      State mostly kept at the sender vs. the receiver
>
> Thoughts?
>
> David: great idea to capture and document.  May turn into an Internet
> draft down the road.
>
> Will any factorization be a problem?  Loss or delay, do you care?
>
> David L.: yes, there are user considerations.  VLBI runs mixed use for
> production transfer.  Maintenance and installation becomes an issue,
> especially on kernel if it is remote.
>
> Shawn: wants something that works.  If it▓s difficult to get all
> required framework to run, then it▓s harder to get adoption.  Ease of
> use is important.  Customized kernel makes it tough.  If it works, he
> does not really care about other details.
>
> Are there other design choices that need to be spelled out?  Loss
> based congestion control algorithm. The rest is explicitly mentioned.
> Stas thinks it would be good to have a design paper.
>
> Xinwei asked if transportation instrument needs to be on the kernel
> side?  File system interface.  David L. said that ideally if it▓s the
> same interface, then there is no change to existing software. He
> agrees with others value in layering out design space and
> understanding options.
>
> Action Item Review:
>
>     * Stas will review and edit minutes and then will post them to the list.
>
>     * Stas will discuss future call agenda.
>
>     * Stas will begin outline for design document.
>
>     * David H. will investigation possible licensing conflicts and
>       conditions for UDT.
>
>     * All to keep eyes open for relevant funding opportunities.
>
> Stas thanked folks for joining the call and agreeing to participate.
> Call ended at 3:59PM.

--
_____________________________________________
Steven Low                assoc prof, cs & ee
netlab.caltech.edu        tel: (626) 395-6767

          fax: (626) 568-3603