Transport protocols and bulk file transfer

[stanislav shalunov <>]

Yunhong Gu 
<>
 writes:

> This week and next week I am in Pittsburgh for SC04 (so do Xinwei
> and Dave H.), so I have to miss the phone conf again.

Talk to you next time and thanks for the comments in your email.

> I have read your design document and I think it covers most design
> issues.  From the design I read that you prefer a user space
> implementation with mixed loss and delay based control algorithm. I
> agree with this idea and I did this in a previous version of UDT
> (unfortunately our paper on this version of UDT never got published
> :)). I removed the delay based mechanish from UDT because it is hard
> to eliminate the estimation errors of delay variation. I may resume
> this mechanism in next version of UDT (UDT3) if I can find a more
> robust approach.

Could you tell us more about the approach that you tried?  Did you
look at delay-base_delay, packet train dispersion, or something else?

> About the in-bound/out-bound and path MTU issues, I think currently we
> do not need to care about them. They are independent components and
> are not critical.

I was trying to cover the design space.  Whatever decisions need to be
made.  There are relatively few dependencies (I try to mention them).
Eventually, PMTUD would need to be addressed and it might be easier to
keep it in mind when making other decisions -- which doesn't mean it
needs to be implemented right away.  In-band/out-of-band signaling is
something that could be hard to reverse later as it'd require changes
in the wire protocol.

> About the TCP friendliness issue.  IMO this is not a big problem as
> long as the new protocol does not starve a concurrent TCP to near 0
> throughput, nor it is starved by a concurrent TCP to near 0
> throughput.

Right, with an emphasis on the first part.

> About API, I'd prefer to use socket API plus overlapped IO and file
> transfer support.  The Windows Socket2 API is a good example.

I'm not familiar with Windows.  What's overlapped I/O and file
transfer?

At the last call, we were talking about interactive applications.
Socket API doesn't give the application feedback on the sending rate
-- instead, TCP just blocks (or doesn't accept data if you set the
nonblocking option) if it can't put it in the sending buffer.  This is
fine for certain things.  I think an additional advisory mechanism is
needed for interactive applications.  In addition, a socket-like API
(connect/accept then write/read) limits the implementation choices and
makes copies that are often unnecessary; sendfile() is a good idea (as
it lets you share disk and network buffers instead of duplicating them
and copying needlessly), but sockets don't let you have corresponding
receivefile().

-- 
Stanislav Shalunov              http://www.internet2.edu/~shalunov/

This message is designed to be viewed with 0.06479891g of NaCl.