perfsonar development work

[Jason Zurawski <>]

Jochen & All;

> I'm mainly responsible for the rsync and Perl Storable part of the story, that 
> is and will not be part of perfSONAR. For the perfSONAR specific stuff others 
> are more competent and should answer the questions.
>   

From a protocol standpoint, I am really interested in seeing what 
messages will be used to actually convey the message that an 'out of 
band transfer' is necessary.  Even though the focus is currently on the 
behind the scenes work, there needs to be some message kicking off this 
action that someone will need to formulate. 

If you haven't thought much about what the message looks like yet, I 
would suggest something similar to this:

<nmwg:message type="SetupDataRequest" 
xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/";>

 <nmwg:metadata id="m-1">
   <!-- whatever it looks like -->
 </nmwg:metadata>

 <nmwg:data id="d-1" metadataIdRef="m-1">
   <nmwg:key id="k-1">
     <nmwg:parameters id="pk-1">
       <nmwg:parameter name="rsync">something...</nmwg:parameter>
     </nmwg:parameters>
   </nmwg:key>
 </nmwg:data>

</nmwg:message>


Even though this is a Hades only extension for now, properly specifying 
what modifications are to be made is pretty important.  Especially for 
other MA developers who are Jaded on XML parsing. 

-jason


> regards,
> Jochen
>
> On Tuesday 06 May 2008 16:07, Jeff W. Boote wrote:
>   
> > On May 6, 2008, at 3:37 AM, Jochen Reinwand wrote:
> >     
> > > On Monday 05 May 2008 17:46, Joe Metzger wrote:
> > >       
> > > > Jeff & Jochen,
> > > > I am not too concerned about the technical details about how
> > > > to use messages to exchange RSYC URLs.  I am concerned
> > > > about the big picture and how this decision affects the
> > > > future of the perfSONAR protocols.
> > > >         
> > > The only thing that is important for the perfSONAR protocol is the
> > > way the
> > > rsync URLs are put into the perfSONAR messages. What this URLs mean
> > > is out of
> > > scope for perfSONAR. This is the basic idea: Make your data
> > > available via
> > > some (most likely otherwise standardised) mechanism and use the
> > > power of
> > > perfSONAR, especially regarding meta data, to point the other
> > > communication
> > > partners to that source.
> > >       
> > To a large extent I agree with you.
> >
> > The 'however' is that I think we need to come up with a language
> > independent perfSONAR based efficient data transport mechanism.
> > Otherwise, other services will create something similar to what you
> > are doing here and over time it could become very difficult to write
> > client applications that can consume multiple sources of data.
> > (Consider: how easy will it be for pS-UI to consume data using this
> > mechanism given the perl objects?) Given that the over-arching goal of
> > perfSONAR is to make many sources of data available in a common
> > infrastructure, surely you can see how this language specific
> > transport is at least somewhat counter to that goal.
> >
> > But, I do see that this trade-off is needed for you in the short term.
> > This is why I would like to hear more details on how much of the issue
> > is with parsing, and how much has to do with in-lining the data with
> > the control communication. So we can attempt to rectify that situation
> > in the medium to long term.
> >
> > The other concern I have is simply that I would like to see
> > specifically how you intend to put the URL's into the messages, and I
> > guess I would like to see the eventType clearly indicate that this
> > type of data transport is outside the scope of perfSONAR.
> >
> > Each new development group that comes on board looks at existing
> > services for the example on how to write services. This particular
> > example trades interoperability for performance. This is reasonable
> > given your goals, but we need to clearly explain the limits of this
> > type of transport to ensure this method is not copied by those that
> > don't understand the limitations.
> >
> > jeff
> >
> >     
> > > > Can you please clarify for me if this will be a private,
> > > > internal interface for your service, or if you expect this to be a
> > > > change
> > > > to the public perfSONAR protocol that will be used by GUI's and
> > > > other services to access the data stored in your service?
> > > >         
> > > It's an enhancement to the protocol and another way of accessing
> > > Hades (and
> > > perhaps even other) data. The first one is a simple enhancement
> > > possibly
> > > useful in some other cases. The later is a new functionality that
> > > has nothing
> > > to do with perfSONAR, but that we should not hide from users either.
> > >
> > >       
> > > > What I hope you are doing is using a perfSONAR-like protocol internal
> > > > to a perfSONAR service to move data around.  If this is what
> > > > is going on, then great, I have no problem. Developers must be able
> > > > to architect the internals of their services in any way that
> > > > meets their requirements, and passing around language specific
> > > > serialized objects is expected internal to a service.
> > > >         
> > > Perhaps you like this description better: It is a perfSONAR
> > > compliant service
> > > enhancement that allows fast data access via Perl storables and
> > > rsync. It is
> > > especially useful for a specific purpose (fast interaction between to
> > > specific perfSONAR tools), but can be useful in other cases, too.
> > >
> > > Can you please give a good reason why it shouldn't be possible to
> > > include a
> > > URL as data in perfSONAR messages? I simply don't see the point...
> > >
> > > Jochen
> > >
> > > --
> > > Jochen Reinwand, DFN-Labor
> > > Friedrich-Alexander-Universität Erlangen-Nürnberg
> > > Regionales RechenZentrum Erlangen (RRZE)
> > > Martensstraße 1, 91058 Erlangen, Germany
> > > Tel. +49 9131 85-28689, -28800, Fax +49 9131 302941
> > >
> > > www.win-labor.dfn.de