perfsonar development work

["Jeff W. Boote" <>]

On May 6, 2008, at 3:37 AM, Jochen Reinwand wrote:

> On Monday 05 May 2008 17:46, Joe Metzger wrote:
> > Jeff & Jochen,
> > I am not too concerned about the technical details about how
> > to use messages to exchange RSYC URLs.  I am concerned
> > about the big picture and how this decision affects the
> > future of the perfSONAR protocols.
>
> The only thing that is important for the perfSONAR protocol is the  
> way the
> rsync URLs are put into the perfSONAR messages. What this URLs mean  
> is out of
> scope for perfSONAR. This is the basic idea: Make your data  
> available via
> some (most likely otherwise standardised) mechanism and use the  
> power of
> perfSONAR, especially regarding meta data, to point the other  
> communication
> partners to that source.

To a large extent I agree with you.

The 'however' is that I think we need to come up with a language  
independent perfSONAR based efficient data transport mechanism.  
Otherwise, other services will create something similar to what you  
are doing here and over time it could become very difficult to write  
client applications that can consume multiple sources of data.  
(Consider: how easy will it be for pS-UI to consume data using this  
mechanism given the perl objects?) Given that the over-arching goal of  
perfSONAR is to make many sources of data available in a common  
infrastructure, surely you can see how this language specific  
transport is at least somewhat counter to that goal.

But, I do see that this trade-off is needed for you in the short term.  
This is why I would like to hear more details on how much of the issue  
is with parsing, and how much has to do with in-lining the data with  
the control communication. So we can attempt to rectify that situation  
in the medium to long term.

The other concern I have is simply that I would like to see  
specifically how you intend to put the URL's into the messages, and I  
guess I would like to see the eventType clearly indicate that this  
type of data transport is outside the scope of perfSONAR.

Each new development group that comes on board looks at existing  
services for the example on how to write services. This particular  
example trades interoperability for performance. This is reasonable  
given your goals, but we need to clearly explain the limits of this  
type of transport to ensure this method is not copied by those that  
don't understand the limitations.

jeff

> > Can you please clarify for me if this will be a private,
> > internal interface for your service, or if you expect this to be a
> > change
> > to the public perfSONAR protocol that will be used by GUI's and
> > other services to access the data stored in your service?
>
> It's an enhancement to the protocol and another way of accessing  
> Hades (and
> perhaps even other) data. The first one is a simple enhancement  
> possibly
> useful in some other cases. The later is a new functionality that  
> has nothing
> to do with perfSONAR, but that we should not hide from users either.
>
> > What I hope you are doing is using a perfSONAR-like protocol internal
> > to a perfSONAR service to move data around.  If this is what
> > is going on, then great, I have no problem. Developers must be able
> > to architect the internals of their services in any way that
> > meets their requirements, and passing around language specific
> > serialized objects is expected internal to a service.
>
> Perhaps you like this description better: It is a perfSONAR  
> compliant service
> enhancement that allows fast data access via Perl storables and  
> rsync. It is
> especially useful for a specific purpose (fast interaction between to
> specific perfSONAR tools), but can be useful in other cases, too.
>
> Can you please give a good reason why it shouldn't be possible to  
> include a
> URL as data in perfSONAR messages? I simply don't see the point...
>
> Jochen
>
> --
> Jochen Reinwand, DFN-Labor
> Friedrich-Alexander-Universität Erlangen-Nürnberg
> Regionales RechenZentrum Erlangen (RRZE)
> Martensstraße 1, 91058 Erlangen, Germany
> Tel. +49 9131 85-28689, -28800, Fax +49 9131 302941
>
> www.win-labor.dfn.de