perfSONAR User Q&A and Other Discussion

[Jason Zurawski <>]

Hi Shiraz;

Comments inline:

On 11/17/10 8:09 AM, shiraz memon wrote:
> Dear Jason;
>
> Many thanks for your reply, it is indeed very insightful in getting
> closer to the decision of PerfSONAR's LS endorsement. However, few
> comments from my side are inline below:
>
> On Mon, Nov 15, 2010 at 3:06 PM, Jason Zurawski 
> <
> <mailto:>>
>  wrote:
>
>     Hi Shiraz;
>
>     The perfSONAR LS is a very general framework when it comes to
>     accepting information registration - any sort of 'service' that you
>     can represent in XML formats can be be registered using some of the
>     schemata we have defined for use in the project.  We normally break
>     information into 2 categories:
>
>       - Metadata e.g. information about the service such as how to
>     contact it
>       - Data e.g. capabilities of the service such as the form of data
>     it controls
>
>
>  From the desired perspective, both type of info. will be
> published/discovered.
>
>
>     As an example we use the LS to register traditional perfSONAR
>     services (e.g. Measurement Points, Measurement Archives) but have
>     also added the ability to insert other entities that are not
>     directly related to perfSONAR (e.g. GridFTP servers, Pheobus
>     Gateways, REDDnet depots). This is still fledgling functionality and
>     we do not have many more uses cases other than 'finding' these types
>     of services.  Parsing over registered information is possible, just
>     not fully explored.  Extending to some of the ideas you list below
>     may require you to add extensions to the LS.
>
>     I would encourage you to look at the 'LSRegistrationDaemon' package
>     
> (http://anonsvn.internet2.edu/svn/perfSONAR-PS/trunk/perfSONAR_PS-LSRegistrationDaemon/)
>     as a first step to see the process we have taken to implement
>     registration functionality.  It is written in perl and acts as a
>     broker between the LS and other forms of services (usually by doing
>     simple TCP connects to check availability of a target service).
>
> Sorry for my dearth of knowledge on Perl, however, Isn't it written
> entirely on Java (according to the mdm admin guide)?

There are two projects that have implemented the perfSONAR protocols to 
develop measurement frameworks:

 - perfSONAR-MDM implemented by the Geant2/Geant3 project and written 
in JAVA
 - perfSONAR-PS implemented by the perfSONAR-PS consortium 
(http://psps.perfsonar.net/) and written in Perl

I was giving you an example from the pSPS implementation since I am 
affiliated with that project, I do not believe there is a similar 
service available in the MDM implementation but I will let developers 
from that project answer.

>     The only real complexity that I can see currently with your ideas is
>     that while it is very simple to get the XML information into your
>     hLS, the gLS would need to know a little bit more about the data
>     types you would be registering for summarization purposes.
>       Currently this is something that the perfSONAR developers would
>     need to be aware of to adjust the algorithms, but we are working on
>     ways to completely automate this step.
>
> Are the developers already working on the extensible summarisation
> process/plug-ability?

Yes, there is joint work with the University of Delaware to simplify and 
extend the summarization algorithms in the current version of the 
perfSONAR-PS LS code.

> Finally, which license is being endorsed to distribute the PerfSONAR's
> LS (or the other components)?

I can't speak for the license in the MDM project, but the perfSONAR-PS 
project license is BSD compatible and is available here:

http://anonsvn.internet2.edu/svn/perfSONAR-PS/trunk/perfSONAR_PS-LookupService/doc/LICENSE

Thanks;

-jason

>     Hope this helps, thanks;
>
>     -jason
>
>
> Cheers,
> Shiraz
>
>     On 11/15/10 3:25 AM, shiraz memon wrote:
>
>         Hi Folks,
>
>         I am investigating LS currently and trying to find the potential
>         feasibility whether it could be used to discover the Grid
>         Services such
>         as Job Management, Storage Management, Computing etc... on "a large
>         scale" deployment. The aforementioned services are exposed as Web or
>         WS-RF compliant services, thus metadata defined would be in XML
>         format.
>
>         Here are some of the high level factors, that I would use to
>         asses the
>         viability of LS within Grids
>         1. enable the service providers (any, but some are mentioned
>         above) to
>         specify the metadata easily, that can be pulled/pushed from/into LS
>                ** How easy it would be to extend the LS, so as to
>         support the
>         info. publishing services, which are not a part of PerfSONAR's
>         network?
>         2. acquaintance of new services to LS
>                ** Can the LS simply allows the foreign services to
>         publish their
>         metadata
>         3. querying the stored information at hLS or gLS using standardised
>         language, such as XPath or XQuery
>                ** I think this is there, but how would the LS handles
>         the large
>         amount of query results?
>                ** Does it provide the iterating capabilities, if the
>         large set
>         of xml documents are returned from an XMLDB?
>         4. to publish and retrieve the information in GLUE2 format
>                ** INFO: GLUE2 is a standard from OGF, the model
>         encapsulates the
>         Grid entities. the Glue-WG also have released the renderings in
>         SQL, XML
>         and LDAP format.
>                ** How challenging and realistic is to add support for
>         GLUE2 model?
>         5. Is there any integration or extension available that can be
>         endorsed
>         out-of-the box?
>
>         Thanks in advance,
>         Shiraz