perfSONAR User Q&A and Other Discussion

[Jason Zurawski <>]

Hi Shiraz;

On 11/18/10 7:09 AM, shiraz memon wrote:
> Hi,
>
> Sorry for prolonging this thread :)
>
> It is good to know that there are two flavors of LS, and having them
> would bring an added value to the project. However, it is important to
> know whether the two flavors are being synchronised in terms of the
> features/functionality or driven by the projects respectively?

Both LS instances are compatible at the protocol level.  There may be 
differences in things like the 'packaging' itself or the configuration 
steps and files, so its important to read the documentation before you 
proceed with either.

> Is there a separate mailing list for the mdm (geant2/3)? I wonder if
> they are reading this email conversation.


This is a shared project wide mailing list, I expect they are.

Thanks;

-jason


> Cheers,
> Shiraz
>
> On Wed, Nov 17, 2010 at 4:24 PM, Jason Zurawski 
> <
> <mailto:>>
>  wrote:
>
>     Hi Shiraz;
>
>     Comments inline:
>
>
>     On 11/17/10 8:09 AM, shiraz memon wrote:
>
>         Dear Jason;
>
>         Many thanks for your reply, it is indeed very insightful in getting
>         closer to the decision of PerfSONAR's LS endorsement. However, few
>         comments from my side are inline below:
>
>         On Mon, Nov 15, 2010 at 3:06 PM, Jason Zurawski
>         
> <
>  
> <mailto:>
>         
> <mailto:
>  
> <mailto:>>>
>         wrote:
>
>             Hi Shiraz;
>
>             The perfSONAR LS is a very general framework when it comes to
>             accepting information registration - any sort of 'service'
>         that you
>             can represent in XML formats can be be registered using some
>         of the
>             schemata we have defined for use in the project.  We
>         normally break
>             information into 2 categories:
>
>               - Metadata e.g. information about the service such as how to
>             contact it
>               - Data e.g. capabilities of the service such as the form
>         of data
>             it controls
>
>
>           From the desired perspective, both type of info. will be
>         published/discovered.
>
>
>             As an example we use the LS to register traditional perfSONAR
>             services (e.g. Measurement Points, Measurement Archives) but
>         have
>             also added the ability to insert other entities that are not
>             directly related to perfSONAR (e.g. GridFTP servers, Pheobus
>             Gateways, REDDnet depots). This is still fledgling
>         functionality and
>             we do not have many more uses cases other than 'finding'
>         these types
>             of services.  Parsing over registered information is
>         possible, just
>             not fully explored.  Extending to some of the ideas you list
>         below
>             may require you to add extensions to the LS.
>
>             I would encourage you to look at the 'LSRegistrationDaemon'
>         package
>
>           
> (http://anonsvn.internet2.edu/svn/perfSONAR-PS/trunk/perfSONAR_PS-LSRegistrationDaemon/)
>             as a first step to see the process we have taken to implement
>             registration functionality.  It is written in perl and acts as a
>             broker between the LS and other forms of services (usually
>         by doing
>             simple TCP connects to check availability of a target service).
>
>         Sorry for my dearth of knowledge on Perl, however, Isn't it written
>         entirely on Java (according to the mdm admin guide)?
>
>
>     There are two projects that have implemented the perfSONAR protocols
>     to develop measurement frameworks:
>
>       - perfSONAR-MDM implemented by the Geant2/Geant3 project and
>     written in JAVA
>       - perfSONAR-PS implemented by the perfSONAR-PS consortium
>     (http://psps.perfsonar.net/) and written in Perl
>
>     I was giving you an example from the pSPS implementation since I am
>     affiliated with that project, I do not believe there is a similar
>     service available in the MDM implementation but I will let
>     developers from that project answer.
>
>
>             The only real complexity that I can see currently with your
>         ideas is
>             that while it is very simple to get the XML information into
>         your
>             hLS, the gLS would need to know a little bit more about the data
>             types you would be registering for summarization purposes.
>               Currently this is something that the perfSONAR developers
>         would
>             need to be aware of to adjust the algorithms, but we are
>         working on
>             ways to completely automate this step.
>
>         Are the developers already working on the extensible summarisation
>         process/plug-ability?
>
>
>     Yes, there is joint work with the University of Delaware to simplify
>     and extend the summarization algorithms in the current version of
>     the perfSONAR-PS LS code.
>
>
>         Finally, which license is being endorsed to distribute the
>         PerfSONAR's
>         LS (or the other components)?
>
>
>     I can't speak for the license in the MDM project, but the
>     perfSONAR-PS project license is BSD compatible and is available here:
>
>     
> http://anonsvn.internet2.edu/svn/perfSONAR-PS/trunk/perfSONAR_PS-LookupService/doc/LICENSE
>
>     Thanks;
>
>     -jason
>
>
>             Hope this helps, thanks;
>
>             -jason
>
>
>         Cheers,
>         Shiraz
>
>             On 11/15/10 3:25 AM, shiraz memon wrote:
>
>                 Hi Folks,
>
>                 I am investigating LS currently and trying to find the
>         potential
>                 feasibility whether it could be used to discover the Grid
>                 Services such
>                 as Job Management, Storage Management, Computing etc...
>         on "a large
>                 scale" deployment. The aforementioned services are
>         exposed as Web or
>                 WS-RF compliant services, thus metadata defined would be
>         in XML
>                 format.
>
>                 Here are some of the high level factors, that I would use to
>                 asses the
>                 viability of LS within Grids
>                 1. enable the service providers (any, but some are mentioned
>                 above) to
>                 specify the metadata easily, that can be pulled/pushed
>         from/into LS
>                        ** How easy it would be to extend the LS, so as to
>                 support the
>                 info. publishing services, which are not a part of
>         PerfSONAR's
>                 network?
>                 2. acquaintance of new services to LS
>                        ** Can the LS simply allows the foreign services to
>                 publish their
>                 metadata
>                 3. querying the stored information at hLS or gLS using
>         standardised
>                 language, such as XPath or XQuery
>                        ** I think this is there, but how would the LS
>         handles
>                 the large
>                 amount of query results?
>                        ** Does it provide the iterating capabilities, if the
>                 large set
>                 of xml documents are returned from an XMLDB?
>                 4. to publish and retrieve the information in GLUE2 format
>                        ** INFO: GLUE2 is a standard from OGF, the model
>                 encapsulates the
>                 Grid entities. the Glue-WG also have released the
>         renderings in
>                 SQL, XML
>                 and LDAP format.
>                        ** How challenging and realistic is to add
>         support for
>                 GLUE2 model?
>                 5. Is there any integration or extension available that
>         can be
>                 endorsed
>                 out-of-the box?
>
>                 Thanks in advance,
>                 Shiraz