perfSONAR User Q&A and Other Discussion

[shiraz memon <>]

Hi,

Sorry for prolonging this thread :)

It is good to know that there are two flavors of LS, and having them would bring an added value to the project. However, it is important to know whether the two flavors are being synchronised in terms of the features/functionality or driven by the projects respectively?

Is there a separate mailing list for the mdm (geant2/3)? I wonder if they are reading this email conversation.

Cheers,

Shiraz

On Wed, Nov 17, 2010 at 4:24 PM, Jason Zurawski <> wrote:

Hi Shiraz;

Comments inline:

On 11/17/10 8:09 AM, shiraz memon wrote:

Dear Jason;

Many thanks for your reply, it is indeed very insightful in getting
closer to the decision of PerfSONAR's LS endorsement. However, few
comments from my side are inline below:

On Mon, Nov 15, 2010 at 3:06 PM, Jason Zurawski <

<mailto:>> wrote:

   Hi Shiraz;

   The perfSONAR LS is a very general framework when it comes to
   accepting information registration - any sort of 'service' that you
   can represent in XML formats can be be registered using some of the
   schemata we have defined for use in the project.  We normally break
   information into 2 categories:

     - Metadata e.g. information about the service such as how to
   contact it
     - Data e.g. capabilities of the service such as the form of data
   it controls


 From the desired perspective, both type of info. will be
published/discovered.


   As an example we use the LS to register traditional perfSONAR
   services (e.g. Measurement Points, Measurement Archives) but have
   also added the ability to insert other entities that are not
   directly related to perfSONAR (e.g. GridFTP servers, Pheobus
   Gateways, REDDnet depots). This is still fledgling functionality and
   we do not have many more uses cases other than 'finding' these types
   of services.  Parsing over registered information is possible, just
   not fully explored.  Extending to some of the ideas you list below
   may require you to add extensions to the LS.

   I would encourage you to look at the 'LSRegistrationDaemon' package
   (http://anonsvn.internet2.edu/svn/perfSONAR-PS/trunk/perfSONAR_PS-LSRegistrationDaemon/)
   as a first step to see the process we have taken to implement
   registration functionality.  It is written in perl and acts as a
   broker between the LS and other forms of services (usually by doing
   simple TCP connects to check availability of a target service).

Sorry for my dearth of knowledge on Perl, however, Isn't it written
entirely on Java (according to the mdm admin guide)?

There are two projects that have implemented the perfSONAR protocols to develop measurement frameworks:

 - perfSONAR-MDM implemented by the Geant2/Geant3 project and written in JAVA
 - perfSONAR-PS implemented by the perfSONAR-PS consortium (http://psps.perfsonar.net/) and written in Perl

I was giving you an example from the pSPS implementation since I am affiliated with that project, I do not believe there is a similar service available in the MDM implementation but I will let developers from that project answer.

   The only real complexity that I can see currently with your ideas is
   that while it is very simple to get the XML information into your
   hLS, the gLS would need to know a little bit more about the data
   types you would be registering for summarization purposes.
     Currently this is something that the perfSONAR developers would
   need to be aware of to adjust the algorithms, but we are working on
   ways to completely automate this step.

Are the developers already working on the extensible summarisation
process/plug-ability?

Yes, there is joint work with the University of Delaware to simplify and extend the summarization algorithms in the current version of the perfSONAR-PS LS code.

Finally, which license is being endorsed to distribute the PerfSONAR's
LS (or the other components)?

I can't speak for the license in the MDM project, but the perfSONAR-PS project license is BSD compatible and is available here:

http://anonsvn.internet2.edu/svn/perfSONAR-PS/trunk/perfSONAR_PS-LookupService/doc/LICENSE

Thanks;

-jason

   Hope this helps, thanks;

   -jason


Cheers,
Shiraz

   On 11/15/10 3:25 AM, shiraz memon wrote:

       Hi Folks,

       I am investigating LS currently and trying to find the potential
       feasibility whether it could be used to discover the Grid
       Services such
       as Job Management, Storage Management, Computing etc... on "a large
       scale" deployment. The aforementioned services are exposed as Web or
       WS-RF compliant services, thus metadata defined would be in XML
       format.

       Here are some of the high level factors, that I would use to
       asses the
       viability of LS within Grids
       1. enable the service providers (any, but some are mentioned
       above) to
       specify the metadata easily, that can be pulled/pushed from/into LS
              ** How easy it would be to extend the LS, so as to
       support the
       info. publishing services, which are not a part of PerfSONAR's
       network?
       2. acquaintance of new services to LS
              ** Can the LS simply allows the foreign services to
       publish their
       metadata
       3. querying the stored information at hLS or gLS using standardised
       language, such as XPath or XQuery
              ** I think this is there, but how would the LS handles
       the large
       amount of query results?
              ** Does it provide the iterating capabilities, if the
       large set
       of xml documents are returned from an XMLDB?
       4. to publish and retrieve the information in GLUE2 format
              ** INFO: GLUE2 is a standard from OGF, the model
       encapsulates the
       Grid entities. the Glue-WG also have released the renderings in
       SQL, XML
       and LDAP format.
              ** How challenging and realistic is to add support for
       GLUE2 model?
       5. Is there any integration or extension available that can be
       endorsed
       out-of-the box?

       Thanks in advance,
       Shiraz