perfsonar development work

[Jason Zurawski <>]

Nina;


> > > The service version, introduced in newer LS versions help, and still 
> > > how do we interpret /servicetype="ma"/ and /service version="3.0"/ ? 
> > > Should it be uniquely recognized as RRD MA from perfsonar 3.0 
> > > distribution? What about flowSA MA ?  Is the expected meaning of 
> > > /serviceName  /defined?
> >
> > These are excellent observations, and points out an obvious flaw in 
> > allowing deployments to define their own keyword structure.  I think 
> > it is much safer to rely on data structure (eventTypes, etc.) to 
> > dictate the structure of that data rather than a set of self 
> > configured strings.
> Sounds reasonable. The following xquery follows this approach - it 
> returns list of available event types per each service, registered in 
> the queried LS.  I wonder if there is an easier way to obtain the same 
> information.
>
>     declare namespace nmwg="http://ggf.org/ns/nmwg/base/2.0/";;
>     declare namespace nmwgt="http://ggf.org/ns/nmwg/topology/2.0/";;
>     declare namespace
>     perfsonar="http://ggf.org/ns/nmwg/tools/org/perfsonar/1.0/";;
>     declare namespace
>     psservice="http://ggf.org/ns/nmwg/tools/org/perfsonar/service/1.0/";;
>
>
>     <nmwg:message>
>     {
>
>         for $y in doc("store.xml")/message/nmwg:store/nmwg:metadata
>            
>     return
>
>     <nmwg:metadata>
>         {$y/perfsonar:subject/psservice:service}
>     {
>
>         let $events :=  distinct-values(
>        
>         for $x in doc("store.xml")/message/nmwg:store/nmwg:data    
>             where $y/@id=$x/@metadataIdRef
>             return $x/nmwg:metadata/nmwg:eventType
>
>         )
>
>         for $e in $events
>         return     <nmwg:eventType>{ $e } </nmwg:eventType>   
>        
>     }   
>     </nmwg:metadata>
>     }
>
>     </nmwg:message>
>
> A similar query can be used to get the list of (filtered) topology 
> elements , together with event types and service addresses.
>
>     declare namespace nmwg="http://ggf.org/ns/nmwg/base/2.0/";;
>     declare namespace nmwgt="http://ggf.org/ns/nmwg/topology/2.0/";;
>     declare namespace
>     netutil="http://ggf.org/ns/nmwg/characteristic/utilization/2.0/";;
>     declare namespace
>     errors="http://ggf.org/ns/nmwg/characteristic/errors/2.0/";;
>     declare namespace
>     discards="http://ggf.org/ns/nmwg/characteristic/discards/2.0/";;
>     declare namespace
>     psservice="http://ggf.org/ns/nmwg/tools/org/perfsonar/service/1.0/";;
>     declare namespace
>     perfsonar="http://ggf.org/ns/nmwg/tools/org/perfsonar/1.0/";;
>
>
>     <nmwg:message>
>     {
>         for $y in doc("store.xml")/message/nmwg:store/nmwg:metadata
>         for $x in doc("store.xml")/message/nmwg:store/nmwg:data    
>         where
>                 $y/@id=$x/@metadataIdRef
>                 and
>     $y/perfsonar:subject/psservice:service/psservice:serviceType='ma'
>                 and (
>                 (
>                
>     $x/nmwg:metadata/nmwg:eventType='http://ggf.org/ns/nmwg/characteristic/discards/2.0'
>                 and            
>                
>     $x/nmwg:metadata/discards:subject/nmwgt:interface/nmwgt:ifAddress='194.141.0.10'
>                 )
>                 or (
>                
>     $x/nmwg:metadata/nmwg:eventType='http://ggf.org/ns/nmwg/characteristic/utilization/2.0'
>                 and            
>                
>     $x/nmwg:metadata/netutil:subject/nmwgt:interface/nmwgt:ifAddress='194.141.0.10'
>                 )
>                 or (
>                
>     $x/nmwg:metadata/nmwg:eventType='http://ggf.org/ns/nmwg/characteristic/utilization/2.0'
>                 and            
>                
>     $x/nmwg:metadata/netutil:subject/nmwgt:interface/nmwgt:ifAddress='194.141.252.41'
>                 )           
>                 )
>                
>         return
>         <row>
>             {$y/perfsonar:subject/psservice:service/psservice:accessPoint}
>             {$x/nmwg:metadata}
>         </row>
>         }
>     </nmwg:message>
>
>
> I am going to include both of queries in perfsonarUI and it would be 
> useful if the common API provides similar functionality.
>
> Michael, could you verify if these type of queries are sufficient for 
> FlowSA plugin?


I think these are good queries, although the second one is currently 
very implementation specific relying on certain eventType names as well 
as the version 2 interface.  You may run into problems where services 
implement the version 3 interface structure or if someone used the snmp 
(http://ggf.org/ns/nmwg/tools/snmp/2.0) eventType.  There are various 
wild-card structures that can be used in XQuery that should help with 
this (local-name is one I can remember off the top of my head, plus 
anything that functions as a regular expression).


> > > I guess event types might be the answer - but this needs to be 
> > > bootstrapped somehow. Can lookup service metadata query be used for 
> > > any type of service and return information in a more or less similar 
> > > way?  Currently, the result of a LS query is returned as *AnyElement*
> > >
> > > XQueryResults =
> > >         element psservice:datum { AnyElement* }
> > >         AnyElement =
> > >   element * {
> > >     (attribute * { text }
> > >      | text
> > >      | AnyElement)*
> > >   }  Provided, that the client should parse the XQuery results in 
> > > order to get the eventtypes, it should expect a well defined syntax. 
> > > Do we have such syntax that is valid for all type of services that 
> > > (at least) exist currently? For PIONIER RRD MA it contains 
> > > nmwgt:interface elements. Is it assumed this is the case for all 
> > > services?
> >
> >
> > I am not sure what type of bootstrapping or well defined syntax you 
> > would be expecting here, the queries that the LS is able to accept 
> > are quiet arbitrary and the resulting data will be as well.  The 
> > purpose of the API was to normalize the behavior of certain calls so 
> > that *every* client/service won't have to:
> >
> > a) create arbitrary XQuery syntax (perhaps redefining well known ways 
> > to get data)
> > b) deal with raw LS results that as you point out can be *anything*
> >
> > Instead of communicating directly with the LS, calling one of the 
> > functions (e.g. getLSLocation or some other proposed call from the 
> > gLS document) and get back an answer that is much more defined than 
> > arbitrary XML.  I am still looking for feedback on this, if one of 
> > the proposed calls doesn't do something you want or need, we should 
> > talk about this very soon.
> Sorry for irrelevant questions, I guess this workflow was not quite 
> clear to me and is somewhat different than currently used in perfsonarUI.
>
> Regarding the API, will it be possible
> - to have as parameters list of IP addresses / domain / event types , 
> rather than single IP address/domain/event type


This would be up to the API implementors, but if it makes sense to do 
this it should be possible.


> - when returning service information, also return relevant event types .


This may make more sense as a separate function.


> - when returning metadata elements, also to return corresponding 
> service information and event types.


Also may make sense a separate function.  I can propose something in the 
current document.


> > > <nmwg:metadata xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/"; 
> > > id="meta36">
> > >         <netutil:subject 
> > > xmlns:netutil="http://ggf.org/ns/nmwg/characteristic/utilization/2.0/"; 
> > > id="subj36">
> > >             <nmwgt:interface 
> > > xmlns:nmwgt="http://ggf.org/ns/nmwg/topology/2.0/";>
> > >                 
> > > <nmwgt:hostName>poznan-gw1.rtr.pionier.gov.pl</nmwgt:hostName>
> > >                 <nmwgt:ifName>xe-5/1/0.128</nmwgt:ifName>
> > >                 <nmwgt:ifDescription>Szczecin at poznan-gw1 
> > > 10GE</nmwgt:ifDescription>
> > >                 <nmwgt:ifAddress 
> > > type="ipv4">212.191.224.170</nmwgt:ifAddress>
> > >                 <nmwgt:direction>out</nmwgt:direction>
> > >                 <nmwgt:authRealm>PIONIER-Public</nmwgt:authRealm>
> > >                 <nmwgt:capacity>1000000000</nmwgt:capacity>
> > >             </nmwgt:interface>
> > >         </netutil:subject>
> > >         
> > > <nmwg:eventType>http://ggf.org/ns/nmwg/characteristic/utilization/2.0</nmwg:eventType> 
> > >
> > >     </nmwg:metadata>
> > >
> > > This is definitely not the first time I am bringing the question how 
> > > a generic client (working with any type of services) could bootstrap 
> > > itself.
> > >
> > > Speaking about LS metadata queries, is there any particular reason 
> > > that metadata query for RRD MA returns only list of metadata 
> > > elements, but not metadata/data pairs? This is the current behaviour 
> > > of PINIER RRD MA service from perfsonar 3.0 distribution 
> > > http://ls.perfsonar.pionier.net.pl:8080/xml-ls/services/LookupService
> > >
> > > The schema seems to specify metadata/data pairs. 
> > > https://svn.internet2.edu/svn/perfsonar/trunk/geant2_java-xml-ls/doc/schema/rnc/LSQueryResponse.rnc 
> >
> >
> > I am not sure what you mean by this, can you paste an example?
> Sorry again, I've expected the xquery results to follow some schema, 
> but then realized this is not true.

It follows about as strict as schema as possible (e.g. the datum element 
will contain well formed XML), but hopefully by using the API we can 
avoid some of the prickly issues of XQuery.  I believe the original 
intention of about 3 years ago was to never allow the XQuery interface 
to get this far, but we are only now realizing the importance of the 
information services.

-jason