perfsonar development work

[Nina Jeliazkova <>]

Jason,

> Nina;
>
>
> > > > The service version, introduced in newer LS versions help, and 
> > > > still how do we interpret /servicetype="ma"/ and /service 
> > > > version="3.0"/ ? Should it be uniquely recognized as RRD MA from 
> > > > perfsonar 3.0 distribution? What about flowSA MA ?  Is the expected 
> > > > meaning of /serviceName  /defined?
> > >
> > > These are excellent observations, and points out an obvious flaw in 
> > > allowing deployments to define their own keyword structure.  I think 
> > > it is much safer to rely on data structure (eventTypes, etc.) to 
> > > dictate the structure of that data rather than a set of self 
> > > configured strings.
> > Sounds reasonable. The following xquery follows this approach - it 
> > returns list of available event types per each service, registered in 
> > the queried LS.  I wonder if there is an easier way to obtain the 
> > same information.
> >
> >     declare namespace nmwg="http://ggf.org/ns/nmwg/base/2.0/";;
> >     declare namespace nmwgt="http://ggf.org/ns/nmwg/topology/2.0/";;
> >     declare namespace
> >     perfsonar="http://ggf.org/ns/nmwg/tools/org/perfsonar/1.0/";;
> >     declare namespace
> >     psservice="http://ggf.org/ns/nmwg/tools/org/perfsonar/service/1.0/";;
> >
> >
> >     <nmwg:message>
> >     {
> >
> >         for $y in doc("store.xml")/message/nmwg:store/nmwg:metadata
> >                return
> >
> >     <nmwg:metadata>
> >         {$y/perfsonar:subject/psservice:service}
> >     {
> >
> >         let $events :=  distinct-values(
> >                for $x in 
> > doc("store.xml")/message/nmwg:store/nmwg:data                where 
> > $y/@id=$x/@metadataIdRef
> >             return $x/nmwg:metadata/nmwg:eventType
> >
> >         )
> >
> >         for $e in $events
> >         return     <nmwg:eventType>{ $e } </nmwg:eventType>          
> >     }       </nmwg:metadata>
> >     }
> >
> >     </nmwg:message>
> >
> > A similar query can be used to get the list of (filtered) topology 
> > elements , together with event types and service addresses.
> >
> >     declare namespace nmwg="http://ggf.org/ns/nmwg/base/2.0/";;
> >     declare namespace nmwgt="http://ggf.org/ns/nmwg/topology/2.0/";;
> >     declare namespace
> >     netutil="http://ggf.org/ns/nmwg/characteristic/utilization/2.0/";;
> >     declare namespace
> >     errors="http://ggf.org/ns/nmwg/characteristic/errors/2.0/";;
> >     declare namespace
> >     discards="http://ggf.org/ns/nmwg/characteristic/discards/2.0/";;
> >     declare namespace
> >     psservice="http://ggf.org/ns/nmwg/tools/org/perfsonar/service/1.0/";;
> >     declare namespace
> >     perfsonar="http://ggf.org/ns/nmwg/tools/org/perfsonar/1.0/";;
> >
> >
> >     <nmwg:message>
> >     {
> >         for $y in doc("store.xml")/message/nmwg:store/nmwg:metadata
> >         for $x in doc("store.xml")/message/nmwg:store/nmwg:data    
> >         where
> >                 $y/@id=$x/@metadataIdRef
> >                 and
> >     $y/perfsonar:subject/psservice:service/psservice:serviceType='ma'
> >                 and (
> >                 (
> >                    
> > $x/nmwg:metadata/nmwg:eventType='http://ggf.org/ns/nmwg/characteristic/discards/2.0' 
> >
> >                 and                               
> > $x/nmwg:metadata/discards:subject/nmwgt:interface/nmwgt:ifAddress='194.141.0.10' 
> >
> >                 )
> >                 or (
> >                    
> > $x/nmwg:metadata/nmwg:eventType='http://ggf.org/ns/nmwg/characteristic/utilization/2.0' 
> >
> >                 and                               
> > $x/nmwg:metadata/netutil:subject/nmwgt:interface/nmwgt:ifAddress='194.141.0.10' 
> >
> >                 )
> >                 or (
> >                    
> > $x/nmwg:metadata/nmwg:eventType='http://ggf.org/ns/nmwg/characteristic/utilization/2.0' 
> >
> >                 and                               
> > $x/nmwg:metadata/netutil:subject/nmwgt:interface/nmwgt:ifAddress='194.141.252.41' 
> >
> >                 )                           )
> >                        return
> >         <row>
> >             
> > {$y/perfsonar:subject/psservice:service/psservice:accessPoint}
> >             {$x/nmwg:metadata}
> >         </row>
> >         }
> >     </nmwg:message>
> >
> >
> > I am going to include both of queries in perfsonarUI and it would be 
> > useful if the common API provides similar functionality.
> >
> > Michael, could you verify if these type of queries are sufficient for 
> > FlowSA plugin?
>
>
> I think these are good queries, although the second one is currently 
> very implementation specific relying on certain eventType names as 
> well as the version 2 interface.  You may run into problems where 
> services implement the version 3 interface structure or if someone 
> used the snmp (http://ggf.org/ns/nmwg/tools/snmp/2.0) eventType.  
> There are various wild-card structures that can be used in XQuery that 
> should help with this (local-name is one I can remember off the top of 
> my head, plus anything that functions as a regular expression).
Yes, the second one is more an example, the intention is to construct 
the query dynamically, given an eventtype and  interface data.


> > > > I guess event types might be the answer - but this needs to be 
> > > > bootstrapped somehow. Can lookup service metadata query be used for 
> > > > any type of service and return information in a more or less 
> > > > similar way?  Currently, the result of a LS query is returned as 
> > > > *AnyElement*
> > > >
> > > > XQueryResults =
> > > >         element psservice:datum { AnyElement* }
> > > >         AnyElement =
> > > >   element * {
> > > >     (attribute * { text }
> > > >      | text
> > > >      | AnyElement)*
> > > >   }  Provided, that the client should parse the XQuery results in 
> > > > order to get the eventtypes, it should expect a well defined 
> > > > syntax. Do we have such syntax that is valid for all type of 
> > > > services that (at least) exist currently? For PIONIER RRD MA it 
> > > > contains nmwgt:interface elements. Is it assumed this is the case 
> > > > for all services?
> > >
> > >
> > > I am not sure what type of bootstrapping or well defined syntax you 
> > > would be expecting here, the queries that the LS is able to accept 
> > > are quiet arbitrary and the resulting data will be as well.  The 
> > > purpose of the API was to normalize the behavior of certain calls so 
> > > that *every* client/service won't have to:
> > >
> > > a) create arbitrary XQuery syntax (perhaps redefining well known 
> > > ways to get data)
> > > b) deal with raw LS results that as you point out can be *anything*
> > >
> > > Instead of communicating directly with the LS, calling one of the 
> > > functions (e.g. getLSLocation or some other proposed call from the 
> > > gLS document) and get back an answer that is much more defined than 
> > > arbitrary XML.  I am still looking for feedback on this, if one of 
> > > the proposed calls doesn't do something you want or need, we should 
> > > talk about this very soon.
> > Sorry for irrelevant questions, I guess this workflow was not quite 
> > clear to me and is somewhat different than currently used in 
> > perfsonarUI.
> >
> > Regarding the API, will it be possible
> > - to have as parameters list of IP addresses / domain / event types , 
> > rather than single IP address/domain/event type
>
>
> This would be up to the API implementors, but if it makes sense to do 
> this it should be possible.
>
>
> > - when returning service information, also return relevant event types .
>
>
> This may make more sense as a separate function.
>
>
> > - when returning metadata elements, also to return corresponding 
> > service information and event types.
>
>
> Also may make sense a separate function.  I can propose something in 
> the current document.
>
>
> > > > <nmwg:metadata xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/"; 
> > > > id="meta36">
> > > >         <netutil:subject 
> > > > xmlns:netutil="http://ggf.org/ns/nmwg/characteristic/utilization/2.0/"; 
> > > > id="subj36">
> > > >             <nmwgt:interface 
> > > > xmlns:nmwgt="http://ggf.org/ns/nmwg/topology/2.0/";>
> > > >                 
> > > > <nmwgt:hostName>poznan-gw1.rtr.pionier.gov.pl</nmwgt:hostName>
> > > >                 <nmwgt:ifName>xe-5/1/0.128</nmwgt:ifName>
> > > >                 <nmwgt:ifDescription>Szczecin at poznan-gw1 
> > > > 10GE</nmwgt:ifDescription>
> > > >                 <nmwgt:ifAddress 
> > > > type="ipv4">212.191.224.170</nmwgt:ifAddress>
> > > >                 <nmwgt:direction>out</nmwgt:direction>
> > > >                 <nmwgt:authRealm>PIONIER-Public</nmwgt:authRealm>
> > > >                 <nmwgt:capacity>1000000000</nmwgt:capacity>
> > > >             </nmwgt:interface>
> > > >         </netutil:subject>
> > > >         
> > > > <nmwg:eventType>http://ggf.org/ns/nmwg/characteristic/utilization/2.0</nmwg:eventType> 
> > > >
> > > >     </nmwg:metadata>
> > > >
> > > > This is definitely not the first time I am bringing the question 
> > > > how a generic client (working with any type of services) could 
> > > > bootstrap itself.
> > > >
> > > > Speaking about LS metadata queries, is there any particular reason 
> > > > that metadata query for RRD MA returns only list of metadata 
> > > > elements, but not metadata/data pairs? This is the current 
> > > > behaviour of PINIER RRD MA service from perfsonar 3.0 distribution 
> > > > http://ls.perfsonar.pionier.net.pl:8080/xml-ls/services/LookupService
> > > >
> > > > The schema seems to specify metadata/data pairs. 
> > > > https://svn.internet2.edu/svn/perfsonar/trunk/geant2_java-xml-ls/doc/schema/rnc/LSQueryResponse.rnc 
> > >
> > >
> > > I am not sure what you mean by this, can you paste an example?
> > Sorry again, I've expected the xquery results to follow some schema, 
> > but then realized this is not true.
>
> It follows about as strict as schema as possible (e.g. the datum 
> element will contain well formed XML), but hopefully by using the API 
> we can avoid some of the prickly issues of XQuery.  I believe the 
> original intention of about 3 years ago was to never allow the XQuery 
> interface to get this far, but we are only now realizing the 
> importance of the information services.
When playing with XQuery, it occurred to me that one can actually 
generate the subsequent message (e.g. setupdata request)  by the xquery 
and submit the result obtained from LS query directly to the information 
service ... thus eliminating all XML parsing and creation again.  Will 
be nice optimization of the client:)

Regards,
Nina
> -jason