perfsonar development work

["Michael Bischoff" <>]

Hello all,

As I said in an other email I'm kinda behind on reading it so excuse me if I 
discuss some old
posts.

> I guess event types might be the answer - but this needs to be
> bootstrapped somehow. Can lookup service metadata query be used for any 
> type of service and
> return information in a more or less similar way? Currently, the result of 
> a LS query is
> returned as *AnyElement*

Xpath results have a predictable 'shape(structure?)'; given you know the 
'shape' of the data
it was preformed upon. And we do know this shape, since Lsstore is (almost 
completely)
formalised. The only part that isn't is the data element but services can 
formalise that in
there own documentation and since the 'client'(Or plugin) is based on(or tied 
too) that
service doing a LookupQuery shouldn't be issue.

---

So this works because there is a contract between the service and the client 
of the service,
however there is no contract between the LS and the client nor between the LS 
and the
service.


> These are excellent observations, and points out an obvious flaw in
> allowing deployments to define their own keyword structure.  I think it
> is much safer to rely on data structure (eventTypes, etc.) to dictate
> the structure of that data rather than a set of self configured strings.

no, we are advocating a fixed documented additional tag in the 
<psservice:service>
container that a correctly configured service registers in a predictable way. 
The keyword
proposal that flew by here on the mailing-list is unrelated.

>> That makes sense, be can we facilitate the whole process currently?
>> Don't we also need to ask
>> (a) service(s) to identify the network topology elements? If not the
>> current structure of the
>> topology itself.
>
>
> The topology elements are tied to the various data schemata through
> eventTypes.

not sure what you mean by this.

>>
>> Use Case:
>>   A client would use a topology service to look
>>   up the identifier for a network element and
>>   then would query a lookup service using the
>>   identifier to find the measurements
>>   associated with that element.
>>
>> Obtaining a topology service must also be retrieved from a LS? right?
>
> It could be done that way, but the Topology service and Lookup service
> are both well known information services (and as such will probbaly be
> combined in the near future).

combining them is a bad idea no matter how you slice it;

- it increases complexity of both the topology service as well as the ls 
unnecessary
- you enforce a implementation detail or have a incomplete spec (if someone 
else does a
implementation the communication between the topology and Ls service is not 
specified in a
perfsonar protocol.)
- it's more difficult to test, debug (you can't test them separately for 
starters)
- it doesn't scale
- The data-storage is crippled because you have to choose to optimise it for 
LS-queries or
Topology service
- etc

> From Maciej's point of view 'anyText' is really all he needs to care
> about.  There is some well formed blob that comes from a service that
> the LS can manage.  It is up to the individual service designer to
> elaborate this as much as possible.
>
> -jason

and thus it's only a contract between the service and it's client's and the 
ls shouldn't
depend on it.(Esp since you don't expect anything to be enforced here by the 
ls) Since it has
been established that for summerisation to work they need certain values. And 
thus the LS
depends on these values, it should be in that part of the schema that is 
formally defined;
the service in data tag.


If it helps us moving forward at this point then I can live with a de-facto 
standard that is
been established by one service (RRD) and needs just as much (or maybe even 
more)
communication effort between us all and relatively more work for service 
developers(the
proposed solution here is needs only a change in the code of the perfsonar 
base and services
should specify a few parameters. The solution thats on the table now needs 
the same plus
additional code in the services.

so if not for 3.1 for 3.2/4.0 this should be rectified.

<nmwg:store type="LSStore">
 <nmwg:metadata id=" ..service_id...">
  <perfsonar:subject>
   <psservice:service id="localhost.4c922942:11a2c3a5abf:-85">
    <psservice:serviceName>perfSONAR PIONIER RRD MA</psservice:serviceName>
    <psservice:accessPoint> ...url...</psservice:accessPoint>
    <psservice:serviceType>ma</psservice:serviceType>
    <psservice:serviceDescription>perfSONAR PIONIER RRD 
MA</psservice:serviceDescription>
    <psservice:serviceVersion>3.0</psservice:serviceVersion>
    <psservice:organization>PIONIER</psservice:organization>
    
<psservice:contactEmail></psservice:contactEmail>
    <psservice:supportedEventTypes>
     
<nmwg:eventType>http://ggf.org/ns/nmwg/characteristic/utilization/2.0</nmwg:eventType>
    </psservice:supportedEventTypes>
    <psservice:relatedTopologyElements>
     <!-- we can ping to anything in this domain-->
     <?:topologyElement urn="urn:ogf:network:domain=Internet2.edu" />
     <!-- we have utilization characteristics of this interface -->
     <?:topologyElement 
urn="urn:ogf:network:domain=internet2.edu:node=packrat:port=eth0" />
    </psservice:relatedTopologyElements>
   </psservice:service>
  </perfsonar:subject>
 </nmwg:metadata>
 <nmwg:data id=" ...id..." metadataIdRef="...id...">
  <nmwg:metadata id="meta36">
   ...anything by contract of the service...
  </nmwg:metadata>
 </nmwg:data>
<nmwg:store>


if the service needs to know more about the topologyElement/interface it 
should query the
topology service with the urn.
if the client needs a service it should query the ls with a urn.
if the client has an set of ips it should query the topology service to 
obtain information
and urn's, it can then use the urn's to find services that 'know' about the
ip's/topologyElements by querying the LS.

Using the above structure we can _enforce_ supportedEventTypes and 
relatedTopologyElements
(might need a rename) being there when the service registers. This is easier 
for all (Ls and
service developers.)

Added advantage is that because the structure is simpler the (x)queries are 
simpler and faster.

imo metadata in data tag should contain the least amount of data to get by.
I'm also against storing too much information in the ls about network 
elements (unless it is
a workaround for the topology service not being avail) the data will get 
outdated and might
conflict with other data being registered by other services storing only an 
id will promote
the single point of definition principle and avoids collisions. Added 
advantage is that
somewhere in the future we should be able to configure service's by just 
specifying the
topology element urn and it will retrieve the rest of the data from the 
topology service.
Domain renames, node renames and changing ip addresses should end up 
requiring very little
effort and ensures better availability of the perfsonar tools.

best regards,

Michael Bischoff

ps. maciej is on vacation so he won't join in soon, during the skypee 
conversations we
touched upon some of the subjects, while he wasn't against it he favoured the 
path of least
resistance(what ever that may be) also considering time constrains. Though I 
elaborated a bit
more on it here.