perfsonar development work

[Jason Zurawski <>]

Roman;


> > > > > > > 6) 
> > > > > > > http://anonsvn.internet2.edu/svn/nmwg/branches/snmp/example_instances/errors/MDKRequest2.xml 
> > > > > > >
> > > > > > >
> > > > > > > Parameters in the key in metadata m3 represent more than one 
> > > > > > > namespace so it should look like this:
> > > > > > >
> > > > > > >    <nmwg:key id="1">
> > > > > > >      <nmwg:parameters id="1">
> > > > > > >        <nmwg:parameter 
> > > > > > > name="supportedEventType">http://ggf.org/ns/nmwg/characteristic/errors/2.0</nmwg:parameter> 
> > > > > > >
> > > > > > >        <nmwg:parameter name="type">rrd</nmwg:parameter>
> > > > > > >        <nmwg:parameter name="valueUnits">Eps</nmwg:parameter>   
> > > > > > >        <nmwg:parameter 
> > > > > > > name="file">/home/jason/perfSONAR-PS/MP/SNMP/mead.rrd</nmwg:parameter> 
> > > > > > >
> > > > > > >        <nmwg:parameter 
> > > > > > > name="dataSource">eth0-ifInErrors</nmwg:parameter>
> > > > > > >      <nmwg:parameters id="2">
> > > > > > >      <select:parameters id="2">
> > > > > > >        <nmwg:parameter name="startTime" value="N-50" />
> > > > > > >        <nmwg:parameter name="endTime" value="N" />
> > > > > > >        <nmwg:parameter name="resolution" value="1" />
> > > > > > >        <nmwg:parameter name="consolidationFunction" 
> > > > > > > value="AVERAGE" />           </select:parameters>
> > > > > > >    </nmwg:key>
> > > > > > >
> > > > > > > This approach should be used in requests and responses.
> > > > > > This is too cumbersome and allowing multiple parameters blocks 
> > > > > > doesn't look correct in this case.  We really should use a chain 
> > > > > > to describe what is going on, or because the contents of the key 
> > > > > > is opaque it should not matter what namespace is used to describe 
> > > > > > the internals.
> > > > > I think it's quite clear and straightforward. Right, what's inside 
> > > > > does not matter for a client but for a service might be 
> > > > > interesting and needed not to lose context of parameters. Having 
> > > > > all parameters in one namespace nmwg, which in fact might be wrong 
> > > > > for some of them, is losing some information.
> > > > There is no information loss because the key has been 'cooked' in 
> > > > one of its previous visits and the server has organized the 
> > > > information in a format it likes, including extra parameters blocks 
> > > > in various namespaces really only aids humans as they examine the 
> > > > key (which they shouldn't do anyway as it is opaque).
> > > > Ideally we shouldn't be passing back a clear key where it is 
> > > > obvious to see what we have done (the key structure if I recall 
> > > > correctly should be used to save state on the server end), we 
> > > > should start moving to an obfuscated key that means nothing by sight.
> > > I agree with you that the key has no meaning for a client but a 
> > > request with key for service side is a different case. Why not to 
> > > keep namespaces as originally? It might be useful for RRD MA (at 
> > > least for Java implementation). It will not be the rule for all 
> > > services of course. Other services might use totally different keys 
> > > (even without parameters :) In fact, client app should not generate 
> > > keys, it should get it from a service.
> > I would still *much* rather we go back to including the entire chain 
> > in these communications, that way we are guaranteed to not lose 
> > anything.  If we are going to treat the entire
> Hmm, but a key can represent a chain of metadata. This is my 
> understanding of a key (something which is just an id; it does not 
> matter how it looks like; I could imaging that a service could have a 
> map structure with pairs key - metadata chain ).


Thats not entirely true, a cooked key gives you access to the combined 
effects of the end of a chain of metadata.  The important thing to 
remember here is that a chain is constructed with an *explicit* ordering 
to get some specific data (i.e. first grab the entire data set, then run 
a filter chain on time, then perhaps do a CDF):


<nmwg:metadata id="1">
 <nmwg:key>
   <nmwg:parameters />
 </nmwg:key>
</nmwg:metadata>

<nmwg:metadata id="2">
 <select:subject metadataIdRef="1"/>
 <nmwg:eventType />
 <select:parameters />
</nmwg:metadata>

<nmwg:metadata id="3">
 <cdf:subject metadataIdRef="2"/>
 <nmwg:eventType />
 <cdf:parameters />
</nmwg:metadata>

<nmwg:data id="d1" metadataIdRef="3" />


Your current cooking procedure should give something like this (?):


<nmwg:metadata id="k1">
 <nmwg:key>
   <nmwg:parameters />
   <select:parameters />
   <cdf:parameters />
 </nmwg:key>
</nmwg:metadata>


My question then becomes, what is the order in which the blocks of 
parameters are evaluated?  It may make a difference when one is done 
before the other, it may not.  We should not get stuck in this trap at 
all, and instead use the entire chain to ensure we doing the right thing 
(in the eyes of the chain architect). 


> > because it is not very natural in my opinion ('cooking' is 'cooking' 
> > no matter what end result looks like).  Does the RRDMA use the above 
> > example now?
>
> Yes. After the last release I implemented this (before that there was 
> a discussion about it on the list). I think it's not a big problem now 
> because the key should be generated be a service and client should use 
> them as a black box (it does not know what's inside). The problem  
> might be on the client side which generates keys (in this case the 
> client must know inside structure).


For now the simple case of dataset+time filter chain will 'work' in a 
key, but as they get more complex this will not be the case.  I would 
prefer we consider the effects of this approach to chaining some more 
(not necessarily before this next deadline) before it comes deeply 
embedded in the way we manage requests and responses.


-jason