perfSONAR User Q&A and Other Discussion

["Jeff W. Boote" <>]

Hi Ivo,

First, please start using the perfsonar-* email aliases. We are trying to 
deprecate the 'proto' list. I'm responding on the -user list because this is a 
usage/configuration question.

As Jason responded, the most 'definitive' answer to how the schemas should work 
is kept in the 'schemas' are of the svn repository. Specifically the examples area:

http://anonsvn.internet2.edu/svn/perfsonar/trunk/perfsonar/schema/example-instances/sonar/

We intend to write more descriptive documentation to explain the examples in 
this area, but for now this is what we have.

In addition to Jason's very good answer below, there is a contrib script that 
was written to translate a 'cricket' configuration to the config file needed by 
RRD. (I don't know if it is completely up to date, but it will at least give you 
another example of how rrd data relates to the schema):

http://anonsvn.internet2.edu/svn/perfsonar/trunk/perfsonar/contrib/CricketParser/

jeff

Jason Zurawski wrote:
> Ivo;
>
> > What I exactly want to know is what do I have to do to have the data 
> > from an RRD-MA.
> > I want to understand how to use that schemas to get data from a RRD-MA.
> >
> > I think I have to follow some steps like that:
> >
> > 1. SetupDataRequest without a key
> > (but I don´t know what parameters is mandatory, maybe none?)
>
>
>
> This is where reading the schema files will come in handy, they are 
> annotated with comments indicating what is required and what is not.  If 
> you are looking at this file 
> (http://wiki.perfsonar.net/jra1-wiki/index.php/Setup_Data_Request_without_a_key) 
> the goal is to use the metadata section of the request to match what 
> appears in the 'storage file' on the RRD MA.  Storage files 
> (http://anonsvn.internet2.edu/svn/perfsonar/trunk/perfsonar/schema/example-instances/sonar/store.xml) 
> are really just the mapping of the metadata about an RRD file (it's 
> descriptive information) to it's physical location on disk so the RRD MA 
> can read it, and deliver information from requests.  For example, in the 
> storage file if there is this information describing some monitored 
> interface:
>
>    <nmwg:metadata id="meta1">
>        <netutil:subject id="subj1">
>            <nmwgt:interface>
>                <nmwgt:hostName>test-hostName</nmwgt:hostName>
>                <nmwgt:ifAddress type="ipv4">10.1.2.3</nmwgt:ifAddress>
>                <nmwgt:ifName>test-0</nmwgt:ifName>
>                <nmwgt:ifDescription>test descripyion</nmwgt:ifDescription>
>                <nmwgt:direction>in</nmwgt:direction>
>                <nmwgt:authRealm>TestRealm</nmwgt:authRealm>
>                <nmwgt:capacity>1000BaseT</nmwgt:capacity>
>            </nmwgt:interface>
>        </netutil:subject>
>        <nmwg:eventType>utilization</nmwg:eventType>
>    </nmwg:metadata>
>
> Then you could either send that entire chunk as is, or just try to match 
> on only a couple of the fields (the preferred method) such as:
>
>    <nmwg:metadata id="meta1">
>        <netutil:subject id="subj1">
>            <nmwgt:interface>
>                <nmwgt:hostName>test-hostName</nmwgt:hostName>
>                <nmwgt:ifAddress type="ipv4">10.1.2.3</nmwgt:ifAddress>
>                <nmwgt:direction>in</nmwgt:direction>
>            </nmwgt:interface>
>        </netutil:subject>
>        <nmwg:eventType>utilization</nmwg:eventType>
>    </nmwg:metadata>
>
>    <nmwg:data id="data1" metadataIdRef="meta1"/>
>
>
> The MA will be compelled to return *all* data that is a match to this 
> metadata, so that can mean getting lots and lots of measurements.  We 
> use a technique called 'filtering' to help limit the results 
> (http://wiki.perfsonar.net/jra1-wiki/index.php/Setup_Data_Request_with_metadata_and_filtering) 
> which basically applies additional criteria (in this case start and end 
> times) so that only a limited amount of data can be returned:
>
>
>    <nmwg:metadata id="meta1">
>        <netutil:subject id="subj1">
>            <nmwgt:interface>
>                <nmwgt:hostName>test-hostName</nmwgt:hostName>
>                <nmwgt:ifAddress type="ipv4">10.1.2.3</nmwgt:ifAddress>
>                <nmwgt:direction>in</nmwgt:direction>
>            </nmwgt:interface>
>        </netutil:subject>
>        <nmwg:eventType>utilization</nmwg:eventType>
>    </nmwg:metadata>
>
>    <nmwg:metadata id="meta2">
>      <select:subject id="sub2" metadataIdRef="meta1"/>
>      <select:parameters id="param2">
>        <nmwg:parameter name="startTime">1148370000</nmwg:parameter>
>        <nmwg:parameter name="endTime">1148373000</nmwg:parameter>
>        <nmwg:parameter 
> name="consolidationFunction">AVERAGE</nmwg:parameter>
>        <nmwg:parameter name="resolution">300</nmwg:parameter>
>      </select:parameters>
>      <nmwg:eventType>select</nmwg:eventType>
>    </nmwg:metadata> 
>  <nmwg:data id="data1" metadataIdRef="meta2"/>
>
>
> > 2. ...
> >
> > or I have to
> > 1. MetaDataKeyRequest
> > (but what is a Key in that context? a rrd file location?)
>
>
>
> A key in this context is a reusable XML chunk that can be used again and 
> again to get at the same metadata mapping.  For example if you look at 
> the key request file 
> (http://wiki.perfsonar.net/jra1-wiki/index.php/Metadata_Key_Request) it 
> has the same basic structure as the SetupDataRequest.  Instead of 
> getting back just data, you will only get a reusable key:
>
>  <nmwg:data id="data1" metadataIdRef="meta1">
>    <nmwg:key id="localhost.-77179f19:10b5b954cea:-7e4f">
>      <nmwg:parameters id="localhost.-77179f19:10b5b954cea:-7e4e">
>        <nmwg:parameter name="dataSource">ds0</nmwg:parameter>
>        <nmwg:parameter 
> name="file">/PATH/TO/SOME_RRD_FILE.rrd</nmwg:paramete r>
>      </nmwg:parameters>
>    </nmwg:key>
>  </nmwg:data>
>
>
> You do not get actual data, but this key can then be used in the ....
>
> > 2. DataSetupRequest with a key..?
>
> ... type of interaction in place of sending the complete metadata 
> description.  This basically just re-uses the key over and over, and 
> gets you data just like in a SetupDataRequest.  This should be enough to 
> get you started, the various design documents and presentations 
> (http://wiki.perfsonar.net/jra1-wiki/index.php/JRA1_Main) may help you 
> more.
> -jason