perfsonar development work

["Vedrin Jeliazkov" <>]

Hi,

Nina and myself agree that:

- 'unit' is too generic and calling it <*>valueUnit (as suggested by Jeff)
would be better;

- it's preferrable to have <*>valueUnit in datum (as suggested by Loukik),
rather than in metadata;

We also feel that opening the door for free text content of <*>valueUnit could
quickly lead to a multitude of variants for a given unit name in different
service instances. It would be much better if the allowed units are well
defined, known by clients and enforced by respective services in some way. We
would very much like to avoid the possibility to have simultaneously:

b/s
bit/sec
bits/sec
bits per second
etc...

all with the same meaning. This would be a nightmare for clients who try to
preserve units in visualizations across different services.

And last, but not least, maybe it would be better to have bytes per second
(instead of bits per second) in the default rrd-database_TEST.xml, because
data sources reporting bytes seem to be more common.

Kind regards,
Vedrin

Loukik Kudarimoti 
<>
 wrote:

> Jeff W. Boote wrote:
> > Roman Lapacz wrote:
> >>
> >> Implemented. Available in SVN (trunk).
> >>
> >
> > You implemented the units for the 'value' field of the datum, but not 
> > the timeValue.
> >
> > This is another area where consistency will be important across 
> > multiple data types. There will be other 'datum' types that have more 
> > than one 'value' in the result datum.
> >
> > I would suggest that 'unit' is too generic of a name in the key. You 
> > need to tie it more closely to the name of the field in the datum that 
> > is being returned. In this case for example: 'valueUnit'. Then, you 
> > can specify the time unit using 'timeValueUnit'. And it is clear for 
> > any other datum fields how this would progress.
> I agree.
> 
> Thinking more about it, if units for' bits per second' is bps, then 
> units for 'bytes per second' would be Bps. There can be a bit 
> misleading. To be on the safe side, my suggestion would be that 
> perfSONAR by default uses 'bits per second' and 'bytes per second' or 
> 'octets per second'.
> 
> >
> > I am curious why it is not sufficient to return the 'Units' in the 
> > metadata of the response instead of including it in every single 
> > datum... (I don't really care that much, but it seems like it would be 
> > nice to have that factored out and that it really could be one of the 
> > parameters of the response metadata.)
> This in my opinion causes uncertainty for the user of nmwg schemas. It 
> should be possible to specify the units of the datum in one place only 
> for all incarnations of the nmwg schema. Otherwise, the schema becomes 
> inherently unstable as same information is allowed to be present in two 
> places.
> 
> If the question is whether it should be in metadata or it should be in 
> datum, I would prefer it to be in datum. From the message size point of 
> view, it wouldn't make a huge impact but on the other hand it would 
> definitely allow situations where different datum in the same response 
> have different units.
> 
> Regards,
> Loukik
> >
> > jeff
> >
> >>
> >> Example of metadata configuration file:
> >>
> >> <nmwg:store >
> >>
> >>    <nmwg:metadata id="meta1">
> >>        ....
> >>    </nmwg:metadata>
> >>
> >>    <nmwg:data id="data1" metadataIdRef="meta1">
> >>        <nmwg:key>
> >>            <nmwg:parameters>
> >>                <nmwg:parameter 
> >> name="file">/rrd/test/test.rrd</nmwg:parameter>
> >>                <nmwg:parameter name="dataSource">ds0</nmwg:parameter>
> >>                <nmwg:parameter name="unit">bps</nmwg:parameter>
> >>            </nmwg:parameters>
> >>        </nmwg:key>
> >>    </nmwg:data>
> >>
> >> </nmwg:store>
> >>
> >>
> >>
> >>
> >> Example of response if the unit is specified in the key:
> >>
> >>
> >> <nmwg:message>
> >>   <nmwg:metadata id="meta1">
> >>      ....
> >>   </nmwg:metadata>
> >>   <nmwg:data id="localhost.27b055c9:10cf803512a:-7fde" 
> >> metadataIdRef="meta1">
> >>      <nmwg:datum timeValue="1121558400" value="189.25857952114652" 
> >> valueUnits="bps" />
> >>      <nmwg:datum timeValue="1121644800" value="157.6300922870065" 
> >> valueUnits="bps" />
> >>      <nmwg:datum timeValue="1121731200" value="341.2686134412051" 
> >> valueUnits="bps" />
> >>      <nmwg:datum timeValue="1121817600" value="376.50555043698813" 
> >> valueUnits="bps" />
> >>      <nmwg:datum timeValue="1121904000" value="320.4919869209144" 
> >> valueUnits="bps" />
> >>   </nmwg:data>
> >> </nmwg:message>
> >>
> >>
> >>
> >> Example of response if the unit is not specified in the key:
> >>
> >> <nmwg:message>
> >>   <nmwg:metadata id="meta1">
> >>      ....
> >>   </nmwg:metadata>
> >>   <nmwg:data id="localhost.-3e0ab627:10cf7f78557:-7fdf" 
> >> metadataIdRef="meta1">
> >>      <nmwg:datum timeValue="1121558400" value="189.25857952114652" 
> >> valueUnits="unknown" />
> >>      <nmwg:datum timeValue="1121644800" value="157.6300922870065" 
> >> valueUnits="unknown" />
> >>      <nmwg:datum timeValue="1121731200" value="341.2686134412051" 
> >> valueUnits="unknown" />
> >>      <nmwg:datum timeValue="1121817600" value="376.50555043698813" 
> >> valueUnits="unknown" />
> >>      <nmwg:datum timeValue="1121904000" value="320.4919869209144" 
> >> valueUnits="unknown" />
> >>   </nmwg:data>
> >> </nmwg:message>
> >>
> >>
> >>
> >> Roman
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>>
> >>> Loukik.
> >>>
> >>>>
> >>>>
> >>>> Roman
> >>>>
> >>>>
> >>>>
> >>>>>
> >>>>> Loukik.
> >>>>>
> >>>>>>
> >>>>>>
> >>>>>> <nmwg:store>
> >>>>>>
> >>>>>>    <nmwg:metadata id="meta1">
> >>>>>>        ....
> >>>>>>    </nmwg:metadata>
> >>>>>>
> >>>>>>    <nmwg:data id="data1" metadataIdRef="meta1">
> >>>>>>        <nmwg:key>
> >>>>>>            <nmwg:parameters>
> >>>>>>                <nmwg:parameter 
> >>>>>> name="file">/test.rrd</nmwg:parameter>
> >>>>>>                <nmwg:parameter 
> >>>>>> name="dataSource">ps0</nmwg:parameter>
> >>>>>>                <nmwg:parameter name="unit">bps</nmwg:parameter>
> >>>>>>            </nmwg:parameters>
> >>>>>>        </nmwg:key>
> >>>>>>    </nmwg:data>
> >>>>>>
> >>>>>> </nmwg:store>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> Roman
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>>
> >>>>>> Loukik Kudarimoti wrote:
> >>>>>>
> >>>>>>> [forgot to cc perfsonar-dev]
> >>>>>>>
> >>>>>>>
> ------------------------------------------------------------------------ 
> >>>>>>>
> >>>>>>>
> >>>>>>> Subject:
> >>>>>>> units expected from RRD MA
> >>>>>>> From:
> >>>>>>> Loukik Kudarimoti 
> >>>>>>> <>
> >>>>>>> Date:
> >>>>>>> Thu, 10 Aug 2006 11:58:25 +0100
> >>>>>>> To:
> >>>>>>> Vedrin Jeliazkov 
> >>>>>>> <>
> >>>>>>>
> >>>>>>> To:
> >>>>>>> Vedrin Jeliazkov 
> >>>>>>> <>
> >>>>>>> CC:
> >>>>>>> Andreas Hanemann 
> >>>>>>> <>,
> >>>>>>>  Roman Lapcz 
> >>>>>>> <>
> >>>>>>>
> >>>>>>>
> >>>>>>> Hi Vedrin and Andreas,
> >>>>>>>
> >>>>>>> I have a query about the units that you assume you are getting 
> >>>>>>> from RRD MA. Basically, the response from RRD MA is as below:
> >>>>>>>
> >>>>>>>
> >>>>>>>  <nmwg:data id="localhost.7e5caec1:10cf76b6c9d:-7fdf" 
> >>>>>>> metadataIdRef="meta1">
> >>>>>>>     <nmwg:datum timeValue="1121558400" 
> >>>>>>> value="189.25857952114652" />
> >>>>>>>     <nmwg:datum timeValue="1121644800" value="157.6300922870065" />
> >>>>>>>     <nmwg:datum timeValue="1121731200" value="341.2686134412051" />
> >>>>>>>     <nmwg:datum timeValue="1121817600" 
> >>>>>>> value="376.50555043698813" />
> >>>>>>>     <nmwg:datum timeValue="1121904000" value="320.4919869209144" />
> >>>>>>>  </nmwg:data>
> >>>>>>>
> >>>>>>>
> >>>>>>> It doesn't say whether the value is in bits per second, Mbits 
> >>>>>>> per second or bytes per second. What do you assume the unit to 
> >>>>>>> be in? do you think it would be better if the response 
> >>>>>>> explicitly mentioned the units and also the timeType?
> >>>>>>>
> >>>>>>> Thanks,
> >>>>>>> Loukik.
> >>>>>>>
> >>>>>>
> >>>>>>
> >>>>>
> >>>>
> >>>>
> >>
> >
>