perfSONAR User Q&A and Other Discussion

[Joe Metzger <>]

On Jul 21, 2009, at 7:49 AM, Piotr Pikusa wrote:

> Hi Ralph!
>
> Ralph Niederberger wrote:
> > I should respecify my question a little better to be more accurate.  
> > Within the CISCO web pages:
> >   
> > http://www.cisco.com/en/US/tech/tk648/tk362/technologies_tech_note09186a008009496e.shtml
> >   "How To Calculate Bandwidth Utilization Using SNMP"
> >
> > I have to calculate the utilization using several requests via  
> > SNMP. In principle it is:
> > "number of packets coming in/out per interval" divided by   
> > intervall-period
> >
> > They provide a percentage value, so they multiply by 100 and divide  
> > by interface speed.
> >
> > Since the sql tables within perfSONAR SQL-MA have a field  
> > valueUnits "Bps". I assume I have to
> > store here the packets (not bits) transfered (in or out) per  
> > second. So to compute those values,
> > I have to  start two snmp  requests, ie. at start-time and end-time  
> > of measurement intervall.
> > It would be much more convenient to store the Interface counters  
> > (in or out) only.
> > A client application could then calculate the utilization itself  
> > getting all the counters within the intervall
> > interested in. It would also allow calculating utilization for any  
> > time intervall, not only the once
> > measured, since the snmp interface values are updated by default in  
> > the machine to which I send
> > my request.

Calculating bps from counter values is simple in the trival case. But
if you try the simple approach for an extended period of time on  
multiple
devices you will run into a large number of edge cases: counters wrap,
counters are reset to 0, counters get frozen or appear to go backwards,
the time that you requested the counter to be read may only loosely  
correlate
with the time that the device actually downloaded the # from the  
interface card, etc.

MTGT, Cacti and most other SNMP polling tools have heuristics  
developed over years
to address all of these edge cases, relying on context available at  
the time the
device is being polled.  Trying to do this inside the perfSONAR MA  
when replying
to a data request would present significant challenges.



> >
>
> Since you have valueUnits field in SQL-MA base, you can store values  
> in any units you want. Bps is only an example.
> This approach is free from hardware specifications and scalable.  
> Network user choose what unit is the best for him.
> Of course you can use counters in constant time and make your own  
> unit.
>
>
> > > > Further is the direction of traffic defined by the metadataId,  
> > > > e.g. by specifying
> > > > metaid-interface-in and metaid-interface-out?
> > I assume that there are different metadata ids for input and output  
> > measurements, since file
> > ....
> > has a field <nmwgt:direction>in</nmwgt:direction>
> > is that correct? Or means utilization always  input+output? But  
> > then the perfsonarUI tool could
> > not differeentiate between input and output.
>
> There are different metadata ids for input and output.
>
> > > > And what do I have to store in the db field "value"? Is it the  
> > > > utilization
> > > > of the interface, which means before inserting data I have to  
> > > > calculate this
> > > > dependend on the last time I stored a value (packets in since  
> > > > last inserting
> > > > devided by time since last inserting?
> > > > Or do I have to store the packet count of the interface only and  
> > > > when
> > > > requesting infos, the utilization will be calculated dependend on  
> > > > the time
> > > > interval I specify within the client request?
> > see same discussion above.
> > I assume valueUnits "Bps" in SQL perfsonar_utilization table means  
> > "Byte per second", not "bit per second".
> > Is this correct?
>
> One more time. You store some values in value filed in units  
> specified in valueUnits. You send information to SQL-MA, that on 21  
> of July at 14:30, utulisation in "in" direction was 5 Mbps. Or more  
> general data per time. So yes, you should calculate it yourself. But  
> if you check it every second (what is standard approach) and your  
> unit in unitValue is based on seconds, you don't have to calculate  
> anything. This is advantage of perfSONAR system - you can store any  
> values in any units, it doesn't force anything.
> When you want to use perfsonarUI to work with data you store, you  
> should use data per second units (not packets per second) because it  
> transforms these units to Mb/s. You can also use % unit option in  
> perfsonarUI Summary - it is based on capacity of the interface.
>
> > > > many thanks in advance
> > > >
> > > > Ralp
>
>
> Thank you for interesting questions.
>
> Regards
> Piotr Pikusa

Joe Metzger