perfsonar development work

[Jochen Reinwand <>]

On Wednesday 05 December 2007 07:27, Nina Jeliazkova wrote:
> Joe Metzger 
> <>
>  wrote:
> > 5 minute data points.) This will still allow you to have some info about
> > each interface, it should respond a whole lot quicker, and it won't blow
> > the heap and crash everytime somebody tries to use it with the ESnet MA.
>
> I have to do some profiling/calucations anyway, but I doubt in this case
> the number of point is crucial, rather this is the number of requests sent
> to the service and DOM/XML that are received and retained simultaneously in
> memory. The reason is for statistics in table/bar chart the values of those
> points are averaged as long as DOM is processed, so this is DOM residing in
> memory.
>
> A reasonable approach (as already suggested by someone) is to limit the
> number of requests sent simultaneously. Currently, one might switch on/off
> sending parallel requests, but apparently better fine tuning is necessary.

A more sophisticated approach, that would make a lot of things easier 
regarding performance, would be to use streaming.

This way we can:

- Minimise memory usage by throwing away this XML garbage as soon as it comes 
in during receiving or create it on the fly throwing away the garbage already 
sent during sending. XML is taking a lot of the memory "for nothing".

- Stop the streaming process, when it becomes obvious that receiving this 
request will lead into an out-of-memory condition.

Streaming doesn't mean that you don't have to set any limits, but it is a lot 
easier to define reasonable values and then enforce them.

regards,
Jochen

-- 
Jochen Reinwand, DFN-Labor
Friedrich-Alexander-Universität Erlangen-Nürnberg
Regionales RechenZentrum Erlangen (RRZE)
Martensstraße 1, 91058 Erlangen, Germany
Tel. +49 9131 85-28689, -28800, Fax +49 9131 302941

www.win-labor.dfn.de