perfsonar development work

[Szymon Trocha <>]

Maciej Glowiak pisze:
> Szymon you should look at the performance problem globally. We've 
> already increased the performance by 20% using new data structures for 
> processing data in psBase2 (using Axis2, getting rid of unnecessary 
> convertions, using class Element instead of old NMWG classes) - Slawek 
> sent those results on the mailing list in October (16th Oct 2009).
>
> Next performance increase was done by using something we called 
> "Multiple Files" where each Lookup Information from single service is in 
> separate structure in the XML database (like separate XML files). It was 
> promising even in LS1 (I was presenting similar results in Montpellier 
> meeting - 3 years ago! ;) Recently, Slawek resolved some synchronization 
> issues and made it working without any problems.

Of course a lot of improvements were made to make the service operation 
more efficient and we learnt a lot. But we still have to take into 
account query requests will probably be the most popular type of query 
sent to the LS. At least I hope this will be the case when LS 
infrastructure is heavily used finally :)

> Regarding XQuery - it was already the fastest part of the LS, so I don't 
> think we could speed it up much more. It's rather the eXist DB 
> performance issue than the LS. But, come on, 55 to 85 milliseconds per 
> query is not so huge time, is it? Query is the fastest operation, so 
> don't expect much more.
>
> Michael's idea is very interesting, as it may be the next step for 
> increasing the speed of the service. I was also thinking about it but 
> for java implemnetation of gLS, as there the structure of data is 
> "static" - all fields are defined and there won't be any unexpected 
> data. In hLS there is opposite situation, because LS doesn't know what 
> the data registered by "a service" will be.
>
> Interesting idea, as we are slowly moving towards queries without using 
> XQuery, so getting rid of XML DB might not be so big problem as before.
>
> Antoine, now we're using XML DB because we store XML data in XML 
> collections. There are some extensions for PostgreSQL and MySQL for 
> stoing XML structures, but they work with XPath, not the XQuery. I think 
> that would be "political" decision in which direction we should go. If 
> we get rid of XML DB we will also get rid of XQuery, but will make the 
> service more efficient. If we still want use XQuery, we can try to 
> optimize the LS code or DB connection or can use Michael's idea. We must 
> also remember that Java LS is not the only one and our hLS must 
> cooperate with perl LS.

You are right, planning the fundamental change we have to consider all 
dependencies.

Anyway I think test are showing us very interesting and useful data and 
a very good basis for discussion.

Regards,
--
Szymon Trocha

Poznan Supercomputing & Networing Center ::: NETWORK OPERATION CENTER
Tel. (+48) 618582022
http://noc.man.poznan.pl  ::: http://noc.pionier.gov.pl