perfsonar development work

[Maciej Glowiak <>]

Hi,

After our last discussion about two options of storing/retrieving data 
in/from LS I made some additional query test.

Basically we had two options:

1. use one XML file in the collection

    - very slow modifications

2. use of separate XML files for each Lookup Information

    - fast insert/remove

We discussed that opt.2 may slow down queries, so I decided to make an 
additional test.

With Jeff's script I generated 10 Lookup Information of MA each 
collecting 100 interfaces. Then I merged them one common file containing 
these 10 Lookup Information.

First I put one, large file into LS collection and run query:

===========================================================================
for $m in /nmwg:store/nmwg:metadata
  let $s := $m/perfsonar:subject/psservice:service
  let $accessPoint := $s/psservice:accessPoint
  let $d := 
/nmwg:store/nmwg:data[@metadataIdRef=data($m/@id)]
  let $datas :=
      for $i in $d
        return <ifAddress>
                 {$i/nmwg:metadata//nmwgt:ifAddress/text()}
               </ifAddress>
  where contains($s/psservice:serviceType,'MA')
        and $datas != ''
  return <service>
            <accessPoint>
               {data($accessPoint)}
            </accessPoint>
            {$datas}
         </service>
===========================================================================

that browses all services and returns a structure of accessPoint and 
stored interfaces addresses, something like this:

===========================================================================
<service>
    <accessPoint>http://....</accessPoint>
    <ifAddress>150.254.161.194</ifAddress>
    <ifAddress>150.254.162.195</ifAddress>
    <ifAddress>150.254.163.196</ifAddress>
</service>
===========================================================================

Then I did the same with separate files (results were exactly the same 
of course):

-----------------+--------------------+-----------------------
                 |   1 large file     |    10 separate files
-----------------+--------------------+-----------------------
size             |      970 KB        |        10 * 97 KB
-----------------+--------------------+-----------------------
results          |       1178         |           392
of several       |       1191         |           391
tests            |       1213         |           394
(ms)             |       1179         |           397
                 |       1183         |           391
-----------------+--------------------+-----------------------
average          |       1189         |           393
-----------------+--------------------+-----------------------
score            |         3          |            1
-----------------+--------------------+-----------------------

So, we thought the query operation would slow down with separate files, 
but the test shows that using separate files may speed up query time 
even 3 times !!!

It's promising.

Maciej


--

 --------------------------------------------------------------------
| Maciej Glowiak        Network Research and Development            ||
| 

     Poznan Supercomputing and Networking Center ||
| (+48 61) 858 2024     http://monstera.man.poznan.pl/              ||
 ====================================================================