perfsonar development work

[Joe Metzger <>]

Hi,
Here is a strawman to start the discussion on benchmarking the LS.


We need to benchmark the lookup service in order to make reasonable 
decisions about how to move forward. IE, how many networks can
register with a signal LS, what should the registration interval
be, and how will they perform under client query loads.



I think the benchmarking should collect the data neccessary to
determine the function terms that define the following
performance criteria:

The time to register an MA in a lookup service should be a function with 
3 terms:
        1. A constant time to process a message.
        2. A variable time relative to the size of the message
        3. A variable time relative to the size of data already in the LS.

The time to make a query is also a function with 3 terms:
        1. A constant time to process a message
        2. A time relative to the size of the database in the LS
        3. A time relative to the size of the query results.



I think the range of the parameter space tested should be large enough
so that we understand how the performance scales in reasonable scenarios:
        Registration Messages with interface counts:
                1, 10, 50, 100, 500, 1000 and 5000
        Queries that retrieve interface counts:
                1, 10, 50, 100, 500, 1000 and 5000
        LS Database sizes of 500 - 10,000 interfaces.




At this point I think we can constrain our analysis to wall-clock time
and completely ignore additional components of performance such as the
network performance between the MA and the LS.  Eventually we may want
to characterize those impacts, as well as processor vs wall clock time
so we can understand how overlapping requests/queries will perform.


I am not familiar enough with the LS architecture to know if there will
be additional data specific aspects that significantly affect scaling. 
IE, will one query that retrieves 100 interfaces perform the same as
one that retrieves a different 100 interfaces?  Do we expect significant
variance based on the data?



--Joe