perfsonar development work

[Jason Zurawski <>]

<re-posting this to pS-Dev>

Roman & Maciej;

> I'd like to ask about the interval time for LS registering. So far
> it's 30 seconds for RRD MA and SQL MA. Do you want to keep this value
> or increase for the release 3.0 (personally I think it should be
> higher value, 10 mins or more; currently LS removes service's
> metadatas if they are not sent/updated in new register request during
> the last 24 hours)?

This is something that we have been exploring a lot recently, so I
wanted to start a pre-meeting discussion on the topic.  I have found
that services with a relatively static data set (something like SNMP/RRD
MA) can get away with performing a registration very infrequently and
relying on keepalive messages to freshen the data set (this required
some additional logic in the service to keep track of the data a little
more closely, but it seems to be a good trade-off). 

Services with very dynamic data (ControlPlane services, Topology
Service, other LS instances) will need to register/re-register/update
the data much more often.  The logic that a service would need to
implement to keep up with task this is rather complex, and it has always
been my general view that the service shouldn't really need to worry
about this (thats what the LS is for). 

There are a couple of things the LS will need to do to get a little bit
smarter about its data management, I think I may have suggested some of
these at the Berkeley meeting but here is a redux:

 * 'Selective' LSDeregister - Be able to pass metadata in the sent data
to remove only certain things on a Deregistration message (Implemented
in perfSONAR_PS, works well especially for ControlPlane services that
will 'tear down' items and let us know they are going away).

 * LSRegister 'Update' improvements - be able to 'add' to an existing
registration without deleting all other things.  This is VERY necessary
for dnyamic services and improves registration times dramatically
(Implemented partially in perfSONAR_PS, I would like a separate LSUpdate
message...)

 * Storage Structure Change - This is the area that I am exploring next
and I think will also be very beneficial.  Currently the service info
and data are all tied to the single control structure for time.  This
means that when the service expires, so does the data.  This works well
for services in which the data will never change, but is problematic for
dynamic data sets/.  The Topology Service for example will experience
constantly entering and leaving information, and shouldn't have to keep
records on what needs deleting from the LS.  It would be much easier to
let the 'data' expire on its own (hence freeing the services from
needing to perform 'deregister bookeeping').  When the dynamic services
'register' or 'update' then, the older stuff will never be touched and
eventually will be removed.  This will require a more frequent update
interval.  This leads to the last item...

 * 'Selective' LSKeepalives - Similar to what is done in LSDeregister
above.  Will allow us to keep individual data alive.

Comments on any of the above welcome.

-jason