perfsonar development work

["Jeff W. Boote" <>]

Loukik Kudarimoti wrote:
> > I would also add that the metadataid should be stable and unique. I 
> > asked developers about a month ago how difficult that would be, and 
> > have yet to hear a single response.
>
> I can think of a few questions related to this topic.
>
> a) What is the significance of having unique ids?
> I am assuming that the ids need to be globally unique. For me, there is 
> no question about ids having to be unique in a message or a store. Since 
> I am not clear about the motive behind unique ids, I would appreciate an 
> answer to this question.

To be honest - Initially, I am not nearly as concerned that they be globally 
unique, as I am concerned that the same identifier be returned for a given 
metadata for the lifetime that metadata is valid in a given service/store.

Please see the thread from last month for a more full discussion of the 
motivation. It can be found in the thread started from:
https://mail.internet2.edu/wws/arc/perfsonar-dev/2006-08/msg00009.html

> b) What is the recommended structure of these ids (which one should 
> follow in order to generate a unique id)? This question is important 
> because clients such as viz. tools will need to generate these unique ids.

I did not suggest a recommended structure. I simply asked how hard it would be 
to keep ids 'stable' in the current services. Once we determine that, we can 
decide if the benefit to clients outweighs the over-head this causes services. 
But, without input from the developers - it may just get specified in the 
normative text of the nmwg schema specification without any developer feedback. 
We have had feedback from the client side indicating it is useful.

> b) Is it necessary to confirm that the id is unique? If so, how do we go 
> about doing that (rather, how can a program go about doing that)?

I personally think 'guaranteed' uniqueness is something of a red-herring. As I 
said, no one has discussed formats for the id's yet, but there are already web 
standards for these. And, I have a little experience with unique identifiers 
from the OWAMP spec as well. It is not that hard to come up with something that 
has a 99.999999999% chance that the id is unique without doing any kind of 
'validation' or confirmation at all. I believe this is more than good enough.

> c) In the scenario where client X sends a request with certain metadata 
> and data elements to Service A and Service A responds with metadata and 
> data elements to client X
> (i) If the service adds new metadata elements, the service generates the 
> ids for these elements?
> (ii) If the service modifies existing metadata or data elements, is it 
> required to modify the ids as well?

Good questions to work out, once we decide if services can present stable 
identifiers.

[I will digress a moment to give you my initial thoughts on this: I was not 
actually thinking the service needed to respond with the same identifiers for 
the metadata objects presented to it in the request. Merely that the service 
needed to respond with the 'same' identifier for the 'same' metadata every time. 
But, in thinking through this further based on your question - there could be 
some benefit to what you suggest. (It might not even be that difficult to do 
this. For example, the identifier could simply be an SHA-1 HMAC of the metadata. 
For a fully populated metadata, this would not change unless the values inside 
the metadata changed.)]

> d) Is it necessary to use unique ids for contents inside a store (i.e. 
> metadata configuration files)?

Yes. This is the main point. Each time a client requests data - the client 
should be able to match the returned metadata with metadata from previous 
requests. That way an analysis tool can easily determine that two requests for 
utilization data are in fact referring to the same interface.

In fact, perhaps an even better question is to determine if two services 
providing data for the exact same subject should somehow use the exact same 
subject id. This would make the overlaying case much, much easier. (Say you 
wanted to overlay achievable bandwidth fetched from one service with utilization 
fetched from another. Currently, I believe it would take a fair amount of work 
and fuzzy logic to reliably determine that the two values refer to traffic over 
the same interface.

All I am getting at, is that the 'identifier' rules are not really specified 
yet. And that they need to be. Now is the time for voices to be heard on how 
they should be specified. The NMWG specifications normative text is being written...

jeff