perfsonar development work

["Michael Bischoff" <>]

> Nina;
>
>
> The mentioned bug did not address the known IP summarization problems.
> There is active work going on at UDel to address that particular problem
> but it has not been incorporated into new releases of the hLS or gLS.
>
> It would be good to record the perfSONAR-UI specific use case (e.g.
> trying to find utilization data by IP address) as well as other potential 
> use cases both for
> GUIs and APIs as a next step.  This will
> require input from users and developers across the perfSONAR spectrum.
>
> We have started a partial list of "Information Service Use Cases" here
> as an example of what should be considered:
>
> https://spaces.internet2.edu/display/ISWG/Use+Cases
>

From that:
"The goal of this message is to return all metadata elements in the XML 
database where the
nmwg:store element has a type equal to LSStore. As noted earlier, this is a 
property of the
actual storage. Note that the contents of the subject may contain any valid 
XQuery or XPath
statement. There are several advantages to this query interface:

   1. Easy to construct APIs that are capable of constructing the proper 
message format
   2. Flexible - allows arbitrarily complex queries based on XQuery and XPath

Drawbacks also exist:

   1. Must be aware of internal storage formats - requires knowledge of XML 
schemata and the
organization of the database
   2. Due to arbitrary nature, APIs cannot offer specific functions but can 
simple expose the
XPath/XQuery to be sent
   3. Cannot construct queries based on higher level needs, e.g. give me 
information on
domain internet2.edu or IP range 192.168.0.0/16

To address some of the drawbacks, the Discovery protocol was designed. The 
Discovery message
has a specific format as seen here:"

With the implementation and use of the discovery messages I think it was 
proven that the most
dynamic part of the LS queries could be captured in relatively solidified 
messages that don't
expose the implementation detail that is the xml database. I think that as we 
study our needs
we can extend the discovery protocol to even fill our not-so-often-used use 
cases and move
towards deprecating xpath/xquery messages. Thereby freeing future 
implementations from having
to use a xml database perse and allow them to use any storage as they see 
fit. (sure one can
rewrite/transelate xpath/xqueries to a form that matches with any arbitrary 
storage but other
then the added complexity that it gives it also leaves for a big surface 
area(there are a lot
of combinations of xqueries that can be formed) to test for.


On a different note I think adding topology services and usage of it by the 
clients is going
to be the main source of use-cases for the LS infrastructure. As process 
there; seems to be
slow(atleast on this side of the pond), I would strongly advice to increase 
efforts.

Other area that might generate new use-cases is the process that surrounds 
drilling down for
information. Finding other services that contain drill down information 
(trend data versus
realtime high detail small snapshots/stream data. But also related influences 
(for example if
a border router fails a router behind will be impacted) and how these 
relations are tracked
and stored in the/a perfsonar network.
>
> Thanks;
>
>
> -jason
>
>
>
>>>> According to my records there was a bug found and fixed in March
>>>> (http://code.google.com/p/perfsonar-ps/issues/detail?id=109) for both
>>>> LS implementations.
>>>>
>>>>
>>>> Since the Geant2 project was in transition at the time, there has
>>>> been no additional communication regarding if this fixes the original 
>>>> problem
>>> I know. We also changed LS and LS API developers. That's why Krzysztof
>>> would like to go back to this discussion.
>>>
>>>> with the GUI or APIs or what next steps may be.  There have not been any 
>>>> more
>>>> discussion regarding requirements or expectations from the GUI (and 
>>>> Service) developers
>>>> on how the information architecture may be improved to answer the 
>>>> appropriate
>>>> questions, but hopefully this can be a priority down the road.
>>> I think we would need some hints here from psUI developers. The goal
>>> should be to make the information provided by LS cloud components 
>>> (including API) useful
>>> for user and tools including performance and accuracy expectations.
>>>
>>> How would you suggest to proceed? May be should (re)run some tests?
>>> What kind?
>>>
>> I've rerun the test
>> (org.perfsonar.perfsonarui.test.ls.TracerouteTest#testNewLookup() ) . It
>> tries to retrieve MA services, containing utilization data for the 
>> following IP addresses.
>>
>> "150.254.185.146",//  (PIONIER)
>> "192.5.40.187",
>> "212.191.224.74",
>> "194.141.0.10"//  (BREN)
>>
>>
>> The result is ESnet MA is found  as a response to the request for
>> 150.254.185.146 and  192.5.40.187 .  The full perfsonar.log with
>> requests/response is attached.
>>
>> [ESnet SNMP MA] 150.254.185.146 N/A N/A
>> ESnet SNMP
>> MA,http://ps3.es.net:8080/perfSONAR_PS/services/snmpMA,http://schemas.perfsonar.net/2.0,,ht
>> tp://ggf.org/ns/nmwg/characteristic/utilization/2.0
>> [ESnet SNMP MA] 192.5.40.187 N/A N/A
>> ESnet SNMP
>> MA,http://ps3.es.net:8080/perfSONAR_PS/services/snmpMA,http://schemas.perfsonar.net/2.0,,ht
>> tp://ggf.org/ns/nmwg/characteristic/utilization/2.0
>> [] 212.191.224.74
>> [] 194.141.0.10
>>
>>
>> Best regards,
>> Nina
>>
>>
>>> Regards,
>>>
>