perfsonar development work

[Jason Zurawski <>]

Nina Jeliazkova wrote:
> Jason,
>
> Jason Zurawski wrote:
> > Nina;
> >
> > > Today experiment with the updated views Jason send, is as follows:
> > >
> > > http://dc211.internet2.edu/cgi-bin/perfSONAR/view.cgi?hls=http://p-mdm.ps-lhcopn.fnal.gov:8080/geant2-java-xml-ls/services/LookupService
> > >
> > >
> > > According to the tree view, 192.16.166.26 is served by
> > > http://p-mdm.ps-lhcopn.fnal.gov:8080/geant2-java-rrd-ma/services/MeasurementArchiveService
> > > ( resolves to 131.225.204.1 ) and I would expect this service to appear
> > > in the results. Unfortunately , this is not the case - could anyone
> > > please provide an explanation?  I assume the tree view is up to date.
> >
> > I have explained this on numerous occasions before, and nothing has
> > really changed: we are aware that the IP summarization is not as tight
> > as some would like them to be.  There is ongoing work with a student
> > at the University of Delaware to improve the algorithm and he has a
> > report on his results so far available here:
> Indeed, we have discussed summarization ‘tightness’ on a number of
> occasions already. However, I would like to emphasize that the suggested
> workaround is not suitable in this specific case, because the returned
> superset of services does not include the specific entry I’m looking for
> (p-mdm.ps-lhcopn.fnal.gov or 131.225.204.1).
>
> That is to say, if I apply the suggested workaround and start querying
> every service from the returned superset list I don’t have any chance to
> query the ‘correct’ one (p-mdm.ps-lhcopn.fnal.gov or 131.225.204.1) ,
> because it is not in the list.


Did you read my response discussing possible reasons *why* things are 
not working as you would expect them to?  There is absolutely nothing 
*anyone* can do if the summarization is not working at the lowest 
levels, I look forward to hearing from the MDM hLS maintainer so we can 
figure out if there is or is not a problem here that needs to be addressed.


> In addition it appears that there’s some inconsistency between the
> output of the directory/tree summary view, which includes a reference to
> 192.16.166.26, attached to p-mdm.ps-lhcopn.fnal.gov/131.225.204.1 and
> the results of direct queries to this particular service that you sent.
> As already stated above, the same kind of inconsistency exists between
> the output of the directory/tree summary view and the results of a
> playground query for this particular IP address (192.16.166.26). I would
> appreciate an explanation of these inconsistencies.


Not sure what you are implying here by "inconsistency? - there are 2 
interfaces on the first GUI for the *utilization eventType* (direction 
"in" and direction "out"), the graphs for these two interfaces reflect 
the data for these two directions.  The GUI *does not* display other 
eventTypes (errors, discards) so it has no reason to display the 
remaining interface definitions that are in the hLS view.

Am I answering your concern correctly?  If not can you try to explain 
what "inconsistency" you are seeing?

-jason


> Directory view
> http://dc211.internet2.edu/cgi-bin/perfSONAR/serviceTest.cgi?url=http://p-mdm.ps-lhcopn.fnal.gov:8080/geant2-java-rrd-ma/services/MeasurementArchiveService&eventType=http://ggf.org/ns/nmwg/characteristic/utilization/2.0
> <http://dc211.internet2.edu/cgi-bin/perfSONAR/serviceTest.cgi?url=http://p-mdm.ps-lhcopn.fnal.gov:8080/geant2-java-rrd-ma/services/MeasurementArchiveService&eventType=http://ggf.org/ns/nmwg/characteristic/utilization/2.0>
>
> Tree view
> http://dc211.internet2.edu/cgi-bin/perfSONAR/view.cgi?hls=http://p-mdm.ps-lhcopn.fnal.gov:8080/geant2-java-xml-ls/services/LookupService
>
> Best regards,
> Nina
> > http://code.google.com/p/perfsonar-ps/wiki/IPSummarization
> >
> > and
> >
> > https://damsl.cis.udel.edu/svn/perl-iptrie/README.html
> >
> > I have not had a chance to incorporate his work into the gLS releases
> > yet, so this is still untested in a production environment.  We will
> > be working towards releasing this before the end of the year.  Until
> > then our suggestion of how to work around this still stands: multiple
> > queries to the result hLSs will be be required to determined if these
> > instances do or do not hold what you are looking for.
> >
> > As for your results below I took a look at the hLS installed on
> > p-mdm.ps-lhcopn.fnal.gov, I would encourage all to see the internal
> > view here:
> >
> > http://dc211.internet2.edu/cgi-bin/pA/view.cgi?hls=http://p-mdm.ps-lhcopn.fnal.gov:8080/geant2-java-xml-ls/services/LookupService
> >
> >
> > This is an MDM hLS, so I am not aware of the specifics of how it works
> > on the inside.  According to the view of the hLS summary set
> > (represented as "Collection:
> > http://ogf.org/ns/nmwg/tools/org/perfsonar/service/lookup/discovery/xquery/2.0";
> > and "Store Type: LSStore-summary" - scroll all the way to the bottom)
> > there is nothing there.  This may mean:
> >
> > 1) I am not querying this set correctly, I will let the MDM hLS
> > maintainer answer if I am doing this correctly.  If there is a problem
> > with my query, that would indicate a bug in my CGIs which I will
> > gladly fix.  I am sending this query:
> >
> > <nmwg:message type="LSQueryRequest"
> >               id="LSQueryRequest"
> >               xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/";
> >
> > xmlns:xquery="http://ggf.org/ns/nmwg/tools/org/perfsonar/service/lookup/xquery/1.0/";>
> >
> >   <nmwg:metadata id="meta1">
> >     <xquery:subject id="sub1">
> >   declare namespace nmwg="http://ggf.org/ns/nmwg/base/2.0/";;
> >   
> > /nmwg:store[@type="LSStore-summary"]/nmwg:metadata
> >     </xquery:subject>
> >
> > <nmwg:eventType>http://ogf.org/ns/nmwg/tools/org/perfsonar/service/lookup/discovery/xquery/2.0</nmwg:eventType>
> >
> >   </nmwg:metadata>
> >   <nmwg:data metadataIdRef="meta1" id="d1"/>
> > </nmwg:message>
> >
> > I do get a response (but it is vaugue):
> >
> > <?xml version="1.0" encoding="UTF-8"?>
> > <soapenv:Envelope
> > xmlns:soapenv="http://schemas.xmlsoap.org/soap/envelope/";
> > xmlns:xsd="http://www.w3.org/2001/XMLSchema";
> > xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance";><soapenv:Body><nmwg:message
> > xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/"; id="LSQueryRequest_resp"
> > messageIdRef="LSQueryRequest" type="LSQueryResponse"><nmwg:metadata
> > id="LSQueryResponseMetadata"><nmwg:eventType>success.ls.query</nmwg:eventType></nmwg:metadata><nmwg:data
> > id="LSQueryResponseData"
> > metadataIdRef="LSQueryResponseMetadata"><psservice:datum
> > xmlns:psservice="http://ggf.org/ns/nmwg/tools/org/perfsonar/service/1.0/"/></nmwg:data></nmwg:message></soapenv:Body></soapenv:Envelope>
> >
> >
> >
> >
> > 2) The set is empty and the service is not summarizing, would indicate
> > a bug in the service
> >
> > 3) The service just came up and did not have a chance to summarize yet
> > (unlikely, this appears to have been registered for a while but I
> > can't prove this from where I sit)
> >
> > -jason
> >
> >
> > > First try:
> > > 40963 ms
> > > Query
> > > urn:ogf:network:node=192.16.166.26
> > > Services
> > >     http://216.255.240.2:8085/perfSONAR_PS/services/pSB
> > >     http://rrdma.net.internet2.edu:8080/perfSONAR_PS/services/IUsnmpMA
> > >     http://rrdma.net.internet2.edu:8080/perfSONAR_PS/services/snmpMA
> > >     http://ps3.es.net:8080/perfSONAR_PS/services/snmpMA
> > >     http://frown.es.net:8085/perfSONAR_PS/services/pSB
> > >     http://perfsonar-1.t2.ucsd.edu:8085/perfSONAR_PS/services/pSB
> > >
> > > Second try:
> > > 41670 ms
> > > Query
> > > urn:ogf:network:node=192.16.166.26
> > > Services
> > >     http://rrdma.net.internet2.edu:8080/perfSONAR_PS/services/IUsnmpMA
> > >     http://rrdma.net.internet2.edu:8080/perfSONAR_PS/services/snmpMA
> > >     http://frown.es.net:8085/perfSONAR_PS/services/pSB
> > >     http://perfsonar-1.t2.ucsd.edu:8085/perfSONAR_PS/services/pSB
> > >     http://ps3.es.net:8080/perfSONAR_PS/services/snmpMA
> > >     http://216.255.240.2:8085/perfSONAR_PS/services/pSB
> > >
> > > Third try:
> > > 433 ms
> > > Query
> > > urn:ogf:network:node=192.16.166.26
> > > Services
> > >
> > > Fourth try:
> > > 3518 ms
> > > Query
> > > urn:ogf:network:node=192.16.166.26
> > > Services
> > >
> > >
> > > Fifth try:
> > > 3518 ms
> > > Query
> > > urn:ogf:network:node=192.16.166.26
> > > Services
> > >
> > > And on subsequent attempts no services were retrieved as well
> > >
> > > Obviously, we need to think of a systematic testing approach.