perfsonar development work

[Jeff W.Boote <>]

On Jan 23, 2009, at 6:56 AM, Szymon Trocha wrote:

> (I'm moving it to ps-dev)
>
> Jeff,
>
> Shouldn't we document the agreement somewhere? I mean I saw you  
> removed bwctl.xml from example instances in SVN so now I'm wodering  
> if ther is some source reference now.

I removed it because it did not follow the agreement... I was planning  
on asking VV to provide an example from her service to put in there  
once she modified the chaining. But, I got distracted yesterday and  
forgot. :)

VV, would you mind taking an example from your service once you change  
it and checking it into SVN in nmwg? (Or just send it to me and I can.)

thanks,
jeff

> regards
> Szymon
>
> Jeff W. Boote pisze:
> > On Jan 22, 2009, at 1:46 AM, Verena Venus wrote:
> > > Hi all,
> > >
> > > Am Wednesday 21 January 2009 19:00:43 schrieb Jeff W. Boote:
> > > > Hi Roman,
> > > >
> > > > I agree, I think it should be function chaining as well. So, it  
> > > > should
> > > > be from the 'subject', not from the 'md'.
> > > >
> > > > VV,
> > > > Where did you get the example you used from? (The one on nmwg svn
> > > > examples is 2 years old... It also doesn't have the correct  
> > > > eventType
> > > > listed. If it was from there, I suspect we forgot to update it over
> > > > time.) Would it cause you much difficulty to change this at this
> > > > point? I think this was just a typo in what we were doing...
> > > I'm not sure, where I have this from (probably from svn sometimes  
> > > in the
> > > past). About the chaining: I remember faintly a discussion about  
> > > it, but I do
> > > not remember the outcome....
> > Ok... that reminded me of the discussion now as well.
> > We talked about it and we still think the 'function chaining'  
> > semantics match best what is happening here. (Where function is  
> > defined as something that takes inputs and produces outputs.)
> > To be clear, I think we all now agree this should be represented as:
> > <nmwg:metadata id="bwctl-metadata">
> > ...
> > </nmwg:metadata>
> > <nmwg:metadata id="iperf-metadata">
> > <iperf:subject id="iperf-subject" metadataIdRef="bwctl-metadata">
> > ....
> > </iperf:subject>
> > ...
> > </nmwg:metadata>
> > (and perhaps even)
> > <nmwg:metdata id="select-metdata">
> > <select:subject id="select-subject" metadataIdRef="iperf-metadata">
> > ...
> > </select:subject>
> > ...
> > </nmwg:metadata>
> > Thanks,
> > jeff
> > > So, I have no problem with changing it, if it is agreed to do so.
> > >
> > > Regards,
> > > Verena
> > > > But, I'm CC'ing Martin just in case he can think of some reason we
> > > > might have used merge chaining here.
> > > >
> > > > jeff
> > > >
> > > > On Jan 21, 2009, at 8:29 AM, Roman Lapacz wrote:
> > > > > Hi Jeff,
> > > > >
> > > > > Jeff W. Boote wrote:
> > > > > > Roman - you are reversing the chaining and that will make your
> > > > > > things incompatible with things we are doing.
> > > > > >
> > > > > > The chaining is in the order it is given by VVs example because  
> > > > > > the
> > > > > > data itself is NOT bwctl data - it is iperf data. So, the  
> > > > > > metadata
> > > > > > referenced by the data is iperf metadata. We allow chaining from
> > > > > > the iperf metadata to a bwctl metadata to indicate information
> > > > > > about how iperf was itself started,
> > > > >
> > > > > OK, I see your point.
> > > > >
> > > > > One additional question: why did you use metadataIdRef attribute  
> > > > > of
> > > > > metadata element (iperf metadata) for chaininng instead of
> > > > > metadataIdRef attribute of iperf:subject element. I thought the
> > > > > chaining used is more for chunked metadatas which can be merged by
> > > > > the service on the fly (that helps to not repeat some parts of
> > > > > metadata in multiple metadatas). The second chaining (through
> > > > > subject element; we used to call it functional chaining) seems  
> > > > > to be
> > > > > more suitable here (chaining the functionalities of iperf and  
> > > > > bwctl
> > > > > tools)
> > > > >
> > > > > example:
> > > > >
> > > > > <nmwg:metadata id="bwctl-metadata">
> > > > > ...
> > > > > </nmwg:metadata>
> > > > >
> > > > > <nmwg:metadata id="iperf-metadata">
> > > > > <iperf:subject id="iperf-subject" metadataIdRef="bwctl-metadata">
> > > > > ....
> > > > > </iperf:subject>
> > > > > ...
> > > > > </nmwg:metadata>
> > > > >
> > > > >
> > > > > (this question is more about chaining definitions and conditions
> > > > > which and when they should be used)
> > > > >
> > > > > > but it is not actually necessary to even put the bwctl metadata
> > > > > > into a database for the data to be useful. (pS-B does not  
> > > > > > currently
> > > > > > do this last chaining step - we only store the iperf part.
> > > > >
> > > > > Right. I've seen it in the doc of pS-B.
> > > > >
> > > > > Roman
> > > > >
> > > > > > It is possible we will put iperf data from raw iperf tests that  
> > > > > > are
> > > > > > not started from bwctl into the database.)
> > > > > >
> > > > > > jeff
> > > > > >
> > > > > > On Jan 21, 2009, at 5:01 AM, Roman Lapacz wrote:
> > > > > > > Jason Zurawski wrote:
> > > > > > > > Roman;
> > > > > > > >
> > > > > > > > Can you be more clear about what you specifically need?  I sent
> > > > > > > > all of the information that I have on pSB last week, but if you
> > > > > > > > need more I can work to give you something.
> > > > > > >
> > > > > > > Some time ago I had a discussion with Verena about store  
> > > > > > > request.
> > > > > > > I wasn't sure the order of metadata elements in the chain was
> > > > > > > correct. In my opinion bwctl metadata should be present as the
> > > > > > > root  in such chain because it's the wrapper around iperf. I  
> > > > > > > would
> > > > > > > propose the following approach:
> > > > > > >
> > > > > > >
> > > > > > > <nmwg:metadata id="iperf-metadata">
> > > > > > > <iperf:subject id="iperf-subject">
> > > > > > >  <nmwgt:endPointPair>
> > > > > > >    <nmwgt:src type="ipv4" value="131.188.81.61"/>
> > > > > > >    <nmwgt:dst type="ipv4" value="131.188.81.12"/>
> > > > > > >  </nmwgt:endPointPair>
> > > > > > > </iperf:subject>
> > > > > > > <iperf:parameters id="iperf-parameters">
> > > > > > >  <nmwg:parameter name="interval" value="2"/>
> > > > > > >  <nmwg:parameter name="protocol" value="tcp"/>
> > > > > > > </iperf:parameters>
> > > > > > > <nmwg:eventType>http://ggf.org/ns/nmwg/tools/iperf/2.0/</
> > > > > > > nmwg:eventType>
> > > > > > > </nmwg:metadata>
> > > > > > >
> > > > > > > <nmwg:metadata id="bwctl-metadata" metadataIdRef="iperf- 
> > > > > > > metadata">
> > > > > > > <bwctl:subject id="bwctl-subject">
> > > > > > >  <nmwgt:endPoint type="ipv4" value="131.188.81.12"/>
> > > > > > > </bwctl:subject>
> > > > > > > <nmwg:eventType>http://ggf.org/ns/nmwg/tools/bwctl/2.0/</
> > > > > > > nmwg:eventType>
> > > > > > > <bwctl:parameters id="bwctl-parameters">
> > > > > > >  <nmwg:parameter name="duration" value="5"/>
> > > > > > > </bwctl:parameters>
> > > > > > > </nmwg:metadata>
> > > > > > >
> > > > > > >
> > > > > > > I'm not saying an example sent by Verena is wrong. I just want  
> > > > > > > to
> > > > > > > clarify this and get the confirmation.
> > > > > > >
> > > > > > >
> > > > > > > Roman
> > > > > > >
> > > > > > > > -jason
> > > > > > > >
> > > > > > > > > Hi again,
> > > > > > > > >
> > > > > > > > > there were no responses about the schema and an example for
> > > > > > > > > store request . Does it mean it's already agreed and  
> > > > > > > > > accepted as
> > > > > > > > > the final version and I can use it in my work?
> > > > > > > > >
> > > > > > > > > Roman
> > > > > > > > >
> > > > > > > > > Roman Lapacz wrote:
> > > > > > > > > > Hi,
> > > > > > > > > >
> > > > > > > > > > Could you confirm that the example below is still valid
> > > > > > > > > > (Verena, we had a discussion about it some time ago and we  
> > > > > > > > > > said
> > > > > > > > > > that this could be refined with Jeff or someone else who is
> > > > > > > > > > familiar with the schema; my main concern was the order of
> > > > > > > > > > metadata elements in the chain). Could you also send me  
> > > > > > > > > > request
> > > > > > > > > > examples for fetching data (Szymon told me that pS Buoy  
> > > > > > > > > > already
> > > > > > > > > > supports store requests so all message examples of it would  
> > > > > > > > > > be
> > > > > > > > > > very helpful for me; I'd like to keep compatibility).
> > > > > > > > > >
> > > > > > > > > >
> > > > > > > > > > Roman
> > > > > > > > > >
> > > > > > > > > > Verena Venus wrote:
> > > > > > > > > > > Hi Nemanja,
> > > > > > > > > > > here is an example message:
> > > > > > > > > > >
> > > > > > > > > > > <?xml version="1.0"?>
> > > > > > > > > > > <nmwg:message
> > > > > > > > > > > xmlns="http://ggf.org/ns/nmwg/tools/org/perfsonar/1.0/ "
> > > > > > > > > > >           xmlns:nmwg="http://ggf.org/ns/nmwg/base/2.0/";
> > > > > > > > > > >           xmlns:nmtm="http://ggf.org/ns/nmwg/time/2.0/";
> > > > > > > > > > >           xmlns:select="http://ggf.org/ns/nmwg/ops/select/
> > > > > > > > > > > 2.0/"
> > > > > > > > > > >           xmlns:bwctl="http://ggf.org/ns/nmwg/tools/bwctl/
> > > > > > > > > > > 2.0/"
> > > > > > > > > > >           xmlns:iperf="http://ggf.org/ns/nmwg/tools/iperf/
> > > > > > > > > > > 2.0/"
> > > > > > > > > > >           xmlns:nmwgt="http://ggf.org/ns/nmwg/topology/2.0/";
> > > > > > > > > > >           type="MeasurementArchiveStoreRequest" id="bwctl-
> > > > > > > > > > > request">
> > > > > > > > > > >
> > > > > > > > > > > <nmwg:metadata id="bwctl-metadata">
> > > > > > > > > > > <bwctl:subject id="bwctl-subject">
> > > > > > > > > > >   <nmwgt:endPoint type="ipv4" value="131.188.81.12"/>
> > > > > > > > > > > </bwctl:subject>
> > > > > > > > > > > <nmwg:eventType>http://ggf.org/ns/nmwg/tools/bwctl/2.0/</
> > > > > > > > > > > nmwg:eventType>
> > > > > > > > > > > <bwctl:parameters id="bwctl-parameters">
> > > > > > > > > > >   <nmwg:parameter name="duration" value="5"/>
> > > > > > > > > > > </bwctl:parameters>
> > > > > > > > > > > </nmwg:metadata>
> > > > > > > > > > >
> > > > > > > > > > > <nmwg:metadata id="iperf-metadata" metadataIdRef="bwctl-
> > > > > > > > > > > metadata">
> > > > > > > > > > > <iperf:subject id="iperf-subject">
> > > > > > > > > > >   <nmwgt:endPointPair>
> > > > > > > > > > >     <nmwgt:src type="ipv4" value="131.188.81.61"/>
> > > > > > > > > > >     <nmwgt:dst type="ipv4" value="131.188.81.12"/>
> > > > > > > > > > >   </nmwgt:endPointPair>
> > > > > > > > > > > </iperf:subject>
> > > > > > > > > > > <iperf:parameters id="iperf-parameters">
> > > > > > > > > > >   <nmwg:parameter name="interval" value="2"/>
> > > > > > > > > > >   <nmwg:parameter name="protocol" value="tcp"/>
> > > > > > > > > > > </iperf:parameters>
> > > > > > > > > > > <nmwg:eventType>http://ggf.org/ns/nmwg/tools/iperf/2.0/</
> > > > > > > > > > > nmwg:eventType>
> > > > > > > > > > > </nmwg:metadata>
> > > > > > > > > > >
> > > > > > > > > > > <!-- triggers to indicate head of chains -->
> > > > > > > > > > >
> > > > > > > > > > > <nmwg:data id="1" metadataIdRef="iperf-metadata">
> > > > > > > > > > > <iperf:datum numBytesUnits="Bytes" time="1197642642"
> > > > > > > > > > > numBytes="23484640" value="93938560" valueUnits="bits/sec"
> > > > > > > > > > > duration="0.0- 2.0"/>
> > > > > > > > > > > <iperf:datum numBytesUnits="Bytes" time="1197642642"
> > > > > > > > > > > numBytes="23525896" value="94103584" valueUnits="bits/sec"
> > > > > > > > > > > duration="2.0- 4.0"/>
> > > > > > > > > > > <iperf:datum numBytesUnits="Bytes" time="1197642642"
> > > > > > > > > > > numBytes="58851328" value="93840271" valueUnits="bits/sec"
> > > > > > > > > > > duration="0.0- 5.0"/>
> > > > > > > > > > > </nmwg:data>
> > > > > > > > > > >
> > > > > > > > > > > </nmwg:message>
> > > > > > > > > > >
> > > > > > > > > > > Am Donnerstag, 4. Dezember 2008 15:32:30 schrieb Nemanja  
> > > > > > > > > > > Zutic:
> > > > > > > > > > > > Hi Verena,
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks for the info. If you have some example requests and
> > > > > > > > > > > > responses send
> > > > > > > > > > > > them to me, it would be of great help
> > > > > > > > > > > >
> > > > > > > > > > > > Thanks again,
> > > > > > > > > > > > Nemanja Zutic
> > > > > > > > > > > >
> > > > > > > > > > > > --------------------------------------------------
> > > > > > > > > > > > From: "Verena Venus" 
> > > > > > > > > > > > <>
> > > > > > > > > > > > Sent: Thursday, December 04, 2008 2:56 PM
> > > > > > > > > > > > To: "Nemanja Zutic" 
> > > > > > > > > > > > <>
> > > > > > > > > > > > Cc: "Slavko Gajin" <>; "Dusan  
> > > > > > > > > > > > Pajin"
> > > > > > > > > > > > <>;
> > > > > > > > > > > >  "Roman Lapacz" 
> > > > > > > > > > > > <
> > > > > > > > > > > >
> > > > > > > > > > > > Subject: Re: BWCTL MA Schema
> > > > > > > > > > > >
> > > > > > > > > > > > > Hi Nemanja,
> > > > > > > > > > > > >
> > > > > > > > > > > > > please use schemata provided by the schema subversion
> > > > > > > > > > > > > repository:
> > > > > > > > > > > > > http://anonsvn.internet2.edu/svn/nmwg/trunk/nmwg/schema/ 
> > > > > > > > > > > > > rnc
> > > > > > > > > > > > >
> > > > > > > > > > > > > You will need bwctl.rnc and iperf.rnc
> > > > > > > > > > > > >
> > > > > > > > > > > > > I can send you example messages, if you like. You can also
> > > > > > > > > > > > > contact Roman
> > > > > > > > > > > > > (in
> > > > > > > > > > > > > cc), who will implement the BWCTL MA interface with SQL  
> > > > > > > > > > > > > MA,
> > > > > > > > > > > > > so you can
> > > > > > > > > > > > > start
> > > > > > > > > > > > > testing, as soon as a service is available.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Regards,
> > > > > > > > > > > > > Verena
> > > > > > > > > > > > >
> > > > > > > > > > > > > Am Montag, 1. Dezember 2008 15:43:31 schrieb Nemanja  
> > > > > > > > > > > > > Zutic:
> > > > > > > > > > > > > > Hi Verena,
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Nicolas asked me to create an additional graph to show  
> > > > > > > > > > > > > > the
> > > > > > > > > > > > > > BWCTL
> > > > > > > > > > > > > > history. For that I need to know the schemas for BWCTL MA
> > > > > > > > > > > > > > requests and
> > > > > > > > > > > > > > responses and
> > > > > > > > > > > > > > he told me to contact you. If I understood correctly  
> > > > > > > > > > > > > > there
> > > > > > > > > > > > > > is no active
> > > > > > > > > > > > > > BWCTL MA service running, but for start I can use the
> > > > > > > > > > > > > > schemas to create
> > > > > > > > > > > > > > dummy responses. Thanks in advance for your help.
>
>
> --
> Szymon Trocha
>
> Poznan Supercomputing & Netw. Center ::: NETWORK OPERATION CENTER
> Tel. +48 618582022 ::: http://noc.man.poznan.pl