perfsonar development work

[Jason Zurawski <>]

Nicolas & All;

The documentation strawman for discussion in Zagreb is here:

http://anonsvn.internet2.edu/svn/perfSONAR-PS/branches/merge/doc/strawman/

There is a small image explaining the structure here:

http://anonsvn.internet2.edu/svn/perfSONAR-PS/branches/merge/doc/strawman/how_it_works.png

A brief overview of the parts I have been able to work on thus far:

- Schema
    * OGF Document - Martin will explain this, it is located on the 
Gridforge site currently.
    * Iperf Profile - Exentension of the base to the iperf tool.  This 
can be used by services such as the perfSONAR-BUOY MA, or Hades.
- Protocol
    * Base Protocol - Basic description of the perfSONAR protocols as 
well as the echo protocol.  There is a place holder for AA
    * MA Protocol - Partially complete (contains only one message type 
currently) description of messages for services of type "MA".  RRDMA, 
SNMP MA, Status MA, etc. will incorporate this document as their base 
making note of changes in the service documentation.
- Service
   perfSONAR-BUOY MA - This makes reference to the MA protocol, and the 
iperf schema profile.  It also includes service details such as 
installation, configuration, and other service specific items.

Comments welcome on this list or in "person" tomorrow.

-jason

> In attachment a representation of the relationship between the 
> different documents for the Hades MA. (this picture is not complete, 
> but highlight the different type of references).
>
> I'd like to thank Verena for her support and explanation in updating 
> the document and in creating this picture.
>
> Best regards,
> Nicolas
>
> Nicolas Simar wrote:
> > Hi,
> >
> > I updated the initial document based on Jason and Jeff's proposal. I 
> > attempted to draw a picture of the new document framework.
> >
> > I suggest people to read that document (without going any further 
> > than the functional specification for next week discussion) along 
> > with the echo document from Jason to have an example: 
> > http://anonsvn.internet2.edu/svn/perfSONAR-PS/branches/merge/doc/protocol/echo/ 
> >
> >
> > Cheers,
> > Nicolas
> >
> > Jason Zurawski wrote:
> >
> > > Nicolas;
> > >
> > >
> > > Also a clarification from my previous email:
> > >
> > > > > - Who is responsible for the maintenance of the "publishable 
> > > > > standard"?
> > > >
> > > >
> > > >
> > > > Martin is the primary author and maintainer. 
> > >
> > >
> > >
> > > I should point out that since this is an OGF doc it will go through 
> > > a vetting process similar to that of other standards organizations 
> > > (IETF, etc).  This will be a community wide effort that includes 
> > > individuals both in and outside of perfSONAR.
> > >
> > > > 1.2.1 Base Protocol
> > > >
> > > > - Is the goal the base protocol to specify the order of messages 
> > > > exchanged between services/clients to perform specific actions. For 
> > > > each of the messages exchanged, it must points towards the profile 
> > > > used?
> > >
> > >
> > >
> > >
> > > This document should contain (and only contain) common elements such 
> > > as the basics of why a Request needs to contain certain things 
> > > (similarly for the Response).  There may be state diagrams that show 
> > > how errors and success fit into an exchange, although these would be 
> > > for illustrative purposes only.
> > >
> > > This document should NOT contain reference to specific services or 
> > > service types.  Each 'type' of service will be explained in protocol 
> > > extensions.  Specific services can go into even finer detail in 
> > > service documentation.
> > >
> > > Alluding to that fact that extensions exist is possible, but we 
> > > should not include references 'backwards' to particular extensions.  
> > > The goal of the base should be to factor out a lot of common writing 
> > > that the service documents currently feature.
> > >
> > >
> > > > - Have you got an example of Base protocol document?
> > >
> > >
> > >
> > >
> > > Jeff will be presenting the strawman in Zagreb.
> > >
> > > > - You define a protocol as a request or response. Isn't the 
> > > > protocol the sequence of message exchanged (i.e. a request followed 
> > > > by a response?)
> > >
> > >
> > >
> > >
> > > Besides the Echo/AA parts (which are really just protocol extensions 
> > > we are choosing to include in the base protocol document) there are 
> > > not services that truly implement the 'base' protocol.  All services 
> > > will implement an extension of it (i.e. the MA protocol is a 
> > > specific instance of the base, the SNMP MA protocol implements 
> > > portions of the MA protocol).  Including a very rudimentary exchange 
> > > in the context of a state diagram may happen, but I think it is more 
> > > useful to describe the pieces of the exchange separately.  This can 
> > > be debated.
> > >
> > >
> > > > - I don't understand the difference between the base protocol and 
> > > > the protocol extension. Can you give an example?
> > >
> > >
> > >
> > >
> > > Base protocol should be the home of common things (the concept of a 
> > > request/response pair).
> > >
> > > An example of an extension would be something such as describing MA 
> > > instances and common things they may wish  to do 
> > > (MetadataKeyRequest/Response for example).  Note that the extension 
> > > borrows concepts from the Base (it is really just a request/response 
> > > paradigm, it contains metadata and data, etc.) but it has special 
> > > features that make it unique that must be described (specific type 
> > > of message, contents of metadata or data can be explained futher, 
> > > success/failure states and messages, etc.).  By separating out the 
> > > common description, we safe a lot of time and space (not every 
> > > service developer should need to explain the concept of 
> > > Metadata/Data over and over, it gets a little tiresome).
> > > The Services may take this one step further and claim which parts of 
> > > Extensions they implement/do not implment.  For example the RRD MA 
> > > implements MetadataKey/SetupData/?Storage? messages from the MA 
> > > protocol, the SNMP MA does not implement the Storage message.
> > >
> > >
> > > > 1.2.2 Protocol Extension
> > > >
> > > > - What is a "type of interaction"?  The eventType, a request, a 
> > > > request-response?
> > >
> > >
> > >
> > > As explained above we have service 'types'.  For example the RRD MA, 
> > > or PingER MA are really just Measurement Archives sharing different 
> > > data types.  These two services, even though the data and backend 
> > > storage may be different, still implement many of the same messages 
> > > types.  Why should the service authors re-invent the wheel each time 
> > > when it comes to re-stating the types of messages?
> > >
> > > The 'MA Protocol Extension' will cover the common message exchanges 
> > > that the different MA instances are able to implement (for example 
> > > the MetadataKeyRequest or SetupDataRequest).  This description will 
> > > be general (i.e. we aren't going to go into details about eventTypes 
> > > or data structure).  We will go into details about expected message 
> > > structure (i.e. for a MetadataKeyRequest it is expected that 
> > > multiple pairs of metadata/data triggers can be sent, there may be 
> > > chaining possible, etc.  The Response would illustrate the vague 
> > > concept of a key on success or possible common error situations).
> > >
> > > > - Whave you got an example of Protocol extension?
> > >
> > >
> > >
> > > Jeff will be presenting the strawman in Zagreb.
> > >
> > > > - Where does that relate with the profiles?
> > >
> > >
> > >
> > >
> > > I am not sure I understand.  Profiles are specific to particular 
> > > data, and will be 'schema' focused documents.  The Extensions are 
> > > data agnostic protocol documents that describe interaction.  These 
> > > really are not related directly, they are related only in the 
> > > context of a service.  Am I missing your question?
> > >
> > >
> > > > 1.3 Individual Service Documents
> > > >
> > > >
> > > > In the schema and protocol documents, it is currently not indicated 
> > > > where the part that must be provided are. I guess that each 
> > > > document should highlight what is a must and what isn't.
> > >
> > >
> > >
> > > Yes, this can be discussed.  The strawman examples will help.
> > >
> > > > For having the web-services inter-acting, we need to have a minimum 
> > > > common ground. E.g. for the timing, where do we indicate which time 
> > > > format we must have for each service to ensure inter-operability?
> > >
> > >
> > >
> > > Services (in their own documentation) will be able to indicate witch 
> > > protocols and profiles they implement.
> > > For example some new Information service may choose to implement ALL 
> > > of the LS protocol, and may perhaps implement the MetadataKeyRequest 
> > > from the MA protocol.  The service documentation will note this.  A 
> > > new MA Service instance may offer different kinds of data, so it 
> > > will be necessary to describe which specific profiles used (or not 
> > > fully used if applicable).
> > >
> > >
> > > -jason
> > >
> > >
> > > > Jason Zurawski wrote:
> > > >
> > > > > All;
> > > > >
> > > > >
> > > > > > Jason and I have worked on a proposal for the document framework.
> > > > > > Jason is going to add some information on this to:
> > > > > >
> > > > > > http://wiki.perfsonar.net/jra1-wiki/index.php/PS_meeting_topics_proposals 
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > >
> > > > > Specifics are here:
> > > > >
> > > > > http://wiki.perfsonar.net/jra1-wiki/index.php/DocumentationProposal
> > > > >
> > > > > Comments encouraged.
> > > > >
> > > > > -jason
>
>
> ------------------------------------------------------------------------


--

Jason Zurawski, Network Software Engineer
Internet2

office: [+1-202-331-5354]
mobile: [+1-734-846-2900]
fax:    [+1-202-872-6648]

Be part of the future!
2008 Internet2 Strategic Planning
http://www.internet2.edu/strategicplanning