perfsonar development work

Author: szymon
Date: 2007-10-08 10:12:12 -0400 (Mon, 08 Oct 2007)
New Revision: 284

Modified:
   trunk/nmwg/doc/dLS/dLS.xml
Log:
Just my first general pass. Still needs clarification and refinement

Modified: trunk/nmwg/doc/dLS/dLS.xml
===================================================================
--- trunk/nmwg/doc/dLS/dLS.xml  2007-10-08 13:52:18 UTC (rev 283)
+++ trunk/nmwg/doc/dLS/dLS.xml  2007-10-08 14:12:12 UTC (rev 284)
@@ -48,19 +48,26 @@
 
     <section anchor="intro" title="Introduction">
       <t>
-        This document describes the Distributed Lookup Service in the 
perfSONAR
+        This document describes the Distributed Lookup Service (dLS) in the 
perfSONAR
         (pS) system.  This functionality extends the basic Lookup Service 
(LS)
         functionality that has been present in the system for some time.  The
         basic LS supports the storing and querying of perfSONAR Service
         information as well as metadata about data stored or gathered by a
         particular pS service instance.  
       </t>
+      <t>
+        From clients' perspective, Lookup Service operation involves 
registration,
+        deregistration, querying and obtaining query results.  Clients want 
to discover
+        the services that are running in the network.  LS enables this by 
gathering 
+        information from the services and then using it to fulfill client 
queries.  
+        Figure below presents basic LS interactions.
+      </t>
       
        <figure anchor="ls-op">
            <preamble>LS Operation</preamble>
            <artwork>
  _____                              __________
-|     |     Register/De-register   | Service  |
+|     |     Register/De-register   |          |
 | LS  | &lt;------------------------&gt; | Service  |
 |_____| &lt;---------                 |__________|
                  |                  _________________
@@ -74,7 +81,8 @@
       
       <t>  
         This document describes the support
-        necessary to extend this service to a distributed mode of operation. 
+        necessary to extend basic Lookup Service to a distributed mode of 
operation.
+        dLS reduces overhead in a single service and maintains consistent 
information.
         There are a few key facets of this mode of operation:
       </t>
       
@@ -95,18 +103,18 @@
       
       <t>
         Additionally we present solutions to issues related to allow 
seamless 
-        operation of this service including bootstrapping and domain specific
-        concerns.  
+        operation of this service including bootstrapping (i.e. how service 
find 
+        other parts of the system) and domain specific concerns.  
       </t>      
     </section>
 
 
-    <section anchor="system" title="system Specific Operation">
+    <section anchor="system" title="System Specific Operation">
 
       <!-- Summarization Section -->
 
       <section anchor="summary" title="Summarization">
-        <t>The first step of information flow is when a service registers 
with
+        <t>The first step of information flow is when a pS service registers 
with
            an LS.  The service may know the name of an LS via static
            configuration (the most common case for legacy deployments), or 
            other forms of bootstrapping such as multicast may occur.  A 
service
@@ -123,7 +131,7 @@
            specialized LS with additional searching capabilities.  While a 
            service instance may support limited searching, this is not 
            necessary as they should be focused on storing or gathering data 
-           and leave the lookup functionality to the LS. Possible exceptions 
+           and leave the lookup functionality to the LS.  Possible 
exceptions 
            are rapidly changing Metadata like the most recent timestamp and 
            full details of data stored in long-term archival MAs.
         </t>
@@ -138,12 +146,15 @@
         
         <t>           
            The construction of such a summary is important to the overall 
-           success of this service; summaries must be general enough to 
allow 
+           success of this service; summaries prevents other LS instances 
+           from being overloaded by information, must be general enough to 
allow 
            for easy creation and exchange but also must retain enough 
            information to provide a rich query interface able to locate the 
-           distributed information.  We start by making an observation that 
-           summarization is best based on scope, simply put this means that 
we
-           should attempt to summarize the "most" the "further" away from 
the 
+           distributed information.  That means service metadata information 
must be filtered 
+           (summarized) as it propagates through the LS cloud.  We start by 
making an 
+           observation that summarization is best based on scope (see also 
+           <xref target="scope" /> for forming scope), simply put this 
+           means that we should attempt to summarize the "most" the 
"further" away from the 
            source that we get.  This creates a smaller data set that travels 
            the furthest away while keeping the larger and more informative 
data
            sets closer to the source.  We present the strategies as such:
@@ -156,7 +167,7 @@
             </list>
             
            We limit the discussion in this case to these two scopes, 
although 
-           extension to "n" scopes is possible.  As the number of of scopes 
+           extension to "n" scopes is possible.  As the number of scopes 
            increases additional "aggregation" will be necessary to combine 
            information thus reducing the size of the data sets further.
         </t>
@@ -255,9 +266,9 @@
         <section anchor="upper_scope_summarization" title="Upper Scope 
Summarization"> 
           <t>
             A designated member of each lower ring will be required to 
interact
-            with the upper level.  The mechanics of how we learn who is the 
+            with the upper scope.  The mechanics of how we learn who is the 
             designated leader is discussed in <xref target="tokens" />.  The 
-            leader of each group (and the designated backup) will be 
responsible
+            leader of each lower scope (and the designated backup) will be 
responsible
             for examining each member's summary information and building a 
             summarization/aggregation that best describes the contents of the
             ring.  
@@ -350,8 +361,12 @@
          
       <!-- Scope Section -->
 
-      <section anchor="scope" title="scope"> 
-
+      <section anchor="scope" title="Scope"> 
+               <t>The architecture of the dLS protocol assumes the existence 
of logical
+                  rings of LS instances.  The current proposal involves two 
levels of rings: 
+                  lower scope and upper scope.
+               </t>   
+               
         <t>The next question is how to form the lower and upper scopes.  The 
            simplest answer is that the lower scope be formed based on the 
domain
            name of the participating systems.  That would allow e.g.
@@ -442,7 +457,7 @@
    |                                                    |
  __V__                      _____                      _V___ 
 |     |                    |     |            (1)     |     |
-| LS3 | &lt;----------------- | LS2 | &lt;----------------&gt; | LS1 | 
+| LS3 | &lt;----------------- | LS2 | &lt;________________&gt; | LS1 | 
 |_____|                    |_____|           (2,3)    |_____|
    |                          ^ ^                       ^ ^
    |                          | |                       | |
@@ -456,6 +471,10 @@
              </artwork>
            </figure>            
 
+           <t>Let's assume LS2, LS3 and LS4 are in the ring. LS1 wants to 
join
+              the dLS cloud.
+           </t>
+           
             <list type="symbols">
                 <t>1. LS1 - "candidate" sends LSControlRequest message with 
                       the http://perfsonar.net/services/LS/join eventType to 
@@ -520,7 +539,7 @@
 
           <t>  
             A common method used in P2P (see <xref target="glossary" />) 
systems
-            such as Gnutella (see <xref target="glossary" />) when forming 
+            such as Gnutella when forming 
             "ultrapeers" is to consider the size of the data that a node is
             serving.  The principle, as described 
             <eref 
target="http://rakjar.de/gnufu/index.php/GnuFU_en#Network_model:_Change_who_calls_whom:_Ultrapeers_and_Leafs";>here</eref>,
 
@@ -577,35 +596,18 @@
 
           <section anchor="token_passing_algorithm" title="Token passing 
algorithm">
  
-            <t>
-            <figure anchor="token-example">
-             <preamble>Illustration of LS Token Passing</preamble>
-             <artwork>
-
- _____                      _____ 
-|     |                    |     |
-| LS1 | &lt;----------------- | LS2 |
-|_____|                    |_____|
-   |                          ^
-   |                          | 
-   |          _____           | 
-   |         |     |          | 
-   |-------&gt; | LS3 | ---------|
-             |_____|
-
-
-             </artwork>
-           </figure>            
-
+            <t>The algorithm for token passing works as follows.
+            
             <list type="symbols">
               <t>0. When any LS receives the token (LSControlRequest message 
                     with the http://perfsonar.net/services/LS/token/... 
                     eventType, we will do the following: </t>
-              <t>1.Update local peer list (from token)</t>
-              <t>2.Send summary to all peers in the lease (excluding 
itself)</t>
-              <t>3.Wait for some amount of time</t>
-              <t>4.Send token to next peer from the list (if it fails, try 
next one)</t>
-            </list>            
+              <t>1. Update local peer list (from token)</t>
+              <t>2. Send summary to all peers in the lease (excluding 
itself)</t>
+              <t>3. Wait for some amount of time</t>
+              <t>4. Send token to next peer from the list (if it fails, try 
next one)</t>
+            </list> 
+            
             </t>
           </section>
           
@@ -1149,10 +1151,23 @@
            <section anchor="glossary" title="Glossary">
              <list style="symbols">
 
-          <t>Service -  A Service is an application that communicates with
-             other perfSONAR Services  via standardized protocol set
-             (SOAP+XML+NMWGv2)</t>
-
+             <t>AuthotitativeLS - </t>
+             
+             <t>Berkeley DB XML - Oracle Berkeley DB XML is an open source, 
+             embeddable XML database with XQuery-based access to documents 
+             stored in containers and indexed based on their content.</t>
+             
+             <t>Bootstraping - </t>
+             
+             <t>eXist XML DB - eXist is an Open Source native XML database 
+             featuring efficient, index-based XQuery processing, automatic 
+             indexing, extensions for full-text search, XUpdate support, 
+             XQuery update extensions and tight integration with existing 
+             XML development tools.</t>
+             
+             <t>Home LS (HLS) - The Home LS of a Service is the LS where the
+             Service registers its Lookup Information</t>
+             
           <t>Lookup Service (LS) - The Lookup Service is a key element of the
              perfSONAR framework because it allows every independent service
              to be a visible part of the system. New services may identify
@@ -1165,21 +1180,33 @@
           <t>Lookup Information - information registered by a Service in the
              Lookup Service</t>
 
-          <t>Summary Information - aggregated information from Lookup
-             Information that is sent by one LS to another</t>
-
+          <t>Lower Scope - The scoping paradigm meant to indicate
+             inter-domain relationships.</t>
+             
+          <t>LSRing - </t>
+                         
           <t>Multidomain / Distributed Lookup Information (mLS) - Lookup
              Service which supports summarization and communication with 
other
              Lookup Services (which might be in the same domain...)</t>
+             
+          <t>P2P - </t>
+                          
+          <t>Service -  A Service is an application that communicates with
+             other perfSONAR Services  via standardized protocol set
+             (SOAP+XML+NMWGv2)</t>
 
-          <t>Home LS (HLS) - The Home LS of a Service is the LS where the
-             Service registers its Lookup Information</t>
+          <t>Summary Information - aggregated information from Lookup
+             Information that is sent by one LS to another</t>
+
+                 <t>Token Ring - A ring network in which the network 
topology features 
+             nodes connected to exactly two other nodes, forming a circular 
+             pathway for signals: a ring. Data travels from node to node, 
+             with each node handling every packet.  We use a logical ring in 
+             which a "token" message is used to synchronize the 
communication 
+             among the nodes.</t>
              
           <t>Upper (Global) Scope - The scoping paradigm meant to indicate
              intra-domain relationships.</t>
-
-          <t>Lower (Local) Scope - The scoping paradigm meant to indicate
-             inter-domain relationships.</t>
            
           <t>XSLT - Extensible Stylesheet Language Transformations is an 
              XML-based language used for the transformation of XML documents 
@@ -1190,23 +1217,7 @@
           <t>XQuery - A query language (with some programming language 
              features) that is designed to query collections of XML data. It 
              is semantically similar to SQL.</t>
-          
-          <t>Token Ring - A ring network in which the network topology 
features 
-             nodes connected to exactly two other nodes, forming a circular 
-             pathway for signals: a ring. Data travels from node to node, 
-             with each node handling every packet.  We use a logical ring in 
-             which a "token" message is used to synchronize the 
communication 
-             among the nodes.</t>
-          
-          <t>Berkeley DB XML - Oracle Berkeley DB XML is an open source, 
-             embeddable XML database with XQuery-based access to documents 
-             stored in containers and indexed based on their content.</t>
-          
-          <t>eXist XML DB - eXist is an Open Source native XML database 
-             featuring efficient, index-based XQuery processing, automatic 
-             indexing, extensions for full-text search, XUpdate support, 
-             XQuery update extensions and tight integration with existing 
-             XML development tools.</t>
+             
              </list>
            </section>