perfsonar development work

Author: roman
Date: 2007-03-06 05:32:50 -0500 (Tue, 06 Mar 2007)
New Revision: 2194

Added:
   branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-manager/filter.py
Removed:
   branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-manager/process.log
Modified:
   branches/JAVA-SQL-MA-STABLE/ant/configure-targets.xml
   branches/JAVA-SQL-MA-STABLE/ant/pre-install-targets.xml
   branches/JAVA-SQL-MA-STABLE/ant/rrdma/rrdma-configure-targets.xml
   branches/JAVA-SQL-MA-STABLE/ant/sqlma/sqlma-configure-targets.xml
   branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-client/config.py
   branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-client/flowmpclient.py
   branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-client/schemas/subscribe.xml
   branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-manager/common.py
   branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-manager/config.py
   branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-manager/manager.py
   
branches/JAVA-SQL-MA-STABLE/doc/ma/sqlTypeMA/Metadata_Configuration_File_Specification.doc
   branches/JAVA-SQL-MA-STABLE/schema/
Log:

the trunk merged with stable branch

  svn merge -r 2163:2193 
https://svn.perfsonar.net/svn/perfsonar/trunk/perfsonar




Modified: branches/JAVA-SQL-MA-STABLE/ant/configure-targets.xml
===================================================================
--- branches/JAVA-SQL-MA-STABLE/ant/configure-targets.xml       2007-03-06 
10:24:58 UTC (rev 2193)
+++ branches/JAVA-SQL-MA-STABLE/ant/configure-targets.xml       2007-03-06 
10:32:50 UTC (rev 2194)
@@ -72,6 +72,9 @@
             addproperty="functionality.register.interval"
             defaultvalue="30000"/>
 
+       <input message="Enter IP or DNS address of the host you are using now 
to deploy the service []:"
+           addproperty="functionality.register.access_point"/>
+
         <input message="Enter the descirption of service that will be send 
to the LS [perfSONAR service]:"
             addproperty="functionality.register.description"
             defaultvalue="perfSONAR service"/>
@@ -79,7 +82,7 @@
         <propertyfile file="conf/service.properties" 
             comment="This file was created by the Ant configuration script.">
 
-            <entry  key="service.r.access_point"         
value="http://${target.server}:${target.port}/${deploy.root}/services/${webservice.name}"/>
+            <entry  key="service.r.access_point"         
value="http://${functionality.register.access_point}:${target.port}/${deploy.root}/services/${webservice.name}"/>
             <entry  key="service.r.service_description"  
value="${functionality.register.description}"/>
             <entry  key="service.r.ls_url"               
value="${functionality.register.ls_address}"/>
             <entry  key="component.registrator.interval" 
value="${functionality.register.interval}"/>

Modified: branches/JAVA-SQL-MA-STABLE/ant/pre-install-targets.xml
===================================================================
--- branches/JAVA-SQL-MA-STABLE/ant/pre-install-targets.xml     2007-03-06 
10:24:58 UTC (rev 2193)
+++ branches/JAVA-SQL-MA-STABLE/ant/pre-install-targets.xml     2007-03-06 
10:32:50 UTC (rev 2194)
@@ -251,7 +251,7 @@
 
     <!-- getting the server address -->
 
-
+<!--
     <target name="get-server-address">
 
         <input message="Enter the address (ip or hostname) of your 
application server Tomcat [localhost]:"
@@ -267,8 +267,20 @@
         </propertyfile>
 
     </target>
+-->
 
+    <target name="get-server-address">
 
+        <propertyfile file="ant/const.properties" 
+            comment="This file was created by the Ant configuration script.">
+
+            <entry  key="target.server" value="127.0.0.1"/>
+
+        </propertyfile>
+
+    </target>
+
+
     <!-- getting the server port -->
 
 

Modified: branches/JAVA-SQL-MA-STABLE/ant/rrdma/rrdma-configure-targets.xml
===================================================================
--- branches/JAVA-SQL-MA-STABLE/ant/rrdma/rrdma-configure-targets.xml   
2007-03-06 10:24:58 UTC (rev 2193)
+++ branches/JAVA-SQL-MA-STABLE/ant/rrdma/rrdma-configure-targets.xml   
2007-03-06 10:32:50 UTC (rev 2194)
@@ -94,7 +94,6 @@
                       comment="This file was created by the Ant 
configuration script.">
 
             <entry  key="service.r.type"                value="ma"/>
-            <entry  key="service.r.access_point"        
value="http://${target.server}:${target.port}/${deploy.root}/services/${webservice.name}"/>
             <entry  key="service.r.service_name"        value="Java RRD MA"/>
             <entry  key="service.r.service_type"        value="RRD MA"/>
             <entry  key="service.r.service_description" value="Java RRD MA, 
perfSONAR project"/>

Modified: branches/JAVA-SQL-MA-STABLE/ant/sqlma/sqlma-configure-targets.xml
===================================================================
--- branches/JAVA-SQL-MA-STABLE/ant/sqlma/sqlma-configure-targets.xml   
2007-03-06 10:24:58 UTC (rev 2193)
+++ branches/JAVA-SQL-MA-STABLE/ant/sqlma/sqlma-configure-targets.xml   
2007-03-06 10:32:50 UTC (rev 2194)
@@ -98,7 +98,6 @@
                         comment="This file was created by the Ant 
configuration script.">
 
             <entry  key="service.r.type"         value="ma"/>
-            <entry  key="service.r.access_point" 
value="http://${target.server}:${target.port}/${deploy.root}/services/${webservice.name}"/>
             <entry  key="service.r.service_name" value="Java SQL MA"/>
             <entry  key="service.r.service_type" value="SQL MA"/>
             <entry  key="service.r.service_description" value="Java SQL MA, 
perfSONAR project"/>

Modified: branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-client/config.py
===================================================================
--- branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-client/config.py 2007-03-06 
10:24:58 UTC (rev 2193)
+++ branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-client/config.py 2007-03-06 
10:32:50 UTC (rev 2194)
@@ -9,21 +9,21 @@
 serviceurl="http://%s:%s%s"; % (serveraddress, serverport, servicepath)
 
 # where do you want to receive your data
-clientaddress="85.144.164.154"
-clientport="2099"
+clientaddress="192.87.102.208"
+clientport="7777"
 
 # where is your zebedee binary
-zebedee="/usr/bin/zebedee"
+zebedee="/usr/local/bin/zebedee"
 
 # keepalive interval
 keepalive=60
 
 # currently only one router at a time is supported
 # surfnet: trillian / ford / zaphod / arthur
-router="zaphod"
+router="ford"
 
-# not yet implemented
-filter=""
+# string to filter flow data
+filter="i'm a evil string"
 
 # not yet implemented
 # flow version (netflow_v9 / netflow_v5 / sflow)
@@ -31,5 +31,5 @@
 
 
 # loglevel (info, debug, warning or error)
-loglevel="info"
+loglevel="debug"
 

Modified: branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-client/flowmpclient.py
===================================================================
--- branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-client/flowmpclient.py   
2007-03-06 10:24:58 UTC (rev 2193)
+++ branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-client/flowmpclient.py   
2007-03-06 10:32:50 UTC (rev 2194)
@@ -36,6 +36,7 @@
         req = req.replace("{clientaddress}", config.clientaddress)
         req = req.replace("{clientport}", config.clientport)
         req = req.replace("{router}", config.router)
+        req = req.replace("{filter}", config.filter)
         logging.debug("using url %s" % config.serviceurl)
         response = self.soapClient.sendSOAP(config.serviceurl, req)
         if not response:
@@ -69,7 +70,7 @@
 
     def keepAlive(self):
         """send keepalive to netflow MP"""
-        logging.info("sending keepalive to service")
+        logging.debug("sending keepalive to service")
         if not self.clientID:
             logging.warning("can't sent keepalive, got no clientID")
             return False

Modified: 
branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-client/schemas/subscribe.xml
===================================================================
--- branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-client/schemas/subscribe.xml   
  2007-03-06 10:24:58 UTC (rev 2193)
+++ branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-client/schemas/subscribe.xml   
  2007-03-06 10:32:50 UTC (rev 2194)
@@ -7,7 +7,7 @@
   <nmwg:metadata id="md1">
     <netflow:parameters id="someid">
       <nmwg:parameter name="anonymizationLevel">1</nmwg:parameter>
-      <nmwg:parameter name="filter"></nmwg:parameter>
+      <nmwg:parameter name="filter">{filter}</nmwg:parameter>
       <nmwg:parameter name="netflowVersion">netflow_v9</nmwg:parameter>
       <nmwg:parameter name="transportProtocol">udp</nmwg:parameter>
     </netflow:parameters>

Modified: branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-manager/common.py
===================================================================
--- branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-manager/common.py        
2007-03-06 10:24:58 UTC (rev 2193)
+++ branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-manager/common.py        
2007-03-06 10:32:50 UTC (rev 2194)
@@ -1,21 +1,24 @@
 
 import os
 import logging
+from elementtree.ElementTree import ElementTree
 
 logging.basicConfig(level=logging.DEBUG,
                     format='%(asctime)s %(levelname)s %(message)s',
                     filename='process.log',
                     filemode='a')
 
-from elementtree.ElementTree import ElementTree
+import config
+import filter
 
-import config
+
   
+fc = filter.FilterChecker()
 
 class Client:
     def __init__(self, address=None, port=None, routers=[], filter=None, \
                    netflow=None, transport=None, zebedeePid=None, 
-                   lastKeepAlive=None, id=None ):
+                   lastKeepAlive=None, id=None, key=None ):
         self.id = id
         self.address = address
         self.port = port
@@ -25,6 +28,7 @@
         self.transport = transport
         self.zebedeePid = zebedeePid
         self.lastKeepAlive = lastKeepAlive
+        self.key = key
 
 
 def readClients():
@@ -46,11 +50,18 @@
                root.find('routers').findall('router')]
 
         f = root.find('filter').text 
-        if f and f.strip(): client.filter = f.strip()
-        else: client.filter = None
+        # check if there is a filter, and if it is okay
+        if f and f.strip():
+            if fc.check(f):
+                client.filter = f.strip()
+            else:
+                logging.warning("evil filter defined!!!")
+        else:
+            client.filter = None
+
         client.flow = root.find('netflowVersion').text.strip()
         client.transport = root.find('transportProtocol').text.strip()
-        #client.zebedeePid = root.find('zebedee-pid').text.strip()
+        client.key = root.find('anonymizingkey').text.strip()
         
         clients.append(client)
 

Modified: branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-manager/config.py
===================================================================
--- branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-manager/config.py        
2007-03-06 10:24:58 UTC (rev 2193)
+++ branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-manager/config.py        
2007-03-06 10:32:50 UTC (rev 2194)
@@ -1,6 +1,6 @@
 #!/usr/bin/env python
 
-propFile = 
"/usr/local/gijs/tomcat/webapps/perfSONAR-FLOWMP-1.1-beta/WEB-INF/classes/perfsonar/conf/service.properties"
+propFile = 
"/usr/local/FlowSubscriptionMP/tomcat/webapps/perfSONAR-FLOWMP-1.1-beta/WEB-INF/classes/perfsonar/conf/service.properties"
 
 import re
 import sys

Copied: branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-manager/filter.py (from 
rev 2193, trunk/perfsonar/contrib/FlowMP-manager/filter.py)

Modified: branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-manager/manager.py
===================================================================
--- branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-manager/manager.py       
2007-03-06 10:24:58 UTC (rev 2193)
+++ branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-manager/manager.py       
2007-03-06 10:32:50 UTC (rev 2194)
@@ -72,9 +72,13 @@
         pFile = open(os.path.join(runDir, pid))
         pNum = int(pFile.readline().strip())
         logging.info("kill " + str(pNum))
-        os.kill(pNum, signal.SIGTERM)
+        try:
+            os.kill(pNum, signal.SIGTERM)
+        except OSError, err:
+            logging.error(err)
 
 
+
 def main():
     try:
         opts, args = getopt.getopt(sys.argv[1:], "hsq", ["help", "start", 
"stop"])

Deleted: branches/JAVA-SQL-MA-STABLE/contrib/FlowMP-manager/process.log

Modified: 
branches/JAVA-SQL-MA-STABLE/doc/ma/sqlTypeMA/Metadata_Configuration_File_Specification.doc
===================================================================
(Binary files differ)


Property changes on: branches/JAVA-SQL-MA-STABLE/schema
___________________________________________________________________
Name: svn:externals
   + nmwg http://anonsvn.internet2.edu/svn/nmwg/trunk/nmwg/schema/