perfsonar development work

["Jeff W. Boote" <>]

Spam detection software, running on the system "basie.internet2.edu", has
identified this incoming email as possible spam.  The original message
has been attached to this so you can view it (if it isn't spam) or label
similar future email.  If you have any questions, see

 for details.

Content preview:  On May 5, 2008, at 3:43 AM, Jochen Reinwand wrote: > On 
Thursday
   01 May 2008 01:47, Jeff W. Boote wrote: >> There are far more people in the
   perfSONAR community than were at >> Zagreb. >> >> It would be good to 
explain
   what it is you are doing, and why for the >> full community. > > I think
  most of what we are doing is already explained in this > thread. Are you >
   missing anything specific? > There was no detailed explanation about how
  the rsync URLs are > included in the > perfSONAR messages. Indeed I'm not
  sure here about how much is > already fixed. > There were ideas to add the
   URL to the data element instead of the > data or > even use the meta data
   key mechanism. But I'm not working on this > issue. > Perhaps Verena or 
David
   (or even Martin) can say more about it. [...] 

Content analysis details:   (8.2 points, 7.0 required)

 pts rule name              description
---- ---------------------- --------------------------------------------------
 4.1 HELO_DYNAMIC_HCC       Relay HELO'd using suspicious hostname (HCC)
 3.8 HELO_DYNAMIC_IPADDR2   Relay HELO'd using suspicious hostname (IP addr
                            2)
 0.8 SARE_LWSHORTT          BODY: SARE_LWSHORTT
-2.6 BAYES_00               BODY: Bayesian spam probability is 0 to 1%
                            [score: 0.0000]
 2.0 RCVD_IN_SORBS_DUL      RBL: SORBS: sent directly from dynamic IP address
                            [24.117.192.51 listed in dnsbl.sorbs.net]

--- Begin Message ---

• From: "Jeff W. Boote" <>
• To: Jochen Reinwand <>
• Cc: Joe Metzger <>, maxim <>, "'Nicolas Simar'" <>, "'Nina Jeliazkova'" <>, "'schmitz'" <>, "'WiN-Labor'" <>,
• Subject: Re: [pS-dev] Efficient large Data transfer
• Date: Mon, 5 May 2008 08:53:28 -0600

On May 5, 2008, at 3:43 AM, Jochen Reinwand wrote:

On Thursday 01 May 2008 01:47, Jeff W. Boote wrote:

There are far more people in the perfSONAR community than were at
Zagreb.

It would be good to explain what it is you are doing, and why for the
full community.

I think most of what we are doing is already explained in this thread. Are you
missing anything specific?
There was no detailed explanation about how the rsync URLs are included in the
perfSONAR messages. Indeed I'm not sure here about how much is already fixed.
There were ideas to add the URL to the data element instead of the data or
even use the meta data key mechanism. But I'm not working on this issue.
Perhaps Verena or David (or even Martin) can say more about it.

Yes - it is specifically these details that I personally would be interested in hearing. (What the messages would look like.)

Also, I thought a few of Joe's questions were very good. I think a careful examination of them is what is needed to determine what ways the pS protocol needs to be extended in the future to handle bulk data.

I realize you have short term goals - and I'm not suggesting you do anything different from your current plan. But, I do think we need to answer these questions in the medium to long term. And since this is a problem you are currently working on, I am asking for your insight.

Is it necessary to change both the transport protocol (rsync), and the
data storage format (Compressed perl Storables) to achieve your performance
objectives?

Do you think we need to incorporate "external" transport and/or data formats
into the perfSONAR standards?

Does this data exchange need to be considered part of the perfSONAR framework? Or, could
it be considered as a 'backed' interface used by a custom measurement system (hades)
to push data into a perfSONAR Measurement Archive. (Similar to the way several
people are using rsync to update the rrd files served by the rrdma.)

jeff


--- End Message ---