perfsonar development work

["Jeff W. Boote" <>]

Roland Karch wrote:
> Hi all,
>
> Jeff W. Boote schrieb:
> > Jochen Reinwand wrote:
> > > Hades utilises POSIX real time capabilities of the operating system, 
> > > so on one hand it is more sensitive to slightly degraded performance, 
> > > but might preempt services that do not run with elevated priority.
> > > But using two interfaces also brings in operating system related 
> > > issues. Perhaps Hades measurements are affected by AMI measurements 
> > > and/or vice versa, although they are running on different interfaces. 
> > > We have no safe knowledge about this issue and it is most likely both 
> > > hardware and operating system dependent.
> >
> > I agree, there are definitely system dependent issues. We won't know 
> > for sure if both systems can coexist on a host until we try some of 
> > these things.
> This is something we could try out within the JRA1 measurement network. 
> If you're interested in it, I'll pick out three MPs which are already 
> part of the GEANT full OWD measurement mesh. Most of them do have two 
> network interfaces connected already, so we could set up an OWAMP daemon 
> with appropriate credentials installed so that you can stress-test it 
> with OWAMP/AMI and do a comparison.

That would be great. I don't have time to do it immediately... If it works for 
you, how about we start this the week before the meeting. Then we can address 
any difficult issues that might come up in person the next week?

> > bwctl/iperf/owamp all allow you to bind to a specific address. In 
> > fact, they each specifically bind to the address that was requested - 
> > it is not an option. So... if the host routing configuration allows 
> > source-based routing to be defined (and Linux at least claims to) the 
> > default route option is not required.
> Linux is indeed a bit tricky in that regard sometimes. For us, it turned 
> out that binding to an IP address does not ensure that packets actually 
> do get sent out of the relevant physical interface. It influences which 
> source address is being used in the IP header of the packet, but a 
> packet with a source IP from eth1 can actually be sent out via eth0 as 
> an example. There is a (Linux-specific?) ioctl which when used ensures 
> that a socket is actually bound to a physical interface, not just its IP 
> address. That's at least what we had to use to make it work.

Right - that is one way. But, Linux now also has some policy routing stuff you 
can use to set up multiple routing tables based on things like the source 
address of the packet. Here is link I found that at least somewhat describes how 
to do it. I was able to get it to work on one system, and not on another last 
year at SC. I never did figure out what I had done differently before we packed 
up the systems...

http://www.linux.org/docs/ldp/howto/Adv-Routing-HOWTO/lartc.rpdb.multiple-links.html

> In any case, if that is necessary I'm sure it's only a minor 
> modification in the code on your part.

Yes - perhaps it would be best to add this functionality directly into owamp.

> > Hmm... Likewise, the specific schedule of packets is configurable for 
> > AMI. So, it would be possible to indicate that packet pairs or even 
> > trains of packets be sent at a given frequency. So, AMI could provide 
> > very close to the same data. I had not considered this before. (Thanks 
> > for the idea. :) )
> Just one small word of caution, because not having a train schedule can 
> literally lead to a train wreck. I.e. not only one measurement packet 
> out of ten a second collide, but one whole train of packets collide. 
> That's after all one of the main reasons why we do schedule our trains. :-)

Ah, makes sense.

> But yes, it would be an interesting possibility to directly compare the 
> systems.

Of course, my data analysis stuff is not setup to look for anything like this at 
the moment either.

> > I think it would be very interesting to compare the Hades data with 
> > the AMI data across the same paths. AMI does rely on the statistical 
> > methods as you state. It was our view that since owamp is a user-space 
> > application it would require some statistical analysis to be useful no 
> > matter how tightly we controlled things (to deal simply with UNIX 
> > scheduling issues). Additionally we didn't want to tightly control 
> > things because we were trying for wide deployments, we did not have an 
> > appliance as a goal.
> Should we take this comparison up as an action until/for the Berkeley 
> meeting? I think the main question we will have to solve there is how we 
> are able to integrate a "GEANT appliance" operating on a set of tier 
> 0/1/(maybe some)2 with an "AMI/owamp distribution" running on a cloud of 
> tier2/3/../n. The current agenda is a bit vague about it. Should we take 
> up the following?
>
>     * Investigation into parallel operation of Hades and AMI/owamp
>           o running as two daemons on the same host and interface
>           o running as two daemons on the same host with separate interfaces
>           o running as two daemons with owamp for on demand purposes and
>             Hades for scheduled operation on the same interface
>     * Evaluate the possibility of unifying the Hades/owamp measurement
>       methodology towards high frequency (~1/s) packet trains
>           o Will visible phenomena change?
>           o Could the same analysis methods be used? (Might allow to use
>             the existing perfSONAR infrastructure for AMI-based OWD
>             measurements)

This sounds reasonable to me. (And it looks like it is already on the agenda. 
:) )

> Me and my team are still lacking a general idea about what exactly we 
> are supposed to discuss in Berkeley. And we'd like to avoid a meeting 
> where there are about 30 minutes of relevant things to discuss, and the 
> rest of the time will be spent reading emails. Does this general outline 
> come close to your expectations?

This works for me. Lets see what others say.

I do need to work with VV at some point so we can make sure we are using as 
similar as possible schema for bwctl and OWD/IPDV - (and summaries/statistics of 
that). This will be critical to make the LHC offering look/act as integrated as 
possible. Especially if we go this route of using both systems. But, hopefully 
we can do that via email/skype. (I do not know yet if I will be at the Oct 30/31 
meeting. I kind of hope not, since that will mean 4 weeks of travel in a row...)

jeff