perfSONAR User Q&A and Other Discussion

[Brian Candler <>]

On 18/05/2017 15:28, Mark Feit wrote:
> (Sorry, this ran very long, but I hope it adequately answers your questions 
> and addresses your concerns.)
Thank you. Lots of detail is definitely good, and I found this very 
helpful in understanding better what's happening and why.

> pScheduler’s database, which is separate from esmond’s, plays two roles.
> The first is as a store for anything that needs to persist, which includes 
> the inventory of available tests, tools and archivers, tasks, the schedule of 
> each run, the results (for as long as we keep them), queues of results that 
> need to be archived and a handful of other administrative items.
Does this mean that all test results get put into postgres before being 
handed over to the archiver?

I can see benefits of that, especially where the archiver is remote and 
may not always be online or reachable. ISTR that perfsonar had some sort 
of file-based queue before, and this sounds like a a much more robust 
way of processing the data.

>    Everything pScheduler knows will persist in a consistent state across 
> software upgrades, system crashes and reboots, which means the single ad hoc 
> test you scheduled for next Wednesday at 3:15 will still run as if nothing 
> had happened.
I get the impression that a passive node - one which never schedules a 
test by itself, and never takes the "lead" role - isn't going to be 
using the database very much, except for keeping an inventory of 
available test modules (which is essentially static).

>      Question 4: owamp has its own control protocol on well-known port (861).
>      Is a perfsonar 4.x node able to run owamp tests to a remote node which
>      is running owamp-server but *not* running pscheduler or bwctl?
>
> It is, but that’s because the latency test was designed to be 
> single-participant (as are rtt and trace).  I’d have preferred that the test 
> be two-participant so the OWAMP tool plugin only starts owampd when needed, 
> but I didn’t make that decision and there are valid reasons for it to work 
> the way it does.
Thanks, that may be sufficient for me.

Basically I want to be able to install a handful of packages on an 
existing server to have it participate in some sort of periodic network 
tests in addition to its normal workload.  Installing ~200 packages [^1] 
is too intrusive.  Turning it into a docker host is also too intrusive.  
Running owamp-server gets me 50% of what I want in a single package.

As a specific example: I have a number of Mac Minis running macOS which 
are acting as virtual "remote users" for network troubleshooting - they 
are used as remote desktops.  I have added bwctl+owamp+iperf from 
homebrew, so that I can also schedule tests to these servers from 
central Perfsonar nodes.

There's no pscheduler in homebrew yet, and it sounds like it might be 
difficult to do given the number of dependencies.

> If BWCTL suits your needs, we’re not going seek out and destroy every copy of 
> the source code we can find.  The license doesn’t prevent anyone from 
> continuing to build, maintain, use or distribute it.
Sure. I totally get what you're saying about community requirements.  
I'm probably a very oddball user, compared to the majority who are 
deploying dedicating testing nodes on their 100G backbones :-)

My concern is how long before bwctl fallback support is stripped out of 
perfsonar - if that happens, bwctl's utility will be limited to ad-hoc 
testing.

Having said that, given pscheduler's pluggable architecture, maybe it's 
not too hard to write a separately-maintained bwctl plugin.

Alternatively, maybe I could write a pscheduler plugin which runs a 
one-participant test talking to an iperf3 server (i.e. assumes "iperf3 
-s" is already running at the far end).  This loses the ability of bwctl 
to prevent two tests being scheduled concurrently, but for occasional 
tests it might be good enough.

Regards,

Brian.

[^1] I thought I'd measure it properly, starting from a clean ubuntu 
16.04 lxd container, and adding the perfsonar-jessie-release repository.

"apt-get install perfsonar-tools pscheduler-server" wants to install 259 
new packages, using 409MB of disk space.

"apt-get install perfsonar-tools pscheduler-server 
--no-install-recommends" wants to install 182 new packages, using 311MB 
of disk space.  This still includes x11-common, and many qt and gtk 
libraries. Maybe getting rid of the hard dependency on gnuplut would 
help, which is where this thread started :-)