perfSONAR User Q&A and Other Discussion

[Mark Feit <>]

Peter Lambrechtsen writes:

 

The only issue I have is the requirement to run postgres and the additional load that creates.

 

As has been mentioned before, we don’t officially support perfSONAR on SBCs, but we’re always interested in what people are doing with them.  That disclaimer aside, I can give you some guidance about how the new software should fit into that environment:

 

PostgreSQL:  Having a database on a small system shouldn’t be much of a problem.  I’ve been around PostgreSQL and its ancestors for almost 30 years.  The current code base and the people who oversee it date back to the constraints of the systems that were available 20 years ago and do a stellar job keeping resource consumption down.  The default configuration does reasonably well and can be re-tuned for a smaller environment (with some trade-offs) if you find it’s needed.  I think what we got in trade for the extra load in terms of ease of development and operational robustness is well worth it.

 

Cores:  We officially recommend four and most of us develop with two and a lot of our test bed runs in that configuration.  pScheduler will run on one if the level of activity on the system is very low.

 

Memory:  We officially recommend 4 GiB.  I develop with half that.  The footprint for my relatively-quiet development system runs a combined 165 MiB for PostgreSQL, pScheduler and Apache.  Our testbed systems tip the scales at about 800 MiB, but that includes a fairly heavy workload and whatever Esmond adds to the picture.  The actual figure for a lightly-loaded SBC is probably going to be much less than half that.  (All of this is with the out-of-the-box configuration.)

 

Storage:   We haven’t run the system on flash-based devices much, so I don’t have a good picture of how long the media will last.  There are some things that can be done to cut down on how much the flash gets abused, like sending the syslogs to another system or using a small RAM disk.

 

Pi and Others:  The Pi 3B or 2B should have enough horsepower to run the new software if you’re not trying to use it for an enterprise-grade workload.  The biggest thorn in our side during testing was systems with only one core, so a Pi 2B or 3B should do fine in that department.  Obviously, measurements requiring high performance aren’t going to happen any more than before.  Personally, I think the best SBC out there to run 4.0 is going to be the ODroid C2, which has 2 GB of RAM and four cores.  That comes close to the systems where the development is done, which have 2 GB RAM and two cores. 

 

If it were me I would allow for offline testing to just be written to a filesystem results file then pushed up to a central server.

 

I’m not sure what you mean by offline testing; pScheduler is completely self-contained and doesn’t require support from other hosts to run.  The archiving mechanism has provisions for reliability, so any measurement result that can’t be sent off immediately will be held and retried later.  It’s also extensible, so if you want to spool your results to a file and do your own thing with them, an archiver to do that can be put together in an hour or two once you understand how everything works.

 

--Mark