perfSONAR User Q&A and Other Discussion

[Mark Feit <>]

on behalf of Hyojoon Kim writes:

One colleague of mine (back at Georgia Tech) gave an idea about running perfSONAR and the OS on an F2FS (Flash-Friendly File System) for the Cubox. This is because F2FS has some good features for doing more efficient I/O on flash storage (e.g., microSD). It’s just an idea, and both of us don’t know if this will have a positive of negative effect. It’s just an idea, which is not tested yet (I’m unlikely to try it unless I have time). 

You may find the effect neutral, but it’s certainly worth experimentation.

Most of what you experienced is probably related to the continuous, linear, all-output nature of the logging being done by the system.  It will run well until the unused RAM commandeered by the kernel fills.  Then it gets turned int a big FIFO constrained by the rate the card can ingest the old pages as they’re written out.  Couple that with the tendency of flash devices to slow down as they wear and you’ll eventually end up with a system that doesn’t perform well.

Logging to RAM is a good choice if you’ve got it to spare and don’t mind losing it if the system dies, but shipping it across the network to a logging server with magnetic storage is better.

The only good, non-magnetic solution for keeping it on the system is to use devices with single-level-cell (SLC) flash instead of the multi-level-cell (MLC) you find in products aimed at the consumer market.  MLC wins out on density and cost per bit, but SLC does better on speed, durability and, if you care about it, power consumption.  Sadly, SLC is unbelievably expensive and doesn’t do much to further the cause of inexpensive perfSONAR nodes.  The 3D XPoint memory that Intel and Micron announced this spring will alleviate those problems, but I wouldn’t expect to see it in SD cards for another year to 18 months.

—Mark