perfSONAR User Q&A and Other Discussion

["Christopher J. Tengi" <>]

Andy,
        Thank-you for the info.  I had taken a look at that section of the 
doc, and decided, for now, not to try clustering.  The MA is actually running 
the full perfSONAR toolkit (although not running any tests itself), so all of 
the automatic deletion setup should already be in place - we just need to 
decide on the policies we want.  And that is where we intersect with disk 
requirements.  Given your numbers, I can now start thinking about the math.

                Thanks,
                        /Chris


> On Jan 12, 2016, at 9:57 AM, Andrew Lake 
> <>
>  wrote:
> 
> Hi,
> 
> 
> The specs you mention(4 cores and 8GB) I *think* will be fine, though it 
> may be worth bumping the RAM to 16GB if you can. It also depends on how 
> many tests each host will be running. The same is true for the disk space. 
> The latency tests are especially important since they are storing new data 
> every minute, as opposed to throughput tests which store just a few data 
> points a day. Doing some very rough math based on the current disk usage of 
> a few of our hosts you are probably looking at around 5MB per month for 
> each latency test, and about 1MB per month for each throughput test. 
> Cassandra does a pretty good job compressing, so it’s not huge, but can add 
> up obviously. It’s not uncommon for latency hosts to run a couple hundred 
> tests, so if you are gonna do a full mesh or similar, the results could add 
> up over time.
> 
> You may have seen this, but there’s some good info on running a central MA 
> such as clustering and managing the data under this heading: 
> http://docs.perfsonar.net/index.html#central-measurement-archive
> 
> Hope that helps,
> Andy
> 
> 
> 
> 
> On January 11, 2016 at 2:22:55 PM, Christopher J. Tengi 
> ()
>  wrote:
> 
>> Greetings, 
>> We have been running a few toolkit hosts and a measurement archive for a 
>> few months now, and plan to scale things *way* up in the not-too-distant 
>> future. I have the MA on a fairly hefty Dell server, and would like to 
>> move it to a VM - it does no measurements of its own. The hardware that 
>> the MA currently lives on has 2, 8 core CPUs (2.4 GHz), 24 GB of RAM, and 
>> around 7 TB of disk. Kind of overkill for what we’re doing, but what I had 
>> not in use at the time. :-) 
>> 
>> I’m looking for a more realistic configuration on a VM for a system that 
>> will act as the MA for around 200 (eventually) low-cost mesh nodes at 
>> building entry points, as well as a few higher-performance hosts in the 
>> data centers and on the Science DMZ, once we deploy one. The VM would also 
>> be running MaDDash for the on-campus mesh as well as one or two (or 3) 
>> wide-area meshes. Note that the members of the wide-are meshes would be 
>> using their own MA, rather than this VM. Based on what I’ve read so far, 
>> I’m assuming that 4 cores and 8 GB of RAM would be more than sufficient 
>> for these purposes, but I’m not sure about the disk requirements. 
>> 
>> The Toolkit documentation talks about 100 GB of disk space, with the 
>> assumption that old data is not automatically cleaned from the local MA. 
>> If I had a 200 node on-campus mesh doing latency (10pps) and bandwidth 
>> measurements (20s every 2h), is there a rule of thumb I could use to 
>> determine how much disk to put on the MA if I wanted to keep the data for 
>> a year? Six months? Three months? As for the CPU and RAM, if I’ve 
>> drastically over-specified or under-specified, I’d like to know. What has 
>> your experience shown? 
>> 
>> Thanks, 
>> /Chris