perfSONAR User Q&A and Other Discussion

[Aaron Brown <>]

Hey Roderick,

On Aug 6, 2014, at 3:17 AM, Roderick Mooi 
<>
 wrote:

> Hi Aaron
> 
>> So you want the hosts to save the results 2 measurement archives: the
> one 
>> local to the machine running the test, as well as a central archive?
> 
> Yes, that's the basic idea. But there are probably some options here
> and hence my email. Let me provide the reasoning and perhaps you can
> advise further.
> 1. Our MPs are not redundant. If a hard drive fails, for example, we
> lose the data.
> Therefore our primary requirement is to have a backup of the data (test
> results).
> 2. Accessing the test results from a central location will be handy
> (e.g. for MADDASH). Also, if a node is isolated we can still access the
> last test results from that node.
> 
> My present thinking revolves around 2 options:
> 
> option 1. Set up the nodes and mesh to store test results locally. Have
> a separate cron job that runs say once a day and copies the data to a
> central MA. This will meet the backup requirement. MADDASH will get data
> directly from the MPs.
> Questions: How to go about this? Ideally, if we could graph the data on
> the central MA that would be great too.

If you’re using 3.3.2, what Ben said about mysql replication is probably the 
best solution. For 3.4, the best solution is probably to have the clients 
store to multiple MAs (one central, one remote). In the mesh, you could just 
set the “read_url” part of the MA to be the central MA, and it should 
theoretically work.

> option 2 (preferred). Set up the nodes and mesh to store test results
> on the central MA. Point the local host to the central MA so that the
> graphs for that host are available through it's own web interface (if
> possible). Backup the central MA data periodically.
> This is the closest to what we currently have = mesh configured with
> central MA - and probably the simplest approach.

To do this, you’ll need to use 3.4, and will have to do some hand 
configuration of the regular testing, and MA installations. You’d then set 
the “read_url” like above, and the graphing should work for MaDDash.

> Which option (or another alternative) would you suggest? Which is
> closest to perfSONAR-ps / Internet2 / ESnet best practice (or way you do
> things)?

Internet2 does a centralized setup, but it’s based on only having a central 
MA. It’s also currently hand configured instead of using the mesh 
configuration to set it up, as Ben noted. I think ESnet is using the mesh 
configuration, but their setup has an MA on each host that they run instead 
of centralizing it.

If you want to try the local MA + central MA regular testing setup, I can try 
to writeup some documentation to get you started on that.

Cheers,
Aaron

> 
> Thanks very much,
> 
> Roderick
> 
>>>> On 2014-08-05 at 19:25, Aaron Brown 
>>>> <>
>>>>  wrote:
>> Hey Roderick,
>> 
>> So you want the hosts to save the results 2 measurement archives: the
> one 
>> local to the machine running the test, as well as a central archive?
> If so, 
>> this will only work on hosts running 3.4-rc, and you’ll need to do
> some hand 
>> configuration, unfortunately. I’m not sure there are good
> instructions for 
>> how to do that, though we could probably give you a quick and dirty
> set of 
>> steps to follow.
>> 
>> Cheers,
>> Aaron
>> 
>> On Aug 5, 2014, at 10:36 AM, Roderick Mooi 
>> <>
>>  wrote:
>> 
>>> Good day
>>> 
>>> We would like to replicate our individual measurement point archives
> to a 
>> central MA server.
>>> The central server has been setup as a toolkit node but will not be
> used for 
>> testing directly.
>>> 
>>> To clarify, we would like both the databases and the graphs to be
> duplicated 
>> - appearing on both the individual node as well as the central
> server.
>>> 
>>> Can you please give us advice on where or how to start?
>>> 
>>> (we are using the mesh config if that helps)
>>> 
>>> Thanks and regards,
>>> 
>>> Roderick
>>> 
>>> 
>>> -- 
>>> This message is subject to the CSIR's copyright terms and
> conditions, e-mail 
>> legal notice, and implemented Open Document Format (ODF) standard. 
>>> The full disclaimer details can be found at 
>> http://www.csir.co.za/disclaimer.html.
>>> 
>>> This message has been scanned for viruses and dangerous content by 
>> MailScanner, 
>>> and is believed to be clean.
>>> 
>>> Please consider the environment before printing this email.
>>> 
>> 
>> 
>> -- 
>> This message is subject to the CSIR's copyright terms and conditions,
> e-mail 
>> legal notice, and implemented Open Document Format (ODF) standard. 
>> The full disclaimer details can be found at 
>> http://www.csir.co.za/disclaimer.html.
>> 
>> This message has been scanned for viruses and dangerous content by 
>> MailScanner, 
>> and is believed to be clean.
>> 
>> Please consider the environment before printing this email.
> 
> -- 
> This message is subject to the CSIR's copyright terms and conditions, 
> e-mail legal notice, and implemented Open Document Format (ODF) standard. 
> The full disclaimer details can be found at 
> http://www.csir.co.za/disclaimer.html.
> 
> This message has been scanned for viruses and dangerous content by 
> MailScanner, 
> and is believed to be clean.
> 
> Please consider the environment before printing this email.
>