perfSONAR User Q&A and Other Discussion

[Aaron Brown <>]

Hey Roderick,

On Aug 7, 2014, at 5:46 AM, Roderick Mooi 
<>
 wrote:

> Hi Aaron
> 
> Thanks very much. We'll figure something out. What is the planned
> release date for 3.4. production?

Assuming all goes well, we’ll be at the final release stage by early 
september.

Cheers,
Aaron

> 
> Regards,
> 
> Roderick
> 
>>>> On 2014-08-06 at 21:50, Aaron Brown 
>>>> <>
>>>>  wrote:
>> Hey Roderick,
>> 
>> On Aug 6, 2014, at 3:17 AM, Roderick Mooi 
>> <>
>>  wrote:
>> 
>>> Hi Aaron
>>> 
>>>> So you want the hosts to save the results 2 measurement archives:
> the
>>> one 
>>>> local to the machine running the test, as well as a central
> archive?
>>> 
>>> Yes, that's the basic idea. But there are probably some options
> here
>>> and hence my email. Let me provide the reasoning and perhaps you
> can
>>> advise further.
>>> 1. Our MPs are not redundant. If a hard drive fails, for example,
> we
>>> lose the data.
>>> Therefore our primary requirement is to have a backup of the data
> (test
>>> results).
>>> 2. Accessing the test results from a central location will be handy
>>> (e.g. for MADDASH). Also, if a node is isolated we can still access
> the
>>> last test results from that node.
>>> 
>>> My present thinking revolves around 2 options:
>>> 
>>> option 1. Set up the nodes and mesh to store test results locally.
> Have
>>> a separate cron job that runs say once a day and copies the data to
> a
>>> central MA. This will meet the backup requirement. MADDASH will get
> data
>>> directly from the MPs.
>>> Questions: How to go about this? Ideally, if we could graph the data
> on
>>> the central MA that would be great too.
>> 
>> If you’re using 3.3.2, what Ben said about mysql replication is
> probably the 
>> best solution. For 3.4, the best solution is probably to have the
> clients 
>> store to multiple MAs (one central, one remote). In the mesh, you
> could just 
>> set the “read_url” part of the MA to be the central MA, and it
> should 
>> theoretically work.
>> 
>>> option 2 (preferred). Set up the nodes and mesh to store test
> results
>>> on the central MA. Point the local host to the central MA so that
> the
>>> graphs for that host are available through it's own web interface
> (if
>>> possible). Backup the central MA data periodically.
>>> This is the closest to what we currently have = mesh configured
> with
>>> central MA - and probably the simplest approach.
>> 
>> To do this, you’ll need to use 3.4, and will have to do some hand 
>> configuration of the regular testing, and MA installations. You’d
> then set 
>> the “read_url” like above, and the graphing should work for
> MaDDash.
>> 
>>> Which option (or another alternative) would you suggest? Which is
>>> closest to perfSONAR-ps / Internet2 / ESnet best practice (or way
> you do
>>> things)?
>> 
>> Internet2 does a centralized setup, but it’s based on only having a
> central 
>> MA. It’s also currently hand configured instead of using the mesh 
>> configuration to set it up, as Ben noted. I think ESnet is using the
> mesh 
>> configuration, but their setup has an MA on each host that they run
> instead 
>> of centralizing it.
>> 
>> If you want to try the local MA + central MA regular testing setup, I
> can 
>> try to writeup some documentation to get you started on that.
>> 
>> Cheers,
>> Aaron
>> 
>>> 
>>> Thanks very much,
>>> 
>>> Roderick
>>> 
>>>>>> On 2014-08-05 at 19:25, Aaron Brown 
>>>>>> <>
>>>>>>  wrote:
>>>> Hey Roderick,
>>>> 
>>>> So you want the hosts to save the results 2 measurement archives:
> the
>>> one 
>>>> local to the machine running the test, as well as a central
> archive?
>>> If so, 
>>>> this will only work on hosts running 3.4-rc, and you’ll need to
> do
>>> some hand 
>>>> configuration, unfortunately. I’m not sure there are good
>>> instructions for 
>>>> how to do that, though we could probably give you a quick and
> dirty
>>> set of 
>>>> steps to follow.
>>>> 
>>>> Cheers,
>>>> Aaron
>>>> 
>>>> On Aug 5, 2014, at 10:36 AM, Roderick Mooi 
>>>> <>
> wrote:
>>>> 
>>>>> Good day
>>>>> 
>>>>> We would like to replicate our individual measurement point
> archives
>>> to a 
>>>> central MA server.
>>>>> The central server has been setup as a toolkit node but will not
> be
>>> used for 
>>>> testing directly.
>>>>> 
>>>>> To clarify, we would like both the databases and the graphs to be
>>> duplicated 
>>>> - appearing on both the individual node as well as the central
>>> server.
>>>>> 
>>>>> Can you please give us advice on where or how to start?
>>>>> 
>>>>> (we are using the mesh config if that helps)
>>>>> 
>>>>> Thanks and regards,
>>>>> 
>>>>> Roderick
>>>>> 
>>>>> 
>>>>> -- 
>>>>> This message is subject to the CSIR's copyright terms and
>>> conditions, e-mail 
>>>> legal notice, and implemented Open Document Format (ODF) standard.
> 
>>>>> The full disclaimer details can be found at 
>>>> http://www.csir.co.za/disclaimer.html.
>>>>> 
>>>>> This message has been scanned for viruses and dangerous content by
> 
>>>> MailScanner, 
>>>>> and is believed to be clean.
>>>>> 
>>>>> Please consider the environment before printing this email.
>>>>> 
>>>> 
>>>> 
>>>> -- 
>>>> This message is subject to the CSIR's copyright terms and
> conditions,
>>> e-mail 
>>>> legal notice, and implemented Open Document Format (ODF) standard.
> 
>>>> The full disclaimer details can be found at 
>>>> http://www.csir.co.za/disclaimer.html.
>>>> 
>>>> This message has been scanned for viruses and dangerous content by
> 
>>>> MailScanner, 
>>>> and is believed to be clean.
>>>> 
>>>> Please consider the environment before printing this email.
>>> 
>>> -- 
>>> This message is subject to the CSIR's copyright terms and
> conditions, e-mail 
>> legal notice, and implemented Open Document Format (ODF) standard. 
>>> The full disclaimer details can be found at 
>> http://www.csir.co.za/disclaimer.html.
>>> 
>>> This message has been scanned for viruses and dangerous content by 
>> MailScanner, 
>>> and is believed to be clean.
>>> 
>>> Please consider the environment before printing this email.
>>> 
>> 
>> 
>> -- 
>> This message is subject to the CSIR's copyright terms and conditions,
> e-mail 
>> legal notice, and implemented Open Document Format (ODF) standard. 
>> The full disclaimer details can be found at 
>> http://www.csir.co.za/disclaimer.html.
>> 
>> This message has been scanned for viruses and dangerous content by 
>> MailScanner, 
>> and is believed to be clean.
>> 
>> Please consider the environment before printing this email.
> 
> -- 
> This message is subject to the CSIR's copyright terms and conditions, 
> e-mail legal notice, and implemented Open Document Format (ODF) standard. 
> The full disclaimer details can be found at 
> http://www.csir.co.za/disclaimer.html.
> 
> This message has been scanned for viruses and dangerous content by 
> MailScanner, 
> and is believed to be clean.
> 
> Please consider the environment before printing this email.
>