perfSONAR User Q&A and Other Discussion

[Brian Tierney <>]

On Thu, Aug 7, 2014 at 10:37 AM, Aaron Brown <> wrote:

Hey Roderick,

On Aug 7, 2014, at 5:46 AM, Roderick Mooi <> wrote:

> Hi Aaron
>

> Thanks very much. We'll figure something out. What is the planned
> release date for 3.4. production?

Assuming all goes well, we’ll be at the final release stage by early september.

The roadmap is here:

https://code.google.com/p/perfsonar-ps/wiki/RoadMap

3.4 plans are at the bottom of the page.

 

Cheers,
Aaron

>
> Regards,
>
> Roderick
>
>>>> On 2014-08-06 at 21:50, Aaron Brown <> wrote:
>> Hey Roderick,
>>
>> On Aug 6, 2014, at 3:17 AM, Roderick Mooi <> wrote:
>>
>>> Hi Aaron
>>>
>>>> So you want the hosts to save the results 2 measurement archives:
> the
>>> one
>>>> local to the machine running the test, as well as a central
> archive?
>>>
>>> Yes, that's the basic idea. But there are probably some options
> here
>>> and hence my email. Let me provide the reasoning and perhaps you
> can
>>> advise further.
>>> 1. Our MPs are not redundant. If a hard drive fails, for example,
> we
>>> lose the data.
>>> Therefore our primary requirement is to have a backup of the data
> (test
>>> results).
>>> 2. Accessing the test results from a central location will be handy
>>> (e.g. for MADDASH). Also, if a node is isolated we can still access
> the
>>> last test results from that node.
>>>
>>> My present thinking revolves around 2 options:
>>>
>>> option 1. Set up the nodes and mesh to store test results locally.
> Have
>>> a separate cron job that runs say once a day and copies the data to
> a
>>> central MA. This will meet the backup requirement. MADDASH will get
> data
>>> directly from the MPs.
>>> Questions: How to go about this? Ideally, if we could graph the data
> on
>>> the central MA that would be great too.
>>
>> If you’re using 3.3.2, what Ben said about mysql replication is
> probably the
>> best solution. For 3.4, the best solution is probably to have the
> clients
>> store to multiple MAs (one central, one remote). In the mesh, you
> could just
>> set the “read_url” part of the MA to be the central MA, and it
> should
>> theoretically work.
>>
>>> option 2 (preferred). Set up the nodes and mesh to store test
> results
>>> on the central MA. Point the local host to the central MA so that
> the
>>> graphs for that host are available through it's own web interface
> (if
>>> possible). Backup the central MA data periodically.
>>> This is the closest to what we currently have = mesh configured
> with
>>> central MA - and probably the simplest approach.
>>
>> To do this, you’ll need to use 3.4, and will have to do some hand
>> configuration of the regular testing, and MA installations. You’d
> then set
>> the “read_url” like above, and the graphing should work for
> MaDDash.
>>
>>> Which option (or another alternative) would you suggest? Which is
>>> closest to perfSONAR-ps / Internet2 / ESnet best practice (or way
> you do
>>> things)?
>>
>> Internet2 does a centralized setup, but it’s based on only having a
> central
>> MA. It’s also currently hand configured instead of using the mesh
>> configuration to set it up, as Ben noted. I think ESnet is using the
> mesh
>> configuration, but their setup has an MA on each host that they run
> instead
>> of centralizing it.
>>
>> If you want to try the local MA + central MA regular testing setup, I
> can
>> try to writeup some documentation to get you started on that.
>>
>> Cheers,
>> Aaron
>>
>>>
>>> Thanks very much,
>>>
>>> Roderick
>>>
>>>>>> On 2014-08-05 at 19:25, Aaron Brown <> wrote:
>>>> Hey Roderick,
>>>>
>>>> So you want the hosts to save the results 2 measurement archives:
> the
>>> one
>>>> local to the machine running the test, as well as a central
> archive?
>>> If so,
>>>> this will only work on hosts running 3.4-rc, and you’ll need to
> do
>>> some hand
>>>> configuration, unfortunately. I’m not sure there are good
>>> instructions for
>>>> how to do that, though we could probably give you a quick and
> dirty
>>> set of
>>>> steps to follow.
>>>>
>>>> Cheers,
>>>> Aaron
>>>>
>>>> On Aug 5, 2014, at 10:36 AM, Roderick Mooi <>
> wrote:
>>>>
>>>>> Good day
>>>>>
>>>>> We would like to replicate our individual measurement point
> archives
>>> to a
>>>> central MA server.
>>>>> The central server has been setup as a toolkit node but will not
> be
>>> used for
>>>> testing directly.
>>>>>
>>>>> To clarify, we would like both the databases and the graphs to be
>>> duplicated
>>>> - appearing on both the individual node as well as the central
>>> server.
>>>>>
>>>>> Can you please give us advice on where or how to start?
>>>>>
>>>>> (we are using the mesh config if that helps)
>>>>>
>>>>> Thanks and regards,
>>>>>
>>>>> Roderick
>>>>>
>>>>>
>>>>> --
>>>>> This message is subject to the CSIR's copyright terms and
>>> conditions, e-mail
>>>> legal notice, and implemented Open Document Format (ODF) standard.
>
>>>>> The full disclaimer details can be found at
>>>> http://www.csir.co.za/disclaimer.html.
>>>>>
>>>>> This message has been scanned for viruses and dangerous content by
>
>>>> MailScanner,
>>>>> and is believed to be clean.
>>>>>
>>>>> Please consider the environment before printing this email.
>>>>>
>>>>
>>>>
>>>> --
>>>> This message is subject to the CSIR's copyright terms and
> conditions,
>>> e-mail
>>>> legal notice, and implemented Open Document Format (ODF) standard.
>
>>>> The full disclaimer details can be found at
>>>> http://www.csir.co.za/disclaimer.html.
>>>>
>>>> This message has been scanned for viruses and dangerous content by
>
>>>> MailScanner,
>>>> and is believed to be clean.
>>>>
>>>> Please consider the environment before printing this email.
>>>
>>> --
>>> This message is subject to the CSIR's copyright terms and
> conditions, e-mail
>> legal notice, and implemented Open Document Format (ODF) standard.
>>> The full disclaimer details can be found at
>> http://www.csir.co.za/disclaimer.html.
>>>
>>> This message has been scanned for viruses and dangerous content by
>> MailScanner,
>>> and is believed to be clean.
>>>
>>> Please consider the environment before printing this email.
>>>
>>
>>
>> --
>> This message is subject to the CSIR's copyright terms and conditions,
> e-mail
>> legal notice, and implemented Open Document Format (ODF) standard.
>> The full disclaimer details can be found at
>> http://www.csir.co.za/disclaimer.html.
>>
>> This message has been scanned for viruses and dangerous content by
>> MailScanner,
>> and is believed to be clean.
>>
>> Please consider the environment before printing this email.
>
> --
> This message is subject to the CSIR's copyright terms and conditions, e-mail legal notice, and implemented Open Document Format (ODF) standard.
> The full disclaimer details can be found at http://www.csir.co.za/disclaimer.html.
>
> This message has been scanned for viruses and dangerous content by MailScanner,
> and is believed to be clean.
>
> Please consider the environment before printing this email.
>

--
Brian Tierney, http://www.es.net/tierney
Energy Sciences Network (ESnet), Berkeley National Lab
http://fasterdata.es.net