perfSONAR User Q&A and Other Discussion

[Aaron Brown <>]

Hey George,

Looking at the output, it looks pretty much identical to the issue roderick 
is having. Would you be able to send the .json mesh config file, and let me 
know the IP(s) of the working host, and the failing host. Also, if you could 
try the 3.4 meshconfig RPM update, that would be helpful as well.

Cheers,
Aaron

On Jul 29, 2014, at 10:12 AM, Uhl, George D. (GSFC-423.0)[ARTS] 
<>
 wrote:

> I apologize for interjecting myself into this thread butŠ
> 
> I'm experiencing the same, or at least a similar, issue as Roderick.  I
> have an existing mesh of non-toolkit PS servers and I've added a server
> running 3.3.2 toolkit for some OWAMP tests.  Tests are only working in one
> direction.  When I run the mesh "generate_configuration" script on the
> toolkit server, the owmesh.conf remains unchanged and I don't see the
> powstream daemons running for the reverse direction tests.
> 
> I've attached the output of "generate_configuration -v" for both the
> non-toolkit server and the toolkit server.
> 
> -George
> 
> On 7/29/14 9:51 AM, "Roderick Mooi" 
> <>
>  wrote:
> 
>> Hi Aaron
>> 
>> These are production hosts so if it's possible to use the 3.4 mesh config
>> with 3.3.2 toolkit that will be fine. Where do I get it from?
>> 
>> Thanks,
>> 
>> Roderick 
>> 
>>>>> On 2014-07-29 at 15:41, Aaron Brown 
>>>>> <>
>>>>>  wrote:
>>> Hey Roderick,
>>> 
>>> That is bizarre. I tried it locally with the 3.4 series of Mesh Config,
>>> and 
>>> it generated the owmesh.conf just fine for the test1.conf. Would you be
>>> able 
>>> to update to the 3.4 Mesh Configuration, or would that cause problems
>>> on your 
>>> side?
>>> 
>>> Cheers,
>>> Aaron
>>> 
>>> On Jul 25, 2014, at 5:33 AM, Roderick Mooi 
>>> <>
>>>  wrote:
>>> 
>>>> Hi Aaron et al
>>>> 
>>>> We are having problems using MeshConfig despite using it successfully
>>>> in the 
>>> past.
>>>> This is on production hosts with the latest perfSONAR PS (3.3.2)
>>>> installed 
>>> and up to date. We removed all manual tests from these hosts.
>>>> 
>>>> We installed MeshConfig and JSONBuilder versions:
>>>> perl-perfSONAR_PS-MeshConfig-Shared-3.3.2-3.pSPS.noarch
>>>> perl-perfSONAR_PS-MeshConfig-Agent-3.3.2-3.pSPS.noarch
>>>> perl-perfSONAR_PS-MeshConfig-JSONBuilder-3.3.2-3.pSPS.noarch
>>>> 
>>>> Setup our mesh .conf file, ran build_json and configured
>>> agent_configuration.conf to point to it.
>>>> 
>>>> Running generate_configuration gave no errors but also no tests.
>>>> 
>>>> To aid troubleshooting we tried a very simple config with only 2 hosts
>>>> and 
>>> bwctl test - still didn't work - no tests appeared on either host (we
>>> ran 
>>> generate_configuration on both). I also saw that the owmesh.conf files
>>> aren't 
>>> updated. A scan of the perfSONAR logs didn't show anything that
>>> appeared to 
>>> be related.
>>>> 
>>>> I attach the following:
>>>> 1. test1.conf - the simplified configuration we tried when the full
>>>> one 
>>> wasn't working
>>>> 2. test2.conf - replaced only essentials from example.conf - e.g.
>>>> swapping 
>>> your servers with ours to see if that helps - same result
>>>> 3. output from generate_configuration --verbose
>>>> 4. owmesh.conf after running generate_configuration on test1.conf =
>>> owmesh1.conf
>>>> 5. owmesh.conf after running generate_configuration on test2.conf =
>>> owmesh2.conf
>>>> 
>>>> Can you please help us track down the problem?
>>>> 
>>>> Thanks very much,
>>>> 
>>>> Roderick
>>>> 
>>>> 
>>>> 
>>>>>>> On 2014-06-11 at 19:02, Jeremy Palmer 
>>>>>>> <>
>>>>>>> wrote:
>>>>> I actually just converted all our toolkit hosts to a 'mesh' this
>>>>> week. Here 
>>>>> are 
>>>>> a few of the things that I noted from my work.
>>>>> 
>>>>> - I do have an example.conf in my /opt/perfsonar_ps/mesh_config/doc/
>>>>> directory. 
>>>>> I'm guessing it was installed when I installed the below JSON builder
>>>>> package.
>>>>> 
>>>>> - Install JSON Builder (not installed by default):
>>>>> yum install perl-perfSONAR_PS-MeshConfig-JSONBuilder
>>>>> 
>>>>> - None of my mesh tests would work bi-directioinally until I added
>>>>> the 
>>>>> following 
>>>>> line under the owamp test_spec section of my mesh config. I did not
>>>>> see any 
>>>>> documentation on this and its not in the example config. I just
>>>>> happened to 
>>>>> find 
>>>>> it somewhere in a google search.
>>>>> force_bidirectional 1
>>>>> 
>>>>> I did remove all manually-created tests on my toolkit hosts prior to
>>>>> setting 
>>>>> up 
>>>>> the mesh. I don't think this is required tho since the mesh creates
>>>>> its own 
>>>>> scheduled tests on each host.
>>>>> 
>>>>> I also had an issue where none of my mesh tests would populate any
>>>>> graph 
>>>>> data in 
>>>>> the webUI. I ended up re-initializing my owamp databases using the
>>>>> following 
>>>>> command. This seemed to fix the issue.
>>>>> /opt/perfsonar_ps/perfsonarbuoy_ma/bin/owdb.pl -i root
>>>>> 
>>>>> One last thing I found odd: in the example.conf config, the
>>>>> owamp_test is 
>>>>> set 
>>>>> for 100 packets-per-second (0.01) even though the comment says "Send
>>>>> 10 
>>>>> packets 
>>>>> every second". I changed my setting back to 10pps (0.1) since 100pps
>>>>> will 
>>>>> use up 
>>>>> a lot more bandwidth if you are doing lots of tests.
>>>>> 
>>>>> Jeremy Palmer
>>>>> Senior Network Engineer
>>>>> ViaWest, Inc.
>>>>> Office: 720.891.1045
>>>>> Fax: 303-874-5236
>>>>> http://www.viawest.com
>>>>> 
>>>>> On 6/11/2014 10:21 AM, Kathy Benninger wrote:
>>>>>> I want to move the XSEDE perfSONARs from manually configured testing
>>>>>> to mesh
>>>>>> config based testing and have come up with a few questions/issues
>>>>>> while
>>>>>> following the instructions at
>>>>>> 
>>>>>> https://code.google.com/p/perfsonar-ps/wiki/MeshConfigurationInstallat
>>>>>> ion 
>>>>>> 
>>>>>> /opt/perfsonar_ps/mesh_config/doc/
>>>>>> as installed doesn't seem to include example.conf , but I was able
>>>>>> to grab 
>>>>> the
>>>>>> file via the online link.
>>>>>> 
>>>>>> I can't find build_json
>>>>>> 
>>>>>> Do I need to disable, delete, or otherwise clean out existing manual
>>>>>> testing 
>>> 
>>>>> and
>>>>>> its associated databases before trying to start the mesh config
>>>>>> invoked
>>>>>> testing?  The mesh config tests will be duplicating what was
>>>>>> previously manu
>>>>>> 
>>>>>> Is there a way to store results both on the individual MP/MA hosts
>>>>>> and a 
>>>>> central
>>>>>> server (e.g., 2 write lines in the config)?
>>>>>> 
>>>>>> Thanks!
>>>>>> Kathy
>>>>>> 
>>>>> 
>>>>> -- 
>>>>> This message is subject to the CSIR's copyright terms and conditions,
>>>>> e-mail 
>>>>> legal notice, and implemented Open Document Format (ODF) standard.
>>>>> The full disclaimer details can be found at
>>>>> http://www.csir.co.za/disclaimer.html.
>>>>> 
>>>>> This message has been scanned for viruses and dangerous content by
>>>>> MailScanner, 
>>>>> and is believed to be clean.
>>>>> 
>>>>> Please consider the environment before printing this email.
>>>> 
>>>> -- 
>>>> This message is subject to the CSIR's copyright terms and conditions,
>>>> e-mail 
>>> legal notice, and implemented Open Document Format (ODF) standard.
>>>> The full disclaimer details can be found at
>>> http://www.csir.co.za/disclaimer.html.
>>>> 
>>>> This message has been scanned for viruses and dangerous content by
>>> MailScanner, 
>>>> and is believed to be clean.
>>>> 
>>>> Please consider the environment before printing this email.
>>>> 
>>>> 
>>>> <generate_configuration.out><owmesh1.conf><owmesh2.conf><test1.conf><tes
>>>> t2.conf>
>>> 
>>> 
>>> -- 
>>> This message is subject to the CSIR's copyright terms and conditions,
>>> e-mail 
>>> legal notice, and implemented Open Document Format (ODF) standard.
>>> The full disclaimer details can be found at
>>> http://www.csir.co.za/disclaimer.html.
>>> 
>>> This message has been scanned for viruses and dangerous content by
>>> MailScanner, 
>>> and is believed to be clean.
>>> 
>>> Please consider the environment before printing this email.
>> 
>> -- 
>> This message is subject to the CSIR's copyright terms and conditions,
>> e-mail legal notice, and implemented Open Document Format (ODF) standard.
>> The full disclaimer details can be found at
>> http://www.csir.co.za/disclaimer.html.
>> 
>> This message has been scanned for viruses and dangerous content by
>> MailScanner, 
>> and is believed to be clean.
>> 
>> Please consider the environment before printing this email.
>> 
> 
> <not working ps-toolkit mesh gen_config output><working non-toolkit sever 
> mesh gen_config output>