perfSONAR User Q&A and Other Discussion

[Amit Khare <>]

Hi Aaron,

Sorry for the delayed reply, please find the attached .xml and .conf files as per your email. This host is configured to use mesh url http://otwa3pfs1.canarie.ca/mesh.json.Thanks,

Amit 

----------------------------------------------------------------------------

Amit Khare | Network Engineer | CANARIE Inc | 45 O'Connor St., Suite 500, Ottawa, ON K1P 1A4 |  Office: 613-943-5377│Cell: 613-404-8696│CANARIE NOC: 613-944-5612│www.canarie.ca

From: Aaron Brown <>
Date: Friday, August 1, 2014 at 11:59 AM
To: Amit Khare <>
Cc: "" <>, Andrew Lake <>
Subject: Re: [perfsonar-user] perfSONAR Toolkit 3.4rc2 available for testing

Hey Amit,

Actually, it looks like the ‘description’ field had a newline in it, and that was causing the problem. However, I think it got the newline because the regular testing config file parser wasn’t parsing “#” correctly when it was escaped like in "Traceroute Test Between All OSG Latency Hosts\#vo_all/traceroute”. It had the description as "Traceroute Test Between All OSG Latency Hosts\” which Config::General output as that multiline thing. I’ve got a fix for that specific issue, though given that that test seems to have been added by the mesh config, I’m not sure why it was upgraded. Those files I listed below would be helpful for discovering the latter problem :)

Cheers,

Aaron

On Aug 1, 2014, at 11:09 AM, Aaron Brown <> wrote:

Hey Amit,

There should be files named like:

/opt/perfsonar_ps/PingER/etc/pinger-landmarks.xml-1

/opt/perfsonar_ps/perfsonarbuoy_ma/etc/owmesh.conf-1

or similar (may have higher numbers). Could you send those to us?

Also, what mesh(es) is this host configured to use?

Cheers,

Aaron

On Aug 1, 2014, at 10:58 AM, Andrew Lake <> wrote:

Hi Amit,

Adding the perfsonar-user list to CC so other developers and other 3.4rc2 testers can see this…

Looks like somehow some multi-line EOF blocks got added to your regular_testing file. See lines 104 and 172 for an example:

Line 104:

    description   <<EOF

    Traceroute Test Between All OSG Latency Hosts\

    EOF

Line 172:

description   <<EOF

    OWAMP Test Between All OSG Latency Hosts\

    EOF

Not sure what would cause this but i am guessing a bug in the upgrade script. You can manually fix by manually editing the file and getting rid of "<<EOF" and "EOF" and making the description one line:

Line 104:

    description  Traceroute Test Between All OSG Latency Hosts

Line 172:

description  OWAMP Test Between All OSG Latency Hosts

I'll create an issue for this and we'll try to get it fixed by the next RC. Thanks for reporting this!

Andy

On Jul 31, 2014, at 4:55 PM, Amit Khare <> wrote:

Hi Andrew,

Please find the attached regular_testing.conf. Thanks a lot for all your help.

Amit 

----------------------------------------------------------------------------

Amit Khare | Network Engineer | CANARIE Inc | 45 O'Connor St., Suite 500, Ottawa, ON K1P 1A4 |  Office: 613-943-5377│Cell: 613-404-8696│CANARIE NOC: 613-944-5612│www.canarie.ca

From: Andrew Lake <>
Date: Thursday, July 31, 2014 at 4:48 PM
To: Amit Khare <>
Subject: Re: [perfsonar-user] perfSONAR Toolkit 3.4rc2 available for testing

Hi,

Can you send your /opt/perfsonar_ps/regular_testing/etc/regular_testing.conf file? 

Thanks,

Andy

On Jul 31, 2014, at 4:43 PM, Amit Khare <> wrote:

Hi Andrew,

I upgraded our test perfSONAR box to 3.4rc2 using yum. All the latency tests got transferred over to the new instance and had been running for a week. All of sudden, tests disappeared from the psGraph.cgi page and now the when I got the regulat_testing page the following  error is showing up. 

(https://otwa3pfs1.canarie.ca/toolkit/admin/regular_testing/)

Scheduled Tests Configuration Tool

Problem reading testing configuration: Problem reading Regular Testing Configuration: Problem reading Regular Testing configurat: Can't use string ("Line 104 malformed") as a HASH ref while "strict refs" in use at /opt/perfsonar_ps/toolkit/web/root/admin/regular_testing/../../../../lib/perfSONAR_PS/RegularTesting/Utils/SerializableObject.pm line 110.

The on-disk configuration has changed. Any changes you made have been lost.

Do you know if anyone else had a similar issue. 

Thanks,

Amit 

----------------------------------------------------------------------------

Amit Khare | Network Engineer | CANARIE Inc | 45 O'Connor St., Suite 500, Ottawa, ON K1P 1A4 |  Office: 613-943-5377│Cell: 613-404-8696│CANARIE NOC: 613-944-5612│www.canarie.ca

From: Andrew Lake <>
Date: Monday, July 21, 2014 at 12:14 PM
To: "" <>, perfsonar-announce <>
Cc: "" <>
Subject: [perfsonar-user] perfSONAR Toolkit 3.4rc2 available for testing

All, 

A new release candidate is available for the next major version of the perfSONAR Toolkit. This release has a major overhaul of several components, and we need your help to test these components. This version of the toolkit is still in the beta phase and SHOULD NOT be placed on production measurement hosts. Production hosts should continue to use 3.3.2. We are making 3.4rc2  available to those with spare hosts that they can use for testing with a goal of a final release in the coming months. 

This version contains many new features, notably a brand new Measurement Archive that provides huge performance gains on queries (especially for large data sets like owamp data), replaces the existing SOAP interface with a new REST interface and stores new data types (such as tcp retransmits for iperf3 and MTU for tracepath results). The new MA also keeps 5 min, 1 hour, and 1 day summary data by default allowing you to quickly view historical data. You will also notice new graphs that display some of these new data points where applicable and show latency, loss, and throughput on the same graphs for nodes running both types of tests. For a full list of features and changes see the release notes: http://psps.perfsonar.net/toolkit/releasenotes/pspt-3_4rc2.html

Currently 3.4rc2 is only available to NetInstall users of the toolkit. ISO images for fresh installs can be found here: http://software.internet2.edu/pSPT3.4rc2/

For users testing an upgrade from the 3.4rc1, you may run "yum update" and then reboot your host. 

For users testing an upgrade from the 3.3.x line, you can upgrade with the following steps:

1. Download http://software.internet2.edu/branches/release-3.4/rpms/el6/x86_64/RPMS.main/Internet2-repo-0.5-2.noarch.rpm (NOTE: this will work for both i386 and x86_64)

2. Run "yum localinstall Internet2-repo-0.5-2.noarch.rpm". This will point your server at the 3.4 testing repository

3. Run "yum clean all"

4. Run "yum update"

5. Reboot your host

These should be the only steps for upgraders. On upgraded hosts, a daemon will run in the background that will automatically move over your tests from the old MA to the new MA. Depending on the amount of data on your server this can range from a several second process to several hours, but eventually it should appear in the new graphs. You can monitor the progress of this script by looking in /var/log/perfsonar/psb_to_esmond.log.

It would be appreciated if all testers take a few moments to run through our testing checklist found here http://code.google.com/p/perfsonar-ps/wiki/PerformanceToolkitTestingChecklist.

Thanks ahead of time to everyone that is able to test. The more issues we found now, the better the final release will be so all help is greatly appreciated. Please email any questions or issues you find to . 

Thank you,

The perfSONAR Development Team

<regular_testing.conf>