perfSONAR User Q&A and Other Discussion

[Jim Nauer CWRU <>]

I am not seeing the HTTP 500 error, nor did I see the postgres "No old data to migrate" error that Pedro Reis saw in the yum history.

Our system is a vanilla PS-on-CentOS 6 setup (installed from the PS net install image), upgraded from 3.4->3.5 last year, and 3.5->4.0 today.

All test results are showing "Error loading data / Error retrieveing data / No data found in the measurement archive" under the "Details" link. Tests that included 'traceroute' results show results for up to about 4:10am today (~25 mins. after the yum updates completed - not sure about timing relative to the PS upgrade scripts, though)

On Tue, Apr 18, 2017 at 1:04 PM, Andrew Lake <> wrote:

Hi Pedro,

That makes sense, if postgresql is in a non-standard spot it won't find it. You should be able to run the scripts by hand to migrate it. I’ll send you a note off-list since it’ll probably involve details specific to your setup, and I don’t want anyone stumbling onto the commands out of context and breaking their host :)

Thanks,

Andy

On April 18, 2017 at 12:17:59 PM, Pedro Reis () wrote:

Hello Andy, All

Regarding the "Server Error (500)" of the Central MA I think I might
found the reason for it in the yum update history.

"Scriptlet output:
1 No old data to migrate in /var/lib/pgsql/data, init
/var/lib/pgsql/9.5/data."

Well the main problem is that the data was in a different directory
"/store/pgsql/data" and since the script didn't find it it just created
a new one in "/var/lib/pgsql/9.5/data"

Can you tell me how to migrate the data to the new postgresql?
And should both new and old versions of postgresql be running or only
the new one?

Thanks in adanced!


Com meus melhores cumprimentos | Best Regards
Pedro Reis
Área de Serviços de Rede | Network Services Area
FCT|FCCN
Av. do Brasil, n.º 101
1700-066 Lisboa - Portugal
Telefone|Phone +351 218 440 100; Fax +351 218 472 167
www.fccn.pt

On 2017/04/18 16:14, Pedro Reis wrote:
> Hello Andy,
>
> Well let me just explain the set up a bit.
>
> I have 2 main toolkit installations that run tests to ± 30 testpoint
> machines. The results of the tests are then forwarded from the toolkits
> to a central MA. The hosts are all CentOS 6. All tests manually configured.
>
> At the moment, and after the upgrade my MA is giving a <h1>Server Error
> (500)</h1>, still trying to solve that one.
>
> On one of the toolkits (let's call it TK1) I can see the tests results
> in the webgui but when click the "Details" link under it takes me to the
> graph page but says
> "Error retrieving data
> No data found in the measurement archive"
>
> Now the interesting thing is that I'm accessing it via HTTPS, but if I
> change it to HTTP it already shows me the graph with the current data
> being retrieved.
>
>
> In the other toolkit (TK2) even though tests are configured there is no
> test results in the webgui and if you curl to it, it just replies with
> an empty set:
> curl https://<TK2>/esmond/perfsonar/archive/
> []
>
>
> On both hosts I'm seeing logs like the ones attached, probably due to
> the 500 error of the MA.
>
>
> Another situation, on booth toolkits I'm getting the following error
> when trying to run pscheduler monitor
> Traceback (most recent call last):
> File "/usr/libexec/pscheduler/commands/monitor", line 262, in <module>
> curses.curs_set(0)
> _curses.error: curs_set() returned ERR
>
>
> So.... there are a bunch of things I'm trying to tackle at the same
> time, but getting a bit confused... :S
>
> If you need any further logs let me know!
>
> Com meus melhores cumprimentos | Best Regards
> Pedro Reis
> Área de Serviços de Rede | Network Services Area
> FCT|FCCN
> Av. do Brasil, n.º 101
> 1700-066 Lisboa - Portugal
> Telefone|Phone +351 218 440 100; Fax +351 218 472 167
> www.fccn.pt
>
> On 2017/04/18 13:49, Andrew Lake wrote:
>> Hi Pedro,
>>
>> Is this just an upgrade of a CentOS 6 host? If that’s the case your
>> certs do not need to be migrated to httpd24, httpd still runs and is
>> what the world uses to access the UI, esmond, pscheduler, etc. The
>> httpd24 you see running only listens on localhost and is just there to
>> run esmond to get around some weird stuff we previously had to do with
>> python27 and mod_wsgi on CentOS 6.
>>
>> Are you not seeing any of your old test results from before the upgrade?
>> or is it just the new ones that are missing?
>>
>> Thanks,
>> Andy
>>
>> On April 18, 2017 at 7:15:11 AM, Pedro Reis (
>> <mailto:>) wrote:
>>
>>> Hello all,
>>>
>>> Looks like unfortunately I wasn't blessed with a smooth upgrade :(
>>>
>>> It seams that there might be a problem with the migration of SSL certs
>>> from httpd to httpd24 and in one of the toolkits no results appear,
>>> although there are about 30 tests scheduled and enabled.
>>>
>>> I'm still investigating further, and I'll try to give some updates later.
>>>
>>>
>>> Com meus melhores cumprimentos | Best Regards
>>> Pedro Reis
>>> Área de Serviços de Rede | Network Services Area
>>> FCT|FCCN
>>> Av. do Brasil, n.º 101
>>> 1700-066 Lisboa - Portugal
>>> Telefone|Phone +351 218 440 100; Fax +351 218 472 167
>>> www.fccn.pt <http://www.fccn.pt>
>>>
>>> On 2017/04/17 19:15, Andrew Lake wrote:
>>>> All,
>>>>
>>>> perfSONAR 4.0 is now available! perfSONAR 4.0 is a significant update to
>>>> the perfSONAR software and brings about many changes we hope will
>>>> strengthen the foundation on which the project is built going forward.
>>>> Thank you to all those in the community that helped test and provide
>>>> feedback over the past few months. We would also like to thank everyone
>>>> for their patience. This release incorporates possibly the largest
>>>> single change in perfSONAR’s 15-year history, and we spent considerable
>>>> time making sure it was right. Please see the rest of this email for
>>>> details on changes, getting the update and other important information.
>>>>
>>>> ============================================
>>>> Changes since 3.5.1
>>>> ============================================
>>>> Highlights include:
>>>>
>>>> - New scheduling software called pScheduler that provides increased
>>>> visibility, extensibility and control in the measurement process.
>>>> - CentOS 7 support
>>>> - Debian perfsonar-toolkit bundle support
>>>> - Updated graphs
>>>> - Support for email alerting in MaDDash
>>>>
>>>> For a more complete list of changes, see the full release notes at:
>>>> http://www.perfsonar.net/release-notes/version-4-0/
>>>>
>>>> ==============================================
>>>> Getting 4.0
>>>> ==============================================
>>>> As stated in our reminders over the last two weeks, if you are running
>>>> auto-updates you should get perfSONAR 4.0 automatically. For others see
>>>> http://docs.perfsonar.net/install_getting.html. Also, please review our
>>>> hardware requirements, in particular with respect to CPU, as there have
>>>> been some changes, specifically for those hosts planning to run and
>>>> store a large number of tests:
>>>> http://docs.perfsonar.net/install_hardware.html.
>>>>
>>>> ============================================
>>>> BWCTL Backward Compatibility
>>>> ============================================
>>>> pScheduler is a completely new codebase and protocol. It does NOT speak
>>>> the BWCTL protocol natively but it does provide a backward compatibility
>>>> strategy with BWCTL. In general you should not have to take any extra
>>>> steps to continue testing to sites running BWCTL and vice versa. For
>>>> more information see the document at
>>>> http://docs.perfsonar.net/pscheduler_intro.html#bwctl-backward-compatibility.
>>>
>>>>
>>>> Also, we HIGHLY recommend you start using the new pscheduler command
>>>> instead of the bwctl for all command-line tests since doing so has the
>>>> potential to create scheduling conflicts. See
>>>> http://docs.perfsonar.net/pscheduler_ref_bwctl.html for a guide on how
>>>> to convert BWCTL commands to pScheduler.
>>>>
>>>> ============================================
>>>> Important Upgrade Notes
>>>> ============================================
>>>> - All your data and tests should be preserved during the upgrade of an
>>>> existing 3.5 host. If for whatever reason this does not happen, please
>>>> contact our user list if you need assistance.
>>>>
>>>> - Debian/Ubuntu users of perfSONAR 3.5.1 will not get this update
>>>> automatically. See our Debian upgrade guide for information on updating
>>>> your host:
>>>> http://docs.perfsonar.net/install_debian.html#upgrading-from-3-5-1
>>>>
>>>> - For CentOS users running auto-updates, it may take up to 72 hours for
>>>> your system to get the release depending on when mirrors sync and
>>>> yum-cron runs.
>>>>
>>>> - You may want to reboot your host after update. While not strictly
>>>> required, this may help with occasional issues we have see with httpd
>>>> processes not restarting properly and similar
>>>>
>>>> - If you were pointing at the staging repository on a production host,
>>>> you should remove it now. You can do so by running "yum remove
>>>> Internet2-repo-staging"
>>>>
>>>> - perfSONAR 4.0 requires more CPU than previous versions as a result of
>>>> some of the increased functionality. Multiple warning about this were
>>>> shared with auto-updaters prior to the release. See our hardware
>>>> requirements page for more information:
>>>> http://docs.perfsonar.net/install_hardware.html
>>>>
>>>> - Web100 kernels are not provided for CentOS 7. The plan is to continue
>>>> to provide Web100 kernels for CentOS 6 until October 17, 2017.
>>>>
>>>> - NDT and NPAD are no longer installed for new install since they
>>>> require Web100. They will NOT be removed from existing systems
>>>> automatically, but require a web100 kernel to operate so will either
>>>> require an outdated kernel or cease to function after October 17, 2017
>>>>
>>>> - The perfsonar-regular testing package is gone in perfSONAR 4.0. This
>>>> means your /etc/perfsonar/regulartesting.conf file has been moved to
>>>> /etc/perfsonar/meshconfig-agent-tasks.conf. It also means you no longer
>>>> have a perfsonar-regulartesting daemon running
>>>>
>>>> - The /usr/lib/perfsonar/bin/generate_configuration script and
>>>> associated cron are gone and now handled by the new
>>>> perfsonar-meshconfig-agent daemon
>>>>
>>>> - The /usr/lib/perfsonar/bin/generate_gui_configuration script and
>>>> associated cron are gone and now handled by the new
>>>> perfsonar-meshconfig-guiagent daemon. This daemon will now create
>>>> traceroute and ping dashboard by default if you have the tests defined.
>>>>
>>>> ======================================
>>>> More Information
>>>> ======================================
>>>> Don't forget we have a webinar on perfSONAR 4.0 scheduled for Thursday
>>>> April 20th. For past talks on perfSONAR 4.0 given at the 2016
>>>> Technology Exchange see below:
>>>>
>>>> http://meetings.internet2.edu/2016-technology-exchange/detail/10004309/
>>>> http://meetings.internet2.edu/2016-technology-exchange/detail/10004491/
>>>> http://meetings.internet2.edu/2016-technology-exchange/detail/10004321/
>>>>
>>>> Thank you,
>>>> The perfSONAR Development Team
>>>>
>>>>
>>>>
>>>

--

James A. Nauer                   | "I shall not yield one whit of  maturity,
Engineer III, Enterprise Systems | not grace, not respectability, to  the
University Technology, [U]Tech   | passing of time. I declare that I  shall
Case Western Reserve University  | forever be, if not a child,  certainly
(216) 368-MACS  (368-6227)       | childish"  --Kennet Shardik
USPA D-25604