perfSONAR User Q&A and Other Discussion

[Mark Feit <>]

Contardo, Gianni Carlo writes:

 

Since we upgraded to release 5.0.7 we've had issues.  We are running on EL 7.9 and things were working fine for months.  The only recent changes have been the upgrade and a reboot.  Things were fine until we rebooted and then we noticed that pscheduler shows up as "Not Running" in the toolkit web page.

…
We manually started the services and then pscheduler showed up as “Running” in the toolkit web page. Everything appears to be in the “Running” status on the toolkit web page.  Pscheduler monitor shows tasks in the Finished, Running, & Pending states …

 

There are a couple of things going on:

 

One is that 5.0.7, made changes to the Systemd service files that start the pScheduler services to resolve a problem.   The distributions we support ship with a wide variety of versions of Systemd.  EL7 brings up the rear and doesn’t support one of the changes, which causes the services not to start at boot.  I either neglected to check that or misread the documentation about when it became available, so that one’s on me.

 

The other is that we’re starting to see systems where PostgreSQL, the database the underpins pScheduler, take longer to start up than it used to.  I’m not yet sure of the exact cause; the versions of CentOS we’ve supported over the years have sometimes slipped in changes that cause problems.  Anyway, the side effect is that Systemd thinks PostgreSQL is ready when it actually isn’t, repeatedly tries to start the pScheduler services and eventually throws in the towel because they keep failing to start.  Otto’s suggestion of setting StartLimitIntervalSec and StartLimitBurst does not work on CentOS 7’s Systemd even though the documentation says it should.

 

Fixes for both are in the pipeline for 5.0.8, which we anticipate releasing the week of February 5.

 

Meanwhile, if you have a CentOS 7 system that’s affected by the reboot problem, executing these three commands as root will back out that part of the change and get pScheduler back on its feet:

 

• sed -i -e 's/^Type=.*$/Type=simple/' /usr/lib/systemd/system/pscheduler-*.service
• systemctl daemon-reload
• pscheduler internal service restart

 

This is also a good time to reiterate that the 5.0.x releases are the last ones that will support CentOS 7 or Rocky/Alma 8.  Upgrading to Alma/Rocky 9 is strongly recommended.

 

… but the MaDDash dashboards, which were previously working fine, show “Unable to find test data.”

 

That will take a bit more trouble than will work in email.  Drop me a line next week and we can have a look at it.

 

--Mark