perfSONAR User Q&A and Other Discussion

["Maciolek, Mark" <>]

Hi,

 

A few python exceptions and my other server did not complain about “isn’t signed with proper key”

Apr 19 13:54:37 perf-rcc bwctld[9278]: FILE=sapi.c, LINE=391, BWLControlAccept(): Unable to read ClientGreeting message

Apr 19 13:54:41 perf-rcc bwctld[9306]: FILE=sapi.c, LINE=391, BWLControlAccept(): Unable to read ClientGreeting message

Apr 19 13:58:41 perf-rcc fail2ban.filter[3494]: INFO [sshd] Found 122.224.40.84

Apr 19 14:14:28 perf-rcc bwctld[9969]: FILE=sapi.c, LINE=987, BWLStopSessionWait: Control socket closed: No child processes

Apr 19 14:14:28 perf-rcc bwctld[9969]: FILE=bwctld.c, LINE=1472, Control session terminated abnormally...

Apr 19 14:16:35 perf-rcc bwctld[10021]: FILE=sapi.c, LINE=987, BWLStopSessionWait: Control socket closed: No child processes

Apr 19 14:16:35 perf-rcc bwctld[10021]: FILE=bwctld.c, LINE=1472, Control session terminated abnormally...

Apr 19 14:23:12 perf-rcc abrt: detected unhandled Python exception in '/usr/libexec/pscheduler/daemons/ticker'

Apr 19 14:23:12 perf-rcc abrt-server[10335]: Saved Python crash dump of pid 3518 to /var/spool/abrt/pyhook-2017-04-19-14:23:12-3518

Apr 19 14:23:12 perf-rcc abrtd: Directory 'pyhook-2017-04-19-14:23:12-3518' creation detected

Apr 19 14:23:12 perf-rcc abrtd: Package 'pscheduler-server' isn't signed with proper key

Apr 19 14:23:12 perf-rcc abrtd: 'post-create' on '/var/spool/abrt/pyhook-2017-04-19-14:23:12-3518' exited with 1

Apr 19 14:23:12 perf-rcc abrtd: Deleting problem directory '/var/spool/abrt/pyhook-2017-04-19-14:23:12-3518'

Apr 19 14:23:15 perf-rcc runner INFO     Exiting on signal 15

Apr 19 14:23:15 perf-rcc abrt: detected unhandled Python exception in '/usr/libexec/pscheduler/daemons/runner'

Apr 19 14:23:15 perf-rcc abrt-server[10355]: Saved Python crash dump of pid 3522 to /var/spool/abrt/pyhook-2017-04-19-14:23:15-3522

Apr 19 14:23:15 perf-rcc abrtd: Directory 'pyhook-2017-04-19-14:23:15-3522' creation detected

Apr 19 14:23:15 perf-rcc abrtd: Package 'pscheduler-server' isn't signed with proper key

Apr 19 14:23:15 perf-rcc abrtd: 'post-create' on '/var/spool/abrt/pyhook-2017-04-19-14:23:15-3522' exited with 1

Apr 19 14:23:15 perf-rcc abrtd: Deleting problem directory '/var/spool/abrt/pyhook-2017-04-19-14:23:15-3522'

Apr 19 14:23:18 perf-rcc archiver INFO     Exiting on signal 15

Apr 19 14:23:18 perf-rcc abrt: detected unhandled Python exception in '/usr/libexec/pscheduler/daemons/archiver'

Apr 19 14:23:18 perf-rcc abrt-server[10379]: Saved Python crash dump of pid 3519 to /var/spool/abrt/pyhook-2017-04-19-14:23:18-3519

Apr 19 14:23:18 perf-rcc abrtd: Directory 'pyhook-2017-04-19-14:23:18-3519' creation detected

Apr 19 14:23:18 perf-rcc abrtd: Package 'pscheduler-server' isn't signed with proper key

Apr 19 14:23:18 perf-rcc abrtd: 'post-create' on '/var/spool/abrt/pyhook-2017-04-19-14:23:18-3519' exited with 1

Apr 19 14:23:18 perf-rcc abrtd: Deleting problem directory '/var/spool/abrt/pyhook-2017-04-19-14:23:18-3519'

Apr 19 14:23:21 perf-rcc abrt: detected unhandled Python exception in '/usr/libexec/pscheduler/daemons/scheduler'

Apr 19 14:23:21 perf-rcc abrt-server[10398]: Saved Python crash dump of pid 3520 to /var/spool/abrt/pyhook-2017-04-19-14:23:21-3520

Apr 19 14:23:21 perf-rcc abrtd: Directory 'pyhook-2017-04-19-14:23:21-3520' creation detected

Apr 19 14:23:21 perf-rcc abrtd: Package 'pscheduler-server' isn't signed with proper key

Apr 19 14:23:21 perf-rcc abrtd: 'post-create' on '/var/spool/abrt/pyhook-2017-04-19-14:23:21-3520' exited with 1

Apr 19 14:23:21 perf-rcc abrtd: Deleting problem directory '/var/spool/abrt/pyhook-2017-04-19-14:23:21-3520'

Apr 19 14:23:28 perf-rcc runner INFO     Started

Apr 19 14:23:28 perf-rcc archiver INFO     Started

Apr 19 14:23:28 perf-rcc scheduler INFO     Started

Apr 19 14:23:31 perf-rcc yum[10274]: Installed: pscheduler-server-1.0-1.el6.noarch

Apr 19 14:23:34 perf-rcc pscheduler-api INFO     Started

Apr 19 14:23:34 perf-rcc pscheduler-api CRITICAL Failed to load limit file /etc/pscheduler/limits.conf:

Apr 19 14:24:27 perf-rcc bwctld[11431]: FILE=sapi.c, LINE=391, BWLControlAccept(): Unable to read ClientGreeting message

Apr 19 14:24:28 perf-rcc bwctld[11458]: FILE=sapi.c, LINE=391, BWLControlAccept(): Unable to read ClientGreeting message

Apr 19 14:24:29 perf-rcc bwctld[11481]: FILE=sapi.c, LINE=391, BWLControlAccept(): Unable to read ClientGreeting message

Apr 19 14:24:30 perf-rcc bwctld[11515]: FILE=sapi.c, LINE=391, BWLControlAccept(): Unable to read ClientGreeting message

Apr 19 14:24:31 perf-rcc bwctld[11542]: FILE=sapi.c, LINE=391, BWLControlAccept(): Unable to read ClientGreeting message

Apr 19 14:24:31 perf-rcc bwctld[10315]: FILE=sapi.c, LINE=987, BWLStopSessionWait: Control socket closed: No child processes

Apr 19 14:24:31 perf-rcc bwctld[10315]: FILE=bwctld.c, LINE=1472, Control session terminated abnormally...

Apr 19 14:24:32 perf-rcc bwctld[11569]: FILE=sapi.c, LINE=391, BWLControlAccept(): Unable to read ClientGreeting message

Apr 19 14:24:33 perf-rcc bwctld[11590]: FILE=sapi.c, LINE=391, BWLControlAccept(): Unable to read ClientGreeting message

Apr 19 14:24:34 perf-rcc bwctld[11624]: FILE=sapi.c, LINE=391, BWLControlAccept(): Unable to read ClientGreeting message

 

 

My working perfsonar box has Scientific Linux release 6.9 (Carbon)

The one having issues has CentOS release 6.9 (Final)

 

 

Mark

 

--Mark Maciolek

Network Administrator

Morse Hall Rm 338

http://www.unh.edu/research/support-units/research-computing-center

 

 

 

 

From: Brian Tierney [mailto:]
Sent: Wednesday, April 19, 2017 2:22 PM
To: Maciolek, Mark <>
Cc: Andrew Lake <>;
Subject: Re: [perfsonar-user] update to 4.0

 

 

That is odd.

 

Can you try this, and see if that fixes it:

 

yum reinstall pscheduler-server

On Wednesday, April 19, 2017, Maciolek, Mark <> wrote:

Hi,

 

2017/04/19 13:07:38 (9225) WARN> perfsonar_meshconfig_agent:430 main:: - Problem adding test, continuing with rest of config: 422 UNPROCESSABLE ENTITY: Unable to complete request: Limit processor is not initialized: Limit processor is not initialized.  See system logs.

2017/04/19 13:07:38 (9225) WARN> perfsonar_meshconfig_agent:430 main:: - Problem adding test, continuing with rest of config: 500 INTERNAL SERVER ERROR: Error while tasking 132.177.100.142: Unable to post task to 132.177.100.142: Unable to complete request: Limit processor is not initialized: Limit processor is not initialized.  See system logs.

 

 

pscheduler-api CRITICAL Failed to load limit file /etc/pscheduler/limits.conf

 

The limits.conf file is there and it’s the same size as the file on my second perfsonar box where the tests are running.

6150 Apr 18 04:40 limits.conf

 

https://perf-rcc.unh.edu/toolkit/

 

Funny thing my other perfsonar box updated yesterday and its tests are working

 

Mark

 

--Mark Maciolek

Network Administrator

Morse Hall Rm 338

http://www.unh.edu/research/support-units/research-computing-center

 

From: Andrew Lake [mailto:]
Sent: Wednesday, April 19, 2017 1:22 PM
To: Maciolek, Mark <>;
Subject: Re: [perfsonar-user] update to 4.0

 

Hi,

 

What is in /var/log/pscheduler/pscheduler.log and /var/log/perfsonar/meshconfig-agent.log? Also,does the host have a public address so I can look at the web frontend?

 

Thanks,

Andy

 

 

 

On April 19, 2017 at 11:57:59 AM, Mark Maciolek () wrote:

Hi,

I let the system auto update to version 4 yesterday. I rebooted the server
yesterday. Since the reboot the tests have not run.

This is from the logs:

perf-rcc bwctld[13065]: FILE=sapi.c, LINE=391, BWLControlAccept(): Unable to
read ClientGreeting message

Do I have to remake all the tests?

Mark

--
Brian Tierney, http://www.es.net/tierney
Energy Sciences Network (ESnet), Berkeley National Lab
http://fasterdata.es.net