perfSONAR User Q&A and Other Discussion

[Marian Babik <>]

Hi Mark,
coming back to this, the node is 4.0.2 on SL6. Last Friday I did the 
following:
[root@perfsonar-lt ~]# service pscheduler-runner stop
Starting pScheduler runner:                                [  OK  ]
[root@perfsonar-lt ~]# pkill -9 -f powstream
[root@perfsonar-lt ~]# service pscheduler-runner start
[root@perfsonar-lt ~]# ps -eTf | grep `id -u pscheduler` | wc -l
1644
[root@perfsonar-lt ~]# ps -eTf | grep `id -u pscheduler` | grep "powstream" | 
wc -l
379
[root@perfsonar-lt ~]# ps -eTf | grep `id -u pscheduler` | grep "runner" | wc 
-l
727

Today - so 4 days later I have:
[root@perfsonar-lt ~]# ps -eTf | grep `id -u pscheduler` | wc -l
2802
[root@perfsonar-lt ~]# ps -eTf | grep `id -u pscheduler` | grep "powstream" | 
wc -l
1113
[root@perfsonar-lt ~]# ps -eTf | grep `id -u pscheduler` | grep "runner" | wc 
-l
1033
[root@perfsonar-lt ~]# ps -eTf | grep `id -u pscheduler` | grep "\[run\]" | 
wc -l
619

The 619 [run] threads are all defunct, they show up like this:
501        774   774  9560  0 Nov29 ?        00:00:05 [run] <defunct>
etc.

I have also done reboots in the past and it didn’t help. It could be that 
this is SL6-only issue as Pete is also running SL6. 

@Pete - user id of pscheduler could be different on your nodes, so grep 501 
might not work, I have updated the commands to grep the correct id now

Cheers,
Marian

> On Dec 7, 2017, at 9:36 PM, Mark Feit 
> <>
>  wrote:
> 
> Marian Babik writes:
> 
>     …as stopping pscheduler runner leaves quite a number of powstream 
> processes around…
> 
> 4.0.2 includes some changes to the way the daemons and forked programs are 
> started and stopped that should improve that situation.  Runners started by 
> older versions won’t get swept up during upgrades because those versions 
> don’t include the clean-ups, so you may have to go in and kill off the 
> stragglers manually or just reboot.
> 
> If you continue to see that on 4.0.2 systems after a flush-out of the old 
> processes, drop me a line and we can have a look at it.
> 
> --Mark
> 
>

Attachment: smime.p7s
Description: S/MIME cryptographic signature