perfSONAR User Q&A and Other Discussion

[Andrew Lake <>]

On November 28, 2023 at 3:25:19 PM, Phil Reese () wrote:

Ok, I'm being honest and letting you know that the issue was a typo in my .json file, just two characters swapped.  Naturally when I rebuilt the system, I used the same, wrong .json file, and got the same blank grid page.  Note to self- don't make json changes at the same time as a version update.

I do think I found a couple of things that might be worth a look:

1. The maddash-server service no longer comes up running in Rocky9,  I either need to restart it or use Shawn's valid suggestion of restarting the system after an update.

I don’t think this is specific to Rocky 9. I ran through a Rocky 9 upgrade on a test host today and the restart happened as it should. This just had a small mesh, so I suspect that had something to do with my different results. I haven’t had a chance to test yet, but I suspect it’s something along the lines of the update happening while the maddash-server is actively polling hosts or similar. In the case of Shawn’s mesh, it is pretty big so maddash-server will be in that state 100% of the time, possible yours is too. There’s a bunch of threading and so maybe some bad signal handlers or something that are leaving things stuck. Just a theory, I’ll try to do some more testing. We might just update systemd to be more aggressive in killing the existing process on restart or similar. I am just speculating though which is always dangerous :) 

2. In my thrashing on the subject, I ended up in a few cases where the psconfig-maddash-agent would be stopped and just wouldn't restart. No helpful messages were in the Journalctl output.  

Did you look in /var/log/perfsonar/psconfig-maddash-agent.log? Not really much that can prevent the agent from starting beyond bad config files or similar, but should report it in the logs. 

Phil

On 11/27/23 7:23 AM, Andrew Lake wrote:

Hi,

I filed a bug on this: https://github.com/perfsonar/maddash/issues/115

We build every package every time there is an update and it gets restarted on update, sounds like something is going wrong with that process. There’s no change in the code. There are also some updates we could explore to the systemd unit file to auto-restart things when it does crash.

Thanks,

Andy