perfSONAR User Q&A and Other Discussion

[Mark Foster <>]

patch wasn't very happy with the .patch file, but I applied the diff 
contained therein to
/opt/perfsonar_ps/ls_registration_daemonlib/perfSONAR_PS/Utils/LookupService.pm

-            $logger->error("Problem retrieving $url: " . 
$http_response->code);
+            $logger->error("Problem retrieving $url: " . 
$http_response->status_line);

and that did yield a couple of additional bits of info. After one reboot, 
followed by
another a couple minutes later, the 
/var/log/perfsonar/ls_registration_daemon.log contained

2015/01/23 17:14:54 (1633) ERROR> LookupService.pm:72 
perfSONAR_PS::Utils::LookupService::discover_lookup_services - Problem 
retrieving http://ps1.es.net:8096/lookup/activehosts.json: 500 Can't connect to 
ps1.es.net:8096 (connect: No route to host)
2015/01/23 17:14:54 (1633) ERROR> daemon.pl:167 main:: - Unable to determine 
ls_instance
2015/01/23 17:35:49 (1624) ERROR> LookupService.pm:72 
perfSONAR_PS::Utils::LookupService::discover_lookup_services - Problem 
retrieving http://ps1.es.net:8096/lookup/activehosts.json: 500 Can't connect to 
ps1.es.net:8096 (Bad hostname 'ps1.es.net')
2015/01/23 17:35:49 (1624) ERROR> daemon.pl:167 main:: - Unable to determine 
ls_instance

So, it looks kind of like the network isn't fully up by the time the daemon 
is started.
Strange.

I rebooted a third time, and things ran normally, with the following two lines
the first two entries in the ls_registration_daemon.log:

2015/01/23 17:40:00 (1634) INFO> daemon.pl:170 main:: - Initial LS URL set to 
http://ps-west.es.net:8090/lookup/records
2015/01/23 17:40:00 (1634) INFO> daemon.pl:183 main:: - No LS key database 
found

I'll keep digging around to see if I can better characterize the fault 
condition(s).

-- Mark

On 1/23/2015 5:53 AM, Aaron Brown wrote:
> Hey Mark,
>
> Could you try applying this patch: 
> http://ndb1.internet2.edu/~aaron/improve_discover_lookup_service_error.patch
>
> You’ll need to download it, and then apply it by running:
>
> patch -d /opt/perfsonar_ps/ls_registration_daemon -p1 -i 
> /path/to/improve_discover_lookup_service_error.patch
>
> That will give us a better understanding of the error that you’re seeing. It 
> may be something related to network connectivity on startup.
>
> Cheers,
> Aaron
>
> > On Jan 22, 2015, at 6:03 PM, Mark Foster 
> > <
> >  
> > <mailto:>>
> >  wrote:
> >
> > It seems like with increasing frequency, the ls_registration_daemon fails to 
> > start when one of my systems is rebooted.  I can manually start the daemon, 
> > and that appears to solve the problem at least until the next reboot.
> >
> > This system is running 3.4.1.-1.psPS, as are my other systems. This 
> > netinstall system was upgraded from an earlier release, while the others were 
> > installed with 3.4 from the beginning. That's about the only difference I can 
> > find.
> >
> > I do see the following from the ls_registration_daemon.log
> >
> > 2015/01/22 14:54:14 (1623) ERROR> LookupService.pm:72 
> > perfSONAR_PS::Utils::LookupService::discover_lookup_services - Problem 
> > retrieving http://ps1.es.net:8096/lookup/activehosts.json: 500
> > 2015/01/22 14:54:14 (1623) ERROR> daemon.pl:167 main:: - Unable to determine 
> > ls_instance
> >
> > (I can wget the specific json url just fine, so it's not actual access to 
> > that file...)
> >
> > I dug around a bit in the config files and couldn't find anything obvious....
> >
> > -- Mark