perfSONAR User Q&A and Other Discussion

[Winnie Lacesso <>]

Good afternoon,

Now having started the PingER service, its logs seem full of error:
root@lcgnetmon>
 tail /var/log/perfsonar/pinger.log
2015/02/03 15:03:24 (30993) ERROR> Remote.pm:373 
perfSONAR_PS::Client::LS::Remote::getKey - Key not found.
2015/02/03 15:03:24 (30993) ERROR> Remote.pm:692 
perfSONAR_PS::Client::LS::Remote::sendKeepalive - Key value not supplied.
2015/02/03 15:03:56 (30992) INFO> PingER.pm:287 
perfSONAR_PS::Services::MP::PingER::registerLS - Registering PingER MP with LS
2015/02/03 15:03:56 (30992) INFO> PingER.pm:287 
perfSONAR_PS::Services::MP::PingER::registerLS - Registering PingER MP with LS
2015/02/03 15:03:56 (30992) ERROR> Remote.pm:373 
perfSONAR_PS::Client::LS::Remote::getKey - Key not found.
2015/02/03 15:03:56 (30992) ERROR> Remote.pm:692 
perfSONAR_PS::Client::LS::Remote::sendKeepalive - Key value not supplied.
2015/02/03 15:04:24 (30993) INFO> PingER.pm:300 
perfSONAR_PS::Services::MA::PingER::registerLS - Registering PingER MA with LS
2015/02/03 15:04:24 (30993) INFO> PingER.pm:300 
perfSONAR_PS::Services::MA::PingER::registerLS - Registering PingER MA with LS
2015/02/03 15:04:25 (30993) ERROR> Remote.pm:373 
perfSONAR_PS::Client::LS::Remote::getKey - Key not found.
2015/02/03 15:04:25 (30993) ERROR> Remote.pm:692 
perfSONAR_PS::Client::LS::Remote::sendKeepalive - Key value not supplied.

What is this Key it's wanting, whence should it be got?

Also yesterday these services were started in the order that rc3.d would 
start them (in a little "for" loop)
traceroute_ma
traceroute_master
traceroute_ondemand_mp
traceroute_scheduler

No obvious errors at that time: (lost output for traceroute_ma startup)
on traceroute_master
chkconfig traceroute_master on
service traceroute_master start
/opt/perfsonar_ps/traceroute_ma/bin/traceroute_master.pl 
--config=/opt/perfsonar_ps/traceroute_ma/etc/traceroute-master.conf 
--pidfile=/var/run/traceroute-master.pid 
--logger=/opt/perfsonar_ps/traceroute_ma/etc/traceroute-master_logger.conf 
--user=perfsonar --group=perfsonar
/etc/init.d/traceroute_master start: Traceroute Master started

on traceroute_ondemand_mp
chkconfig traceroute_ondemand_mp on
service traceroute_ondemand_mp start
/opt/perfsonar_ps/traceroute_ma/bin/daemon.pl 
--config=/opt/perfsonar_ps/traceroute_ma/etc/ondemand_mp-daemon.conf 
--pidfile=traceroute_ondemand_mp.pid --piddir=/var/run 
--logger=/opt/perfsonar_ps/traceroute_ma/etc/ondemand_mp-daemon_logger.conf 
--user=perfsonar --group=perfsonar
/etc/init.d/traceroute_ondemand_mp start: perfSONAR Traceroute On-Demand MP 
Service started

on traceroute_scheduler
chkconfig traceroute_scheduler on
service traceroute_scheduler start
/opt/perfsonar_ps/traceroute_ma/bin/traceroute_scheduler.pl 
--owmesh=/opt/perfsonar_ps/traceroute_ma/etc 
--pidfile=/var/run/traceroute-scheduler.pid 
--logger=/opt/perfsonar_ps/traceroute_ma/etc/traceroute-scheduler_logger.conf 
--user=perfsonar --group=perfsonar
/etc/init.d/traceroute_scheduler start: Traceroute Scheduler started

but the logs are full of errors:

root@lcgnetmon>
 tail -5 /var/log/perfsonar/traceroute_ondemand_mp.log
(timestamped today)
2015/02/03 15:03:15 (11289) DEBUG> daemon.pl:569 main::psService - Accept 
returned nothing, likely a timeout occurred or a child exited
2015/02/03 15:03:35 (11289) DEBUG> daemon.pl:569 main::psService - Accept 
returned nothing, likely a timeout occurred or a child exited
2015/02/03 15:03:55 (11289) DEBUG> daemon.pl:569 main::psService - Accept 
returned nothing, likely a timeout occurred or a child exited
2015/02/03 15:04:15 (11289) DEBUG> daemon.pl:569 main::psService - Accept 
returned nothing, likely a timeout occurred or a child exited
2015/02/03 15:04:35 (11289) DEBUG> daemon.pl:569 main::psService - Accept 
returned nothing, likely a timeout occurred or a child exited

But, our working Bandwidth box is logging exactly the same so all that noise
must be "normal noise"?! (SIGH, wish for _real_ info/data in logs?!)

Also from yesterday startup, errors in logs:
     
in traceroute_ma.log

2015/02/02 15:32:43 (11237) WARN> daemon.pl:272 main:: - Enabling echo 
service for each endpoint unless specified otherwise
2015/02/02 15:32:43 (11237) INFO> Traceroute.pm:111 
perfSONAR_PS::Services::MA::Traceroute::init - Setting service access point 
to http://lcgnetmon.phy.bris.ac.uk:8086/perfSONAR_PS/services/tracerouteMA
2015/02/02 15:32:43 (11237) WARN> Traceroute.pm:126 
perfSONAR_PS::Services::MA::Traceroute::init - Setting 'service_description' 
to 'perfSONAR_PS Traceroute MA at UKI-SOUTHGRID-BRIS-HEP'.
2015/02/02 15:32:43 (11237) WARN> Traceroute.pm:133 
perfSONAR_PS::Services::MA::Traceroute::init - Setting 'service_name' to 
'Traceroute MA'.
2015/02/02 15:32:43 (11237) WARN> Traceroute.pm:140 
perfSONAR_PS::Services::MA::Traceroute::init - Setting 'service_type' to 'MA'.
2015/02/02 15:32:43 (11258) ERROR> SQL.pm:166 perfSONAR_PS::DB::SQL::openDB - 
Open error "DBI connect('traceroute_ma:localhost','perfsonar',...) failed: 
Can't connect to local MySQL server through socket 
'/var/lib/mysql/mysql.sock' (2) at 
/opt/perfsonar_ps/traceroute_ma/bin/../lib/perfSONAR_PS/DB/SQL.pm line 163
".
2015/02/02 15:32:43 (11258) ERROR> SQL.pm:218 perfSONAR_PS::DB::SQL::query - 
Query error "Can't call method "prepare" on an undefined value at 
/opt/perfsonar_ps/traceroute_ma/bin/../lib/perfSONAR_PS/DB/SQL.pm line 212.
" on statement "SELECT year, month, day FROM DATES".
2015/02/02 15:32:43 (11258) ERROR> daemon.pl:670 main::registerLS - Problem 
running register LS: Can't use string ("-1") as an ARRAY ref while "strict 
refs" in use at 
/opt/perfsonar_ps/traceroute_ma/bin/../lib/perfSONAR_PS/Services/MA/Traceroute.pm
 line 757.

(note that mysql started right AFTER that, according to the order in rc3.d)

Again, our working Bandwidth box is logging exactly the same so all that noise
must be "normal errors"?! (SIGH, wish for _real_ info/data in logs?!)

in traceroute_scheduler.log:

2015/01/26 14:12:04 (1820) ERROR> TracerouteScheduler.pm:35 
perfSONAR_PS::Services::MP::TracerouteScheduler::new - Error loading test 
config: Conf::must_get_val(ATTR=>NODE, ) undefined, called from 
/opt/perfsonar_ps/traceroute_ma/bin/../lib/perfSONAR_PS/Services/MP/TracerouteScheduler.pm:52
2015/01/26 14:52:03 (2006) ERROR> TracerouteScheduler.pm:35 
perfSONAR_PS::Services::MP::TracerouteScheduler::new - Error loading test 
config: Conf::must_get_val(ATTR=>NODE, ) undefined, called from 
/opt/perfsonar_ps/traceroute_ma/bin/../lib/perfSONAR_PS/Services/MP/TracerouteScheduler.pm:52
2015/02/02 15:32:47 (11306) ERROR> TracerouteScheduler.pm:35 
perfSONAR_PS::Services::MP::TracerouteScheduler::new - Error loading test 
config: Conf::must_get_val(ATTR=>NODE, ) undefined, called from 
/opt/perfsonar_ps/traceroute_ma/bin/../lib/perfSONAR_PS/Services/MP/TracerouteScheduler.pm:52

Again, our working Bandwidth box is logging exactly the same so all that noise
must be "normal errors"?! (SIGH, wish for _real_ info/data in logs?!)

Next to start is regular_testing & hope it doesn't implode again....