perfSONAR User Q&A and Other Discussion

[Shawn McKee <>]

Hi  Philippe,

Have  you checked  the tips  and  tools  here  http://www.usatlas.bnl.gov/twiki/bin/view/Projects/LHCperfSONAR

The  first  issue  is  recent  and  involves  logging  that  should  have  been  DEBUG  but  was  incorrectly set as INFO.   Not  sure  this  is your  issue though.

The  third  item is the  command  Dave  Lesny  found  to improve  the  responsiveness  of  the  latency graphing  (actually just  to  get it  to  work  for  longer graphs).    It  is  different  than  the  myisamchk using  mysqlcheck  -o -A  instead.

Shawn

On Tue, Mar 25, 2014 at 7:24 PM, Philippe Laurens <> wrote:

Hello,

1) Does anybody know what's making my latency node's
perfsonarbuoy_ma.log very chatty.  I am wondering if there is
some debug or verbose flag that was left on from some ancient
debugging session, but I can't find it.  The config files
I know about are
/opt/perfsonar_ps/perfsonarbuoy_ma/etc/owmesh.conf
/etc/owampd/owampd.conf
and I don't see any such flag.
"verbose" was ON in owampd.conf, but switching it off didn't help.

This is the kind of stuff I see by the boatload on this node
(psmsu01.aglt2.org), but not on another (psum01.aglt2.org) :

2014/03/25 18:23:59 (1481) INFO> owhdb.pm:339 perfSONAR_PS::DB::owhdb::fetchSummaryData - Processing DB results
2014/03/25 18:23:59 (1481) INFO> owhdb.pm:339 perfSONAR_PS::DB::owhdb::fetchSummaryData - Processing DB results
2014/03/25 18:23:59 (1481) INFO> owhdb.pm:339 perfSONAR_PS::DB::owhdb::fetchSummaryData - Processing DB results
2014/03/25 18:23:59 (1481) INFO> owhdb.pm:339 perfSONAR_PS::DB::owhdb::fetchSummaryData - Processing DB results
2014/03/25 18:23:59 (1481) INFO> owhdb.pm:339 perfSONAR_PS::DB::owhdb::fetchSummaryData - Processing DB results
2014/03/25 18:23:59 (1481) INFO> owhdb.pm:339 perfSONAR_PS::DB::owhdb::fetchSummaryData - Processing DB results
2014/03/25 18:23:59 (1481) INFO> owhdb.pm:339 perfSONAR_PS::DB::owhdb::fetchSummaryData - Processing DB results
2014/03/25 18:23:59 (1481) INFO> owhdb.pm:339 perfSONAR_PS::DB::owhdb::fetchSummaryData - Processing DB results
2014/03/25 18:23:59 (1481) INFO> owhdb.pm:339 perfSONAR_PS::DB::owhdb::fetchSummaryData - Processing DB results
2014/03/25 18:23:59 (1481) INFO> owhdb.pm:339 perfSONAR_PS::DB::owhdb::fetchSummaryData - Processing DB results
2014/03/25 18:23:59 (1481) INFO> owhdb.pm:339 perfSONAR_PS::DB::owhdb::fetchSummaryData - Processing DB results
2014/03/25 18:23:59 (1481) INFO> owhdb.pm:339 perfSONAR_PS::DB::owhdb::fetchSummaryData - Processing DB results
2014/03/25 18:23:59 (1481) INFO> owhdb.pm:339 perfSONAR_PS::DB::owhdb::fetchSummaryData - Processing DB results
2014/03/25 18:23:59 (1481) INFO> owhdb.pm:339 perfSONAR_PS::DB::owhdb::fetchSummaryData - Processing DB results
2014/03/25 18:23:59 (1481) INFO> owhdb.pm:339 perfSONAR_PS::DB::owhdb::fetchSummaryData - Processing DB results
2014/03/25 18:23:59 (1481) INFO> perfSONARBUOY.pm:3091 perfSONAR_PS::Services::MA::perfSONARBUOY::retrieveSQL - Query "" formed.
2014/03/25 18:23:59 (1481) INFO> perfSONARBUOY.pm:3015 perfSONAR_PS::Services::MA::perfSONARBUOY::retrieveSQL - DATES table query range:15482387032176590848-15482390897647157248
2014/03/25 18:23:59 (1481) INFO> owhdb.pm:74 perfSONAR_PS::DB::owhdb::getDateTableList - About to process querySELECT year, month, day
                                 FROM DATES;
2014/03/25 18:23:59 (1481) INFO> perfSONARBUOY.pm:3025 perfSONAR_PS::Services::MA::perfSONARBUOY::retrieveSQL - NODES and DATA queries
2014/03/25 18:23:59 (1481) INFO> owhdb.pm:117 perfSONAR_PS::DB::owhdb::getNodeIdsFromIp - About to process querySELECT node_id
                                 FROM 20140325_NODES
                                 WHERE addr=?
2014/03/25 18:23:59 (1481) INFO> owhdb.pm:117 perfSONAR_PS::DB::owhdb::getNodeIdsFromIp - About to process querySELECT node_id
                                 FROM 20140325_NODES
                                 WHERE addr=?
2014/03/25 18:23:59 (1481) INFO> owhdb.pm:327 perfSONAR_PS::DB::owhdb::fetchSummaryData - About to process querySELECT a.si, a.ei, a.stimestamp, a.etimestamp, a.start_time, a.end_time,
                                        a.min, a.max, a.minttl, a.maxttl,
                                        a.sent, a.lost, a.dups, a.maxerr
                         FROM 20140325_DATA AS a
                         WHERE a.si>? AND a.ei<? AND  (  a.tspec_id="2875318245" )  AND  (  a.send_id="387540855" )  AND  (  a.recv_id="584641430" or  a.recv_id="1725791178" ) ORDER BY a.si;

the logfile is "only" 100s of megabytes on psum01

[root@psum01 ~]# ls -lh /var/log/perfsonar/perfsonarbuoy_ma.log
-rw-r--r-- 1 perfsonar perfsonar 136M Mar 25 18:47 /var/log/perfsonar/perfsonarbuoy_ma.log

vs gygabytes on psmsu01

[root@psmsu01 ~]# ls -lh /var/log/perfsonar/perfsonarbuoy_ma.log
-rw-r--r-- 1 perfsonar perfsonar 2.2G Mar 25 18:42 /var/log/perfsonar/perfsonarbuoy_ma.log

2) I have graphs which are listed in the GUI, and even show a summary
packet loss so we know that they are collecting data, but the graphs
do not display any content.  I have tried stopping services and running
'myisamchk -er' on the tables, but that didn't help.  Where should I
be looking for the source of the trouble?
For example: the first graph for atlas-npt1.bu.edu here
http://psmsu01.aglt2.org/serviceTest/index.cgi?eventType=owamp


Thanks,

Philippe