perfSONAR User Q&A and Other Discussion

[Andrew Lake <>]

Hi,

I can help with a couple of these:

- For the maddash issue, see this FAQ http://psps.perfsonar.net/toolkit/FAQs.html#Q71

- For the traceroute issue, what is in traceroute_master.log and traceroute_scheduler.netlogger.log on one of the test nodes ? Its possible the traceroute tests are not completing for some reason (e.g. ICMP blocked). Those may shed some more light. I think otherwise your traceroute test config looks fine (though I may be missing something)

Thanks,

Andy

On Mar 31, 2014, at 10:16 AM, "Uhl, George D. (GSFC-423.0)[ARTS]" <> wrote:

All,

I did an yum update on my Measurement Archive server over the weekend  and now the maddish page on the server is broken as well as the traceroute_ma service graphs from the toolkit page.  The maddish page displays the page header and returns nothing else.  The traceroute service graph page displays  "Error: No Measurement Archives available."  This is a custom built measurement archive so perhaps not all the packages that might be available from a PS toolkit yum update are going to be available.

From the yum update log on the MA sever:

# cat yum.log | grep perf

Mar 28 14:58:37 Installed: perl-perfSONAR_PS-MeshConfig-GUIAgent-3.3.2-3.pSPS.noarch

Mar 29 21:22:45 Updated: perl-perfSONAR_PS-TracerouteMA-config-3.3.2-1.pSPS.noarch

Mar 29 21:23:07 Updated: perl-perfSONAR_PS-perfSONARBUOY-config-3.3.2-1.pSPS.noarch

Mar 29 21:23:07 Updated: iperf-2.0.5-11.el6.x86_64

Mar 29 21:23:58 Updated: perl-perfSONAR_PS-serviceTest-3.3.2-4.pSPS.noarch

Mar 29 21:23:58 Updated: perl-perfSONAR_PS-SimpleLS-BootStrap-client-3.3.2-1.pSPS.noarch

Mar 29 21:23:58 Updated: perl-perfSONAR_PS-SNMPMA-3.3-4.pSPS.noarch

Mar 29 21:23:58 Updated: perl-perfSONAR_PS-TracerouteMA-server-3.3.2-1.pSPS.noarch

Mar 29 21:23:59 Updated: perl-perfSONAR_PS-LSRegistrationDaemon-3.3.2-1.pSPS.noarch

Mar 29 21:23:59 Updated: perl-perfSONAR_PS-perfSONARBUOY-server-3.3.2-1.pSPS.noarch

Mar 29 21:23:59 Updated: perl-perfSONAR_PS-perfSONARBUOY-client-3.3.2-1.pSPS.noarch

Mar 29 21:23:59 Updated: perl-perfSONAR_PS-TracerouteMA-client-3.3.2-1.pSPS.noarch

Mar 29 21:24:07 Installed: perl-perfSONAR_PS-Toolkit-3.3.2-13.pSPS.noarch

Mar 29 21:24:07 Updated: perl-perfSONAR_PS-MeshConfig-Shared-3.3.2-3.pSPS.noarch

Mar 29 21:24:07 Updated: perl-perfSONAR_PS-Nagios-3.3.2-2.pSPS.noarch

Mar 29 21:25:10 Updated: perl-perfSONAR_PS-MeshConfig-JSONBuilder-3.3.2-3.pSPS.noarch

Mar 29 21:25:23 Updated: perf-2.6.32-431.11.2.el6.x86_64

Mar 29 21:25:38 Erased: perl-perfSONAR_PS-TopologyService

From the yum log of one of the test nodes:

#  cat /var/log/yum.log | grep perf

Mar 20 15:16:45 Updated: perl-perfSONAR_PS-TracerouteMA-config-3.3.2-1.pSPS.noarch

Mar 20 15:17:50 Updated: perl-perfSONAR_PS-MeshConfig-Shared-3.3.2-3.pSPS.noarch

Mar 20 15:17:51 Updated: perl-perfSONAR_PS-SimpleLS-BootStrap-client-3.3.1-1.pSPS.noarch

Mar 20 15:18:02 Updated: perl-perfSONAR_PS-LSRegistrationDaemon-3.3.2-1.pSPS.noarch

Mar 20 15:18:12 Updated: perl-perfSONAR_PS-perfSONARBUOY-config-3.3.2-1.pSPS.noarch

Mar 20 15:18:17 Updated: iperf-2.0.5-11.el6.x86_64

Mar 20 15:20:55 Updated: perl-perfSONAR_PS-perfSONARBUOY-server-3.3.2-1.pSPS.noarch

Mar 20 15:20:56 Updated: perl-perfSONAR_PS-TracerouteMA-client-3.3.2-1.pSPS.noarch

Mar 20 15:20:57 Updated: perl-perfSONAR_PS-MeshConfig-Agent-3.3.2-3.pSPS.noarch

Mar 20 15:20:58 Updated: perl-perfSONAR_PS-perfSONARBUOY-client-3.3.2-1.pSPS.noarch

Mar 20 15:22:54 Updated: perf-2.6.32-431.5.1.el6.x86_64

Mar 26 12:08:12 Updated: perf-2.6.32-431.11.2.el6.x86_64

Mar 26 12:08:15 Updated: perl-perfSONAR_PS-SimpleLS-BootStrap-client-3.3.2-1.pSPS.noarch

From the maddash-server.netlogger.log:

level=INFO ts=2014-03-30T02:18:02.536201Z event=maddash.init.start guid=9de487d2-8c5e-4574-8125-3faf7e97d381 

level=ERROR ts=2014-03-30T02:18:06.292021Z event=maddash.init.end guid=9de487d2-8c5e-4574-8125-3faf7e97d381 status=-1 msg="Error loading database: Column 'TEMPLATENAME' is either not in any table in the FROM list or appears within a join specification and is outside the scope of the join specification or appears in a HAVING clause and is not in the GROUP BY list. If this is a CREATE or ALTER TABLE  statement then 'TEMPLATENAME' is not a column in the target table." 

level=INFO ts=2014-03-31T01:04:00.152609Z event=maddash.init.start guid=78268c7e-f7c9-43a0-81eb-402faaf6ab07 

level=ERROR ts=2014-03-31T01:04:02.302381Z event=maddash.init.end guid=78268c7e-f7c9-43a0-81eb-402faaf6ab07 status=-1 msg="Error loading database: Column 'TEMPLATENAME' is either not in any table in the FROM list or appears within a join specification and is outside the scope of the join specification or appears in a HAVING clause and is not in the GROUP BY list. If this is a CREATE or ALTER TABLE  statement then 'TEMPLATENAME' is not a column in the target table." 

The maddish yaml file (which define the maddish pages of two meshes) was built from the MeshConfig GUI Agent and can be viewed here : https://ensight.eos.nasa.gov/maddash.yaml 

The json file that defines one off the meshes can be viewed here:  https://ensight.eos.nasa.gov/enpl.json

-----

Before I did the yum update, I did a clean_pSBdb of the traceroute_ma database to clear out old data.  This left the database with only an empty DATES table.  However over time the traceroute_ma database never was repopulated with the daily tables containing traceroute test results and remains empty.

When a traceroute node reports into the MA, the MA's traceroute_ma.log reports:

2014/03/30 11:44:47 (26029) INFO> Traceroute.pm:111 perfSONAR_PS::Services::MA::Traceroute::init - Setting service access point to http://archive.eos.nasa.gov:8086/perfSONAR_PS/services/tracerouteMA

2014/03/30 11:44:47 (26029) WARN> Traceroute.pm:126 perfSONAR_PS::Services::MA::Traceroute::init - Setting 'service_description' to 'perfSONAR_PS Traceroute MA at NASA ESDIS Network Prototyping Lab'.

2014/03/30 11:44:47 (26029) WARN> Traceroute.pm:133 perfSONAR_PS::Services::MA::Traceroute::init - Setting 'service_name' to 'Traceroute MA'.

2014/03/30 11:44:47 (26029) WARN> Traceroute.pm:140 perfSONAR_PS::Services::MA::Traceroute::init - Setting 'service_type' to 'MA'.

2014/03/30 11:44:47 (26033) ERROR> daemon.pl:670 main::registerLS - Problem running register LS: No data found in given time range

2014/03/30 11:53:37 (26032) INFO> daemon.pl:572 main::psService - Received incoming connection from: 198.119.22.34

2014/03/30 11:53:42 (26118) ERROR> daemon.pl:582 main::psService - No HTTP Request received from host: 198.119.22.34

The owmesh.conf for the MA can be viewed here: https://ensight.eos.nasa.gov/ma_owmesh.conf

The owmesh.conf for one of the test nodes can be viewed here: https://ensight.eos.nasa.gov/node_owmesh.conf

The output of a "generate_configuration –verbose" can be viewed here: https://ensight.eos.nasa.gov/node_gen_config

Please let me know if you need more information.

Thanks,

George Uhl

NASA ESDIS / SGT Inc.

Code 423

NASA Goddard Space Flight Center

Greenbelt, MD 20771

Office: 301-614-5155

Fax: 301-614-5700

email:

--