perfSONAR User Q&A and Other Discussion

["Remsik,Robert" <>]

Hello!  

Your lookup_file generates the same .json file I'm getting on my end which is awesome!  Thank you.

When using your .json or the one on my end I'm still getting the error when MadDash attempts to consume the .json and build charts.  I've verified that the maddash-server.yaml isn't getting properly updated(and so the web-page isn't being updated).  Are you able to generate a mad-dash dashboard using the generated .json?

Could it be it's something to do with pointing at perfsonar1.research-lan.colostate.edu for all the hosts as the measurement_archive instead of asking them to referencing their individual archives?  I'm not sure if the later is even possible.

015/10/26 10:46:13 (5158) ERROR> GUIAgent.pm:285 perfSONAR_PS::MeshConfig::GUIAgent::__generate_maddash_config - Problem generating maddash configuration: Couldn't find ma for host: perfSONAR_PS::MeshConfig::Config::Address=HASH(0xaa9f130) at /opt/perfsonar_ps/mesh_config/bin/../lib/perfSONAR_PS/MeshConfig/Generators/MaDDash.pm line 291.

2015/10/26 10:46:13 (5158) ERROR> GUIAgent.pm:231 perfSONAR_PS::MeshConfig::GUIAgent::__configure_guis - Problem generating maddash configuration: Problem generating maddash configuration: Couldn't find ma for host: perfSONAR_PS::MeshConfig::Config::Address=HASH(0xaa9f130) at /opt/perfsonar_ps/mesh_config/bin/../lib/perfSONAR_PS/MeshConfig/Generators/MaDDash.pm line 291.

Thank you in advance,

Robert

Robert Remsik

ACNS

Cell: 970 213 7219

---

From: Andrew Lake <>
Sent: Monday, October 26, 2015 6:56 AM
To: Remsik,Robert
Cc:
Subject: Re: [perfsonar-user] Can't find MA for host using dynamic mesh

 

Hi Robert,

The queries in lookup_hosts.conf are case sensitive (because in the LS they are cases sensitive). The web page caches all the LS data and does it’s own searching basically, so it’s not case sensitive.  If I added another search for the capitalized version to lookup_hosts.conf I get 22 hosts which match exactly what is on the GUI. The archives look right too as far as I can tell. See attached .conf file and the JSON it output.

Thanks,

Andy

On Fri, Oct 23, 2015 at 4:46 PM, Remsik,Robert <> wrote:

By copying and pasting your lookup file I'm able to generate the same auto.json.  Thank you.

<OutlookEmoji-😊.png>

It seems there are 2 groups for the FRGP(frgp, and FRGP).  When I add additional queries(copy+paste changing the filter value only) or change the filter value to the capitalized spelling I again get the error.  Is there something special about searching with capital letters?

I'm using as a reference of the amount of hosts I'm expecting.

http://stats.es.net/ServicesDirectory/#search=frgp&uri=lookup%2Fservice%2F4222734d-0bde-4e4f-9b67-79319bcfeebe

Thank you again for the help,

Robert

Robert Remsik

ACNS

Cell: 970 213 7219

---

From: Andrew Lake <>
Sent: Friday, October 23, 2015 2:04 PM
To: Remsik,Robert
Cc:
Subject: Re: [perfsonar-user] Can't find MA for host using dynamic mesh

 

Hi Robert,

It looks like it’s complaining because the hosts at https://perfsonar1.research-lan.colostate.edu/auto.json don't have measurement_archives for the owamp or ping types. As a result when you try to define a ping or owamp mesh, it complains about not being able to find an MA. 

For the OWAMP MA, I actually ran the lookup_hosts scripts locally using your exact lookup_hosts.conf, and it set the OWAMP MAs properly. Maybe you just need to run it again to get those?

For the ping tests you need to add “type pinger” to the <measurement_archive> blocks of one of the <query> blocks. Attached is the lookup_hosts.conf I used and corresponding auto.json that got generated. Let me know if you get different results. 

Hope that helps,

Andy

 

On Fri, Oct 23, 2015 at 3:24 PM, Remsik,Robert <> wrote:

My statically defined json(in case we need a reference):

http://perfsonar1.research-lan.colostate.edu/research.json

Dynamically created Json:

http://perfsonar1.research-lan.colostate.edu/dyn.json

json for the hosts:

https://perfsonar1.research-lan.colostate.edu/auto.json

No problem!

Robert Remsik

ACNS

Cell: 970 213 7219

---

From: Andrew Lake <>
Sent: Friday, October 23, 2015 12:59 PM
To: Remsik,Robert
Cc:
Subject: Re: [perfsonar-user] Can't find MA for host using dynamic mesh

 

Hi Robert,

Could you send me a link to the generated JSON (or attach it into a file if its firewalled off)? It looks like in theory it should be able to find an MA as you said.

Thanks,

Andy

On Fri, Oct 23, 2015 at 1:32 PM, Remsik,Robert <> wrote:

Hello!

I'm trying to use a dynamic mesh to run tests on and update maddash to display with no luck so far.  Any help is appreciated.

I'm getting the below error when I try to run ( /opt/perfsonar_ps/mesh_config/bin/generate_gui_configuration ) to update maddash.  When I run generate_configuration I don't notice the tests being created.  My hunch is it's the lookup_hosts file not providing the correct measurement archive, but I think I've asked all the hosts to point to perfsonar1-research-lan.colostate.edu instead of using their local MAs(which would be cool to do as well).

perfsonar1.research-lan.colostate.edu/dyn.json - dynamic tests

perfsonar1.research-lan.colostate.edu/auto.json - list of hosts

2015/10/23 11:25:52 (8008) ERROR> GUIAgent.pm:285 perfSONAR_PS::MeshConfig::GUIAgent::__generate_maddash_config - Problem generating maddash configuration: Couldn't find ma for host: perfSONAR_PS::MeshConfig::Config::Address=HASH(0xa951770) at /opt/perfsonar_ps/mesh_config/bin/../lib/perfSONAR_PS/MeshConfig/Generators/MaDDash.pm line 291.

2015/10/23 11:25:52 (8008) ERROR> GUIAgent.pm:231 perfSONAR_PS::MeshConfig::GUIAgent::__configure_guis - Problem generating maddash configuration: Problem generating maddash configuration: Couldn't find ma for host: perfSONAR_PS::MeshConfig::Config::Address=HASH(0xa951770) at /opt/perfsonar_ps/mesh_config/bin/../lib/perfSONAR_PS/MeshConfig/Generators/MaDDash.pm line 291.

lookup_hosts.conf file:

ls_instance http://ps-west.es.net:8090/lookup/records

ls_instance http://ps-east.es.net:8090/lookup/records

<query>

    service_type bwctl

    <filter>

        filter_key group-communities

        filter_value frgp

    </filter>

    <output_settings>

        organization_name FRGP

        <measurement_archive>

            type perfsonarbuoy/bwctl

            type traceroute

            read_url http://perfsonar1.research-lan.colostate.edu/esmond/perfsonar/archive

            write_url http://perfsonar1.research-lan.colostate.edu/esmond/perfsonar/archive

        </measurement_archive>

        organization_tag frgp

        site_tag bwctl_site

        host_tag linux

        host_tag throughput_host

        address_tag bwctl

    </output_settings>

</query>

<query>

    service_type owamp

    <filter>

        filter_key group-communities

        filter_value frgp

    </filter>

    <output_settings>

        organization_name FRGP

        <measurement_archive>

            type perfsonarbuoy/owamp

            type traceroute

            read_url http://perfsonar1.research-lan.colostate.edu/esmond/perfsonar/archive

            write_url http://perfsonar1.research-lan.colostate.edu/esmond/perfsonar/archive

        </measurement_archive>

        address_tag owamp

        site_tag owamp_site

        host_tag linux

        host_tag latency_host

        organization_tag frgp

    </output_settings>

</query>

Batch file I'm using to update all the configs and restart services for testing:

#!/bin/bash

cd /opt/perfsonar_ps/mesh_config/doc

#/opt/perfsonar_ps/mesh_config/bin/lookup_hosts --input /opt/perfsonar_ps/mesh_config/etc/lookup_hosts.conf --output /var/www/html/dynamic_host_list.json

echo "Generating .json configs"

echo "Generating research.json"

/opt/perfsonar_ps/mesh_config/bin/build_json -o /var/www/html/research.json research.conf

echo "Generating auto.json"

/opt/perfsonar_ps/mesh_config/bin/lookup_hosts --input /opt/perfsonar_ps/mesh_config/etc/lookup_hosts.conf --output /var/www/html/auto.json

echo "Generating dyn.json"

/opt/perfsonar_ps/mesh_config/bin/build_json -o /var/www/html/dyn.json dyn.conf

echo "Generating .yaml config"

echo "Agent Configuration"

/opt/perfsonar_ps/mesh_config/bin/generate_configuration

echo "GUIConfiguration"

/opt/perfsonar_ps/mesh_config/bin/generate_gui_configuration

echo "Restarting Services"

service maddash-server restart

~                                                                                                                                                                                                                                            

~                                                                                                                                                                                                                                            

~                                                                                                                                                                                                                                            

~                                             

Robert Remsik

ACNS

Cell: 970 213 7219