perfSONAR User Q&A and Other Discussion

["C. Aaron Smith" <>]

Hi again Andy,

Forgot to copy the list on my first reply.  Thanks for taking a look.
Here is the /etc/perfsonar/meshconfig-agent.conf.  There are manual
tests running on the test nodes to a set of nodes not part of the mesh
config.  Those tests show up in the meshconfig-agent-tasks.conf file as
expected.

Thanks,
Aaron

On 5/4/2017 10:47 AM, Andrew Lake wrote:
> Hi Aaron,
> 
> May seem like a silly question, but in your
> /etc/perfsonar/meshconfig-agent.conf are they pointing at
> http://128.91.196.207/mesh.json? If they are, could you send me
> /etc/perfsonar/meshconfig-agent.conf and I can double-check for anything
> else out of place. Looks like its just not reading that URL. It looks
> like what I assume are manual tests or tests from another mesh are
> getting created fine since if I go
> to https://165.123.50.178/pscheduler/tasks and dig down into one of the
> URLs its a new task, but looks like its running to a set of hosts not in
> the mesh shared.
> 
> Thanks,
> Andy
> 
> 
> On May 4, 2017 at 10:07:58 AM, C. Aaron Smith 
> (
> <mailto:>)
>  wrote:
> 
>> Hello all,
>>
>> I am setting up a mesh of test nodes as a proof of concept and running
>> into a problem. The testing nodes are not getting tests from the mesh.
>>
>> The test nodes are toolkit iso installs on centos 6, with perfsonar
>> upgraded to 4.0. The central mesh server is centos 7 toolkit iso
>> install of 4.0.
>>
>> The mesh configuration is derived from a conf file and can be found
>> here: http://128.91.196.207/mesh.json
>>
>> As per http://docs.perfsonar.net/multi_mesh_agent_config.html the test
>> nodes have /etc/perfsonar/meshconfig-agent.conf updated to use the above
>> file.
>>
>> /var/log/meshconfig-agent.log has updates like these:
>> 2017/05/03 10:37:01 (1107) INFO> perfsonar_meshconfig_agent:453 main:: -
>> Configuration file change detected, refreshing records.
>> 2017/05/03 10:39:32 (1107) INFO> perfsonar_meshconfig_agent:438 main:: -
>> Added 12 new tasks, and deleted 36 old tasks
>> 2017/05/03 10:51:32 (1107) INFO> perfsonar_meshconfig_agent:453 main:: -
>> Configuration file change detected, refreshing records.
>> 2017/05/03 10:52:18 (1107) INFO> perfsonar_meshconfig_agent:438 main:: -
>> Added 3 new tasks, and deleted 0 old tasks
>> 2017/05/04 03:53:51 (1107) INFO> perfsonar_meshconfig_agent:438 main:: -
>> Added 12 new tasks, and deleted 0 old tasks
>> 2017/05/04 04:52:19 (1107) INFO> perfsonar_meshconfig_agent:438 main:: -
>> Added 3 new tasks, and deleted 0 old tasks
>> 2017/05/04 09:19:54 (1107) INFO> perfsonar_meshconfig_agent:453 main:: -
>> Configuration file change detected, refreshing records.
>>
>> But the /etc/perfsonar/meshconfig-agent-tasks.conf on the test node is
>> not getting updated with the tasks from the mesh config.
>>
>> Where do I look on the test node to see if the mesh.json file is being
>> read by the test node?
>>
>> Thanks,
>> Aaron
>>
>> -- 
>> C. Aaron Smith
>> Senior Network Engineer
>> ISC Tech Services
>> University of Pennsylvania
>> OpenPGP Key ID: 0xC311AB25

-- 
C. Aaron Smith
Senior Network Engineer
ISC Tech Services
University of Pennsylvania
OpenPGP Key ID: 0xC311AB25
## Use 'mesh' blocks to to specify each mesh that the agent should
## configure the host against
#<mesh>
#    ## Use 'configuration_url' to specify the URL where the agent should 
obtain
#    ## the mesh configuration from
#    configuration_url             https://host.domain.edu/example.json
#
#    ## To ensure that the configuration is trusted, you can set the
#    ## 'validate_certificate' option to 1. This will validate that the 
certificate
#    ## is valid, and matches the hostname. If the 'validate_certificate' 
option is
#    ## set to 1, the 'ca_certificate_file' option must be set.
#    #validate_certificate         0
#
#    ## The 'ca_certificate_file' specifies which CAs to use to validate the
#    ## certificates.
#    #ca_certificate_file          /etc/pki/tls/certs/ca-bundle.crt    # the 
default RedHat 6 CAs
#
#    ## The 'required' field specifies that if this mesh isn't available, or 
has
#    ## problems, to skip making any changes to the host configuration.
#    #required              1
#
#    ## The 'permit_non_participation' specifies that if the host isn't
#    ## included in this mesh, the agent shouldn't thrown an error.
#    #permit_non_participation      1
#</mesh>
<mesh>
configuration_url       http://128.91.196.207/mesh.json
</mesh>

## You can define more meshes to configure against by adding more 'mesh' 
blocks.
#<mesh>
#    configuration_url            https://host.domain.edu/example2.json
#    #validate_certificate         0
#    #ca_certificate_file          /etc/pki/tls/certs/ca-bundle.crt
#</mesh>

# You can use the 'configure_archives' variable to have the mesh 
configuration generate
# measurement_archive blocks using the write_url from your configuration 
file. This is disabled
# by default but you can enable it by setting to to 1.
configure_archives 1

## Use the 'tasks_conf' variable to change the location of the
## meshconfig-agent-tasks.conf file. This is where you can define tasks 
locally
## and where tests from remote meshes will be populated. This file contains a 
list of all the
## tests you want run. In a normal configuration, these should be left as 
what's included below.
tasks_conf        /etc/perfsonar/meshconfig-agent-tasks.conf

## Frequency with which to check the remote meshes, local tasks_conf file, 
and pScheduler
## server for changes. If any found it will attempt to build any new tasks 
and delete any old
## ones as required. See also check_config_interval which is the frequency 
with which to
## check for local configuration file changes
check_interval 3600

## Frequency in seconds with which to poll this file and the tasks_conf file 
for changes.
## If any changes are detected, it will attempt to build any new tasks and 
delete any old
## ones as required. Should generally be less than check_interval to have any 
value.
check_config_interval 60

# You can include one or more 'address' options to specify which address
## corresponds to this host. By default, the agent will attempt to guess the
## address using the hostname of the host, as well as the hostname IPs.
#address                      client-host.domain.edu

## You can use the "local_host" directive to define properties for a host
## you'd normally define in a MeshConfig host directive. This allows you to 
tag the host
## and addresses locally, which is especially useful for meshes dynamically 
generated
## using host tags. You can also define <measurement_archive> blocks to 
define the
## archives you want tho host to use.
# <local_host>
#     tag myorganization
#
#     <address>
#         address 10.0.1.1
#         tag     latency
#     </address>
#     <address>
#         address 10.0.1.2
#         tag     throughput
#     </address>
#
#     <measurement_archive>
#         type        perfsonarbuoy/bwctl
#         read_url    http://client-ma.domain.edu/esmond/perfsonar/archive
#         write_url   http://client-ma.domain.edu/esmond/perfsonar/archive
#     </measurement_archive>
# </local_host>

# The 'send_error_emails' option can be used to tell the agent to send an 
email
# (via sendmail) when an error occurs. These emails will be sent to the
# applicable administrators (e.g. the local administrator(s), the mesh
# administrator(s), the site administrator(s), and/or the host
# administrator(s).
#send_error_emails             1

## By default, the from address for the error emails will be
## mesh_agent@[hostname]. However, if you'd like to specify a different email
## address, you can do so here.
#from_address           


## One or more 'admin_email' options can be included to specify email 
addresses
## to send errors to. These email addresses will receive all errors (including
## parsing errors). This is in addition to the emails that will be sent to the
## various affected administrators (e.g. the mesh administrator or the site
## administrator).
#admin_email         

#admin_email         


## The 'skip_redundant_tests' can be used so that if the meshes end up 
creating
## multiple tests between hosts with the exact same test parameters, it will
## only setup the hosts to only run one test. If you enable this, you won't 
see
## the skipped tests in the Toolkit GUI
skip_redundant_tests        1

## The file that stores a UUID included in every pScheduler task. This UUID 
is used to
## track tasks that this agent has created.
client_uuid_file /var/lib/perfsonar/meshconfig/client_uuid

## The URL of the pScheduler instance to use as the 'assist' server to 
determine where
## to schedule tasks. Generally this is a local pScheduler instance that will 
be the lead
## participant on many of the tasks submitted, though technically speaking it 
is not required
## to be a participant in any task
#pscheduler_url https://127.0.0.1/pscheduler

## The file where the agent tracks which pScheduler servers it has previously 
submitted tasks
#pscheduler_tracker_file /var/lib/perfsonar/meshconfig/psc_tracker

## The number of times to try to connect to the local pScheduler when it 
fails after
# only waiting check_config_interval as opposed to the generally much larger 
check_interval.
#pscheduler_fail_attempts 5

## The minimum lifetime in seconds to set for a new test created. The greater 
of this value and
## that calculated from task_min_runs will be used to determine end time of 
the test. It is recommended
## to set this to a reasonable value as it greatly reduces the cost of an 
orphaned test
##(i.e. a test you create will live on if your host goes away)
#task_min_ttl 86400

## The minimum number of times a task should run before expiring. If you have 
a test
## scheduler to run once an hour and set this value to 2, then the minimum 
end time will
## be determined to be the greater of task_min_ttl and 2 hours.
#task_min_runs 2

## The percentage of check_interval after the next scheduled check that a 
task's end time
## must be below before you create a new task. For example if this is .25 and
## check_interval is 1 hour, than a task's end time must be 75 minutes (1.25 
* 1 hour) in
## the future or else a new task will be created where the start time is the 
end time of
## the old.
#task_renewal_fudge_factor .25