perfSONAR User Q&A and Other Discussion

[Murilo Vetter <>]

Hi Andrew,

Now I changed the configuration of my perfsonar hosts and I am having the following:

REMOTE PS CONFIG HOST:

--

[root@monipe-sc-perfsonar /]# psconfig remote list
=== pScheduler Agent ===
[
   {
      "url" : "http://monipe-central.rnp.br:8000/pub/config/matriz?format=psconfig",
      "configure-archives" : true
   }
]

--

LOGS:

--

[root@monipe-sc-perfsonar /]# tail -f /var/log/perfsonar/psconfig-pscheduler-*
==> /var/log/perfsonar/psconfig-pscheduler-agent-tasks.log <==

==> /var/log/perfsonar/psconfig-pscheduler-agent-transactions.log <==
2019/04/03 15:16:00 INFO guid=8474DB58-563C-11E9-BEE2-83364A7EE038 action="create" msg=Creating tasks
2019/04/03 15:16:00 INFO guid=8474DB58-563C-11E9-BEE2-83364A7EE038 action="create" msg=No tasks to create
2019/04/03 15:16:00 INFO guid=8474DB58-563C-11E9-BEE2-83364A7EE038 action="create" msg=Done creating tasks
2019/04/03 15:20:00 INFO guid=17CD80A8-563D-11E9-96CC-6C374A7EE038 action="list" url="https://localhost/pscheduler" msg=Getting task list from https://localhost/pscheduler
2019/04/03 15:20:02 INFO guid=17CD80A8-563D-11E9-96CC-6C374A7EE038 action="delete" msg=Deleting tasks
2019/04/03 15:20:02 INFO guid=17CD80A8-563D-11E9-96CC-6C374A7EE038 action="delete" msg=No tasks marked for deletion
2019/04/03 15:20:02 INFO guid=17CD80A8-563D-11E9-96CC-6C374A7EE038 action="delete" msg=Done deleting tasks
2019/04/03 15:20:02 INFO guid=17CD80A8-563D-11E9-96CC-6C374A7EE038 action="create" msg=Creating tasks
2019/04/03 15:20:02 INFO guid=17CD80A8-563D-11E9-96CC-6C374A7EE038 action="create" msg=No tasks to create
2019/04/03 15:20:02 INFO guid=17CD80A8-563D-11E9-96CC-6C374A7EE038 action="create" msg=Done creating tasks

==> /var/log/perfsonar/psconfig-pscheduler-agent.log <==
2019/04/03 15:20:02 ERROR pid=233 prog=perfSONAR_PS::PSConfig::PScheduler::Agent::_run_handle_psconfig line=188 task=ATRASO,addresses=monipe-sc-atraso.rnp.br->monipe-pe-atraso.rnp.br,error=Unable to find archive defined in task: config-archive0
2019/04/03 15:20:02 ERROR pid=233 prog=perfSONAR_PS::PSConfig::PScheduler::Agent::_run_handle_psconfig line=188 task=ATRASO,addresses=monipe-sc-atraso.rnp.br->monipe-ro-atraso.rnp.br,error=Unable to find archive defined in task: config-archive0
2019/04/03 15:20:02 ERROR pid=233 prog=perfSONAR_PS::PSConfig::PScheduler::Agent::_run_handle_psconfig line=188 task=ATRASO,addresses=monipe-sc-atraso.rnp.br->monipe-go-atraso.rnp.br,error=Unable to find archive defined in task: config-archive0
2019/04/03 15:20:02 ERROR pid=233 prog=perfSONAR_PS::PSConfig::PScheduler::Agent::_run_handle_psconfig line=188 task=ATRASO,addresses=monipe-sc-atraso.rnp.br->monipe-pi-atraso.rnp.br,error=Unable to find archive defined in task: config-archive0
2019/04/03 15:20:02 ERROR pid=233 prog=perfSONAR_PS::PSConfig::PScheduler::Agent::_run_handle_psconfig line=188 task=ATRASO,addresses=monipe-sc-atraso.rnp.br->monipe-rr-atraso.rnp.br,error=Unable to find archive defined in task: config-archive0
2019/04/03 15:20:02 ERROR pid=233 prog=perfSONAR_PS::PSConfig::PScheduler::Agent::_run_handle_psconfig line=188 task=ATRASO,addresses=monipe-sc-atraso.rnp.br->monipe-se-atraso.rnp.br,error=Unable to find archive defined in task: config-archive0
2019/04/03 15:20:02 ERROR pid=233 prog=perfSONAR_PS::PSConfig::PScheduler::Agent::_run_handle_psconfig line=188 task=ATRASO,addresses=monipe-sc-atraso.rnp.br->monipe-rn-atraso.rnp.br,error=Unable to find archive defined in task: config-archive0
2019/04/03 15:20:02 ERROR pid=233 prog=perfSONAR_PS::PSConfig::PScheduler::Agent::_run_handle_psconfig line=188 task=ATRASO,addresses=monipe-sc-atraso.rnp.br->monipe-pr-atraso.rnp.br,error=Unable to find archive defined in task: config-archive0
2019/04/03 15:20:02 INFO pid=233 prog=main:: line=178 guid=17CD80A8-563D-11E9-96CC-6C374A7EE038 msg=Agent completed running
2019/04/03 15:20:02 INFO pid=233 prog=main:: line=189 guid=17CD80A8-563D-11E9-96CC-6C374A7EE038 msg=Time until next record refresh is 3598 seconds

==> /var/log/perfsonar/psconfig-pscheduler-migrate.log <==

/etc/perfsonar/meshconfig-agent.conf does not exist, so nothing to translate. That probably means this is a new install or has the host already been upgraded.

/etc/perfsonar/meshconfig-agent-tasks.conf does not exist, so nothing to translate. That probably means this is a new install or has the host already been upgraded.

/var/lib/perfsonar/meshconfig/client_uuid does not exist, nothing to migrate

/var/lib/perfsonar/meshconfig/psc_tracker does not exist, nothing to migrate

[Mon Oct 29 15:07:31 UTC 2018] Migration complete
--

Now every hosts are showing these problems. Any help?

Regards,

--

Murilo Vetter
Analista REMEP-FLN/PoP-SC
PoP-SC - Ponto de Presença da RNP em Santa Catarina
REMEP-FLN - Rede Metropolitana de Educação e Pesquisa da Região de Florianópolis
RNP - Rede Nacional de Ensino e Pesquisa
UFSC - Universidade Federal de Santa Catarina
SETIC - Superintendência de Governança Eletrônica e Tecnologia da Informação e Comunicação
Fundações de Apoio FEESC e FEPESE
--/--

www.pop-sc.rnp.br
remep.pop-sc.rnp.br
CHAMADOS:
+55(48)3721-3000
INOC: 11242*100

---

De: "Andrew Lake" <>
Para: "Ivan Garnizov" <>, "Murilo Vetter" <>
Cc: "perfsonar-user" <>, "Mark Feit" <>
Enviadas: Quarta-feira, 3 de abril de 2019 12:10:55
Assunto: Re: [perfsonar-user] Tests are not scheduling on pscheduler

Hi Murilio,

Let me try to summarize the steps you have run and what should happen so you can correct me if I missed anything and then we can delve into how to figure out what is going wrong:

1. You bring up a new perfsolnar-testpoint container

2. Inside the testpoint container you run “psconfig remote add --configure-archives http://monipe-central.rnp.br:8000/pub/config/matriz”

3. Within 60 seconds the psconfig agent should pickup the new URL and start looking for tasks (i.e. no service restart needed)

4. It starts creating tasks.

It looks like at either steps 3 or 4 things are tripping-up. The most common cause for the type of behavior you are seeing is that the container has an IP address that does not map to anything in your remote JSON file. Specifically, if you were to do a DNS lookup any of the hostnames in the “addresses” section of your remote JSON, does it map to an IP address you see when you run “ifconfig” inside the container? If the answer is no, that is likely the issue and you need some additional configuration (that I can help you with if this is indeed the case).

If the hostnames do map to an IP address inside the container, then there are a couple things you can check. For one, it looks like you are using the pSConfig Web administrator, which still defaults to the old MeshConfig JSON format for backward compatibility reasons. You can get the newer format directly by running the following:

psconfig remote delete http://monipe-central.rnp.br:8000/pub/config/matriz

psconfig remote add --configure-archives "http://monipe-central.rnp.br:8000/pub/config/matriz?format=psconfig"

This might fix things if you are running into some type of translation error. If not, it will take a translation error out of the equation at least. 

If that still does not fix things you can turn on debugging in the logs by replacing INFO with DEBUG in /etc/perfsonar/psconfig/pscheduler-agent-logger.conf. We can go from there. In particular I’d be interested if we see any messages in the logs indicating it even tried to download the JSON file if it it turns out none of the above helps. 

Hope that helps get you further along.

Thanks,

Andy

On April 3, 2019 at 8:59:35 AM, Murilo Vetter () wrote:

 the system operates without the knowledge of the mesh configuration, which is added during the operation of the pS container….

 

In your case, I would suggest to prepare the psconfig configuration for  the meshes with the building of the container. Thus when started with the container psconfig will automatically parse it and configure pscheduler.

 

There might be other mean to restart a service without the use of the ‘systemctl’ command. Of course I can’t recommend that, but I guess in your case, there is nothing to loose.