perfSONAR User Q&A and Other Discussion

[Murilo Vetter <>]

Hi Ezra,

In the mentioned file no errors information:

2019/04/01 13:25:16 INFO guid=B9C249C4-549A-11E9-B3C8-E524A98EDC83 action="delete" msg=Deleting tasks
2019/04/01 13:25:16 INFO guid=B9C249C4-549A-11E9-B3C8-E524A98EDC83 action="delete" msg=No tasks marked for deletion
2019/04/01 13:25:16 INFO guid=B9C249C4-549A-11E9-B3C8-E524A98EDC83 action="delete" msg=Done deleting tasks
2019/04/01 13:25:16 INFO guid=B9C249C4-549A-11E9-B3C8-E524A98EDC83 action="create" msg=Creating tasks
2019/04/01 13:25:16 INFO guid=B9C249C4-549A-11E9-B3C8-E524A98EDC83 action="create" msg=No tasks to create
2019/04/01 13:25:16 INFO guid=B9C249C4-549A-11E9-B3C8-E524A98EDC83 action="create" msg=Done creating tasks
2019/04/01 14:25:16 INFO guid=1D833F88-54A3-11E9-B3C8-E524A98EDC83 action="list" url="https://localhost/pscheduler" msg=Getting task list from https://localhost/pscheduler
2019/04/01 14:25:19 INFO guid=1D833F88-54A3-11E9-B3C8-E524A98EDC83 action="delete" msg=Deleting tasks
2019/04/01 14:25:19 INFO guid=1D833F88-54A3-11E9-B3C8-E524A98EDC83 action="delete" msg=No tasks marked for deletion
2019/04/01 14:25:19 INFO guid=1D833F88-54A3-11E9-B3C8-E524A98EDC83 action="delete" msg=Done deleting tasks
2019/04/01 14:25:19 INFO guid=1D833F88-54A3-11E9-B3C8-E524A98EDC83 action="create" msg=Creating tasks
2019/04/01 14:25:19 INFO guid=1D833F88-54A3-11E9-B3C8-E524A98EDC83 action="create" msg=No tasks to create
2019/04/01 14:25:19 INFO guid=1D833F88-54A3-11E9-B3C8-E524A98EDC83 action="create" msg=Done creating tasks
2019/04/01 15:25:19 INFO guid=812B4A46-54AB-11E9-B3C8-E524A98EDC83 action="list" url="https://localhost/pscheduler" msg=Getting task list from https://localhost/pscheduler
2019/04/01 15:25:21 INFO guid=812B4A46-54AB-11E9-B3C8-E524A98EDC83 action="delete" msg=Deleting tasks
2019/04/01 15:25:21 INFO guid=812B4A46-54AB-11E9-B3C8-E524A98EDC83 action="delete" msg=No tasks marked for deletion
2019/04/01 15:25:21 INFO guid=812B4A46-54AB-11E9-B3C8-E524A98EDC83 action="delete" msg=Done deleting tasks
2019/04/01 15:25:21 INFO guid=812B4A46-54AB-11E9-B3C8-E524A98EDC83 action="create" msg=Creating tasks
2019/04/01 15:25:21 INFO guid=812B4A46-54AB-11E9-B3C8-E524A98EDC83 action="create" msg=No tasks to create
2019/04/01 15:25:21 INFO guid=812B4A46-54AB-11E9-B3C8-E524A98EDC83 action="create" msg=Done creating tasks
2019/04/01 16:25:22 INFO guid=E44FDD0A-54B3-11E9-B3C8-E524A98EDC83 action="list" url="https://localhost/pscheduler" msg=Getting task list from https://localhost/pscheduler
2019/04/01 16:25:25 INFO guid=E44FDD0A-54B3-11E9-B3C8-E524A98EDC83 action="delete" msg=Deleting tasks
2019/04/01 16:25:25 INFO guid=E44FDD0A-54B3-11E9-B3C8-E524A98EDC83 action="delete" msg=No tasks marked for deletion
2019/04/01 16:25:25 INFO guid=E44FDD0A-54B3-11E9-B3C8-E524A98EDC83 action="delete" msg=Done deleting tasks
2019/04/01 16:25:25 INFO guid=E44FDD0A-54B3-11E9-B3C8-E524A98EDC83 action="create" msg=Creating tasks
2019/04/01 16:25:25 INFO guid=E44FDD0A-54B3-11E9-B3C8-E524A98EDC83 action="create" msg=No tasks to create
2019/04/01 16:25:25 INFO guid=E44FDD0A-54B3-11E9-B3C8-E524A98EDC83 action="create" msg=Done creating tasks

I tried to change the configuration of /etc/perfsonar/psconfig/pscheduler-agent.json adding the attribute pscheduler-assist-server but it didn't help.

On thing that the followoing file /var/lib/perfsonar/psconfig/psc_tracker is empty like it:

--

{
   "archives" : {},
   "leads" : {}
}
--

In another host that the scheduling is working, it looks like this:

--

{
   "archives" : {
      "UmrpleC0xJUosfKOeTb7qw" : "UmrpleC0xJUosfKOeTb7qw",
      "SIFdxrCaheQUgBzXkZY5NA" : "SIFdxrCaheQUgBzXkZY5NA"
   },
   "leads" : {
      "https://monipe-ro-banda.rnp.br/pscheduler" : {
         "success_time" : 1554150625
      },
      "https://monipe-ro-atraso.rnp.br/pscheduler" : {
         "success_time" : 1554121653
      }
   }
}

--

What is going on with the problem host? Any clue?

Regards,

--

Murilo Vetter
Analista REMEP-FLN/PoP-SC
PoP-SC - Ponto de Presença da RNP em Santa Catarina
REMEP-FLN - Rede Metropolitana de Educação e Pesquisa da Região de Florianópolis
RNP - Rede Nacional de Ensino e Pesquisa
UFSC - Universidade Federal de Santa Catarina
SETIC - Superintendência de Governança Eletrônica e Tecnologia da Informação e Comunicação
Fundações de Apoio FEESC e FEPESE
--/--

www.pop-sc.rnp.br
remep.pop-sc.rnp.br
CHAMADOS:
+55(48)3721-3000
INOC: 11242*100

---

De: "Kissel, Ezra D" <>
Para: "Murilo Vetter" <>
Cc: "Mark Feit" <>, "perfsonar-user" <>
Enviadas: Segunda-feira, 1 de abril de 2019 17:08:40
Assunto: RE: Tests are not scheduling on pscheduler

Hi Murilo,

 

Are there any errors present in /var/log/perfsonar/psconfig-pscheduler-agent-transactions.log ?

 

My similar issue a while back was due to host resolution, which required some “helper” configuration.  See

 

https://lists.internet2.edu/sympa/arc/perfsonar-user/2019-01/msg00022.html

 

- ezra

 

 

From: <> On Behalf Of Murilo Vetter
Sent: Monday, April 1, 2019 3:57 PM
To: Murilo Vetter <>
Cc: Mark Feit <>; perfsonar-user <>
Subject: Re: [perfsonar-user] Tests are not scheduling on pscheduler

 

Hi list,

 

I would like to understand how the perfSONAR hosts know when to schedule tasks based on psconfig remote agent.

I am trying to debug a wierd behavior that the host appears on the configuration of PWA and in the following json, but it does not schedule on pscheduler.

 

[root@monipe-to-perfsonar /]# psconfig remote list
=== pScheduler Agent ===
[
   {
      "url" : "http://monipe-central.rnp.br:8000/pub/config/matriz",
      "configure-archives" : true
   }
]

If I try to troubleshoot the host, it looks it is ok.

[root@monipe-to-perfsonar /]# pscheduler troubleshoot --host monipe-to-atraso.rnp.br
Performing basic troubleshooting of monipe-to-atraso.rnp.br.

monipe-to-atraso.rnp.br:

  Checking path MTU... 65520+
  Checking for pScheduler... OK.
  Checking clock... OK.
  Idle test.... 12 seconds.... Checking archiving... OK.

pScheduler appears to be functioning normally.
[root@monipe-to-perfsonar /]# pscheduler troubleshoot --host monipe-to-banda.rnp.br
Performing basic troubleshooting of monipe-to-banda.rnp.br.

monipe-to-banda.rnp.br:

  Checking path MTU... 65520+
  Checking for pScheduler... OK.
  Checking clock... OK.
  Idle test.... 13 seconds.... Checking archiving... OK.

pScheduler appears to be functioning normally.

It is resolving the server name:

[root@monipe-to-perfsonar /]# hostname -A
monipe-to-atraso.rnp.br monipe-to-banda.rnp.br monipe-to-perfsonar

What am I missing? Is there a way to troubleshoot it in a better way?

 

Regards,

--

 

Murilo Vetter
Analista REMEP-FLN/PoP-SC
PoP-SC - Ponto de Presença da RNP em Santa Catarina
REMEP-FLN - Rede Metropolitana de Educação e Pesquisa da Região de Florianópolis
RNP - Rede Nacional de Ensino e Pesquisa
UFSC - Universidade Federal de Santa Catarina
SETIC - Superintendência de Governança Eletrônica e Tecnologia da Informação e Comunicação
Fundações de Apoio FEESC e FEPESE
--/--

www.pop-sc.rnp.br
remep.pop-sc.rnp.br
CHAMADOS:
+55(48)3721-3000
INOC: 11242*100

 

---

De: "Murilo Vetter" <>
Para: "Mark Feit" <>
Cc: "perfsonar-user" <>
Enviadas: Sexta-feira, 29 de março de 2019 17:43:52
Assunto: Re: [perfsonar-user] Tests are not scheduling on pscheduler

 

Hi Guys,

 

It is strange that is is not scheduling.

I am using hostnames to try to schedule. Should it works with hostnames or just IP address?

 

Some commands below:

 

[root@monipe-ba-perfsonar tmp]# psconfig remote list
=== pScheduler Agent ===
[
   {
      "url" : "http://monipe-central.rnp.br:8000/pub/config/matriz"
   }
]

[root@monipe-ba-perfsonar tmp]# psconfig pscheduler-tasks
{
   "tasks" : []
}

 

Any help?

 

Regards,

--

 

Murilo Vetter
Analista REMEP-FLN/PoP-SC
PoP-SC - Ponto de Presença da RNP em Santa Catarina
REMEP-FLN - Rede Metropolitana de Educação e Pesquisa da Região de Florianópolis
RNP - Rede Nacional de Ensino e Pesquisa
UFSC - Universidade Federal de Santa Catarina
SETIC - Superintendência de Governança Eletrônica e Tecnologia da Informação e Comunicação
Fundações de Apoio FEESC e FEPESE
--/--

www.pop-sc.rnp.br
remep.pop-sc.rnp.br
CHAMADOS:
+55(48)3721-3000
INOC: 11242*100

 

---

De: "Mark Feit" <>
Para: "Murilo Vetter" <>, "perfsonar-user" <>
Enviadas: Quinta-feira, 28 de março de 2019 18:45:52
Assunto: Re: [perfsonar-user] Tests are not scheduling on pscheduler

 

Murilo Vetter writes:

 

I am trying to schedule some tests from the host: monipe-ba-atraso.rnp.br / monipe-ba-banda.rnp.br ( 2 net interfaces in this host, 1 for ping/trace/owamp and 1 for troughput ).

 

I am getting problems since it does not put on schedule.

 

I already check this issues:

- Hostname:

[root@monipe-ba-perfsonar httpd]# hostname -A
monipe-ba-atraso.rnp.br monipe-ba-banda.rnp.br monipe-ba-perfsonar

 

- Troubleshooting:

[root@monipe-ba-perfsonar httpd]# pscheduler troubleshoot monipe-ba-atraso.rnp.br
…

 

If I understand this correctly, you’re trying to troubleshoot from the local host to itself.  That will (and did) work fine for the single-host idle tests.  Once it tries to do something multi-participant (simplestream, throughput), it’s going to fail because the same machine can’t be both ends of a multi-participant test even if they’re different interfaces.  I was able to run tasks to and from that host with no problem, although I did notice that either the system or the network between here and there was a bit sluggish.

 

localhost and monipe-ba-atraso.rnp.br:

  Checking path MTU... 65520+
  Checking timekeeping... OK.
  Simple stream test... Failed.
Unable to post task: Error while tasking monipe-ba-atraso.rnp.br: Unable to post task to monipe-ba-atraso.rnp.br: Internal problem; see system logs.

 

 

Unless there’s some other problem causing that, you should be seeing a message that says “Task already exists.  All participants must be on separate systems.”  Early versions of pScheduler choked like that, but it was corrected before anything went into production.

 

Is there anything of interest in the logs on that system?

 

--Mark

 

 

--
To unsubscribe from this list: https://lists.internet2.edu/sympa/signoff/perfsonar-user