perfSONAR User Q&A and Other Discussion

[Prannoy Sablok <>]

Hello Antoine,

Found the error below in the logs. Hope this helps.

Jul 26 00:06:08 dtn098 ticker INFO     Started

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR    Program threw an exception after 0:00:00.215390

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR    Exception: psycopg2.InternalError: AttributeError: module 'jsonschema' has no attribute 'Draft7Validator'

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR    CONTEXT:  Traceback (most recent call last):

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR      PL/Python function "json_validate", line 17, in <module>

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR        "required": [ "" ]

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR      PL/Python function "json_validate", line 1411, in json_validate

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR    PL/Python function "json_validate"

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR    PL/pgSQL function test_boot() line 35 at assignment

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR    SQL statement "SELECT test_boot()"

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR    PL/pgSQL function warm_boot() line 4 at PERFORM

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR    SQL statement "SELECT warm_boot()"

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR    PL/pgSQL function cold_boot() line 4 at PERFORM

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR    Traceback (most recent call last):

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR      File "/usr/lib/python3/dist-packages/pscheduler/saferun.py", line 76, in safe_run

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR        function()

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR      File "/usr/lib/pscheduler/daemons/ticker", line 170, in <lambda>

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR        pscheduler.safe_run(lambda: main_program())

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR      File "/usr/lib/pscheduler/daemons/ticker", line 132, in main_program

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR        cursor.execute("SELECT cold_boot()")

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR    psycopg2.InternalError: AttributeError: module 'jsonschema' has no attribute 'Draft7Validator'

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR    CONTEXT:  Traceback (most recent call last):

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR      PL/Python function "json_validate", line 17, in <module>

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR        "required": [ "" ]

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR      PL/Python function "json_validate", line 1411, in json_validate

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR    PL/Python function "json_validate"

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR    PL/pgSQL function test_boot() line 35 at assignment

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR    SQL statement "SELECT test_boot()"

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR    PL/pgSQL function warm_boot() line 4 at PERFORM

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR    SQL statement "SELECT warm_boot()"

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR    PL/pgSQL function cold_boot() line 4 at PERFORM

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR    Waiting 0.25 seconds before restarting

Jul 26 00:06:08 dtn098 safe_run/ticker ERROR    Restarting: ['/usr/lib/pscheduler/daemons/ticker', '--daemon', '--pid-file', '/var/run/pscheduler-server/ticker/pid', '--dsn', '@/etc/pscheduler/database/database-dsn']

Prannoy SABLOK

Technical Specialist

Singapore Advanced Research and Education Network (SingAREN)

    

M  +65 87492366|   E 

www.singaren.net.sg

On Tue, 10 Aug 2021 at 15:34, Prannoy Sablok <> wrote:

Hi Antoine,

Its a different machine that has just recently installed perfsonar-testpoint. Also, its just been updated to pscheduler version 4.4.0-1 but its still facing the same issue.

 

Kind Regards,

Prannoy SABLOK

Technical Specialist

Singapore Advanced Research and Education Network (SingAREN)

    

M  +65 87492366|   E 

www.singaren.net.sg

On Fri, 30 Jul 2021 at 18:24, Antoine Delvaux <> wrote:

Hi Prannoy,

Is it the same machine where you had the iperf3 issue previously?  Or is it a machine that was running fine before?  Or is it a new clean install?

Antoine.

Le 30 juil. 2021 à 04:18, Prannoy Sablok <> a écrit :

Hi Antoine ,

Below is the result from pscheduler version - 

pscheduler-account                       4.3.4-1

pscheduler-archiver-bitbucket            4.3.4-1

pscheduler-archiver-esmond               4.3.4-1

pscheduler-archiver-failer               4.3.4-1

pscheduler-archiver-http                 4.3.4-1

pscheduler-archiver-kafka                4.3.4-1

pscheduler-archiver-rabbitmq             4.3.4-1

pscheduler-archiver-syslog               4.3.4-1

pscheduler-bundle-full                   4.3.4-1

pscheduler-bundle-full-nos3

pscheduler-context-changefail            4.3.4-1

pscheduler-context-changenothing         4.3.4-1

pscheduler-context-linuxnns              4.3.4-1

pscheduler-context-linuxvrf              4.3.4-1

pscheduler-core                          4.3.4-1

pscheduler-docs                          4.3.4-1

pscheduler-jq-library                    4.3.4-1

pscheduler-server                        4.3.4-1

pscheduler-test-clock                    4.3.4-1

pscheduler-test-disk-to-disk             4.3.4-1

pscheduler-test-dns                      4.3.4-1

pscheduler-test-http                     4.3.4-1

pscheduler-test-idle                     4.3.4-1

pscheduler-test-idlebgm                  4.3.4-1

pscheduler-test-idleex                   4.3.4-1

pscheduler-test-latency                  4.3.4-1

pscheduler-test-latencybg                4.3.4-1

pscheduler-test-rtt                      4.3.4-1

pscheduler-test-s3throughput             4.3.4-1

pscheduler-test-simplestream             4.3.4-1

pscheduler-test-throughput               4.3.4-1

pscheduler-test-trace                    4.3.4-1

pscheduler-tool-curl                     4.3.4-1

pscheduler-tool-dnspy                    4.3.4-1

pscheduler-tool-ethr                     4.3.4-1

pscheduler-tool-iperf2                   4.3.4-1

pscheduler-tool-iperf3                   4.3.4-1

pscheduler-tool-nuttcp                   4.3.4-1

pscheduler-tool-owping                   4.3.4-1

pscheduler-tool-paris-traceroute         4.3.4-1

pscheduler-tool-ping                     4.3.4-1

pscheduler-tool-powstream                4.3.4-1

pscheduler-tool-psclock                  4.3.4-1

pscheduler-tool-psurl                    4.3.4-1

pscheduler-tool-s3-benchmark             4.3.4-1

pscheduler-tool-simplestreamer           4.3.4-1

pscheduler-tool-sleep                    4.3.4-1

pscheduler-tool-sleepbgm                 4.3.4-1

pscheduler-tool-snooze                   4.3.4-1

pscheduler-tool-tracepath                4.3.4-1

pscheduler-tool-traceroute               4.3.4-1

pscheduler-tool-twping                   4.3.4-1

The OS is Ubuntu 18.04.5 LTS.

Kind Regards,

Prannoy SABLOK

Technical Specialist

Singapore Advanced Research and Education Network (SingAREN)

    

M  +65 87492366|   E 

www.singaren.net.sg

On Thu, 29 Jul 2021 at 23:19, Antoine Delvaux <> wrote:

Hi Prannoy,

Can you please tell us which version of perfSONAR that is and what is the OS running on this system?

Thanks,

Antoine.

Le 29 juil. 2021 à 06:58, Prannoy Sablok <> a écrit :

Hi all,

I have been trying to run a pscheduler task on a server but it keeps throwing the following error -

~$ pscheduler task throughput dtn.singaren.net.sg

Could not find test throughput on server

This occurs with all pscheduler tasks. I checked service status and there seems to be a problem with the ticker service, as below - 

~$ systemctl status pscheduler-ticker.service

● pscheduler-ticker.service - pScheduler server - ticker

   Loaded: loaded (/lib/systemd/system/pscheduler-ticker.service; enabled; vendor preset: enabled)

   Active: activating (auto-restart) (Result: exit-code) since Thu 2021-07-29 00:43:44 CDT; 679ms ago

  Process: 27223 ExecStopPost=/bin/rm -rf /var/run/pscheduler-server/ticker (code=exited, status=0/SUCCESS)

  Process: 27187 ExecStart=/usr/lib/pscheduler/daemons/ticker --daemon --pid-file /var/run/pscheduler-server/ticker/pid --dsn @/et

  Process: 27178 ExecStartPre=/bin/sh -c if [ -r /etc/pscheduler/daemons/ticker.conf ]; then opts=$(sed -e 's/#.*$//' /etc/pschedu

  Process: 27177 ExecStartPre=/bin/mv /var/run/pscheduler-server/ticker/pid.build /var/run/pscheduler-server/ticker/pid (code=exit

  Process: 27176 ExecStartPre=/bin/chown pscheduler:pscheduler /var/run/pscheduler-server/ticker/pid.build (code=exited, status=0/

  Process: 27175 ExecStartPre=/usr/bin/touch -f /var/run/pscheduler-server/ticker/pid.build (code=exited, status=0/SUCCESS)

  Process: 27174 ExecStartPre=/bin/rm -f /var/run/pscheduler-server/ticker/pid.build (code=exited, status=0/SUCCESS)

  Process: 27173 ExecStartPre=/bin/chown pscheduler:pscheduler /var/run/pscheduler-server/ticker/tmp (code=exited, status=0/SUCCES

  Process: 27172 ExecStartPre=/bin/chmod 700 /var/run/pscheduler-server/ticker/tmp (code=exited, status=0/SUCCESS)

  Process: 27171 ExecStartPre=/bin/mkdir -p /var/run/pscheduler-server/ticker/tmp (code=exited, status=0/SUCCESS)

  Process: 27170 ExecStartPre=/bin/chmod 755 /var/run/pscheduler-server/ticker (code=exited, status=0/SUCCESS)

  Process: 27166 ExecStartPre=/bin/chown pscheduler:pscheduler /var/run/pscheduler-server/ticker (code=exited, status=0/SUCCESS)

  Process: 27158 ExecStartPre=/bin/mkdir -p /var/run/pscheduler-server/ticker (code=exited, status=0/SUCCESS)

 Main PID: 27187 (code=exited, status=1/FAILURE)

Upon further inspecting the pscheduler.log I found the following - 

$ tail -n 20 pscheduler.log

Jul 28 13:02:39 dtn098 safe_run/ticker ERROR    Traceback (most recent call last):

Jul 28 13:02:39 dtn098 safe_run/ticker ERROR      File "/usr/lib/python3/dist-packages/pscheduler/saferun.py", line 76, in safe_run

Jul 28 13:02:39 dtn098 safe_run/ticker ERROR        function()

Jul 28 13:02:39 dtn098 safe_run/ticker ERROR      File "/usr/lib/pscheduler/daemons/ticker", line 170, in <lambda>

Jul 28 13:02:39 dtn098 safe_run/ticker ERROR        pscheduler.safe_run(lambda: main_program())

Jul 28 13:02:39 dtn098 safe_run/ticker ERROR      File "/usr/lib/pscheduler/daemons/ticker", line 132, in main_program

Jul 28 13:02:39 dtn098 safe_run/ticker ERROR        cursor.execute("SELECT cold_boot()")

Jul 28 13:02:39 dtn098 safe_run/ticker ERROR    psycopg2.InternalError: AttributeError: module 'jsonschema' has no attribute 'Draft7Validator'

Jul 28 13:02:39 dtn098 safe_run/ticker ERROR    CONTEXT:  Traceback (most recent call last):

Jul 28 13:02:39 dtn098 safe_run/ticker ERROR      PL/Python function "json_validate", line 17, in <module>

Jul 28 13:02:39 dtn098 safe_run/ticker ERROR        "required": [ "" ]

Jul 28 13:02:39 dtn098 safe_run/ticker ERROR      PL/Python function "json_validate", line 1411, in json_validate

Jul 28 13:02:39 dtn098 safe_run/ticker ERROR    PL/Python function "json_validate"

Jul 28 13:02:39 dtn098 safe_run/ticker ERROR    PL/pgSQL function test_boot() line 35 at assignment

Jul 28 13:02:39 dtn098 safe_run/ticker ERROR    SQL statement "SELECT test_boot()"

Jul 28 13:02:39 dtn098 safe_run/ticker ERROR    PL/pgSQL function warm_boot() line 4 at PERFORM

Jul 28 13:02:39 dtn098 safe_run/ticker ERROR    SQL statement "SELECT warm_boot()"

Jul 28 13:02:39 dtn098 safe_run/ticker ERROR    PL/pgSQL function cold_boot() line 4 at PERFORM

Jul 28 13:02:39 dtn098 safe_run/ticker ERROR    Waiting 0.25 seconds before restarting

Jul 28 13:02:39 dtn098 safe_run/ticker ERROR    Restarting: ['/usr/lib/pscheduler/daemons/ticker', '--daemon', '--pid-file', '/var/run/pscheduler-server/ticker/pid', '--dsn', '@/etc/pscheduler/database/database-dsn']

It seems like there's an issue with a python dependency. Would the best way to fix this be to upgrade the jsonschema module with  "pip install --upgrade jsonschema" or something else?

Kind Regards,

Prannoy SABLOK

Technical Specialist

Singapore Advanced Research and Education Network (SingAREN)

    

M  +65 87492366|   E 

www.singaren.net.sg

<image001.jpg>--
To unsubscribe from this list: https://lists.internet2.edu/sympa/signoff/perfsonar-user

--
To unsubscribe from this list: https://lists.internet2.edu/sympa/signoff/perfsonar-user