perfSONAR User Q&A and Other Discussion

[Joël COSTAMAGNA <>]

Hello,

yes I tried to remove the grids I created and create new one using the example, I have no error when creating the grid but I still see the same error and I think the agent dies before loading the config:

When I reload the agent with

systemctl restart psconfig-maddash-agent

I get this error:

2020/03/02 15:14:40 ERROR pid=2576 prog=main::__ANON__ line=131 guid=89B14436-5C98-11EA-859E-9F47C0EB086C
msg=Died: Can't use an undefined value as an ARRAY reference at /usr/lib/perfsonar/bin/../lib/perfSONAR_PS/PSConfig/MaDDash/Agent.pm line 245.

regards,

joel costamagna

Le 02/03/2020 à 14:27, Garnizov, Ivan a écrit :

Hello Joel,

 

With regards the MaDDash grids please check these guidelines: http://docs.perfsonar.net/psconfig_maddash_agent.html#configuring-grids

 

Please report back any issues you come across with the execution of the commands, including any unexpected results.

 

 

Regards,

Ivan Garnizov

 

GEANT WP6T3: pS development team

GEANT WP7T1: pS deployments GN Operations

GEANT WP9T2: Software governance in GEANT

 

 

 

Von: [] Im Auftrag von Joël COSTAMAGNA
Gesendet: Montag, 2. März 2020 11:01
An: Szymon Trocha
Cc:
Betreff: Re: [perfsonar-user] help with maddash-config

 

I used psconfig web admin to create the template, so I used the URL to the json as the remote, here it is:

{
  "archives": {
    "host-additional-archive0": {
      "archiver": "esmond",
      "data": {
        "url": "https://172.16.XX.12/esmond/perfsonar/archive",
        "measurement-agent": "{% scheduled_by_address %}"
      }
    }
  },
  "addresses": {
    "172.16.XX.13": {
      "address": "172.16.XX.13",
      "host": "172.16.XX.13",
      "_meta": {
        "display-name": "probe number one",
        "display-url": "https://172.16.XX.13/toolkit/"
      }
    },
    "172.16.XX.11": {
      "address": "172.16.XX.11",
      "host": "172.16.XX.11",
      "_meta": {
        "display-name": "main probe",
        "display-url": "https://172.16.XX.11/toolkit/"
      }
    },
    "172.16.XX.14": {
      "address": "172.16.XX.14",
      "host": "172.16.XX.14",
      "_meta": {
        "display-name": "probe number 2",
        "display-url": "https://172.16.XX.14/toolkit/"
      }
    }
  },
  "groups": {
    "throughput": {
      "type": "disjoint",
      "a-addresses": [
        {
          "name": "172.16.XX.13"
        },
        {
          "name": "172.16.XX.11"
        },
        {
          "name": "172.16.XX.14"
        }
      ],
      "b-addresses": [
        {
          "name": "172.16.XX.13"
        },
        {
          "name": "172.16.XX.11"
        },
        {
          "name": "172.16.XX.14"
        }
      ]
    },
    "latency": {
      "type": "disjoint",
      "b-addresses": [
        {
          "name": "172.16.XX.13"
        },
        {
          "name": "172.16.XX.11"
        },
        {
          "name": "172.16.XX.14"
        }
      ],
      "a-addresses": [
        {
          "name": "172.16.XX.13"
        },
        {
          "name": "172.16.XX.11"
        },
        {
          "name": "172.16.XX.14"
        }
      ]
    }
  },
  "tests": {
    "throughput": {
      "type": "throughput",
      "spec": {
        "duration": "PT20S",
        "source": "{% address[0] %}",
        "dest": "{% address[1] %}",
        "probe-type": "tcp",
        "omit": "PT5S",
        "ip-version": 4
      }
    },
    "latency": {
      "type": "latencybg",
      "spec": {
        "source": "{% address[0] %}",
        "dest": "{% address[1] %}",
        "packet-interval": 0.1,
        "bucket-width": 0.001,
        "packet-count": 600,
        "ip-version": 4
      }
    }
  },
  "schedules": {
    "sched-0": {
      "repeat": "PT10M",
      "sliprand": true
    }
  },
  "tasks": {
    "throughput": {
      "group": "throughput",
      "test": "throughput",
      "archives": [
        "config-archive0"
      ],
      "_meta": {
        "display-name": "throughput"
      },
      "schedule": "sched-0",
      "tools": [
        "bwctliperf3",
        "iperf3"
      ]
    },
    "latency": {
      "group": "latency",
      "test": "latency",
      "archives": [
        "config-archive0"
      ],
      "_meta": {
        "display-name": "latency"
      },
      "tools": [
        "owping"
      ]
    }
  },
  "_meta": {
    "display-name": "test config"
  },
  "hosts": {
    "172.16.XX.13": {
      "archives": [
        "host-additional-archive0"
      ]
    },
    "172.16.XX.11": {
      "archives": [
        "host-additional-archive0"
      ]
    },
    "172.16.XX.14": {
      "archives": [
        "host-additional-archive0"
      ]
    }
  }
}

 

and here is the yaml file, I didn't edit it, but I see the grids[] is empty while it shows them in psconfig maddash-grid list, do I need to add them manually?

cat /etc/maddash/maddash-server/maddash.yaml
checks: {}
dashboards: []
database: /var/lib/maddash/
grids: []
groupMembers: []
groups:
  Dashboard_1_-_latency_task-col:
    - probe1
    - probe2
    - toolkit
  Dashboard_1_-_latency_task-row:
    - probe1
    - probe2
    - toolkit
http:
  port: 8881
reports:
  -
    id: loss_mesh_report
    rule:
      rules:
        -
          match:
            status: 3
            type: status
          problem:
            category: CONFIGURATION
            message: Grid is down
            severity: 3
            solutions:
              - 'If you just configured this grid, you may just need to wait as it takes a few minutes for loss data to populate'
              - Verify maddash is configured properly. Look in the files under /var/log/maddash/ for any errors. Things to look for are incorrect paths to checks or connection errors.
              - Verify that the pSConfig MaDDash Agent has run recently and you are looking at an accurate test configuration
              - Verify that your measurement archive(s) are running
              - Verify no firewall is blocking maddash from reaching your measurement archive(s)
              - Verify your hosts are downloading the pSConfig file and that there are tasks listed when you run 'psconfig pscheduler-tasks'
              - Verify that pSConfig pScheduler Agent is running ('systemctl status psconfig-pscheduler-agent')
              - Verify your hosts are able to reach their configured measurement archive and that there are no errors in /var/log/perfsonar/psconfig-pscheduler-agent.log
          selector:
            type: grid
          type: rule
        -
          match:
            status: 0
            type: status
          problem:
            category: PERFORMANCE
            message: Entire grid has OK status
            severity: 0
          selector:
            type: grid
          type: rule
        -
          rule:
            rules:
              -
                match:
                  status: 3
                  type: status
                problem:
                  category: CONFIGURATION
                  message: Site is down
                  severity: 3
                  solutions:
                    - Verify the host is up
                    - 'If recently added to the configuration, verify the pSConfig file has been downloaded by the end-hosts since the update.'
                    - 'If recently removed from the configuration, verify that the pSConfig MaDDash Agent has run recently and you are looking at an accurate test configuration'
                    - Verify the local and remote sites allow access to TCP port 861 and UDP ports 8760-9960
                selector:
                  type: site
                type: rule
              -
                match:
                  status: 3
                  type: status
                problem:
                  category: CONFIGURATION
                  message: Unable to run and/or query any outgoing one-way delay tests.
                  severity: 3
                  solutions:
                    - Verify you are not blocking any of the required outgoing OWAMP ports in your firewall
                    - Verify the remote sites allow your host to access UDP ports 8760-9960
                selector:
                  type: row
                type: rule
              -
                match:
                  status: 3
                  type: status
                problem:
                  category: CONFIGURATION
                  message: Unable to run and/or query any incoming one-way delay tests.
                  severity: 3
                  solutions:
                    - Verify your host and router firewalls are allowing UDP ports 8760-9960
                selector:
                  type: column
                type: rule
              -
                rules:
                  -
                    match:
                      statuses:
                        - 0.0
                        - .5
                        - 1.0
                        - -1.0
                      threshold: .6
                      type: statusWeightedThreshold
                    problem:
                      category: PERFORMANCE
                      message: Outgoing loss is above warning or critical thresholds to a majority of sites
                      severity: 2
                    selector:
                      type: row
                    type: rule
                  -
                    match:
                      statuses:
                        - 0.0
                        - .5
                        - 1.0
                        - -1.0
                      threshold: .6
                      type: statusWeightedThreshold
                    problem:
                      category: PERFORMANCE
                      message: Incoming loss is above warning or critical thresholds to a majority of sites
                      severity: 2
                    selector:
                      type: column
                    type: rule
                type: matchAll
            type: matchFirst
          type: forEachSite
      type: matchFirst
serverHost: localhost

 

Regards,

joel costamagna

 

Le 02/03/2020 à 09:07, Szymon Trocha a écrit :

Hi,

 

W dniu 02.03.2020 o 09:31, Joël COSTAMAGNA pisze:

Hi,

this was a stupid error, I used http for the remote add while it was only available on https. Now it has the right URL, the tests run and I can see the results in the archive, but I still don't see my grid, I have an error in the /var/log/maddash/psconfig-maddash-agent.log:

2020/03/02 07:37:38 ERROR pid=19963 prog=main::__ANON__ line=131 guid=B10490FA-5C58-11EA-B0FC-8A8BC0EB086C

msg=Died: Can't use an undefined value as an ARRAY reference at /usr/lib/perfsonar/bin/../lib/perfSONAR_PS/PSConfig/MaDDash/Agent.pm line 245.

I tried to look at the code but I don't know perl and I don't understand where the problem come from

On the central server the Maddash agent is running and is responsible for automatic generation of MAddash config to /etc/maddash/maddash-server/maddash.yaml.

To verify it correctly points to the config run in your central server:

$ psconfig remote list

It should list the path to your local .json psconfig template ffile with the header "=== MaDDash Agent ==="

You may also want to share the generated maddash.yaml file. Did you modify it manually?

Regards,

--

Szymon Trocha
Poznań Supercomputing & Networking Center

General NOC phone +48 61-858-2015 | noc.pcss.pl
Personal desk phone +48 61-858-2022

Wysłaliśmy do Ciebie ten e-mail w odpowiedzi na Twoje zapytanie lub w związku z oferowaną usługą. Przesłanie korespondencji do Centrum Zarządzania PCSS lub zgłoszenie telefoniczne jest równoznaczne z wyrażeniem zgody na przetwarzanie danych osobowych przez Instytut Chemii Bioorganicznej Polskiej Akademii Nauk w Poznaniu adres: ul. Z. Noskowskiego 12/14, 61-704 Poznań. Szczegółowe informacje znajdują się w naszej Polityce prywatności. | This message has been sent as a part of communication with PSNC NOC or your service request sent to us. For more information read our Privacy Policy.