perfSONAR User Q&A and Other Discussion

[matt fetting <>]

"pscheduler schedule +PT1H" does show all of the pending tests I would expect to see. 

I am running a tail -f /var/log/perfsonar/* so I can see all logs, and have nothing new that's really jumping out since the reboot of the two nodes this morning. However, still don't have data in MaDDash. FWIW, I generated by MaDDash YAML from the central mesh config I am publishing from my MA, which is non participatory. 

Example error from MaDDash:

Unable to find any tests with data in the given time range where source is host1 and destination is host2

Further error when I drill into that test detail (funny, because I dont have filtering between these hosts)

host1

Unable to run and/or query any outgoing one-way delay tests.

Category:CONFIGURATION

Potential Solutions:

• Verify you are not blocking any of the required outgoing OWAMP ports in your firewall
• Verify the remote sites allow your host to access UDP ports 8760-9960

host2

Unable to run and/or query any incoming one-way delay tests.

Category:CONFIGURATION

Potential Solutions:

• Verify your host and router firewalls are allowing UDP ports 8760-9960

On Thu, Jan 25, 2018 at 12:57 PM, David Szydloski <> wrote:

Matt,

You may want to check the " sudo pscheduler schedule +PT1H" to make sure the tests are being scheduled correctly. 

Aside from that, check the status of the cassandra service + logs, esmond logs as well as the postgres logs. If the tests are running correctly but not showing data you could still have an issue with the daemons getting the data into the db.

-D

On Thu, Jan 25, 2018 at 11:51 AM, matt fetting <> wrote:

Thanks David. The pscheduler ping command passed with an "is alive" result. However, the troubleshoot command did produce a failure (see below):

[root@host2]# pscheduler troubleshoot

Performing basic troubleshooting of localhost.

Checking for pScheduler on localhost... OK.

Idle test on localhost.... 9 seconds...404 Resource Not found.

 Failed.

  Did not get a result: Resource Not found.

I rebooted both nodes for grins, and now that troubleshoot command returns a "pScheduler appears to be functioning normally." Not sure if the services were in some broken state based on my past troubleshooting (service restarts, etc). I do not see the 400 errors in my logs anymore, but still don't have any data. Will give it a little time and perhaps start a new thread related to the lack of data if that issue persists. 

mdf

On Thu, Jan 25, 2018 at 11:58 AM, David Szydloski <> wrote:

Matt,

Depending on what information you are trying to get, running toolkit on Node A would likely be sufficient as it should be able to give you one-way latency and, say, TCP throughput both ways as long as minimal tools like perfsonar-testpoint are installed on Node B. (In my experience, this setup is the easiest to start with to familiar yourself with perfSONAR)

1)Have you already tried to see if "sudo pscheduler ping localhost [or a host IP]" and "sudo pscheduler troubleshoot"? Either of those should point you to issues with pscheduler setup. Node A should be able to see pScheduler on itself and Node B.

2) " sudo pscheduler schedule +PT5H" will show you what tests are scheduled based on your test configuration. This is was really helpful to me in troubleshooting problems I had with throughput tests since a pScheduler issue on the remote end was keeping the tests from being scheduled at all.

3) Running "sudo pscheduler debug [whatever service is causing an issue]" will give you more verbose logging in the /var/log/pscheduler/pscheduler.log  

4) trying to run a test manually in debug mode via "sudo pscheduler task --debug [whatever task you want to run]" should give you some more good output to dig through.

Hope that helps,

D

On Thu, Jan 25, 2018 at 10:37 AM, matt fetting <> wrote:

I have 3 centos7 toolkit hosts deployed from the netinstall iso, two of which are intended to run tests and one of which is intended to be a measurement archive. Starting very simple with just owamp, traceroute, and iperf tests between these two testing hosts. My understanding about perfSonar is partial, so please excuse any ignorance here. 

My MaDDash dashboard has some errors around not being able to find data for some tests, and it's been up/running for over 24 hours now without changes. The scheduled tests are frequent and should be showing data by now. pScheduler is running on both hosts, and there is no firewall in between them. I can confirm that with certainty because I am the operator. Since these are internal, I've actually stopped firewalld to make sure there isn't any unintended filtering there. selinux is disabled. There are variations of this error message in the meshconfig-agent.log: 

2018/01/24 10:18:12 (3859) WARN> perfsonar_meshconfig_agent:430 main:: - Problem determining which pscheduler to submit test to for creation, skipping test throughput(host1->host2): 400 BAD REQUEST: Can't find pScheduler or BWCTL on host2

Can someone help get me started on where to look for resolving this? Past research into the list/google didn't prove fruitful. 

Thanks

mdf

--

David Szydloski

Core Deployment Engineer

VidScale, Inc.

--

David Szydloski

Core Deployment Engineer

VidScale, Inc.