perfSONAR User Q&A and Other Discussion

["Weed, Adam" <>]

I do see the tasks listed in the pscheduler monitor. The tests all stay in Running status, then vanish when the latencybg test completes. I'm unable to see the results of any runs.

You asked if adding a source with the dest worked, and it did not. I tried again with just the --dest, and this time it failed as well.

Discussing it with our networking people, it appears that they have been making changes, attempting to resolve some IPv6 issues. I've been shooting at a moving target.

I'll be on break until after the new year, and I'll continue testing after all network issues are resolved. I'll continue this thread if I still have issues.

My thanks to all of you for your assistance. Have a great holiday and new year!

Adam Weed
Database Programmer/Analyst

University of Missouri System - MOREnet

  |  (573) 884-2562

---

From: Andrew Lake <>
Sent: Friday, December 1, 2023 1:35 PM
To: Weed, Adam <>; Shawn McKee <>; Tim Chown <>
Cc: <>
Subject: Re: [perfsonar-user] IPv6 tests not completing

 

Hi,

In one of your earlier notes, you mentioned that your psconfig created tasks never make is to “Finished” status. Does that mean you are seeing them created? What status are they staying in? If you do a “ps auxw | grep powstream” do you see anything that looks like your psconfig-created  IPv6 tests? 

I also saw you mentioned "pscheduler task latencybg —dest ADDRESS” works. What happens if you specify the source address with —source? The psconfig created tests will be specifying the source explicitly, so maybe something is going wrong there. Is this a multi-interface host?

Thanks,

Andy

On November 30, 2023 at 6:01:00 PM, Weed, Adam () wrote:

I'm still unable to get IPv6 working from template.

Expected behavior: for the IPv4 latencybg tests, if I use a task link from the "pscheduler schedule" output, I can use "pscheduler result [task link]" and see all of the jobs of the task that have completed so far. Also, in "pscheduler monitor", the individual results of the IPv4 tasks move into Finished status and scroll up off of the screen. I expect this behavior from the IPv6 latencybg tests.

I've tried with host names and ' "ip-version" : 6 ', and I've tried with IPv6 addresses with and without an ' "ip-version" : 6 ' setting. I even tried IPv4 addresses with ' "ip-version" : 6 ', but I wasn't really expecting anything from it.

This is the simplest version of the file I could get to validate:

{

   "addresses" : {

      "host01" : {

         "address" : "[host name or IPv6 address]"

      },

      "host02" : {

         "address" : "[host name or IPv6 address]"

      }

   },

   "groups" : {

      "simplemesh" : {

         "addresses" : [

            {

               "name" : "host01"

            },

            {

               "name" : "host02"

            }

         ],

         "type" : "mesh"

      }

   },

   "tasks" : {

      "latency6" : {

         "group" : "simplemesh",

         "test" : "latencyipv6"

      }

   },

   "tests" : {

      "latencyipv6" : {

         "spec" : {

            "dest" : "{% address[1] %}",

            "source" : "{% address[0] %}"

         },

         "type" : "latencybg"

      }

   }

}

The "source" line of the "latencyipv6" test spec can be followed by a comma, then ' "ip-version" : 6 ' to formally add the IPv6 parameter. As I wrote, I've tested with and without it.

Would this file be expected to work? If not, why is it validating? If so, why isn't it working on our Ubuntu 20 systems?

Any additional insight would be appreciated.

Adam Weed
Database Programmer/Analyst

University of Missouri System - MOREnet

  |  (573) 884-2562

---

From: <> on behalf of Weed, Adam <>
Sent: Wednesday, November 29, 2023 8:03 AM
To: Shawn McKee <>; Tim Chown <>
Cc: <>
Subject: Re: [perfsonar-user] IPv6 tests not completing

 

It probably would have been helpful to mention that this is on Ubuntu 20. We don't have ncsd installed, all systems are running systemd-resolved.

I have attempted both host name and IPv6 address, neither has worked from template file.

My next step is to get rid of options until I find something that works (making the template as simple as possible). If I can get something working, I'll add parameters until it fails again, that should give me a bit to work from.

Once I've made the template as simple as possible (only two hosts, only one IPv6 test entry) I'll write again if it still fails.

OK, maybe I wasn't completely out of ideas.

Adam Weed
Database Programmer/Analyst

University of Missouri System - MOREnet

  |  (573) 884-2562

---

From: Shawn McKee <>
Sent: Wednesday, November 29, 2023 7:38 AM
To: Tim Chown <>
Cc: Weed, Adam <>; <>
Subject: Re: [perfsonar-user] IPv6 tests not completing

 

Thanks Tim,

I thought I would chime in with what I did earlier this week:

systemctl stop nscd

systemctl disable ncsd

yum install systemd-resolved

systemctl enable  systemd-resolved

systemctl start systemd-resolved

(Works on EL7 and EL9 (assuming yum is aliased to dnf))

Shawn

On Wed, Nov 29, 2023 at 2:28 PM Tim Chown <> wrote:

Hi,

We’ve encountered IPv6 test failures recently and discussed with the devs.  One cause appears to be cache behaviour issues with nscd, so a first step would be to disable that, and if you want cacheing then installed systemd/resolved.  If your test below is to an IPv6 literal address rather than host name, that of course would not use DNS.

It seems likely that a future version of perfSONAR will do both.

Tim

On 29 Nov 2023, at 01:42, Weed, Adam <> wrote:

You don't often get email from . Learn why this is important

I have IPv4 and IPv6 latencybg tests running from a psconfig template file. The difference between the two is "ip-version" : 4 and "ip-version": 6. The tests happen between the members of a mesh.

I can bring up the pscheduler monitor and see all of my tests running. The IPv4 tests work fine; the IPv6 tests never make it to Finished status.

I can retrieve the task link from the schedule, then when the task completes, check "pscheduler result [task_link]" and I get the response:

Run failed.

Error:

  No result was produced.

I have no errors in any of my log files. The limits are correct ("Application PASSES"). I can do a quick "pscheduler task latencybg --dest [IPv6 address]" from any of the systems I'm running to any of the other members in the mesh, and I can see runs complete in the monitor, with "pscheduler result" for the task link showing the results of the runs that have completed.

I don't know how many configuration adjustments I've made to try to get this to work, without success. Currently I've split up the hosts into two groups, removing host names from the addresses and just using IP addresses. Still, the IPv4 tests work fine and the IPv6 tests never finish.

Given that a manually-typed task works fine, and the template does not, I would assume a configuration issue somewhere. What's the best way to troubleshoot this? I'm out of ideas.

Adam Weed
Database Programmer/Analyst

University of Missouri System - MOREnet

  |  (573) 884-2562

--
To unsubscribe from this list: https://lists.internet2.edu/sympa/signoff/perfsonar-user

--
To unsubscribe from this list: https://lists.internet2.edu/sympa/signoff/perfsonar-user

--
To unsubscribe from this list: https://lists.internet2.edu/sympa/signoff/perfsonar-user