perfSONAR User Q&A and Other Discussion

["Curtis, Bruce" <>]

On Sep 27, 2018, at 8:42 AM, Andrew Lake <> wrote:

Hi,

I think ultimately we'll have to look at how variable substitution works in this scenario for templates, but I think you should be able to workaround the issue by replaceing {% scheduled_by_address %} with perfsonar2.it.ndsu.edu in /etc/perfsonar/psconfig/archives.d/esmond_local.json. That should use the hostname in the URL of the archive regardless of the test and get around any bracketing issues. 

Thanks,
Andy

I tried that but in the GUI I see.

 Error loading detailed test summary data: Internal Server Error

And I see this in /var/log/messages.

Oct 12 14:07:47 perfsonar2 journal: archiver WARNING  30056: Failed to archive https://localhost/pscheduler/tasks/241ea7a4-40ff-424d-934b-dd36c59afb67/runs/b25e904c-6a12-482e-8858-dcbcb643dddf to esmond: Archiver permanently abandoned registering test after 1 attempt(s): 400: Invalid JSON returned

Oct 12 14:07:47 perfsonar2 journal: archiver WARNING  30056: Gave up archiving https://localhost/pscheduler/tasks/241ea7a4-40ff-424d-934b-dd36c59afb67/runs/b25e904c-6a12-482e-8858-dcbcb643dddf to esmond

[curtis@perfsonar2 ~]$ cat /etc/perfsonar/psconfig/archives.d/esmond_local.json

{

    "archiver" : "esmond",

    "data" : {

        "url" : "https://perfsonar2.it.ndsu.edu/esmond/perfsonar/archive/",

        "measurement-agent" : "{% scheduled_by_address %}",

        "_auth-token" : "5cc930b9a98c1db5f390ba7b97d3f72e3bd3318e"

    }

}

[curtis@perfsonar2 ~]$ 

On Tue, Sep 25, 2018 at 12:32 PM Curtis, Bruce <> wrote:

On Sep 25, 2018, at 8:40 AM, Andrew Lake <> wrote:

Hi,

Excellent, thanks. If you go to https://134.129.91.125/pscheduler/tasks/ce10e64a-c97e-4c71-bd0a-3f6b584a9851/runs/b1ddc117-a31c-4962-b3ce-30e2ac42e9d5?pretty from the logs (with localhost replaced with the IP for those of us coming in remote) you can see more details about the archiving. My guess is the lack of square brackets in the URL https://2001:4930:91::125/esmond/perfsonar/archive/ is the issue. How are you defining your tests, through the tookit web interface or through a central JSON file?

Thanks,
Andy

Through the toolkit web interface.  The host was added to the test member list as perfsonar.it.ndsu.edu and I checked only the IPv6 box.

On Mon, Sep 24, 2018 at 5:36 PM Curtis, Bruce <> wrote:

On Sep 24, 2018, at 9:31 AM, Andrew Lake <> wrote:

Hi,

Would it be possible to add a few of the IPv6 tests back? It looks like some type of archiving issue. My guess is that its is filling the IPv6 address in the archive URL and either the archive is not listening on IPV6, the v6 filters are blocking 443 or its not formatting the URL properly. If you add the task back I can take a look and see what it does.

Thanks,
Andy

  There is still one IPv6 test running.

Sep 23 09:02:46 perfsonar2 journal: runner INFO     7136: With iperf3: throughput --source 2001:4930:91::125 --ip-version 6 --dest perfsonar.it.ndsu.edu --source-node [2001:4930:91::125] --dest-node perfsonar.it.ndsu.edu

Sep 23 09:02:59 perfsonar2 journal: runner INFO     7136: Run succeeded.

Sep 23 09:03:00 perfsonar2 journal: runner INFO     6297: Running https://134.129.91.125/pscheduler/tasks/7383fd5e-2684-40d6-b3de-d18ea265a54d/runs/d800e213-b89e-433d-a4dd-4167b0d3bb0f

Sep 23 09:03:00 perfsonar2 journal: runner INFO     6297: With traceroute: trace --dest hcc-ps02.unl.edu --ip-version 4 --source 134.129.91.125 --source-node 134.129.91.125

Sep 23 09:03:00 perfsonar2 journal: runner INFO     6297: Run succeeded.

Sep 23 09:03:05 perfsonar2 journal: archiver WARNING  3645: Failed to archive https://localhost/pscheduler/tasks/ac8395ac-ac03-4847-84d7-8e14b6884a92/runs/aeb65b92-974d-4678-b6cd-e7072166b2ae to esmond: Archiver permanently abandoned registering test after 1 attempt(s): 500: Invalid JSON returned

Sep 23 09:03:05 perfsonar2 journal: archiver WARNING  3645: Gave up archiving https://localhost/pscheduler/tasks/ac8395ac-ac03-4847-84d7-8e14b6884a92/runs/aeb65b92-974d-4678-b6cd-e7072166b2ae to esmond

Sep 23 09:03:09 perfsonar2 journal: runner INFO     6888: Running https://134.129.91.125/pscheduler/tasks/3665a1cb-2720-42c7-ab83-51da940a23e2/runs/42f852fe-f3fc-4f87-82b3-c7880f40ec77

Sep 23 09:03:09 perfsonar2 journal: runner INFO     6888: With traceroute: trace --dest perfsonar1.ccast.ndsu.edu --ip-version 4 --source 134.129.91.125 --source-node 134.129.91.125

Sep 23 09:03:09 perfsonar2 journal: runner INFO     6888: Run succeeded.

Sep 23 09:04:18 perfsonar2 journal: runner INFO     6300: Running https://[2001:4930:91::125]/pscheduler/tasks/ce10e64a-c97e-4c71-bd0a-3f6b584a9851/runs/b1ddc117-a31c-4962-b3ce-30e2ac42e9d5

Sep 23 09:04:18 perfsonar2 journal: runner INFO     6300: With traceroute: trace --dest perfsonar.it.ndsu.edu --ip-version 6 --source 2001:4930:91::125 --source-node [2001:4930:91::125]

Sep 23 09:04:18 perfsonar2 journal: runner INFO     6300: Run succeeded.

Sep 23 09:04:18 perfsonar2 journal: archiver WARNING  3647: Failed to archive https://localhost/pscheduler/tasks/ce10e64a-c97e-4c71-bd0a-3f6b584a9851/runs/b1ddc117-a31c-4962-b3ce-30e2ac42e9d5 to esmond: Archiver permanently abandoned registering test after 1 attempt(s): 500: Invalid JSON returned

Sep 23 09:04:18 perfsonar2 journal: archiver WARNING  3647: Gave up archiving https://localhost/pscheduler/tasks/ce10e64a-c97e-4c71-bd0a-3f6b584a9851/runs/b1ddc117-a31c-4962-b3ce-30e2ac42e9d5 to esmond

On Fri, Sep 21, 2018 at 2:58 PM Curtis, Bruce <> wrote:

  We had some problems with IPv6 on our perfsonar nodes so we changed our tests to only use IPv4.

  I thought the problems had been fixed on newer versions but we couldn’t upgrade until we upgraded to Centos 7.

  We have not upgraded to Centos 7 and Perfsonar 4.1 but are still seeing issues with IPv6 tests.


Here is an example from /var/log/messages. It looks like the test succeeds but is not archived.

Sep 20 20:36:49 perfsonar2 journal: runner INFO     2547: With iperf3: throughput --source 2001:4930:91::125 --ip-version 6 --dest perfsonar1.ccast.ndsu.edu --source-node [2001:4930:91::125] --dest-node perfsonar1.ccast.ndsu.edu
Sep 20 20:37:06 perfsonar2 journal: runner INFO     2547: Run succeeded.
Sep 20 20:37:23 perfsonar2 journal: archiver WARNING  207: Failed to archive https://localhost/pscheduler/tasks/4d2fdd27-3278-450b-90da-e212c8fbe525/runs/64524ab9-7d46-40fe-ae5e-d333ebd9a932 to esmond: Archiver permanently abandoned registering test after 1 attempt(s): 500: Invalid JSON returned
Sep 20 20:37:23 perfsonar2 journal: archiver WARNING  207: Gave up archiving https://localhost/pscheduler/tasks/4d2fdd27-3278-450b-90da-e212c8fbe525/runs/64524ab9-7d46-40fe-ae5e-d333ebd9a932 to esmond


---
Bruce Curtis                         
Certified NetAnalyst II                701-231-8527
North Dakota State University       

---
Bruce Curtis                        
Certified NetAnalyst II                701-231-8527
North Dakota State University        

---
Bruce Curtis                        
Certified NetAnalyst II                701-231-8527
North Dakota State University        

---
Bruce Curtis                        
Certified NetAnalyst II                701-231-8527
North Dakota State University