perfSONAR User Q&A and Other Discussion

["Uhl, George D. (GSFC-423.0)[SGT INC]" <>]

Hi Andy,

 

I can’t seem to get the syntax right to make this work.  Can you review please and tell me what I might be missing?  Do I need to specify a lead-bind-address for the single-ended pS host?  FWIW, I’ve removed 172.16.0.0/12 as a bogon in my pscheduler limits.conf file.

 

Thanks,

George

 

I’m getting this error in /var/log/perfsonar/psconfig-pscheduler-agent.log on my single-ended pS node:

 

2019/08/17 09:42:44 WARN pid=2706 prog=perfSONAR_PS::PSConfig::PScheduler::Agent::_run_end line=218 guid=1D0482FA-C0D3-11E9-9727-270833D31611 msg=Problem determining which pscheduler to submit test to for creation, skipping test throughput(esdis-pt1-10g.eosdis.nasa.gov->172.25.8.17): HTTP/1.1 400 BAD REQUEST

: {u'reverse': True, u'single-ended': True, u'dest': u'172.25.8.17', u'mss': u'1448', u'source': u'esdis-pt1-10g.eosdis.nasa.gov', u'ip-version': 4, u'duration': u'PT30S', u'schema': 2} is not valid under any of the given schemas

 

The excerpts from my psconfig json file that specify the single ended throughput tests:

 

    "esdis-pt1-10g.eosdis.nasa.gov": {

      "_meta": {

        "display-name": "esdis-ps"

      },

      "address": "esdis-pt1-10g.eosdis.nasa.gov",

      "host": "esdis-ps"

    },

     "172.25.8.17": {

       "_meta": {

         "display-name": "aws-max-tvif"

       },

       "address": "172.25.8.17",

       "no-agent":true

       "host": "aws-max-tvif"

     },

  "groups": {

     "aws": {

       "a-addresses": [

         {

           "name": "esdis-pt1-10g.eosdis.nasa.gov"

         }

       ],

       "b-addresses": [

         {

           "name": "172.25.8.17"

         }

       ],

      "unidirectional": true,

       "type": "disjoint"

    },

  },

  "hosts": {

    "esdis-ps": {

      "_meta": {

        "display-name": "esdis-ps",

        "organization-display-name": "NASA GSFC"

      }

    },

     "aws-max-tvif": {

       "_meta": {

         "display-name": "aws-max-tvif",

         "organization-display-name": "AWS"

       },

      "no-agent": true

    },

  },

  "tasks": {

     "Open_EBNet_AWS_IPv4_Throughput_Forward": {

       "_meta": {

         "display-name": "Open EBNet AWS IPv4 Throughput Forward"

       },

       "archives": [

         "endpsma01.eosdis.nasa.gov"

       ],

       "group": "aws",

       "schedule": "schedule_1",

       "test": "throughput_singleended_forward",

       "tools": [

         "iperf3"

       ]

     },

     "Open_EBNet_AWS_IPv4_Throughput_Reverse": {

       "_meta": {

         "display-name": "Open EBNet AWS IPv4 Throughput Reverse"

       },

       "archives": [

         "endpsma01.eosdis.nasa.gov"

       ],

       "group": "aws",

       "schedule": "schedule_1",

       "test": "throughput_singleended_reverse",

       "tools": [

         "iperf3"

       ]

     },

  },

  "tests": {

     "throughput_singleended_forward": {

       "spec": {

         "schema": 2,

         "ip-version": 4,

         "source": "{% address[0] %}",

         "dest": "{% address[1] %}",

         "duration": "PT30S",

         "single-ended": true,

         "mss": "1448"

       },

       "type": "throughput"

     },

     "throughput_singleended_reverse": {

       "spec": {

         "schema": 2,

         "ip-version": 4,

         "source": "{% address[0] %}",

         "dest": "{% address[1] %}",

         "duration": "PT30S",

         "single-ended": true,

         "reverse": true,

         "mss": "1448"

      },

      "type": "throughput"

    },

  }

 

 

From: Andrew Lake <>
Date: Wednesday, August 14, 2019 at 9:53 AM
To: "" <>, "George.D.Uhl" <>
Subject: [EXTERNAL] Re: Defining single-ended throughput tests inside a psconfig template

 

Hi George,

 

Yes, you can use all the options of the throughput test in the psconfig template. Just to distinguish for everyone listening, the --single-ended option allows you to run a test to a server that doesn’t have pscheduler running but has an iperf server launched. This is different from the “no-agent” case in that “no-agent” means there is not a psconfig agent reading the template, but throughput tests will still assume pscheduler is on a no-agent host.

 

The tricky part here is making sure the test only uses the options when it needs to, which can also be done but may be less obvious and can be more or less complex depending on your test setup. The simplest option in terms of configuration is shown  in the example here: https://gist.github.com/arlake228/2ea2ee99440f5312cbc2c1e0a2b03439. This example makes a couple assumptions. One is that one side of your test is all your non-perfsonar hosts. It also splits the forward and reverse into separate tests which means they will get drawn in separate grids.If you’re ok with that it should just work. 

 

If you want to mix the non-perfsonar hosts in with perfsonar hosts and/or you want forward and reverse in the same grid, you need to get fancier with JQ. You’ll need to tag the non-perfsonar address objects as such and then use jq to set the source, dest, single-ended and reverse parameters based on the presence of no-agent and the non-perfsonar tag...and whether its on the source of dest. It’s probably doable but will take some creative jq’ing , so will only go down that road if you need me too.

 

Thanks,

Andy

 

 

On August 13, 2019 at 6:42:41 PM, Uhl, George D. (GSFC-423.0)[SGT INC] () wrote:

Hi,

 

I have a non-perfsonar test host that I am able run a single-ended throughput test with from one of my managed hosts by using pscheduler on the CLI.  I am able to test in the reverse direction using the –reverse option although at the cost of getting non-reported retransmits and current window values.  From the documentation I can’t determine if single-ended throughput testing is supported other than using the CLI.   Is it possible to include single-ended tests in a psconfig template?  If so, could the test results be stored in a measurement archive such that the throughput rates are stored in both the forward and reverse directions?  Would I be able to extract this data using a maddash dashboard?

 

Thanks,

George