perfSONAR User Q&A and Other Discussion

["Yamamoto, Miguel" <>]

Hi Mark,

I tried to follow the exact format that was given on the jq tutorial to limit the bandwith but it gaves me a compile error on $max_bw. I also included the whole rewrite

    "#": "-------------------------------------------------------------------",

    "#": "REWRITE:  WHAT CHANGES ARE MADE TO INCOMING TASKS?                 ",

    "#": "                                                                   ",

    "#": "This is a jq transform that makes changes to incoming tasks prior  ",

    "#": "to limit enforcement.                                              ",

    "#": "-------------------------------------------------------------------",

 

    "rewrite": {

    "script": [

        "import \"pscheduler/iso8601\" as iso;",

 

        "# This does nothing but is recommended so the statements below",

        "# all begin with |.  (This makes editing easier.)",

            ".",

 

        "# Hold this for use later.",

        "| .test.type as $testtype",

 

            "# Make some tests run a minimum of 5 seconds",

            "| if ( [\"idle\", \"idlebgm\", \"latency\", \"latencybg\", \"throughput\" ]",

        "       | contains([$testtype]) )",

            "    and .test.spec.duration != null",

            "    and iso::duration_as_seconds(.test.spec.duration) < 5",

            "  then",

            "    .test.spec.duration = \"PT5S\"",

            "    | change(\"Bumped duration to 5-second minimum\")",

            "  else",

            "    .",

            "  end",

 

        "# The end.  (This takes care of the no-comma-at-end problem)"

        ]

    },

 

    "rewrite": {

        "script": [

            "500000000 as $max_bw",

            "| if .test.type == \"throughput\"",

            "    and (",

            "      (.test.spec.bandwidth == null)",

            "      or (.test.spec.bandwidth > $max_bw)",

            "    )",

            "    and (classifiers_has (\"friendlies\")",

            "  then",

            "    .test.spec.bandwidth = 500000000",

            "    | change(\"Limited to \\($max_bw)\")",

            "  else  . ",

            "  end"

        ]

    },

 

The error:

jq: error: syntax error, unexpected then (Unix shell quoting issues?) at <top-level>, line 30:

  then

jq: 1 compile error

 

Thanks,

Miguel

 

From: Mark Feit <>
Date: Tuesday, March 12, 2024 at 4:08 PM
To: Yamamoto, Miguel <>, <>
Subject: Re: Limit TCP Throughput Bandwidth to 500Mb

Yamamoto, Miguel writes: I was wondering how to limit the TCP Throughput Bandwidth of a 1G to just be 500Mb. This is what I have so far (taken from perfSonar documentation: The good news for you and the bad news for me is that there’s a mistake

ZjQcmQRYFpfptBannerStart

This Message Is From an External Sender

This message came from outside your organization.

 

ZjQcmQRYFpfptBannerEnd

Yamamoto, Miguel writes:

 

I was wondering how to limit the TCP Throughput Bandwidth of a 1G to just be 500Mb. This is what I have so far (taken from perfSonar documentation:

 

The good news for you and the bad news for me is that there’s a mistake in the documentation that’s been there since 2018.  The bandwidth parameter of the throughput test is an integer in bits per second rather than a string with an SI suffix.  The CLI takes numbers with SI suffixes as a convenience.  I’ve corrected that and the next feature release will reflect it.  Sorry to have given you a bum steer.  :-)

 

In the first limit, I’ve corrected what you sent to reflect what the docs should have said:

 

{

    "name": "throughput-low-bandwidth",

    "description": "Limit throughput test bandwidth",

    "type": "jq",

    "data": {

        "script": [

            "500000000 as $max_bandwidth",

            "| if .type == \"throughput\" and .spec.bandwidth > $max_bandwidth",

            "  then",

            "    \"Bandwidth is limited to \\($max_bandwidth)\"",

            "  else true",

            "  end"

        ]

    }

}

Note that this will still allow any test where a bandwidth is not specified, which could run faster than that.  You can add some additional logic to also reject tests where .spec.bandwidth is null (unspecified).

 

The second one looks like a limit but has script intended for the rewriter, which is a different part of the limit configuration and works in a different context as far as what’s provided and what functions (like change()) are available to call.  That said, the syntax error in what you sent is in the line that says  if .test.spec. == true; there can’t be a trailing dot there.   Using  the rewriter is the way to go if you want to still allow tests but modify them so they don’t run any faster than your preferred number.  The only down side is that people might not check the diagnostics and see that the speed was limited and go looking for a 500M-wide bottleneck in the network someplace.

 

We’ve done some Webinars that are available on YouTube that might be useful to you:

• The Limit System:  https://www.youtube.com/watch?v=UNItTorQqnw
• Jq Tutorial:  https://www.youtube.com/watch?v=FrT6R75M3BE

 

At about 1:04:00 in the jq tutorial, there’s a brief discussion about the rewriter.  I believe what we have in the docs on that to be substantially complete and correct as well.

 

Hope that helps.

 

--Mark