Skip to Content.
Sympa Menu

perfsonar-user - Re: [perfsonar-user] Testpoints measurement failure

Subject: perfSONAR User Q&A and Other Discussion

List archive

Re: [perfsonar-user] Testpoints measurement failure


Chronological Thread 
  • From: Tadeja Saje <>
  • To: Szymon Trocha <>
  • Cc:
  • Subject: Re: [perfsonar-user] Testpoints measurement failure
  • Date: Tue, 5 Nov 2019 12:02:35 +0100

[root@mp-ijs-pf2 ~]# cat /etc/pscheduler/limits.conf
{
    "#": "-------------------------------------------------------------------",
    "#": "                 PSCHEDULER LIMIT CONFIGURATION                    ",
    "#": "                                                                   ",
    "#": "This file is a sample which contains fairly sane defaults.  It     ",
    "#": "should be customized according to the needs of your site.          ",
    "#": "-------------------------------------------------------------------",

    "schema": 3,

    "#": "-------------------------------------------------------------------",
    "#": "IDENTIFIERS:  WHO'S ASKING?                                        ",
    "#": "                                                                   ",
    "#": "These identify who's asking to run the test.  One requester can    ",
    "#": "map to zero or more identifiers.                                   ",
    "#": "-------------------------------------------------------------------",

    "identifiers": [
    {
        "name": "everybody",
        "description": "An identifier that always identifies",
        "type": "always",
        "data": { }
    },
    {
        "name": "local-interfaces",
        "description": "Requests coming from this system",
        "type": "localif",
        "data": { }
    },
        {
            "#": "NOTE: This only works if the host can resolve DNS",
            "#": "on the public Internet.",

            "name": "bogons",
            "description": "Bogon/Martian IPv4 addresses without private networks",
            "type": "ip-cymru-bogon",
            "data": {
                "exclude": [
                    "10.0.0.0/8",
                    "127.0.0.0/8",
                    "172.16.0.0/12",
                    "192.168.0.0/16",
            "169.254.0.0/16",
                    "::/8",
            "fe80::/10"
                    ],
                "timeout": "PT1S",
                "fail-result": false
            }
        },
    {
        "name": "hackers",
        "description": "Blocks that have tried to hack us (actually TEST-NET-2)",
        "type": "ip-cidr-list",
        "data": {
        "cidrs": [ "198.51.100.0/24" ]
        }
    }
    ],


    "#": "-------------------------------------------------------------------",
    "#": "CLASSIFIERS:  HOW DO WE CLASSIFY THE IDENTIFIERS?                  ",
    "#": "                                                                   ",
    "#": "These collect identifiers into groups.                             ",
    "#": "-------------------------------------------------------------------",

    "classifiers": [
    {
        "name": "default",
        "description": "Everybody",
        "identifiers": [ "everybody" ]
    },
    {
        "name": "friendlies",
        "description": "Identifiers we find friendly",
        "identifiers": [ "local-interfaces" ]
    },
    {
        "name": "hostiles",
        "description": "Identifiers we find unfriendly",
        "identifiers": [ "hackers", "bogons" ]
    }
    ],


    "#": "-------------------------------------------------------------------",
    "#": "REWRITE:  WHAT CHANGES ARE MADE TO INCOMING TASKS?                 ",
    "#": "                                                                   ",
    "#": "This is a jq transform that makes changes to incoming tasks prior  ",
    "#": "to limit enforcement.                                              ",
    "#": "-------------------------------------------------------------------",

    "rewrite": {
    "script": [
        "import \"pscheduler/iso8601\" as iso;",

        "# This does nothing but is recommended so the statements below",
        "# all begin with |.  (This makes editing easier.)",
            ".",

        "# Hold this for use later.",
        "| .test.type as $testtype",

            "# Make some tests run a minimum of 5 seconds",
            "| if ( [\"idle\", \"idlebgm\", \"idleex\", \"latency\", \"latencybg\", \"throughput\" ]",
        "       | contains([$testtype]) )",
            "    and .test.spec.duration != null",
            "    and iso::duration_as_seconds(.test.spec.duration) < 5",
            "  then",
            "    .test.spec.duration = \"PT5S\"",
            "    | change(\"Bumped duration to 5-second minimum\")",
            "  else",
            "    .",
            "  end",

        "# The end.  (This takes care of the no-comma-at-end problem)"
        ]
    },



    "#": "-------------------------------------------------------------------",
    "#": "LIMITS:  WHAT ARE THE RESTRICTIONS?                                ",
    "#": "                                                                   ",
    "#": "These are comparisons made against the type of test being proposed,",
    "#": "the paramaters for the run and when it is proposed to be run.      ",
    "#": "-------------------------------------------------------------------",

    "limits": [
        {
        "name": "always",
        "description": "Always passes",
        "type": "pass-fail",
        "data": {
            "pass": true
        }
    },
        {
        "name": "never",
        "description": "Always fails",
        "type": "pass-fail",
        "data": {
            "pass": false
        }
    },
    {
            "#": "This prevents denial of service by scheduling long tasks.",
        "name": "idleex-default",
        "description": "Default limits for idleex",
        "type": "test",
        "data": {
        "test": "idleex",
        "limit": {
            "duration": {
                        "range": {
                            "lower": "PT1S",
                            "upper": "PT2S"
                        }
                     }
        }
        }
    },

    {
        "name": "innocuous-tests",
        "description": "Tests considered harmless",
        "type": "test-type",
        "data": {
            "#": "Resource hogs, which will be inverted below",
        "types": [ "throughput", "idleex" ]
        },
        "invert": true
    },
    {
        "name": "throughput-default-time",
        "description": "Throughput time limits",
        "type": "test",
        "data": {
        "test": "throughput",
        "limit": {
            "duration": {
                        "range": {
                            "lower": "PT5S",
                            "upper": "PT60S"
                        }
                     },
                     "udp":{
                         "match":false
                     }
        }
        }
    },
        {
            "name": "throughput-default-udp",
            "description": "Throughput UDP limits",
            "type": "test",
            "data": {
                "test": "throughput",
                "limit": {
                    "bandwidth": {
                        "range": {
                            "lower": "1",
                            "upper": "50M"
                        }
                     },
                     "duration": {
                        "range": {
                            "lower": "PT5S",
                            "upper": "PT60S"
                        }
                     },
                     "udp":{
                         "match":true
                     }
                }
            }
        }
    ],


    "#": "-------------------------------------------------------------------",
    "#": "APPLICATIONS:  TO WHOM DO WE APPLY THE LIMITS?                     ",
    "#": "                                                                   ",
    "#": "These are processed in order until one passes all of the           ",
    "#": "requirements.  The run will be rejected if one fails with          ",
    "#": "stop-on-failure set to true or none of them passes.                ",
    "#": "-------------------------------------------------------------------",

    "applications": [
    {
        "description": "Hosts we don't want running any tests",
        "classifier": "hostiles",
        "apply": [
        { "require": "all", "limits": [ "never" ] }
        ],
        "stop-on-failure": true
    },
    {
        "description": "Hosts we trust to do everything",
        "classifier": "friendlies",
        "apply": [
        { "require": "all", "limits": [ "always" ] }
        ]
    },
    {
        "description": "Defaults applied to non-friendly hosts",
        "classifier": "default",
        "apply": [
        {
            "require": "any",
            "limits": [
            "innocuous-tests",
                        "throughput-default-time",
                        "throughput-default-udp",
            "idleex-default"
            ]
        }
        ]
    }
    ],


    "#": "-------------------------------------------------------------------",
    "#": "PRIORITY:  HOW DO WE PRIORITIZE RUNS OF TASKS?                     ",
    "#": "                                                                   ",
    "#": "This is a jq transform that examines a proposed run of a task and  ",
    "#": "produces an integer value indicating its priority.  If this is not ",
    "#": "present, the neutral priority value of 0 will be used.             ",
    "#": "-------------------------------------------------------------------",

    "priority": {
    "script": [
            ".",

            "# Start with the lower of the requested and default priorities",
            "| set(default; \"Initial priority\")",

        "# This is where decisions to change priority would be made,",
        "# as in this commented-out example:",

        "# # Friendly requesters get a small bump in priority.",
        "# | if classifiers_has(\"friendlies\")",
            "#   then adjust(5; \"Friendly requester\") else . end",

        "# The two blocks below implement recommended standard behavior.",

        "# If the requested priority was lower than what we came up",
        "# with, force that.",
        "| if requested != null and requested < priority",
        "  then set(requested; \"Lower requested priority\")",
            "  else . end",

        "# Allow at least the requested priority for those who are",
        "# allowed to do so.  Do this last in case things done",
        "# above push the priority higher than was requested",
        "| if requested != null",
        "     and requested > default",
        "     and requested > priority",
        "     and classifiers_has(\"priority-positive\")",
        "  then set(requested; \"Higher requested priority\")",
            "  else . end",

        "# The end.  (This takes care of the no-comma-at-end problem)"
        ]
    }


On 5. 11. 19 11:08, Szymon Trocha wrote:
W dniu 05.11.2019 o 11:04, Tadeja Saje pisze:

Hi Szymon,

No, did not modify limits configuration to use priorities section in /etc/pscheduler/limits.con. I have  psconfig template(json-mesh configuration). No measuments to testpoints with new http://software.internet2.edu/rpms/el7/x86_64/main/RPMS/perfSONAR-repo-0.9-1.noarch.rpm  could be done.

I try on testpoint with repo.0.9.1:

iperf3 -s

and now it works ok.

Running just iperf test ignores pscheduler and all it's configuration. Can you share your limits.conf file?

Regards,

--
Szymon Trocha
Poznań Supercomputing & Networking Center
General NOC phone +48 61-858-2015 | noc.pcss.pl
Personal desk phone +48 61-858-2022 Wysłaliśmy do Ciebie ten e-mail w odpowiedzi na Twoje zapytanie lub w związku z oferowaną usługą. Przesłanie korespondencji do Centrum Zarządzania PCSS lub zgłoszenie telefoniczne jest równoznaczne z wyrażeniem zgody na przetwarzanie danych osobowych przez Instytut Chemii Bioorganicznej Polskiej Akademii Nauk w Poznaniu adres: ul. Z. Noskowskiego 12/14, 61-704 Poznań. Szczegółowe informacje znajdują się w naszej Polityce prywatności. | This message has been sent as a part of communication with PSNC NOC or your service request sent to us. For more information read our Privacy Policy.

--
To unsubscribe from this list: https://lists.internet2.edu/sympa/signoff/perfsonar-user



Archive powered by MHonArc 2.6.19.

Top of Page