Skip to Content.
Sympa Menu

perfsonar-user - [perfsonar-user] new install of PWA gives a 503 error instead of loading

Subject: perfSONAR User Q&A and Other Discussion

List archive

[perfsonar-user] new install of PWA gives a 503 error instead of loading


Chronological Thread 
    < Chronological >      < Thread >
  • From: Jarett DeAngelis <>
  • To:
  • Subject: [perfsonar-user] new install of PWA gives a 503 error instead of loading
  • Date: Mon, 7 Nov 2022 05:55:14 -0600
Hi,

As far as I can tell I've followed the instructions to set PWA up, but get 503'd when trying to visit the main /pwa/pub link. Nothing is obviously broken in systemd:

[root@localhost httpd]# sudo systemctl status perfsonar-psconfig-web-admin-*
● perfsonar-psconfig-web-admin-auth.service - perfSONAR PWA Authentication API daemon
   Loaded: loaded (/usr/lib/systemd/system/perfsonar-psconfig-web-admin-auth.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2022-11-07 11:18:40 CST; 5h 48min left
 Main PID: 1184 (node)
   CGroup: /system.slice/perfsonar-psconfig-web-admin-auth.service
           └─1184 /usr/bin/node api/auth.js

Nov 07 11:18:40 localhost.localdomain systemd[1]: Started perfSONAR PWA Authentication API daemon.
Nov 07 05:11:35 localhost.localdomain node[1184]: Express server listening on 0.0.0.0:12000
Nov 07 05:11:35 localhost.localdomain node[1184]: service started

● perfsonar-psconfig-web-admin-api.service - perfSONAR PWA API daemon
   Loaded: loaded (/usr/lib/systemd/system/perfsonar-psconfig-web-admin-api.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2022-11-07 11:18:40 CST; 5h 48min left
 Main PID: 1190 (node)
   CGroup: /system.slice/perfsonar-psconfig-web-admin-api.service
           └─1190 /usr/bin/node api/pwaadmin.js

Nov 07 11:18:49 localhost.localdomain node[1190]: port: 8082,
Nov 07 11:18:49 localhost.localdomain node[1190]: url: 'http://10.8.8.70/pwa/pub/',
Nov 07 11:18:49 localhost.localdomain node[1190]: default_config_format: 'psconfig',
Nov 07 11:18:49 localhost.localdomain node[1190]: plugins: {
Nov 07 11:18:49 localhost.localdomain node[1190]: enabled: false,
Nov 07 11:18:49 localhost.localdomain node[1190]: plugin_scripts: [ 'plugins/pub/filter_example.js' ]
Nov 07 11:18:49 localhost.localdomain node[1190]: }
Nov 07 11:18:49 localhost.localdomain node[1190]: }
Nov 07 05:11:35 localhost.localdomain node[1190]: Mon Nov 07 2022 05:11:35 GMT-0600 (Central Standard Time) - info: connected to mongo
Nov 07 05:11:35 localhost.localdomain node[1190]: Mon Nov 07 2022 05:11:35 GMT-0600 (Central Standard Time) - info: admin listening on port 8080 in development mode

● perfsonar-psconfig-web-admin-cache.service - perfSONAR PWA LS Cache daemon
   Loaded: loaded (/usr/lib/systemd/system/perfsonar-psconfig-web-admin-cache.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2022-11-07 11:18:40 CST; 5h 48min left
 Main PID: 1193 (node)
   CGroup: /system.slice/perfsonar-psconfig-web-admin-cache.service
           └─1193 /usr/bin/node ui/api/pwacache.js

Nov 07 05:29:50 localhost.localdomain node[1193]: service['service-host'] [ 'lookup/host/7c2c353b-0d67-427b-8649-a7c7a2ca022d' ]
Nov 07 05:29:50 localhost.localdomain node[1193]: URL http://35.223.142.206:8090/lookup/host/7c2c353b-0d67-427b-8649-a7c7a2ca022d
Nov 07 05:29:52 localhost.localdomain node[1193]: service['service-host'] [ 'lookup/host/d39627fb-6615-4341-8414-2fb2af335827' ]
Nov 07 05:29:52 localhost.localdomain node[1193]: URL http://35.223.142.206:8090/lookup/host/d39627fb-6615-4341-8414-2fb2af335827
Nov 07 05:29:52 localhost.localdomain node[1193]: service['service-host'] [ 'lookup/host/ed8e1030-9e90-481b-bd5d-76d1c57895ba' ]
Nov 07 05:29:52 localhost.localdomain node[1193]: URL http://35.223.142.206:8090/lookup/host/ed8e1030-9e90-481b-bd5d-76d1c57895ba
Nov 07 05:29:52 localhost.localdomain node[1193]: Mon Nov 07 2022 05:29:52 GMT-0600 (Central Standard Time) - error: location-sitename not set!! using mockup name.Measurement Ar...es.brown.edu
Nov 07 05:29:52 localhost.localdomain node[1193]: service['service-host'] [ 'lookup/host/66331ed2-f3ed-4c1c-8dcd-2f97b201200f' ]
Nov 07 05:29:52 localhost.localdomain node[1193]: URL http://35.223.142.206:8090/lookup/host/66331ed2-f3ed-4c1c-8dcd-2f97b201200f
Nov 07 05:29:52 localhost.localdomain node[1193]: Mon Nov 07 2022 05:29:52 GMT-0600 (Central Standard Time) - error: location-sitename not set!! using mockup name.TWAMP Server at ac.cn

● perfsonar-psconfig-web-admin-publisher.service - perfSONAR PWA Publisher daemon
   Loaded: loaded (/usr/lib/systemd/system/perfsonar-psconfig-web-admin-publisher.service; enabled; vendor preset: disabled)
   Active: active (running) since Mon 2022-11-07 11:18:40 CST; 5h 48min left
 Main PID: 1199 (node)
   CGroup: /system.slice/perfsonar-psconfig-web-admin-publisher.service
           └─1199 /usr/bin/node pub/api/pwapub.js

Nov 07 11:18:49 localhost.localdomain node[1199]: port: 8082,
Nov 07 11:18:49 localhost.localdomain node[1199]: url: 'http://10.8.8.70/pwa/pub/',
Nov 07 11:18:49 localhost.localdomain node[1199]: default_config_format: 'psconfig',
Nov 07 11:18:49 localhost.localdomain node[1199]: plugins: {
Nov 07 11:18:49 localhost.localdomain node[1199]: enabled: false,
Nov 07 11:18:49 localhost.localdomain node[1199]: plugin_scripts: [ 'plugins/pub/filter_example.js' ]
Nov 07 11:18:49 localhost.localdomain node[1199]: }
Nov 07 11:18:49 localhost.localdomain node[1199]: }
Nov 07 05:11:35 localhost.localdomain node[1199]: Mon Nov 07 2022 05:11:35 GMT-0600 (Central Standard Time) - info: connected to mongo
Nov 07 05:11:35 localhost.localdomain node[1199]: Mon Nov 07 2022 05:11:35 GMT-0600 (Central Standard Time) - info: pub service running on 10.8.8.70:8082 in development mode
Hint: Some lines were ellipsized, use -l to show in full.

It seems the reverse proxy cannot contact whatever is supposed to serve PWA (node?) as seen in /var/log/httpd/error_log:

[Mon Nov 07 11:14:44.271474 2022] [wsgi:error] [pid 1423] [remote 10.8.8.70:41864] esmond.api.perfsonar.api_v2 [DEBUG] action="query_timeseries.start" md_key=37e05fc9186f49e786ea5fb6799a4bf6 event_type=throughput summ_type=base summ_win=None start=1667214879 end=1667819679 start_millis=1667214879000 end_millis=1667819679000 cf=base_rates datapath=['ps', 'throughput', '37e05fc9186f49e786ea5fb6799a4bf6']
[Mon Nov 07 05:14:45.123557 2022] [wsgi:error] [pid 1426] path= ['/usr/lib/esmond', '/usr/lib/esmond/esmond', '/usr/lib/esmond/lib/python3.6', '/usr/lib/esmond/lib/python3.6/site-packages', '/usr/lib/esmond', '/usr/lib/esmond/esmond_client', '/usr/lib64/python36.zip', '/usr/lib64/python3.6', '/usr/lib64/python3.6/lib-dynload', '/usr/lib64/python3.6/site-packages', '/usr/lib/python3.6/site-packages']
[Mon Nov 07 11:14:46.441683 2022] [wsgi:error] [pid 1426] [remote 10.8.8.70:42032] esmond [INFO] Checking/creating column families
[Mon Nov 07 11:14:46.443193 2022] [wsgi:error] [pid 1426] [remote 10.8.8.70:42032] esmond [INFO] Schema check done
[Mon Nov 07 11:14:46.443250 2022] [wsgi:error] [pid 1426] [remote 10.8.8.70:42032] esmond [DEBUG] Opening ConnectionPool
[Mon Nov 07 11:14:46.740785 2022] [wsgi:error] [pid 1426] [remote 10.8.8.70:42032] esmond [INFO] Connected to ['localhost:9160']
[Mon Nov 07 11:14:46.845531 2022] [wsgi:error] [pid 1426] [remote 10.8.8.70:42032] esmond.api.perfsonar.api_v2 [DEBUG] action="query_timeseries.start" md_key=6c32c32bac764005ba90b4386cbedba4 event_type=throughput summ_type=base summ_win=None start=1667214879 end=1667819679 start_millis=1667214879000 end_millis=1667819679000 cf=base_rates datapath=['ps', 'throughput', '6c32c32bac764005ba90b4386cbedba4']
[Mon Nov 07 05:25:37.766601 2022] [cgi:error] [pid 1443] [client 10.99.0.3:49577] AH01215: Use of uninitialized value $elem in concatenation (.) or string at /usr/lib/perfsonar/lib/SimpleLookupService/QueryObjects/QueryObject.pm line 70., referer: http://10.8.8.70/perfsonar-graphs/?source=10.8.8.70&dest=10.8.8.71&url="http%3A%2F%2F10.8.8.70%2Fesmond%2Fperfsonar%2Farchive%2F&timeframe=1w

Node is definitely listening:


And yet:
--2022-11-07 05:47:01--  http://127.0.0.1:8082/
Connecting to 127.0.0.1:8082... failed: Connection refused.

It listens to connection attempts to 10.8.8.70 instead of 127.0.0.1:

[root@localhost psconfig-web]# wget http://10.8.8.70:8082/pwa/pub
--2022-11-07 05:48:14--  http://10.8.8.70:8082/pwa/pub
Connecting to 10.8.8.70:8082... connected.
HTTP request sent, awaiting response... 404 Not Found
2022-11-07 05:48:14 ERROR 404: Not Found.

But the path is still wrong.

Node seem to be there in a browser, but isn't returning anything useful:

I remembered it doing something with snakeoil certs, so I tried https. Nope.



Contents of /etc/perfsonar/psconfig-web:

"use strict";
const fs = require("fs");
const winston = require("winston");
const os = require("os");

// Publisher api (will run on a different container from admin container)
// Note: you *must* update the "url" with your hostname in place of <hostname>

exports.pub = {
    host: "10.8.8.70",
    port: 8082,
    //url: "http://10.8.8.70/pwa/pub/",
    url: "/pwa/pub/",
    default_config_format: "psconfig",
};

exports.pub.plugins = {
    enabled: false,
    plugin_scripts: ["plugins/pub/filter_example.js"],
};

console.log("exports.pub", exports.pub);

// Mongo DB to use (the default should work fine, unless you specifically need a different db)
exports.mongodb = "mongodb://localhost/pwa";

// PWA general settings

exports.meshconfig = {
    login_url: "/pwa/auth/",

    // service-types to support
    service_types: {
        owamp: { label: "Latency" },
        bwctl: { label: "Throughput" },
        traceroute: { label: "Traceroute" },
        ping: { label: "Ping" },
    },

    // Supported mesh types
    mesh_types: {
        mesh: { label: "Mesh" },
        disjoint: { label: "Disjoint" },
    },

    // Default values for various new entities created via the GUI
    // Each time a new testspec is created, many parameters have default values -- for instance, test duration
    // These defaults are specified here; they should work fine in most cases, but you can tweak them if necessary
    defaults: {
        testspecs: {
            bwctl: {
                tool: "bwctl/iperf3",
                protocol: "tcp",
                interval: 14400,
                duration: 20,
                random_start_percentage: 10,
                omit_interval: 5,
                force_bidirectional: false,
                ipv4_only: true,
            },
            owamp: {
                packet_interval: 0.1,
                sample_count: 600,
                packet_padding: 0,
                bucket_width: 0.001,
                force_bidirectional: false,
                ipv4_only: true,
                interval: 3600,
                duration: 30,
                tool: "owping",
                schedule_type: "continuous",
            },
            traceroute: {
                tool: "traceroute",
                test_interval: 600,
                random_start_percentage: 10,
                protocol: "icmp",
                first_ttl: 1,
                packet_size: 1200,
                force_bidirectional: false,
                ipv4_only: false,
                ipv6_only: false,
                pause: 0,
                waittime: 10,
                timeout: 60,
            },
            ping: {
                test_interval: 1,
            },
        },
    },

    //minumum option version catalog (used to suppress them for known v3 host)
    minver: {
        bwctl: {
            tcp_bandwidth: 4,
            mss: 4,
            dscp: 4,
            dynamic_window_size: 4,
            no_delay: 4,
            congestion: 4,
            flow_label: 4,
            server_cpu_affinity: 4,
            client_cpu_affinity: 4,
        },
        owamp: {
            tos_bits: 4,
            output_raw: 4,
        },
        traceroute: {
            tos_bits: 4,
            algorithm: 4,
            as: 4,
            fragment: 4,
            hostnames: 4,
            probe_type: 4,
            queries: 4,
            sendwait: 4,
            wait: 4,
        },
        ping: {
            tool: 4,
            tos_bits: 4,
            flow_label: 4,
            hostnames: 4,
            suppress_loopback: 4,
            deadline: 4,
            timeout: 4,
        },
    },

    //meshconfig admin loads sls content at startup, and then every once a while with following frequency
    sls_cache_frequency: 60 * 1000 * 10, //every 10 minutes
};

exports.datasource = {
    //amount of time between each cache
    delay: 1000 * 3600,

    //ls endpoints to pull host information from
    lses: {
        // Global LS queries
        // if hostname collision happens, the first datasource will take precedence

        // Example: here, we are creating a label based on querying the Global LS for a given community
        // Fake institution "Widget Factory"
        // Note the "label" should be short as it will display for each host in the GUI and there is limited space
        /*
        "widgetfactory": {
            label: "widget",
            type: "global-sls",
            activehosts_url: "http://ps1.es.net:8096/lookup/activehosts.json",
            query: "?type=service&group-communities=*WIDGET,*widget&group-communities-operator=any",
        },
*/
        // IN:
        /*"gls": {
            label: "GLS",
            type: "global-sls",
            activehosts_url: "http://ps1.es.net:8096/lookup/activehosts.json",
            query: "?type=service&group-communities=Indiana",
        }
        */
        // Global LS instance (by default, this is the only LS specified)

        gls: {
            label: "GLS",
            type: "global-sls",
            activehosts_url: "http://ps1.es.net:8096/lookup/activehosts.json",
            query: "?type=service",
        },

        // Private sLS instance
        // only uncomment this if you are running a private sLS instance
        /*
        "private": {
            label: "private",
            type: "sls",
            url: "http://sls:8090/lookup/records/?type=service",
        },
        */
    },
};

//admin api
exports.admin = {
    //api server host/port (don't change this for docker)
    host: "0.0.0.0",
    port: 8080,

    //authentication service public key to verify jwt token generated by it
    jwt: { pub: fs.readFileSync(__dirname + "/auth/auth.pub") },
};

exports.common = {
    //needed to access auth service to pull profile
    auth_api: "http://localhost:12000",
    //auth_api: "http://sca-auth:8080",
    auth_jwt: fs
        .readFileSync(__dirname + "/auth/user.jwt")
        .toString()
        .trim(),
};

exports.logger = {
    winston: {
        transports: [
            //display all logs to console
            new winston.transports.Console({
                timestamp: function () {
                    var d = new Date();
                    return d.toString();
                },
                level: "info",
                colorize: true,
            }),

            /*
            //store all warnings / errors in error.log
            new (winston.transports.File)({
                filename: "error.log",
                level: "warn"
            })
            */
        ],
    },
};

Kind of at the end of my rope here. Help?

--

 

Jarett T. DeAngelis, MS

Scientific Systems Engineer

Email:  
M: +1.646.417.2165

bioteam.net






  • [perfsonar-user] new install of PWA gives a 503 error instead of loading, Jarett DeAngelis, 11/07/2022

Archive powered by MHonArc 2.6.24.

Top of Page