perfSONAR User Q&A and Other Discussion

[Johann Hugo <>]

Hy Andy

Thanks for the ack.

I've made some progress and managed to get all the containers up and running. I first had to get rid of all the old stuff.

docker ps -aq | xargs docker stop | xargs docker rm

docker system prune -a

rm -rf /usr/local/data

See attached files with the tweaks I made to update_pwa_containers.sh and start_pwa_containers.sh

/etc/pwa/nginx/conf.d/pwa.conf needed some tweaks to stop nginx from restarting all the time. .

https://github.com/perfsonar/psconfig-web/raw/master/psconfig-web-admin/perfsonar-psconfig-web-admin/deploy/docker/pwa.sample.tar.gz

are missing auth.key auth.pub and user.jwt in the /etc/pwa/auth/ directory

I still can't login, looks like a database issue:

docker logs --tail 50 --follow --timestamps sca-auth

2023-07-21T11:50:27.709553000Z PM2     | App [auth:1] exited with code [0] via signal [SIGINT]

2023-07-21T11:50:27.711934000Z PM2     | App [auth:1] starting in -fork mode-

2023-07-21T11:50:27.720583000Z PM2     | App [auth:1] online

2023-07-21T11:50:28.639382000Z 1|auth  | Error [SequelizeConnectionError]: SQLITE_CANTOPEN: unable to open database file

2023-07-21T11:50:28.640595000Z 1|auth  |     at Database.<anonymous> (/app/node_modules/sequelize/lib/dialects/sqlite/connection-manager.js:63:61) {

2023-07-21T11:50:28.641139000Z 1|auth  |   parent: [Error: SQLITE_CANTOPEN: unable to open database file] {

2023-07-21T11:50:28.641794000Z 1|auth  |     errno: 14,

2023-07-21T11:50:28.642262000Z 1|auth  |     code: 'SQLITE_CANTOPEN'

2023-07-21T11:50:28.642709000Z 1|auth  |   },

2023-07-21T11:50:28.643143000Z 1|auth  |   original: [Error: SQLITE_CANTOPEN: unable to open database file] {

2023-07-21T11:50:28.643605000Z 1|auth  |     errno: 14,

2023-07-21T11:50:28.644032000Z 1|auth  |     code: 'SQLITE_CANTOPEN'

2023-07-21T11:50:28.644488000Z 1|auth  |   }

2023-07-21T11:50:28.644939000Z 1|auth  | }

Regards

Johann

On Thu, Jul 20, 2023 at 11:03 PM Andrew Lake <> wrote:

Hi Johann,

Mostly just ack'ing this so you don’t get complete radio silence. The dev team is still trying to fill an expertise gap on the PWA and right now don’t have immediate answer to your questions. Clearly the container is having some trouble finding a few python modules and the mentioned script, so not sure if there is a problem with the build or something else. We’re trying to figure out how to best handle this request, but didn’t want to leave you completely hanging in meantime. If you are able to get these sorted on your own let us know, and sorry we don’t have a quick solution for you at the moment.

Thanks,

Andy

On July 20, 2023 at 4:46:27 AM, Johann Hugo () wrote:

I decided to switch from RPM packages to try the Docker PWA route.

I first tried the docker-compose route using this script, but it's giving a lot of errors:

wget -4 https://raw.githubusercontent.com/perfsonar/psconfig-web/master/psconfig-web-admin/perfsonar-psconfig-web-admin/deploy/docker/docker-compose.yml

docker-compose up

Traceback (most recent call last):

  File "/usr/lib/python3.6/site-packages/jsonschema/_format.py", line 173, in <module>

    import strict_rfc3339

ModuleNotFoundError: No module named 'strict_rfc3339'

During handling of the above exception, another exception occurred:

Traceback (most recent call last):

  File "/bin/docker-compose", line 11, in <module>

    load_entry_point('docker-compose==1.18.0', 'console_scripts', 'docker-compose')()

  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 476, in load_entry_point

    return get_distribution(dist).load_entry_point(group, name)

  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2700, in load_entry_point

    return ep.load()

  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2318, in load

    return self.resolve()

  File "/usr/lib/python3.6/site-packages/pkg_resources/__init__.py", line 2324, in resolve

    module = __import__(self.module_name, fromlist=['__name__'], level=0)

  File "/usr/lib/python3.6/site-packages/compose/cli/main.py", line 22, in <module>

    from ..bundle import get_image_digests

  File "/usr/lib/python3.6/site-packages/compose/bundle.py", line 12, in <module>

    from .config.serialize import denormalize_config

  File "/usr/lib/python3.6/site-packages/compose/config/__init__.py", line 6, in <module>

    from .config import ConfigurationError

  File "/usr/lib/python3.6/site-packages/compose/config/config.py", line 45, in <module>

    from .validation import match_named_volumes

  File "/usr/lib/python3.6/site-packages/compose/config/validation.py", line 12, in <module>

    from jsonschema import Draft4Validator

  File "/usr/lib/python3.6/site-packages/jsonschema/__init__.py", line 15, in <module>

    from jsonschema._format import (

  File "/usr/lib/python3.6/site-packages/jsonschema/_format.py", line 180, in <module>

    @_checks_drafts("date-time", raises=(ValueError, isodate.ISO8601Error))

AttributeError: module 'isodate' has no attribute 'ISO8601Error'

I then switched to using the docker scripts: 

Running /usr/local/sbin/update_pwa_containers.sh downloads all the images, but gives this error at the end.

/usr/local/sbin/update_pwa_containers.sh: line 31: ./start_pwa_containers.sh: No such file or directory

Running the script manually starts all the containers, except for the an error with mongo :

/usr/local/sbin/start_pwa_containers.sh

/usr/bin/docker-current: Error response from daemon: SELinux relabeling of /usr/local/docker/volumes/d69b41dc7fe2f46b7f6b430b85e2855e2d599421c328d67d447caf449f875f26/_data is not allowed: "Relabeling content in /usr is not allowed."

Creating the mongo container manually gives the same error:

docker run \
>         --restart=always \
>         --net pwa \
>         --name mongo \
>         -v /usr/local/data/mongo:/data/db \
>         -d mongo
/usr/bin/docker-current: Error response from daemon: SELinux relabeling of /usr/local/docker/volumes/6f82d4392952d720a9e7fb472f804a49a027780d41275c1eb71310f8aabb111e/_data is not allowed: "Relabeling content in /usr is not allowed.".

Regards

Johann

On Wed, Jul 19, 2023 at 2:09 PM Szymon Trocha <> wrote:

Hi Johann,

Indeed links are not correct. Please see correct links below and we'll correct web pages

Also for some reason downloading over IPv6 semms to not work so please use v4 switch and it works then.

W dniu 19.07.2023 o 13:49, Johann Hugo pisze:

Hi

I'm trying to get a new PWA server running, but no success. Some of the links in the documentation don't work.

https://docs.perfsonar.net/pwa_install_docker.html

 - wget https://github.com/perfsonar/psconfig-web/raw/master/deploy/docker/pwa.sample.tar.gz

wget -4 https://github.com/perfsonar/psconfig-web/raw/master/psconfig-web-admin/perfsonar-psconfig-web-admin/deploy/docker/pwa.sample.tar.gz

https://docs.perfsonar.net/pwa_docker_create.html

- wget https://raw.githubusercontent.com/perfsonar/psconfig-web/master/deploy/docker/docker-compose.yml

wget -4 https://raw.githubusercontent.com/perfsonar/psconfig-web/master/psconfig-web-admin/perfsonar-psconfig-web-admin/deploy/docker/docker-compose.yml

https://docs.perfsonar.net/pwa_install_rpm.html

- rpm -hUv http://software.internet2.edu/rpms/el7/x86_64/latest/packages/perfSONAR-repo-0.10-1.noarch.rpm

refer to repos installation mentioned here: https://docs.perfsonar.net/install_el.html

On https://docs.perfsonar.net/pwa_install_rpm.html , this paragraph seems to be misplaced:

- Before you start the docker engine, you might want to add any VM specific configuration. For example, your VM might be using /usr/local as a primary partition for your VM.

I've made some progress by installing PWA using RPM packages. 

- systemctl status httpd perfsonar-psconfig-web-admin-* shows all the daemons are up and running (httpd was not enabled by default). 

- The only changes I made to the default configs was changing the hostname in /etc/perfsonar/psconfig-web/index.js and /etc/perfsonar/psconfig-web/auth/index.js.

 - I've also added a local PWA user with super-admin rights using "pwa_auth useradd" + "pwa_auth modscope". 

http://pwa-test.sanren.ac.za/ loads the default Apache html page and http://pwa-test.sanren.ac.za/pwa/pub gives a Not Found. 

- Do I need to configure anything in the pwa*.conf files in /etc/httpd/conf.d/ 

- Where can I view the log files for PWA

Regards,

--

Szymon Trocha
Poznań Supercomputing & Networking Center

General NOC phone +48 61-858-2015 | noc.pcss.pl
Personal desk phone +48 61-858-2022

Wysłaliśmy do Ciebie ten e-mail w odpowiedzi na Twoje zapytanie lub w związku z oferowaną usługą. Przesłanie korespondencji do Centrum Zarządzania PCSS lub zgłoszenie telefoniczne jest równoznaczne z wyrażeniem zgody na przetwarzanie danych osobowych przez Instytut Chemii Bioorganicznej Polskiej Akademii Nauk w Poznaniu adres: ul. Z. Noskowskiego 12/14, 61-704 Poznań. Szczegółowe informacje znajdują się w naszej Polityce prywatności. | This message has been sent as a part of communication with PSNC NOC or your service request sent to us. For more information read our Privacy Policy.

--

SANReN Engineer

South African National Research Network (SANReN)

National Integrated Cyber Infrastructure System (NICIS)

CSIR NextGen Enterprises and Institutions Cluster

Office: 012 841 2066Email: , Website: www.sanren.ac.za / www.csir.co.za

--
To unsubscribe from this list: https://lists.internet2.edu/sympa/signoff/perfsonar-user

--

SANReN Engineer

South African National Research Network (SANReN)

National Integrated Cyber Infrastructure System (NICIS)

CSIR NextGen Enterprises and Institutions Cluster

Office: 012 841 2066Email: , Website: www.sanren.ac.za / www.csir.co.za

Attachment: pwa.conf
Description: Binary data

#!/usr/bin/bash

# Start mongo Docker container
docker run \
		--privileged \
        --restart=always \
        --net pwa \
        --name mongo \
        -v /usr/local/data/mongo:/data/db \
        -d mongo

# Start Authentication Service container
docker run \
    --restart=always \
    --net pwa \
    --name sca-auth \
    -v /etc/pwa/auth:/app/api/config \
    -v /usr/local/data/auth:/db \
    -d perfsonar/sca-auth

# Start PWA Application container
docker run \
    --restart=always \
    --net pwa \
    --name pwa-admin1 \
    -v /etc/pwa:/app/api/config:ro \
    -d perfsonar/pwa-admin

# Start pwa-pub1 container
docker run \
   --restart=always \
   --net pwa \
   --name pwa-pub1 \
   -v /etc/pwa:/app/api/config:ro \
   -d perfsonar/pwa-pub

# Uncomment following lines for additional publishers (you can add as many as you like)
# Make sure to update the nginx config as well, if you modify this

#docker run \
#   --restart=always \
#   --net pwa \
#   --name pwa-pub2 \
#   -v /etc/pwa:/app/api/config:ro \
#   -d perfsonar/pwa-pub
#
#docker run \
#   --restart=always \
#   --net pwa \
#   --name pwa-pub3 \
#   -v /etc/pwa:/app/api/config:ro \
#   -d perfsonar/pwa-pub

# Start nginx container
docker run \
    --restart=always \
    --net pwa \
    --name nginx \
    -v /etc/pwa/shared:/shared:ro \
    -v /etc/pwa/nginx:/etc/nginx:ro \
    -v /etc/pwa/auth:/certs:ro \
    -p 80:80 \
    -p 443:443 \
    -p 9443:9443 \
    -d nginx

# Docker postfix container (optional)
# Uncomment this if you need to run a local mail server in a container, rather than specifying an SMTP server

#docker run \
#    --network pwa \
#    -d --name postfix \
#    -p 587:25 \
#    --restart always \
#    yorkshirekev/postfix HOSTNAME

#!/usr/bin/bash

# Make sure the containers stopped/deleted/updated here match
# those started in "start_pwa_containers.sh"

docker stop pwa-admin1
docker stop sca-auth
docker stop pwa-pub1
#docker stop pwa-pub2
#docker stop pwa-pub3
docker stop mongo
docker stop nginx
#docker stop postfix

docker rm pwa-admin1
docker rm sca-auth
docker rm pwa-pub1
#docker rm pwa-pub2
#docker rm pwa-pub3
docker rm mongo
docker rm nginx
#docker rm postfix

docker pull perfsonar/pwa-admin
docker pull perfsonar/pwa-pub
docker pull perfsonar/sca-auth
docker pull mongo
docker pull nginx
docker pull yorkshirekev/postfix

/etc/pwa/scripts/start_pwa_containers.sh