perfSONAR User Q&A and Other Discussion

[Edward Colone <>]

Full disclosure: I have no idea how running Ansible roles against the ISO would turn out.  Although they have the same end result, they are completely opposite methods.  The ISO is a canned image, ready to use.  The Ansible scripts simulate a user starting from a bare OS and typing in the commands to provision perfSONAR (yum, apt-get, etc.).  There shouldn't be any harm in running Ansible against the ISO, and in fact it could potentially do some config for you (importing schedules, authenticating interfaces to the archiver, etc.).

To my knowledge only the toolkit is available as an ISO.  So none of the other components (PWA, MadDash, etc.) are available as ISOs.

Thanks,

-Ed

On Wed, Sep 16, 2020 at 10:38 AM Paul Manomaitis <> wrote:

Thank you Edward for the clarification.

 

I will have to weigh options available for how to handle building a template/baseline image.

 

Would there be any conflicts from layering Ansible installs on top of the aforementioned ‘pS-Toolkit-4.2.4-CentOS7-FullInstall-x86_64-2020Mar27.iso’ ?

 

//Reply to previous email in thread.

 

I started with the editing of the raw JSON file but then saw that it would work better leveraging PWA, I am interested in learning about the Ansible role.

 

 

--

Also, you do not require PWA, although it is a nicety.  The "old" method is to create a JSON test schedule by hand, publish it via a web server, and point all testpoints and dashboards at the URL.  Here's an example file that is commonly used as a starting point for a JSON schedule:

 

https://github.com/perfsonar/psconfig/blob/master/doc/skeleton.json

 

There is also an Ansible role to help with that process.

 

Thanks,

 

-Ed

 

--

 

 

From: Edward Colone <>
Sent: Wednesday, September 16, 2020 7:52 AM
To: Paul Manomaitis <>
Cc: ; Josh Olson <>; John Urbano <>; Todd Retterer <>
Subject: [External] Re: Re: [perfsonar-user] Offline/Air gapped install want PWA and Mddash

 

By "full install" I meant not just testpoints, but PWA, Archiving, MadDash, and everything integrated together.  Sorry for the confusion.

 

The ISO is one way to get a testpoint/toolkit.  The Ansible solution does not use the ISO, it uses yum/apt-get to install specific packages for each component.

 

I have some pS nodes & infrastructure on semi-air-gapped private networks.  My strategy has been to grant each node outgoing access to the greater internet, build each node from scratch (install OS, use Ansible to install & configure components), and then resume the air gap (remove Internet access).

 

The BB Talk is Friday at 2PM EST.  I believe it will be hosted on Youtube afterwards.  Here is the schedule for more information:

 

http://scientificnetworking.lbl.gov/science-engagement/ci-engineering-lunch-and-learn-series/

 

Thanks,

 

-Ed

 

 

On Wed, Sep 16, 2020 at 9:46 AM Paul Manomaitis <> wrote:

Thank you Edward,

 

Where will your brownbag be published?

 

Also, to get the "full install" with the dashboard and everything: - Are you referring to ‘CentOS 7 Full Install’ from https://docs.perfsonar.net/install_getting.html#gettingchooseinstall ?

 

That is the foundation for my node installs the ‘pS-Toolkit-4.2.4-CentOS7-FullInstall-x86_64-2020Mar27.iso’ All my testing now is being conducted on the same.

 

I still fear that the perl-perfSONAR_PS-SimpleLS-BootStrap-server being out of date / not supported will prevent PWA from working in an air gapped deployment.

 

 

From: Edward Colone <>
Sent: Wednesday, September 16, 2020 6:10 AM
To: Paul Manomaitis <>
Cc: ; Josh Olson <>; John Urbano <>; Todd Retterer <>
Subject: [External] Re: [perfsonar-user] Offline/Air gapped install want PWA and Mddash

 

Also, to get the "full install" with the dashboard and everything:

 

10 testpoints

1 PWA node

1 Esmond Archiver

1 MadDash dashboard

 

  - install the archiver

  - install the testpoints

  - tell the archiver to accept data from each testpoint IP Addr

  - install MadDash

  - install PWA

  - create a PWA user for yourself

  - possibly add the 10 testpoints manually to your PWA instance (I don't know much about private lookup services, it might be easiest to just add the testpoints to PWA by hand)

  - use PWA to create a test schedule

  - tell all 10 testpoints and MadDash to read the published URL for the schedule

 

Any thoughts from anyone welcome.

 

Thanks,

 

-Ed

 

 

On Wed, Sep 16, 2020 at 7:50 AM Edward Colone <> wrote:

Hi Paul,

 

While I must admit that I'm not 100% certain if this will be of any help to you, I can offer that each perfSONAR component (PWA, testpoints, MadDash, Esmond Archive) can be discreetly installed and managed by the perfsonar Ansible scripts.  In fact I will be giving a tutorial this Friday at the ES Net CI Engineering Brown Bag talk.

 

https://github.com/perfsonar/ansible-playbook-perfsonar

 

Using the Ansible scripts you don't have to install the monolithic centralmanagement bundle.  The PWA script might need an additional repo to work correctly, I will address that this week.

 

If you want a hand installing perfSONAR via Ansible, let me know and I'll see if I can help.

 

Thanks,

 

-Ed

 

 

On Wed, Sep 16, 2020 at 7:39 AM Paul Manomaitis <> wrote:

Thanks again, * Again my end goal is to have a cluster of PerfSonar Nodes centrally managed, without an internet connection, airgapped.

– Is there a Perfsonar Dev that can chime in if this is possible to do in 4.2.4 ?

 

So new issue. Seems that for my airgapped/private cluster of PerfSonar nodes, I am possibly ‘SOL’ on private SlS. See Ø below

Going down the rabbit hole of configuring the PWA and associated services, https://docs.perfsonar.net/pwa_configure.html *

 

‘Most of the configuration files for PWA can be found in the /etc/perfsonar/psconfig-web directory.

At a minimum, you need to configure your hostname, configure the datasources for your instance, configure the authentication module, and create a user before you can start using PWA.’

 

I find I will need to construct a private SLS. (configure the datasources for your instance) This is a airgapped deployment.

 

 

0.       Editing /etc/perfsonar/psconfig-web/index.js as per * links instructions, Datasources section Leads me to -> ‘1.’

1.       https://code.google.com/archive/p/perfsonar-ps/wikis/PrivateLookupService.wiki     - outdated

2.       which leads to also following -> https://github.com/esnet/simple-lookup-service/wiki/LSInstallation

3.       and then -> https://docs.mongodb.com/manual/tutorial/install-mongodb-on-red-hat/

 

 

From ‘3.’ I skipped step 1. Which was a mistake.. mongodb-org package won’t install without creating the PMS for YUM first. Got this working

Ø From ‘1.’ Step 3. The command ‘-y install perl-perfSONAR_PS-SimpleLS-BootStrap-server’ will not work due to this package is missing.

 

I find this alternative which is a version 3.x.x

https://github.com/perfsonar/simple-ls-bootstrap - Ø Installed from here but cannot start the service since it is not located in.

 

 

‼ I also installed yum install perfsonar-centralmanagement since it includes serveral needed items. https://docs.perfsonar.net/install_centos.html

 

 

 

 

 

From: Mike Ware <>
Sent: Tuesday, September 15, 2020 4:53 PM
To: Paul Manomaitis <>
Cc:
Subject: [External] Re: Re: Re: [perfsonar-user] Offline/Air gapped install want PWA and Mddash

 

I'm not certain on that but I'm sure someone on the list knows. Glad to help and good luck.

Mike

 

On Tue, Sep 15, 2020 at 3:50 PM Paul Manomaitis <> wrote:

Haha, I was just reading that exact page already. Thank you again.

Also, yes that is a possible scenario.

 

Any idea if I need to install the PWA RPM/Dependencies on ALL of my nodes ( clients ) or just on one ( Server ) that will control the rest ?

Same question for MaDDash

 

Thank you again for your time.

 

From: Mike Ware <>
Sent: Tuesday, September 15, 2020 4:47 PM
To: Paul Manomaitis <>
Cc:
Subject: [External] Re: Re: [perfsonar-user] Offline/Air gapped install want PWA and Mddash

 

https://phoenixnap.com/kb/create-local-yum-repository-centos

 

You could in theory just build a vm with all the repos cached and then migrate the vm into the secure network then point your /etc/yum/repos.d on your other machines to pull packages from that system.

 

 

On Tue, Sep 15, 2020, 15:42 Paul Manomaitis <> wrote:

Thank you Mike, I spun up an instance @ my house where I have access to the net to demo this exact theory.

 

14 Dependencies and two updates tree off the two .RPMs for the PWA install alone.

 

Any suggestions on the method of packaging up the yum repo, will look this up as well.

 

From: Mike Ware <>
Sent: Tuesday, September 15, 2020 4:37 PM
To: Paul Manomaitis <>
Cc:
Subject: [External] Re: [perfsonar-user] Offline/Air gapped install want PWA and Mddash

 

You could make local yum repos and then move them via sneakernet to a repo server on your airgapped segment.

 

Mike

 

On Tue, Sep 15, 2020, 15:31 Paul Manomaitis <> wrote:

Greetings,

 

I have the CentOS 7 Full Install    x86_64 .iso and will be installing 10+ Nodes.

End goal/state is to have PWA and MaDDash dashboard and the node status checkerboard for C2. Also the graphing functionality, OFFLINE/AirGapped.

 

Is there a bundle I can grab or will I have to grab many individual RPMs from different sources and manually build the dependencies to accomplish the subject lines goals?

 

I have started pulling down .rpm files from http://software.internet2.edu/rpms/el7/x86_64/latest/packages/?C=S;O=D

 

I would need to install the PWA and mddash to at least one of these nodes or to a docket instance to centrally control the 10+ nodes.

Correct ?

 

The guides on the site have been built to support an internet connected install, I have run into issues trying to get PWA to come online.

e.g.

[root@localhost tmp]# rpm -ivh perfsonar-psconfig-web-admin-ui-4.2.4-1.el7.x86_64.rpm

error: Failed dependencies:

   /usr/bin/node is needed by perfsonar-psconfig-web-admin-ui-4.2.4-1.el7.x86_64

   mongodb is needed by perfsonar-psconfig-web-admin-ui-4.2.4-1.el7.x86_64

   mongodb-server is needed by perfsonar-psconfig-web-admin-ui-4.2.4-1.el7.x86_64

   nodejs is needed by perfsonar-psconfig-web-admin-ui-4.2.4-1.el7.x86_64

   perfsonar-psconfig-web-admin-auth is needed by perfsonar-psconfig-web-admin-ui-4.2.4-1.el7.x86_64

   perfsonar-psconfig-web-admin-shared is needed by perfsonar-psconfig-web-admin-ui-4.2.4-1.el7.x86_64

 

Also, have overcome some issues with host ip not showing by editing, using private address space.

/usr/lib/perfsonar/web-ng/etc/web_admin.conf

allow_internal_addresses  1

 

 

Thank you for your time

__________________________________________________________________________

Paul Manomaitis                                                                                                                 

Sr. Network Engineer II                                                                                                       

720-216-8557 Office                                                                                                            

 

Raytheon Technologies

Intelligence, Information, and Service

16800 East CentreTech Parkway

Aurora, Colorado 80011

S77/3M-74A

                                                                                                                                                                                                                                      

CCNP R&S, CCNA R&S, CASP, Network+, Linux+, Server+, Sec+, A+

__________________________________________________________________________

 

 

--
To unsubscribe from this list: https://lists.internet2.edu/sympa/signoff/perfsonar-user

 

--

Michael Ware
Network Security Engineer
UCSC
406-210-4725

--
To unsubscribe from this list: https://lists.internet2.edu/sympa/signoff/perfsonar-user