perfSONAR User Q&A and Other Discussion

["Fedorka, Shayne" <>]

Thank you Adam, I can confirm this workaround fixes the issue on my systems as well.

 

From: <> on behalf of Weed, Adam <>
Date: Tuesday, September 19, 2023 at 12:14 PM
To: <>
Subject: [perfsonar-user] Opensearch/Logstash environment and 503 Service Unavailable for archiver

Some people who received this message don't often get email from . Learn why this is important

CAUTION: This email originated from outside of NREL. Do not click links or open attachments unless you recognize the sender and know the content is safe.

I was getting 503 errors when attempting to archive, similar to other users. My installation is perfsonar-toolkit on Ubuntu 20.

 

The /var/log/logstash/logstash.plain log was a continuous series of the service starting, erroring, and stopping. The error was:

 

Failed to execute action {:action=">LogStash::PipelineAction::Create/pipeline_id:pscheduler," :exception=>"Java::JavaLang::IllegalStateException", :message=>"Unable to configure plugins: Cannot evaluate `${opensearch_output_host}`. Replacement variable `opensearch_output_host` is not defined in a Logstash secret store or an environment entry and there is no default value given."

 

The "opensearch_output_host" value is contained in /etc/perfsonar/logstash/logstash_sysconfig, which also contains opensearch_logstash_user and opensearch_logstash_password.

 

These variables are referenced in /usr/lib/perfsonar/logstash/pipeline/99-outputs.conf. If I replace the variables in 99-outputs.conf with the values in logstash_sysconfig and restart the logstash service, the error goes away and is replaced with a 401 error when attempting to contact OpenSearch at the given address (https://localhost:9200).

 

If I then replace the user and password values in 99-outputs.conf with the OpenSearch login contained in /etc/perfsonar/opensearch/opensearch_login, then upon restart the logstash service works correctly and I have records displayed in the Test Results section of the web UI.

 

What can I do to properly configure the environment so that the values in logstash_sysconfig and opensearch_auth are found without having to manually replace their variable references?

 

Any information would be greatly appreciated.

 

Adam Weed
Database Programmer/Analyst

University of Missouri System - MOREnet

  |  (573) 884-2562