perfSONAR User Q&A and Other Discussion

[Andrew Lake <andy@es.net>]

Hi Daneil,

Cassandra might look like its running but it can sometime get stuck. Try the following to kill it and force esmond to reconnect: 

pkill -9 -f java

systemctl restart cassandra

systemctl restart httpd

Thanks,

Andy

On June 29, 2017 at 9:41:12 AM, Daniel Manzo (daniel.manzo@bayer.com) wrote:

Hi Ivan,

 

The error from ‘clean_esmond_db.log’ is still showing up as of 2:30 am EST today (6/29). The last time the error from ‘meshconfig-agent.log’ showed up was 3:50 pm EST yesterday (6/28) and the error from ‘esmond.log’ is still showing up as of 9:22 am EST today.

 

I am unable to access the local Esmond instance. Here is the error:

 

In my Cassandra.log file, there is only one line stating “This account is currently not available.” Not sure which account it is referring to.

 

The pscheduler.log seems pretty normal. I don’t see any warnings and the last two entries are from yesterday stating “Started” and “Limits loaded from /etc/pscheduler/limits.conf” which makes sense since I did try restarting the pscheduler services yesterday at that time.

 

For the meshconfig messages, my version of perfsonar-meshconfig-agent is 4.0.0.2-1.

 

Thank you for the help,

Dan Manzo

 

From: Garnizov, Ivan (RRZE) [mailto:ivan.garnizov@fau.de]
Sent: Thursday, June 29, 2017 3:50 AM
To: Daniel Manzo; perfsonar-user
Subject: RE: Internal Server Error

 

Hello Daniel,

 

Please tell if the reports in the event log, you have shared, are aligned in time with your checks. Could these be old messages?

Please tell if you are able to access your local Esmond instance. Open <host>/esmond/perfsonar/archive/

Please check your Cassandra log. Tell if there is a constant flow on “INFO” events or if there is something suspicious.

Check your pscheduler.log for warning events from the archiver daemon.

 

With regards to the meshconfig messages, please share the meshconfig version. Package: perfsonar-meshconfig-agent.

 

Regards,

Ivan Garnizov

 

GEANT SA1T2: pS deployments GN Operations

GEANT SA2T3: pS development team

GEANT SA3T5: eduPERT team

 

 

 

 

 

From: perfsonar-user-request@internet2.edu [mailto:perfsonar-user-request@internet2.edu] On Behalf Of Daniel Manzo
Sent: Mittwoch, 28. Juni 2017 20:27
To: perfsonar-user
Subject: [perfsonar-user] Internal Server Error

 

Hi all,

 

I am currently running perfSONAR v4.0 on RHEL 7.3. I was previously having problems with postgresql-9.5 and pscheduler, but now those are resolved and both are running smoothly. However, I cannot access the toolkit from HTTP, only HTTPS. Also, when I access the web page, I am getting “Error loading test listing: Internal Server Error” underneath the “Test Results” section. I have checked the logs, and I’m not seeing any real evidence where this problem could be located.

 

Here is something that does not look good to me, especially since Cassandra is running (clean_esmond_db.log):

No handlers could be found for logger "esmond.api.perfsonar.api_v2"

Unable to connect - presuming stand-alone testing mode...

Traceback (most recent call last):

  File "./util/ps_remove_data.py", line 74, in <module>

    from esmond.api.perfsonar.api_v2 import EVENT_TYPE_CF_MAP

  File "/usr/lib/esmond/esmond/api/perfsonar/api_v2.py", line 65, in <module>

    raise ConnectionException(error_msg)

esmond.cassandra.ConnectionException: 'Unable to connect to cassandra. Please verify cassandra is running.'

This also worries me (from meshconfig-agent.log):

2017/06/28 14:16:55 (16717) WARN> perfsonar_meshconfig_agent:145 main::__ANON__ - Warned: Use of uninitialized value $address in exists at /usr/lib/perfsonar/bin/../lib/perfSONAR_PS/RegularTesting/Tests/BwctlBase.pm line 384.

Also, esmond.log shows this error multiple times:

2017-06-28 14:22:17,198 [ERROR] /usr/lib/esmond/esmond/api/perfsonar/api_v2.py: Unable to connect to cassandra. Please verify cassandra is running.

 

Any and all help is appreciated. I’m guessing there must be a problem with esmond, but unsure where to start.

 

Thank you,

Dan Manzo