perfsonar-user - Re: [perfsonar-user] perfSONAR Grafana Integration
Subject: perfSONAR User Q&A and Other Discussion
List archive
- From: Shawn McKee <>
- To: Mark Feit <>
- Cc: Murilo Vetter <>, perfsonar-user <>, monipe-des <>
- Subject: Re: [perfsonar-user] perfSONAR Grafana Integration
- Date: Wed, 17 Jul 2019 14:25:28 -0700
Shawn
--Murilo Vetter writes:
Does anyone know how was made the integration of perfSONAR measurements to Grafana?
I found something in this document:
Shawn can speak directly to what OSG/WLCG is doing, but I can give you some more-general information.
Historically, the only method for getting perfSONAR data into other systems has been to send it to Esmond and write programs to pull it from there.
One of the things pScheduler brought with it from day one is archiver plugins that can send results as they’re produced. MeshConfig had been reprogrammed to use the Esmond plugin, as does the out-of-the-box pSConfig template. However, there are also plugins that will send results to syslog, HTTP(S), RabbitMQ and, if you’re a glutton for punishment, SNMP traps. A plugin for Kafka is in the pipeline for release with 4.2.0. We’ve tried to cover as many of the common cases as we could, but it’s easy to write new ones for things we haven’t.
All of the plugins receive a blob of JSON that includes the result and as much information about the task and run as possible. There’s a mechanism for the JSON to be transformed using a jq script before it’s handed to the plugin, so if you need special formatting, that can be done by pScheduler on your perfSONAR nodes and you don’t have to run a separate system to do it. (Some of this is discussed in a webinar we did on jq last year: https://youtu.be/FrT6R75M3BE?t=2678.)
I did some throwaway some experimentation last year that sent data into ElasticSearch using the HTTP plugin and pulled it for display using Grafana. It wasn’t much and the data probably wasn’t very accurate, but it was enough to prove the concept. This is how it looked: https://slack-files.com/T073A2Z0D-FEM1QA4LC-04a7cd4665.
The development team is in the early stages of looking at ways to improve the visualization component of perfSONAR. What we have now is mostly home-grown code that requires changing to add new types of tests which, as we found out with MeshConfig, isn’t such a good thing. Expect to hear more about this at TechEx in December.
Hope that’s useful.
--Mark
To unsubscribe from this list: https://lists.internet2.edu/sympa/signoff/perfsonar-user
- [perfsonar-user] perfSONAR Grafana Integration, Murilo Vetter, 07/17/2019
- <Possible follow-up(s)>
- Re: [perfsonar-user] perfSONAR Grafana Integration, Mark Feit, 07/17/2019
- Re: [perfsonar-user] perfSONAR Grafana Integration, Shawn McKee, 07/17/2019
- Re: [perfsonar-user] perfSONAR Grafana Integration, Murilo Vetter, 07/18/2019
- Re: [perfsonar-user] perfSONAR Grafana Integration, Michael Johnson, 07/18/2019
- Re: [perfsonar-user] perfSONAR Grafana Integration, Murilo Vetter, 07/18/2019
- Re: [perfsonar-user] perfSONAR Grafana Integration, Michael Johnson, 07/18/2019
- Re: [perfsonar-user] perfSONAR Grafana Integration, Murilo Vetter, 07/18/2019
- Re: [perfsonar-user] perfSONAR Grafana Integration, Shawn McKee, 07/17/2019
Archive powered by MHonArc 2.6.19.