perfSONAR User Q&A and Other Discussion

[Shawn McKee <>]

Hi Murilo,

Sorry for the delay in responding!

The setup to get the perfSONAR data from WLCG/OSG into Grafana is a little complicated.    There are various presentations which have described the details (URL below).

Basically this is the "pipeline" we have setup to get data from our toolkit instances and get the data to various destinations:

The code which gathers the data and puts it on the bus, as well as the code which reads data off the bus and puts it into Elasticsearch is on GitHub at https://github.com/sand-ci

The most recent relevant presentation was the the 6th SIG PMV meeting at the beginning of the month:

https://wiki.geant.org/display/PMV/6th+SIG-PMV+Meeting+@+Dublin  

My talk is at https://wiki.geant.org/download/attachments/123774072/SIG-PMV-DUblin-SAND-McKee-02-Jul-2019.pdf?version=2&modificationDate=1562073472330&api=v2

We can take this discussion off to a separate thread if you want to follow up or have detailed questions about what we have done.

Thanks,

Shawn

On Wed, Jul 17, 2019 at 1:56 PM Mark Feit <> wrote:

Murilo Vetter writes:

 

Does anyone know how was made the integration of perfSONAR measurements to Grafana?

 

I found something in this document:

https://indico.cern.ch/event/637013/contributions/2739243/attachments/1539101/2412730/HEPiX_WLCG-neworking_fall2017.pdf

 

Shawn can speak directly to what OSG/WLCG is doing, but I can give you some more-general information.

 

Historically, the only method for getting perfSONAR data into other systems has been to send it to Esmond and write programs to pull it from there.

 

One of the things pScheduler brought with it from day one is archiver plugins that can send results as they’re produced.  MeshConfig had been reprogrammed to use the Esmond plugin, as does the out-of-the-box pSConfig template.  However, there are also plugins that will send results to syslog, HTTP(S), RabbitMQ and, if you’re a glutton for punishment, SNMP traps.  A plugin for Kafka is in the pipeline for release with 4.2.0.  We’ve tried to cover as many of the common cases as we could, but it’s easy to write new ones for things we haven’t.

 

All of the plugins receive a blob of JSON that includes the result and as much information about the task and run as possible.  There’s a mechanism for the JSON to be transformed using a jq script before it’s handed to the plugin, so if you need special formatting, that can be done by pScheduler on your perfSONAR nodes and you don’t have to run a separate system to do it.  (Some of this is discussed in a webinar we did on jq  last year:  https://youtu.be/FrT6R75M3BE?t=2678.)

 

I did some throwaway some experimentation last year that sent data into ElasticSearch using the HTTP plugin and pulled it for display using Grafana.  It wasn’t much and the data probably wasn’t very accurate, but it was enough to prove the concept.  This is how it looked:  https://slack-files.com/T073A2Z0D-FEM1QA4LC-04a7cd4665.

 

The development team is in the early stages of looking at ways to improve the visualization component of perfSONAR.  What we have now is mostly home-grown code that requires changing to add new types of tests which, as we found out with MeshConfig, isn’t such a good thing.  Expect to hear more about this at TechEx in December.

 

Hope that’s useful.

 

--Mark

 

--
To unsubscribe from this list: https://lists.internet2.edu/sympa/signoff/perfsonar-user