perfSONAR User Q&A and Other Discussion

[Michael Johnson <>]

Hi Murilo,

As others have mentioned, this is something we intended to support in 
perfSONAR in the future.

We also have an experiment here at IU doing this -- Shawn's pipeline is 
complicated, but getting ps data into elasticsearch doesn't have to be. You 
could use the http archiver to post data to ES directly, or use the rabbitmq 
archiver to put the data in a rabbit queue, which you then ingest with 
logstash. We are doing the latter for our experiment, and it's working well 
so far.

I don't have anything readily sharable at the moment ... but, stay tuned.

Thanks,
Michael

On Thu, Jul 18, 2019 at 08:33:22AM -0300, Murilo Vetter wrote:
> Hi Shawn/Mark,
>
> I am impressed with the results of perfSONAR integration.
> In this first moment I was wondering how things evolved and find better ways 
> to represent perfSONAR data.
> Now I have new possibilities to look forward.
> In the right moment, I will come back to discuss this topic.
> For now, thanks for you information.
>
> Regards,
> Murilo Vetter
> RNP/Brazil
>
>
>
> De: "Shawn McKee" <>
> Para: "Mark Feit" <>
> Cc: "Murilo Vetter" <>, "perfsonar-user" 
> <>, "monipe-des" <>
> Enviadas: Quarta-feira, 17 de julho de 2019 18:25:28
> Assunto: Re: [perfsonar-user] perfSONAR Grafana Integration
>
> Hi Murilo,
> Sorry for the delay in responding!
>
> The setup to get the perfSONAR data from WLCG/OSG into Grafana is a little 
> complicated. There are various presentations which have described the details 
> (URL below).
>
> Basically this is the "pipeline" we have setup to get data from our toolkit 
> instances and get the data to various destinations:
>
>
>
> The code which gathers the data and puts it on the bus, as well as the code 
> which reads data off the bus and puts it into Elasticsearch is on GitHub at [ 
> https://github.com/sand-ci | https://github.com/sand-ci ]
>
> The most recent relevant presentation was the the 6th SIG PMV meeting at the 
> beginning of the month:
> [ https://wiki.geant.org/display/PMV/6th+SIG-PMV+Meeting+@+Dublin | 
> https://wiki.geant.org/display/PMV/6th+SIG-PMV+Meeting+@+Dublin ]
> My talk is at [ 
> https://wiki.geant.org/download/attachments/123774072/SIG-PMV-DUblin-SAND-McKee-02-Jul-2019.pdf?version=2&modificationDate=1562073472330&api=v2
>  | 
> https://wiki.geant.org/download/attachments/123774072/SIG-PMV-DUblin-SAND-McKee-02-Jul-2019.pdf?version=2&modificationDate=1562073472330&api=v2
>  ]
>
> We can take this discussion off to a separate thread if you want to follow up 
> or have detailed questions about what we have done.
>
> Thanks,
>
> Shawn
>
>
> On Wed, Jul 17, 2019 at 1:56 PM Mark Feit < [  | 
>  ] > wrote:
>
>
>
>
>
> Murilo Vetter writes:
>
>
>
>
>
> Does anyone know how was made the integration of perfSONAR measurements to 
> Grafana?
>
>
>
>
>
> I found something in this document:
>
>
> [ 
> https://indico.cern.ch/event/637013/contributions/2739243/attachments/1539101/2412730/HEPiX_WLCG-neworking_fall2017.pdf
>  | 
> https://indico.cern.ch/event/637013/contributions/2739243/attachments/1539101/2412730/HEPiX_WLCG-neworking_fall2017.pdf
>  ]
>
>
>
>
> Shawn can speak directly to what OSG/WLCG is doing, but I can give you some 
> more-general information.
>
>
>
> Historically, the only method for getting perfSONAR data into other systems 
> has been to send it to Esmond and write programs to pull it from there.
>
>
>
> One of the things pScheduler brought with it from day one is archiver plugins 
> that can send results as they’re produced. MeshConfig had been reprogrammed 
> to use the Esmond plugin, as does the out-of-the-box pSConfig template. 
> However, there are also plugins that will send results to syslog, HTTP(S), 
> RabbitMQ and, if you’re a glutton for punishment, SNMP traps. A plugin for 
> Kafka is in the pipeline for release with 4.2.0. We’ve tried to cover as many 
> of the common cases as we could, but it’s easy to write new ones for things 
> we haven’t.
>
>
>
> All of the plugins receive a blob of JSON that includes the result and as 
> much information about the task and run as possible. There’s a mechanism for 
> the JSON to be transformed using a jq script before it’s handed to the 
> plugin, so if you need special formatting, that can be done by pScheduler on 
> your perfSONAR nodes and you don’t have to run a separate system to do it. 
> (Some of this is discussed in a webinar we did on jq last year: [ 
> https://youtu.be/FrT6R75M3BE?t=2678 | https://youtu.be/FrT6R75M3BE?t=2678 ] .)
>
>
>
> I did some throwaway some experimentation last year that sent data into 
> ElasticSearch using the HTTP plugin and pulled it for display using Grafana. 
> It wasn’t much and the data probably wasn’t very accurate, but it was enough 
> to prove the concept. This is how it looked: [ 
> https://slack-files.com/T073A2Z0D-FEM1QA4LC-04a7cd4665 |
> https://slack-files.com/T073A2Z0D-FEM1QA4LC-04a7cd4665 ] .
>
>
>
> The development team is in the early stages of looking at ways to improve the 
> visualization component of perfSONAR. What we have now is mostly home-grown 
> code that requires changing to add new types of tests which, as we found out 
> with MeshConfig, isn’t such a good thing. Expect to hear more about this at 
> TechEx in December.
>
>
>
> Hope that’s useful.
>
>
>
> --Mark
>
>
> --
> To unsubscribe from this list: [ 
> https://lists.internet2.edu/sympa/signoff/perfsonar-user | 
> https://lists.internet2.edu/sympa/signoff/perfsonar-user ]

> --
> To unsubscribe from this list: 
> https://lists.internet2.edu/sympa/signoff/perfsonar-user


--
Michael Johnson
GlobalNOC DevOps Engineer

Attachment: smime.p7s
Description: S/MIME cryptographic signature