Skip to Content.
Sympa Menu

grouper-users - [grouper-users] Grouper ops and insight from a DB perspective

Subject: Grouper Users - Open Discussion List

List archive

[grouper-users] Grouper ops and insight from a DB perspective

Chronological Thread 
  • From: "Gettes, Michael" <>
  • To: "" <>
  • Subject: [grouper-users] Grouper ops and insight from a DB perspective
  • Date: Mon, 18 Jun 2018 19:22:44 +0000
  • Accept-language: en-US
  • Ironport-phdr: 9a23:/MSvahBhomvPY6dvX44SUyQJP3N1i/DPJgcQr6AfoPdwSPT7o8bcNUDSrc9gkEXOFd2Cra4c1qyO6+jJYi8p2d65qncMcZhBBVcuqP49uEgeOvODElDxN/XwbiY3T4xoXV5h+GynYwAOQJ6tL1LdrWev4jEMBx7xKRR6JvjvGo7Vks+7y/2+94fcbglUhDexe69+IAmrpgjNq8cahpdvJLwswRXTuHtIfOpWxWJsJV2Nmhv3+9m98p1+/SlOovwt78FPX7n0cKQ+VrxYES8pM3sp683xtBnMVhWA630BWWgLiBVIAgzF7BbnXpfttybxq+Rw1DWGMcDwULs5Xymp4aV2Rx/ykCoJNyA3/nzLisJ+j6xVpxOhqABwzIPPZYGZKP9+c7ndfd8GR2dMWNtaWSxbAoO7aosCF+QNM+dGoInhu1sBswC+CRGxD+3oyz9Hmn/20rck3OQlHw3NwQstH90Qv3TIsNX6KrwSXPuozKnGyTXPde1Z1irg6IXRdB0qvPKCXapofMbM10UiFBnJgkuOpYHnJT+Y1OsAv3KF4+dgWu+jk3Mrpx1rrjWhx8ogkJTFi4wax1ze6Cl0wJ45JdOmR0N4fdKpFZ5duiKeOodqXswvRn1ntz0gxbAGvJO3YioHx485yxPadvOIaJaE7xDlWe2MOzl3nmhld6i6hxuq8Uiv1On8Vs6s3VZRqypFjsHMumoQ1xzd7siLUOVy8Vq82TqX1gDT7P9LIVwsmKbGNZIswaQ8moQNvUnBBCP7mkr7gLWIekgk9eWk8+Hnba/npp+YOY90kAb+MqE2l8ywH+s4LhQOUHaA+emnyb3j80n5TK9UgfIrj6nVqIraKtgDpq6lHw9V1Z4u6xmlDzehzdQYhWcILEhcdBKalIjmJUvOIOvjAPe7glSsiytrx+vYMrH7A5XNKGTDn6n7fbZ79UFc1BQ/wcpB6J1JF7FSaM70DwX+rtvFFhIjdhGvzvz8INR7yo4EX2+TWOmUPL6Y+QuH/OUyO+SWIZIOtSzmA/kj+/P0i3IlwxkQcbT/jrUNb3XtVNtrP1mUej6kp9YKEW4HukB2GOPnlEGPShZOaXL0Uq4hsGJoQLm6BJvOE9j+yIeK2z22S8Vb

Over the last several weeks I have been doing a bit of testing and demo using
the TIER grouper package. Through this effort (and based on my prior
experiences with Grouper) I needed to be able to better “see" what was going
on. I offer up these 2 bits of SQL. I am using mySQL (MariaDB from centos
as packaged by TIER) and a separate instance of MariaDB v10 as the grouper
DB. (I have two TIER/grouper instances on my laptop).

Overall, I recommend setting changeLog.changeLogConsumerBatchSize = 100 (or
50) in which helps with the updating of information
of where the consumers are at which helps with both SQL queries. Chris, is
there something that influences the BatchSize for the temp changelog which
appears to be 1000? I couldn’t seem to locate it. Also, how can I tell
where in processing is the tempChangelog consumer - like how many to go?

The first is tiny and shows the consumers and where they are in processing.
When they last ran (timestamp and age) and how many entries to go to be all
caught up. For me, this takes about 3-5ms to run. This might also be useful
from a monitoring perspective - maybe send notices over certain thresholds
for certain consumers when so “old” and connected to Nagios and such.

The second is similar to what Chris installed in the latest grouper under
Miscellaneous >> loader jobs which I had shared as part of the Grouper
Deployment Guide work. Chris has broken it out on a per loader job basis
which is definitely useful. But, for me, from an operations perspective, you
lose the larger view. There’s more than just those specific loader jobs.
This SQL shows what’s going on and if you run it like every 10 seconds (like
the first can be run) you start to see how things change and the behavior of
grouper itself. It tries to thin out the noise of grouper consumers but show
the real activity.

I hope others find these useful and maybe Chris and crew would consider some
sort of ops display (auto-updating) of what’s happening with grouper.

And, for the record, anyone who says grouper isn’t so useful or the code is
bad or whatever - well, they just don’t know what they’re talking about.
Grouper can be the basis of change for any IT org let alone a
college/University. No software is perfect but Grouper definitely addresses
many issues in Higher Ed. You just have to dive in. I, for one, very much
appreciate Chris, Shilen, Bert (and I am sure there are others and I don’t
know who you are but please don’t be insulted if I am not listing you). You
guys do great stuff. THANK YOU! (Looking forward to 2.4). Also, a
shout-out to Hubing, Caskey, Gasper, Jokl and whomever else is involved in
the TIER packaging. You guys also rock! The combination of all this makes
figuring things out fun… especially when you don’t have the resources to get
it done - but, that’s another story!

The Grouper-Consumers SQL generates something like the attached image. The
Grouper-Loader-ops.sql is a bit more involved in what it displays so you just
have to run it. Both of these are simple queries - nothing is modified. And
if you’re interested, I use DBeaver-CE as my SQL tool of choice.

Attachment: Grouper-Consumers.sql
Description: Grouper-Consumers.sql

Attachment: Grouper-Loader-ops.sql
Description: Grouper-Loader-ops.sql

Attachment: PastedGraphic-1.png
Description: PastedGraphic-1.png

  • [grouper-users] Grouper ops and insight from a DB perspective, Gettes, Michael, 06/18/2018

Archive powered by MHonArc 2.6.19.

Top of Page