Subject: Grouper Users - Open Discussion List
[grouper-users] Decoupling PIT table updates and change log consumers
- From: Peter DiCamillo <>
- To: "" <>
- Subject: [grouper-users] Decoupling PIT table updates and change log consumers
- Date: Tue, 18 Aug 2015 10:32:32 -0400
We use a change log consumer in Grouper for real-time provisioning, and it's key part of our provisioning process. Ordinarily that works very well. However, we've had problems recently and in the past where events are not sent to the change log consumer for minutes, or even hours after they occurred. My understanding is that this is caused by the Grouper loader doing at least two things with each change before going on to the next one: updating the Point-In-Time (PIT) tables, and sending the change to the change log consumer. The delays happen when a change requires extensive updates to the PIT tables. That can happen for us, since we have extensive nesting of groups, a very large number of groups, and some groups with a very large number of memberships.
In our case, it's not a problem for us if it takes hours for the PIT tables to be updated. Our main use of them is checking the audit logs when debugging provisioning problems. However, as we become more dependent on real-time provisioning, delays in it can cause significant problems for us and for our users.
It occurred to me that perhaps it's not necessary for Grouper to couple the PIT table updating to sending changes to a change log consumer. As a future enhancement, could those functions be separated? Maybe maintain separate queues of changes for PIT table updates and for the change log consumer, instead of one shared table? That would allow the change log consumer process events as fast as possible. Also, we could ensure that we don't lose any change log consumer events, even if there's some problem with the PIT table updating.
- [grouper-users] Decoupling PIT table updates and change log consumers, Peter DiCamillo, 08/18/2015
- Re: [grouper-users] Decoupling PIT table updates and change log consumers, Shilen Patel, 08/18/2015
Archive powered by MHonArc 2.6.16.