Grouper Users - Open Discussion List

[Shilen Patel <>]

Hi Peter,

So the two are coupled together because the point in time data is used to
determine the flattened membership, privilege, and permission changes.

Usually it¹s that part (determining the flattened changes) that takes that
longest.  And if there are a lot of flattened changes, there would be a
lot of inserts into the change log table.

But I¹ll take a look at the code to see what can be done.  I may ping you
about getting more information about your environment so I can replicate
exactly what you¹re seeing.

Thanks!

- Shilen


On 8/18/15, 10:32 AM, "Peter DiCamillo" 
<>
 wrote:

>We use a change log consumer in Grouper for real-time provisioning, and
>it's key part of our provisioning process. Ordinarily that works very
>well. However, we've had problems recently and in the past where events
>are not sent to the change log consumer for minutes, or        even hours
>after they occurred. My understanding is that this is caused by the
>Grouper loader doing at least two things with each change before going
>on to the next one: updating the Point-In-Time (PIT) tables, and sending
>the change to the change log consumer. The delays happen when a change
>requires extensive updates to the PIT tables. That can happen for us,
>since we have extensive nesting of groups, a very large number of
>groups, and some groups with a very large number of memberships.
>
>In our case, it's not a problem for us if it takes hours for the PIT
>tables to be updated. Our main use of them is checking the audit logs
>when debugging provisioning problems. However, as we become more
>dependent on real-time provisioning, delays in it can cause significant
>problems for us and for our users.
>
>It occurred to me that perhaps it's not necessary for Grouper to couple
>the PIT table updating to sending changes to a change log consumer. As a
>future enhancement, could those functions be separated? Maybe maintain
>separate queues of changes for PIT table updates and for the change log
>consumer, instead of one shared table? That would allow the change log
>consumer process events as fast as possible. Also, we could ensure that
>we don't lose any change log consumer events, even if there's some
>problem with the PIT table updating.
>
>Peter