Grouper Users - Open Discussion List

[Shilen Patel <>]

Hi,

Just to be sure, you restarted the daemon after making the
includeFlattenedPrivileges config change?  Also, while the temp change log
is processing (at the 15 records per minute rate), are records being
inserted into grouper_change_log_entry?  If so, which categories/actions
are being inserted there?

Thanks!

-- Shilen

On 10/7/14 5:11 PM, "Mike Roszkowski" 
<>
 wrote:

>Hi,
>
>(Sorry if this is a duplicate message, I tried to send to the list and
>forgot
>that
>my email address had changed due to an Office 365 migration.)
>
>After upgrading to 2.1.5, we made some sweeping permission changes to our
>groups and as a result, we have about 115,000 records in
>grouper_change_log_entry_temp waiting to be processed into
>grouper_change_log_entry.
>
>We have a pair of VMs running grouper, with one machine running the
>grouper loader.
>
>We¹re seeing performance of about 15 records per minute going from the
>temp log to the change log, which means we¹re looking at 127 hours to get
>through the changes already queued (and there are more changes that are
>going to be made to group permissions which will increase the backlog of
>records even further).
>
>The majority of changes (> 100000) are permission changes, which we don¹t
>even end up consuming.
>
>So, a few questions:
>
>- Any guidance on what we might be able to do to speed up processing of
>change log changes? For example, would increasing the threads in the
>thread pool help? More memory for the grouper loader process (we¹re
>running with -Xms1024m and -Xmx2048m now)?
>
>- Since we don¹t really care about consuming the privilege changes, would
>it be safe to shut down and remove all the records in
>grouper_change_log_entry_temp with change_log_category='privilege¹?
>
>The (I think) relevant settings in grouper-loader.properties are:
>
>loader.use.transactions = false
>loader.thread.pool.size=10
>changeLog.changeLogTempToChangeLog.enable = true
>changeLog.includeFlattenedMemberships = true
>changeLog.includeFlattenedPrivileges = false
>changeLog.includeRolesWithPermissionChanges = false
>changeLog.includeNonFlattenedMemberships = false
>changeLog.includeNonFlattenedPrivileges = false
>
>We changed includeFlattenedPrivileges to false after the 115000 changes
>were queued, thinking that might help reduce the number of priv changes
>for our next set of changes.
>
>Thanks for any advice you can offer.
>Mike Roszkowski
>DoIT Middleware, University of Wisconsin-Madison