Grouper Users - Open Discussion List

[Ryan Rumbaugh <>]

Hi all, just wanted to bump this thread and see if there were any suggestions?

 

At this point we’re going to look at scheduling full sync’s for the weekend and have the incremental process take care of weekday updates.

 

Thanks again.

 

--

Ryan Rumbaugh

 

 

 

From: <> on behalf of Ryan Rumbaugh <>
Reply-To: Ryan Rumbaugh <>
Date: Thursday, September 12, 2019 at 9:07 AM
To: "" <>
Subject: [grouper-users] Slow PSPNG Full Sync

 

Hi all,

 

We are using ITAP versions of Grouper and after recently upgrading from 2.4.0-a11-u5-w0-p0-181201-test to 2.4.0-a47-u25-w5-p6-20190515-rc1 our pspng full sync’s have been running incredibly slow. As I write this we have one AD full sync that is still running after 3 days.

 

The logs below are a typical collection of errors and warnings when Grouper is attempting to sync very large groups. The group in this case, app:nu:unl:Directory:DirectoryVisible, has 48k memberships for example.

 

grouper-api;grouper_error.log;;;2019-09-12 13:49:49,917: [FullSyncer(pspng_ad_lincoln)-Thread] WARN LdapProvisioner.makeCoalescedLdapChanges(651) - - (THIS WILL BE RETRIED) Problem doing coalesced ldap modification

 

grouper-api;grouper_error.log;;;2019-09-12 13:49:49,917: [FullSyncer(pspng_ad_lincoln)-Thread] WARN LdapProvisioner.finishProvisioningBatch(449) - - (THIS WILL BE RETRIED) Optimized, coalesced ldap provisioning failed: Coalesced LDAP Modification failed: javax.naming.NameAlreadyBoundException: [LDAP: error code 68 - 00000562: UpdErr: DSID-031A1261, problem 6005 (ENTRY_EXISTS), data 0

 

grouper-api;grouper_error.log;;;2019-09-12 13:49:49,917: [FullSyncer(pspng_ad_lincoln)-Thread] WARN LdapProvisioner.finishProvisioningBatch(450) - - RETRYING: Performing slower, unoptimized ldap provisioning after optimized provisioning failed

 

grouper-api;grouper_error.log;;;2019-09-12 13:50:19,623: [FullSyncer(pspng_ad_lincoln)-Thread] WARN LdapSystem.performLdapModify(405) - - ad_lincoln: Problem while modifying ldap system based on grouper expectations. Starting to perform adaptive modifications based on data already on server:

 

grouper-api;grouper_error.log;;;2019-09-12 13:50:20,719: [FullSyncer(pspng_ad_lincoln)-Thread] ERROR FullSyncProvisioner.fullSyncGroup(739) - - FullSyncer(pspng_ad_lincoln): Problem doing full sync. Requeuing group app:nu:unl:Directory:DirectoryVisible

 

It does make sense that the full sync process is running so long if Grouper is continually trying to requeue these large groups. Is there a configuration setting I may have missed or maybe a setting on AD that needs to be adjusted in order for the optimized coalesced sync to function?

 

Also, very oddly, I see the warning log entry below about the cache being very full and this particular log file was supposed to be corrected in pspng patch 6…I don’t know why I still see it in my logs. When I access a running container and run gsh I see it recognizing that I’m on pspng patch level 6. *shrug*

 

Sep 12 08:49:31 localhost abf31ba01def[1245]: grouper-api;grouper_error.log;;;2019-09-12 13:49:31,971: [DefaultQuartzScheduler_Worker-8] WARN Provisioner.warnAboutCacheSizeConcerns(628) - - Cache is very full (%.0f%%). Performance is much higher if 100.0 is large enough to hold the number provisioned groups or subjects

 

Thanks!

--

Ryan Rumbaugh