Skip to Content.
Sympa Menu

grouper-users - [grouper-users] Slow PSPNG Full Sync

Subject: Grouper Users - Open Discussion List

List archive

[grouper-users] Slow PSPNG Full Sync

Chronological Thread 
  • From: Ryan Rumbaugh <>
  • To: "" <>
  • Subject: [grouper-users] Slow PSPNG Full Sync
  • Date: Thu, 12 Sep 2019 14:06:47 +0000
  • Arc-authentication-results: i=1; 1; spf=pass; dmarc=pass action=none; dkim=pass; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed;; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=nHUTm1LzZKC0dqJg1MyPPd0SnyrS6Wc3IOAeZ4Oc82A=; b=Ru6QZsVfBUiudwEjHuX8ENQocRAZEITOBnsJ6aWliE1KKt/qOa30n2RudZebP0HeWZYBewGsAz01XF8ucU4YEeJ+BpZiFGJHZaaCV00PM+KIeyf0yYOJ45SrWEPMtQDeNUF9orADHS9pIpr1TprE/cgmoszuDrCEDlmVjCdsBigY630X/3yEzaO+bNdUqXtzT5ubA9Yf+hd2OwZ5LiFZZCOysdC4qmDgBRB4pahten+ZQq9AVfngMZqO63XF0Ix+ImZkqSDdiBmbee773FrjKQnDXvzarO12cBnr5BWhqoXpFiCHVGP4intEvgevj4KGwa2PoDOX0zfXOo6XVp1z4w==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901;; cv=none; b=GWjoJuuWt9kKsH2fF9egt3t2BP/XO4lq3wRkOwvJqHmszL9GJxwUSkwpUJxSLdawZmBvO8faId4TQEf7Cl/v9xY5OrXlcVgJIDWjBo035XI2c1+/b39djfmb+zZfjckVsCjUcKL51mfr00YLfuDzdSlLhdT5cjPa2u3R1gMN/LQQN6hNgaXCqLqeLbVmsx2yjZijj4bc58r/KHJ66RBNhx6l/Buk5Hurarma+67Z30wCcXCNdXJXAQbvmylyfi0E/wibHPriT19PfH5ZhzbhYM5vDdvlNtHc8zCRjd01480v1B9DxH8BMPosMDoJxFYigXLYnDxLDvd9jEgrffDAdQ==

Hi all,


We are using ITAP versions of Grouper and after recently upgrading from 2.4.0-a11-u5-w0-p0-181201-test to 2.4.0-a47-u25-w5-p6-20190515-rc1 our pspng full sync’s have been running incredibly slow. As I write this we have one AD full sync that is still running after 3 days.


The logs below are a typical collection of errors and warnings when Grouper is attempting to sync very large groups. The group in this case, app:nu:unl:Directory:DirectoryVisible, has 48k memberships for example.


grouper-api;grouper_error.log;;;2019-09-12 13:49:49,917: [FullSyncer(pspng_ad_lincoln)-Thread] WARN LdapProvisioner.makeCoalescedLdapChanges(651) - - (THIS WILL BE RETRIED) Problem doing coalesced ldap modification


grouper-api;grouper_error.log;;;2019-09-12 13:49:49,917: [FullSyncer(pspng_ad_lincoln)-Thread] WARN LdapProvisioner.finishProvisioningBatch(449) - - (THIS WILL BE RETRIED) Optimized, coalesced ldap provisioning failed: Coalesced LDAP Modification failed: javax.naming.NameAlreadyBoundException: [LDAP: error code 68 - 00000562: UpdErr: DSID-031A1261, problem 6005 (ENTRY_EXISTS), data 0


grouper-api;grouper_error.log;;;2019-09-12 13:49:49,917: [FullSyncer(pspng_ad_lincoln)-Thread] WARN LdapProvisioner.finishProvisioningBatch(450) - - RETRYING: Performing slower, unoptimized ldap provisioning after optimized provisioning failed


grouper-api;grouper_error.log;;;2019-09-12 13:50:19,623: [FullSyncer(pspng_ad_lincoln)-Thread] WARN LdapSystem.performLdapModify(405) - - ad_lincoln: Problem while modifying ldap system based on grouper expectations. Starting to perform adaptive modifications based on data already on server:


grouper-api;grouper_error.log;;;2019-09-12 13:50:20,719: [FullSyncer(pspng_ad_lincoln)-Thread] ERROR FullSyncProvisioner.fullSyncGroup(739) - - FullSyncer(pspng_ad_lincoln): Problem doing full sync. Requeuing group app:nu:unl:Directory:DirectoryVisible


It does make sense that the full sync process is running so long if Grouper is continually trying to requeue these large groups. Is there a configuration setting I may have missed or maybe a setting on AD that needs to be adjusted in order for the optimized coalesced sync to function?


Also, very oddly, I see the warning log entry below about the cache being very full and this particular log file was supposed to be corrected in pspng patch 6…I don’t know why I still see it in my logs. When I access a running container and run gsh I see it recognizing that I’m on pspng patch level 6. *shrug*


Sep 12 08:49:31 localhost abf31ba01def[1245]: grouper-api;grouper_error.log;;;2019-09-12 13:49:31,971: [DefaultQuartzScheduler_Worker-8] WARN Provisioner.warnAboutCacheSizeConcerns(628) - - Cache is very full (%.0f%%). Performance is much higher if 100.0 is large enough to hold the number provisioned groups or subjects




Ryan Rumbaugh


Archive powered by MHonArc 2.6.19.

Top of Page