Grouper Developers Forum

["Gettes, Michael" <>]

I have tried to DISABLE all PSPNG related jobs in the grouper misc UI - this 
background fullsyncer continues.  There appears to be no way to stop this 
from happening.  The problem moved to a different group now - a smaller one 
with about 2000 memberships.  Again, running with latest tier/grouper - will 
update to a42/p5 when that is made available - hopefully today.

/mrg

> On Apr 26, 2019, at 11:04 PM, Gettes, Michael <> wrote:
> 
> Going against AD - this error keeps repeating.  It’s a group with 17K 
> members.  my pspng config is below.  This is on tier/grouper:latest 
> published today (2.4.0-a42-u23-w5-p4-20190426-rc1).
> 
> 2019-04-27T02:19:09+00:00 DAEMON:dev 
> grouper-api;grouper.log;grouper_dev;daemon;2019-04-26 22:19:09,381: 
> [FullSyncer(psp_UFADdev)-Thread] WARN  LdapSystem.performLdapModify(405) -  
> - UFADdev: Problem while modifying ldap system based on grouper 
> expectations. Starting to perform adaptive modifications based on data 
> already on server: [org.ldaptive.ModifyRequest@1988375000::modifyDn
> … (long list of DNs) … 
> controls=null, referralHandler=null, intermediateResponseHandlers=null]: 
> ENTRY_ALREADY_EXISTS
> 2019-04-27T02:15:38+00:00 DAEMON:dev 
> grouper-api;grouper.log;grouper_dev;daemon;2019-04-26 22:15:38,044: 
> [FullSyncer(psp_UFADdev)-Thread] ERROR 
> FullSyncProvisioner.fullSyncGroup(739) -  - FullSyncer(psp_UFADdev): 
> Problem doing full sync. Requeuing group 
> App:UFAD:UF:Groups:Services:Zoom:Service uflphi
> 2019-04-27T02:15:38+00:00 DAEMON:dev java.lang.IllegalStateException: The 
> active provisioner should be defined when creating LdapObjects
> 2019-04-27T02:15:38+00:00 DAEMON:dev  at 
> edu.internet2.middleware.grouper.pspng.LdapObject.<init>(LdapObject.java:95)
> 2019-04-27T02:15:38+00:00 DAEMON:dev  at 
> edu.internet2.middleware.grouper.pspng.LdapSystem.performLdapRead(LdapSystem.java:626)
> 2019-04-27T02:15:38+00:00 DAEMON:dev  at 
> edu.internet2.middleware.grouper.pspng.LdapSystem.performLdapRead(LdapSystem.java:596)
> 2019-04-27T02:15:38+00:00 DAEMON:dev  at 
> edu.internet2.middleware.grouper.pspng.LdapSystem.performLdapModify(LdapSystem.java:448)
> 2019-04-27T02:15:38+00:00 DAEMON:dev  at 
> edu.internet2.middleware.grouper.pspng.LdapSystem.performLdapModify(LdapSystem.java:377)
> 2019-04-27T02:15:38+00:00 DAEMON:dev  at 
> edu.internet2.middleware.grouper.pspng.LdapProvisioner.makeIndividualLdapChanges(LdapProvisioner.java:713)
> 2019-04-27T02:15:38+00:00 DAEMON:dev  at 
> edu.internet2.middleware.grouper.pspng.LdapProvisioner.finishProvisioningBatch(LdapProvisioner.java:455)
> 2019-04-27T02:15:38+00:00 DAEMON:dev  at 
> edu.internet2.middleware.grouper.pspng.FullSyncProvisioner.fullSyncGroup(FullSyncProvisioner.java:721)
> 2019-04-27T02:15:38+00:00 DAEMON:dev  at 
> edu.internet2.middleware.grouper.pspng.FullSyncProvisioner.processQueueItem(FullSyncProvisioner.java:433)
> 9-04-27T02:19:09+00:00 DAEMON:dev     at 
> edu.internet2.middleware.grouper.pspng.FullSyncProvisioner.thread_manageFullSyncProcessing(FullSyncProvisioner.java:254)
> 2019-04-27T02:19:09+00:00 DAEMON:dev  at 
> edu.internet2.middleware.grouper.pspng.FullSyncProvisioner$1.run(FullSyncProvisioner.java:146)
> 2019-04-27T02:19:09+00:00 DAEMON:dev  at 
> java.lang.Thread.run(Thread.java:748)
> 2019-04-27T02:53:48+00:00 DAEMON:dev 
> grouper-api;grouper.log;grouper_dev;daemon;2019-04-26 22:53:48,840: 
> [FullSyncer(psp_UFADdev)-Thread] WARN  
> ProvisionerCoordinator$ProvisioningStatus.lockForFullSyncWhenNoIncrementalIsUnderway(73)
>  -  - psp_UFADdev: Cannot start FullSync of 
> App:UFAD:UF:Groups:Services:Zoom:Service uflphi. Incremental provisioning 
> underway since Fri Apr 26 22:53:48 EDT 2019. We'll give up and move ahead 
> anyway in 300 seconds.
> 
> ———
> 
> AND this is a poorly formatted message that comes next:  not sure what i 
> should do to address it if anything.
> 
> 2019-04-27T02:19:09+00:00 DAEMON:dev 
> grouper-api;grouper.log;grouper_dev;daemon;2019-04-26 22:19:09,461: 
> [FullSyncer(psp_UFADdev)-Thread] WARN  
> Provisioner.warnAboutCacheSizeConcerns(620) -  - Cache is very full 
> (%.0f%%). Performance is much higher if 100.0 is large enough to hold the 
> number provisioned groups or subjects
> 
> ———
> 
> pspng config:
> 
> ldap.UFADdev.pass = XXXXXXXXXXXX
> ldap.UFADdev.user = XXXXXXXXXXXXXXXXXXXXXXXXX
> ldap.UFADdev.url = ldap://ufadXXXXXXXXXXXXXXXXX.ufl.edu
> ldap.UFADdev.pagedResultsSize = 1000
> ldap.UFADdev.timeout = 100000
> ldap.UFADdev.tls = false
> ldap.UFADdev.searchResultHandlers=org.ldaptive.handler.DnAttributeEntryHandler,edu.internet2.middleware.grouper.ldap.ldaptive.GrouperRangeEntryHandler
> # comma-delimited list of result codes (org.ldaptive.ResultCode) to ignore, 
> e.g. TIME_LIMIT_EXCEEDED, SIZE_LIMIT_EXCEEDED, PARTIAL_RESULTS
> ldap.UFADdev.searchIgnoreResultCodes=SIZE_LIMIT_EXCEEDED,ATTRIBUTE_OR_VALUE_EXISTS
> 
> changeLog.consumer.psp_UFADdev.provisionerName = psp_UFADdev
> changeLog.consumer.psp_UFADdev.class = 
> edu.internet2.middleware.grouper.pspng.PspChangelogConsumerShim
> changeLog.consumer.psp_UFADdev.type = 
> edu.internet2.middleware.grouper.pspng.LdapGroupProvisioner
> changeLog.consumer.psp_UFADdev.quartzCron = 0/15 * * * * ?
> changeLog.consumer.psp_UFADdev.ldapPoolName = UFADdev
> changeLog.consumer.psp_UFADdev.retryOnError = false
> changeLog.consumer.psp_UFADdev.isActiveDirectory = true
> changeLog.consumer.psp_UFADdev.grouperIsAuthoritative = true
> changeLog.consumer.psp_UFADdev.memberAttributeName = member
> changeLog.consumer.psp_UFADdev.memberAttributeValueFormat = 
> ${ldapUser.getDn()}
> changeLog.consumer.psp_UFADdev.groupSearchBaseDn = 
> OU=Grouper,OU=Groups,OU=UF,DC=dev-ad,DC=ufl,DC=edu
> changeLog.consumer.psp_UFADdev.groupCreationBaseDn = 
> OU=Grouper,OU=Groups,OU=UF,DC=dev-ad,DC=ufl,DC=edu
> changeLog.consumer.psp_UFADdev.allGroupsSearchFilter = objectclass=group
> changeLog.consumer.psp_UFADdev.singleGroupSearchFilter = 
> (&(objectclass=group)(cn=${group.name.replaceAll("(App:.+:UF:Groups:)(.+):(.*)","$3").replace(":","_")}))
> changeLog.consumer.psp_UFADdev.groupSearchAttributes = 
> cn,gidNumber,objectclass,samAccountName,name
> changeLog.consumer.psp_UFADdev.groupCreationLdifTemplate = dn: 
> ${utils.bushyDn("${group.name.replaceAll('(App:.+:UF:Groups:)(.+):(.*)','$2:'+'$3'.replace(':','_'))}","cn","ou")}||objectclass:
>  group
> changeLog.consumer.psp_UFADdev.userSearchBaseDn = 
> OU=People,OU=UF,DC=dev-ad,DC=ufl,DC=edu
> changeLog.consumer.psp_UFADdev.userSearchFilter = 
> userPrincipalName=${subject.id}
> changeLog.consumer.psp_UFADdev.userSearchAttributes = 
> dn,cn,uid,mail,samAccountName,uidNumber,objectclass,memberOf,userPrincipalName
> changeLog.consumer.psp_UFADdev.groupSearch_batchSize = 100
> changeLog.consumer.psp_UFADdev.ldapSearchResultPagingSize = 1000
> changelog.consumer.psp_UFADdev.grouperSubjectCacheSize = 400000
> changelog.consumer.psp_UFADdev.grouperGroupCacheSize = 50000
> changelog.consumer.psp_UFADdev.ldapUserCacheSize = 200000
> changelog.consumer.psp_UFADdev.ldapUserCacheTime_secs = 3600
> changelog.consumer.psp_UFADdev.maxValuesToChangePerOperation = 500
> changeLog.consumer.psp_UFADdev.needsTargetSystemUsers = true
> changeLog.consumer.psp_UFADdev.supportsEmptyGroups = true
> changeLog.consumer.psp_UFADdev.createMissingUsers = true
> changeLog.consumer.psp_UFADdev.userCreationBaseDn = 
> OU=People,OU=UF,DC=dev-ad,DC=ufl,DC=edu
> changeLog.consumer.psp_UFADdev.userCreationLdifTemplate = dn: 
> CN=${subject.getId().toLowerCase().replaceAll('@ufl.edu','').replaceAll('\\.$','')}||sAMAccountName:
>  
> ${subject.getId().toLowerCase().replaceAll('@ufl.edu','').replaceAll('\\.$','')}||objectclass:
>  top||objectclass: person||objectclass: user||sn: 
> ${subject.getId().toLowerCase().replaceAll('@ufl.edu','')}||userPrincipalName:
>  ${subject.getId()}||displayName: ${subject.name}||cn: 
> ${subject.getId().toLowerCase().replaceAll('@ufl.edu','')}