Grouper Developers Forum

["Gettes, Michael" <>]

Now running with latest TIER package - confirmed a42 and p5.  No change in 
behavior of all aspects of problems reported in this thread.

/mrg

> On Apr 29, 2019, at 8:37 AM, Gettes, Michael <> wrote:
> 
> I have tried to DISABLE all PSPNG related jobs in the grouper misc UI - 
> this background fullsyncer continues.  There appears to be no way to stop 
> this from happening.  The problem moved to a different group now - a 
> smaller one with about 2000 memberships.  Again, running with latest 
> tier/grouper - will update to a42/p5 when that is made available - 
> hopefully today.
> 
> /mrg
> 
>> On Apr 26, 2019, at 11:04 PM, Gettes, Michael <> wrote:
>> 
>> Going against AD - this error keeps repeating.  It’s a group with 17K 
>> members.  my pspng config is below.  This is on tier/grouper:latest 
>> published today (2.4.0-a42-u23-w5-p4-20190426-rc1).
>> 
>> 2019-04-27T02:19:09+00:00 DAEMON:dev 
>> grouper-api;grouper.log;grouper_dev;daemon;2019-04-26 22:19:09,381: 
>> [FullSyncer(psp_UFADdev)-Thread] WARN  LdapSystem.performLdapModify(405) - 
>>  - UFADdev: Problem while modifying ldap system based on grouper 
>> expectations. Starting to perform adaptive modifications based on data 
>> already on server: [org.ldaptive.ModifyRequest@1988375000::modifyDn
>> … (long list of DNs) … 
>> controls=null, referralHandler=null, intermediateResponseHandlers=null]: 
>> ENTRY_ALREADY_EXISTS
>> 2019-04-27T02:15:38+00:00 DAEMON:dev 
>> grouper-api;grouper.log;grouper_dev;daemon;2019-04-26 22:15:38,044: 
>> [FullSyncer(psp_UFADdev)-Thread] ERROR 
>> FullSyncProvisioner.fullSyncGroup(739) -  - FullSyncer(psp_UFADdev): 
>> Problem doing full sync. Requeuing group 
>> App:UFAD:UF:Groups:Services:Zoom:Service uflphi
>> 2019-04-27T02:15:38+00:00 DAEMON:dev java.lang.IllegalStateException: The 
>> active provisioner should be defined when creating LdapObjects
>> 2019-04-27T02:15:38+00:00 DAEMON:dev         at 
>> edu.internet2.middleware.grouper.pspng.LdapObject.<init>(LdapObject.java:95)
>> 2019-04-27T02:15:38+00:00 DAEMON:dev         at 
>> edu.internet2.middleware.grouper.pspng.LdapSystem.performLdapRead(LdapSystem.java:626)
>> 2019-04-27T02:15:38+00:00 DAEMON:dev         at 
>> edu.internet2.middleware.grouper.pspng.LdapSystem.performLdapRead(LdapSystem.java:596)
>> 2019-04-27T02:15:38+00:00 DAEMON:dev         at 
>> edu.internet2.middleware.grouper.pspng.LdapSystem.performLdapModify(LdapSystem.java:448)
>> 2019-04-27T02:15:38+00:00 DAEMON:dev         at 
>> edu.internet2.middleware.grouper.pspng.LdapSystem.performLdapModify(LdapSystem.java:377)
>> 2019-04-27T02:15:38+00:00 DAEMON:dev         at 
>> edu.internet2.middleware.grouper.pspng.LdapProvisioner.makeIndividualLdapChanges(LdapProvisioner.java:713)
>> 2019-04-27T02:15:38+00:00 DAEMON:dev         at 
>> edu.internet2.middleware.grouper.pspng.LdapProvisioner.finishProvisioningBatch(LdapProvisioner.java:455)
>> 2019-04-27T02:15:38+00:00 DAEMON:dev         at 
>> edu.internet2.middleware.grouper.pspng.FullSyncProvisioner.fullSyncGroup(FullSyncProvisioner.java:721)
>> 2019-04-27T02:15:38+00:00 DAEMON:dev         at 
>> edu.internet2.middleware.grouper.pspng.FullSyncProvisioner.processQueueItem(FullSyncProvisioner.java:433)
>> 9-04-27T02:19:09+00:00 DAEMON:dev    at 
>> edu.internet2.middleware.grouper.pspng.FullSyncProvisioner.thread_manageFullSyncProcessing(FullSyncProvisioner.java:254)
>> 2019-04-27T02:19:09+00:00 DAEMON:dev         at 
>> edu.internet2.middleware.grouper.pspng.FullSyncProvisioner$1.run(FullSyncProvisioner.java:146)
>> 2019-04-27T02:19:09+00:00 DAEMON:dev         at 
>> java.lang.Thread.run(Thread.java:748)
>> 2019-04-27T02:53:48+00:00 DAEMON:dev 
>> grouper-api;grouper.log;grouper_dev;daemon;2019-04-26 22:53:48,840: 
>> [FullSyncer(psp_UFADdev)-Thread] WARN  
>> ProvisionerCoordinator$ProvisioningStatus.lockForFullSyncWhenNoIncrementalIsUnderway(73)
>>  -  - psp_UFADdev: Cannot start FullSync of 
>> App:UFAD:UF:Groups:Services:Zoom:Service uflphi. Incremental provisioning 
>> underway since Fri Apr 26 22:53:48 EDT 2019. We'll give up and move ahead 
>> anyway in 300 seconds.
>> 
>> ———
>> 
>> AND this is a poorly formatted message that comes next:  not sure what i 
>> should do to address it if anything.
>> 
>> 2019-04-27T02:19:09+00:00 DAEMON:dev 
>> grouper-api;grouper.log;grouper_dev;daemon;2019-04-26 22:19:09,461: 
>> [FullSyncer(psp_UFADdev)-Thread] WARN  
>> Provisioner.warnAboutCacheSizeConcerns(620) -  - Cache is very full 
>> (%.0f%%). Performance is much higher if 100.0 is large enough to hold the 
>> number provisioned groups or subjects
>> 
>> ———
>> 
>> pspng config:
>> 
>> ldap.UFADdev.pass = XXXXXXXXXXXX
>> ldap.UFADdev.user = XXXXXXXXXXXXXXXXXXXXXXXXX
>> ldap.UFADdev.url = ldap://ufadXXXXXXXXXXXXXXXXX.ufl.edu
>> ldap.UFADdev.pagedResultsSize = 1000
>> ldap.UFADdev.timeout = 100000
>> ldap.UFADdev.tls = false
>> ldap.UFADdev.searchResultHandlers=org.ldaptive.handler.DnAttributeEntryHandler,edu.internet2.middleware.grouper.ldap.ldaptive.GrouperRangeEntryHandler
>> # comma-delimited list of result codes (org.ldaptive.ResultCode) to 
>> ignore, e.g. TIME_LIMIT_EXCEEDED, SIZE_LIMIT_EXCEEDED, PARTIAL_RESULTS
>> ldap.UFADdev.searchIgnoreResultCodes=SIZE_LIMIT_EXCEEDED,ATTRIBUTE_OR_VALUE_EXISTS
>> 
>> changeLog.consumer.psp_UFADdev.provisionerName = psp_UFADdev
>> changeLog.consumer.psp_UFADdev.class = 
>> edu.internet2.middleware.grouper.pspng.PspChangelogConsumerShim
>> changeLog.consumer.psp_UFADdev.type = 
>> edu.internet2.middleware.grouper.pspng.LdapGroupProvisioner
>> changeLog.consumer.psp_UFADdev.quartzCron = 0/15 * * * * ?
>> changeLog.consumer.psp_UFADdev.ldapPoolName = UFADdev
>> changeLog.consumer.psp_UFADdev.retryOnError = false
>> changeLog.consumer.psp_UFADdev.isActiveDirectory = true
>> changeLog.consumer.psp_UFADdev.grouperIsAuthoritative = true
>> changeLog.consumer.psp_UFADdev.memberAttributeName = member
>> changeLog.consumer.psp_UFADdev.memberAttributeValueFormat = 
>> ${ldapUser.getDn()}
>> changeLog.consumer.psp_UFADdev.groupSearchBaseDn = 
>> OU=Grouper,OU=Groups,OU=UF,DC=dev-ad,DC=ufl,DC=edu
>> changeLog.consumer.psp_UFADdev.groupCreationBaseDn = 
>> OU=Grouper,OU=Groups,OU=UF,DC=dev-ad,DC=ufl,DC=edu
>> changeLog.consumer.psp_UFADdev.allGroupsSearchFilter = objectclass=group
>> changeLog.consumer.psp_UFADdev.singleGroupSearchFilter = 
>> (&(objectclass=group)(cn=${group.name.replaceAll("(App:.+:UF:Groups:)(.+):(.*)","$3").replace(":","_")}))
>> changeLog.consumer.psp_UFADdev.groupSearchAttributes = 
>> cn,gidNumber,objectclass,samAccountName,name
>> changeLog.consumer.psp_UFADdev.groupCreationLdifTemplate = dn: 
>> ${utils.bushyDn("${group.name.replaceAll('(App:.+:UF:Groups:)(.+):(.*)','$2:'+'$3'.replace(':','_'))}","cn","ou")}||objectclass:
>>  group
>> changeLog.consumer.psp_UFADdev.userSearchBaseDn = 
>> OU=People,OU=UF,DC=dev-ad,DC=ufl,DC=edu
>> changeLog.consumer.psp_UFADdev.userSearchFilter = 
>> userPrincipalName=${subject.id}
>> changeLog.consumer.psp_UFADdev.userSearchAttributes = 
>> dn,cn,uid,mail,samAccountName,uidNumber,objectclass,memberOf,userPrincipalName
>> changeLog.consumer.psp_UFADdev.groupSearch_batchSize = 100
>> changeLog.consumer.psp_UFADdev.ldapSearchResultPagingSize = 1000
>> changelog.consumer.psp_UFADdev.grouperSubjectCacheSize = 400000
>> changelog.consumer.psp_UFADdev.grouperGroupCacheSize = 50000
>> changelog.consumer.psp_UFADdev.ldapUserCacheSize = 200000
>> changelog.consumer.psp_UFADdev.ldapUserCacheTime_secs = 3600
>> changelog.consumer.psp_UFADdev.maxValuesToChangePerOperation = 500
>> changeLog.consumer.psp_UFADdev.needsTargetSystemUsers = true
>> changeLog.consumer.psp_UFADdev.supportsEmptyGroups = true
>> changeLog.consumer.psp_UFADdev.createMissingUsers = true
>> changeLog.consumer.psp_UFADdev.userCreationBaseDn = 
>> OU=People,OU=UF,DC=dev-ad,DC=ufl,DC=edu
>> changeLog.consumer.psp_UFADdev.userCreationLdifTemplate = dn: 
>> CN=${subject.getId().toLowerCase().replaceAll('@ufl.edu','').replaceAll('\\.$','')}||sAMAccountName:
>>  
>> ${subject.getId().toLowerCase().replaceAll('@ufl.edu','').replaceAll('\\.$','')}||objectclass:
>>  top||objectclass: person||objectclass: user||sn: 
>> ${subject.getId().toLowerCase().replaceAll('@ufl.edu','')}||userPrincipalName:
>>  ${subject.getId()}||displayName: ${subject.name}||cn: 
>> ${subject.getId().toLowerCase().replaceAll('@ufl.edu','')}
>