Grouper Developers Forum

["Bee-Lindgren, Bert" <>]

Hello,

1) Please add the top objectclass to your group template, though this problem looks different to me

2) Can you privately send me what you replaced with ' > … (long list of DNs) … '?

3) (particularly with PSPNG p5), Can you run it with INFO or DEBUG log level and send the result to me?

4) I would suggest adding gidNumber to your groupCreationTemplate and then (after gidNumbers are fully provisioned by a full-sync sweep) using gidNumber-based searching in your singleGroupSearchFilter

Thanks,

  Bert

---

From: <> on behalf of Gettes, Michael <>
Sent: Friday, April 26, 2019 11:04 PM
To:
Subject: [grouper-dev] PSPNG error repeating (latest)

 

Going against AD - this error keeps repeating.  It’s a group with 17K members.  my pspng config is below.  This is on tier/grouper:latest published today (2.4.0-a42-u23-w5-p4-20190426-rc1).

2019-04-27T02:19:09+00:00 DAEMON:dev grouper-api;grouper.log;grouper_dev;daemon;2019-04-26 22:19:09,381: [FullSyncer(psp_UFADdev)-Thread] WARN  LdapSystem.performLdapModify(405) -  - UFADdev: Problem while modifying ldap system based on grouper expectations. Starting to perform adaptive modifications based on data already on server: [org.ldaptive.ModifyRequest@1988375000::modifyDn
… (long list of DNs) …
controls=null, referralHandler=null, intermediateResponseHandlers=null]: ENTRY_ALREADY_EXISTS
2019-04-27T02:15:38+00:00 DAEMON:dev grouper-api;grouper.log;grouper_dev;daemon;2019-04-26 22:15:38,044: [FullSyncer(psp_UFADdev)-Thread] ERROR FullSyncProvisioner.fullSyncGroup(739) -  - FullSyncer(psp_UFADdev): Problem doing full sync. Requeuing group App:UFAD:UF:Groups:Services:Zoom:Service uflphi
2019-04-27T02:15:38+00:00 DAEMON:dev java.lang.IllegalStateException: The active provisioner should be defined when creating LdapObjects
2019-04-27T02:15:38+00:00 DAEMON:dev     at edu.internet2.middleware.grouper.pspng.LdapObject.<init>(LdapObject.java:95)
2019-04-27T02:15:38+00:00 DAEMON:dev     at edu.internet2.middleware.grouper.pspng.LdapSystem.performLdapRead(LdapSystem.java:626)
2019-04-27T02:15:38+00:00 DAEMON:dev     at edu.internet2.middleware.grouper.pspng.LdapSystem.performLdapRead(LdapSystem.java:596)
2019-04-27T02:15:38+00:00 DAEMON:dev     at edu.internet2.middleware.grouper.pspng.LdapSystem.performLdapModify(LdapSystem.java:448)
2019-04-27T02:15:38+00:00 DAEMON:dev     at edu.internet2.middleware.grouper.pspng.LdapSystem.performLdapModify(LdapSystem.java:377)
2019-04-27T02:15:38+00:00 DAEMON:dev     at edu.internet2.middleware.grouper.pspng.LdapProvisioner.makeIndividualLdapChanges(LdapProvisioner.java:713)
2019-04-27T02:15:38+00:00 DAEMON:dev     at edu.internet2.middleware.grouper.pspng.LdapProvisioner.finishProvisioningBatch(LdapProvisioner.java:455)
2019-04-27T02:15:38+00:00 DAEMON:dev     at edu.internet2.middleware.grouper.pspng.FullSyncProvisioner.fullSyncGroup(FullSyncProvisioner.java:721)
2019-04-27T02:15:38+00:00 DAEMON:dev     at edu.internet2.middleware.grouper.pspng.FullSyncProvisioner.processQueueItem(FullSyncProvisioner.java:433)
9-04-27T02:19:09+00:00 DAEMON:dev        at edu.internet2.middleware.grouper.pspng.FullSyncProvisioner.thread_manageFullSyncProcessing(FullSyncProvisioner.java:254)
2019-04-27T02:19:09+00:00 DAEMON:dev     at edu.internet2.middleware.grouper.pspng.FullSyncProvisioner$1.run(FullSyncProvisioner.java:146)
2019-04-27T02:19:09+00:00 DAEMON:dev     at java.lang.Thread.run(Thread.java:748)
2019-04-27T02:53:48+00:00 DAEMON:dev grouper-api;grouper.log;grouper_dev;daemon;2019-04-26 22:53:48,840: [FullSyncer(psp_UFADdev)-Thread] WARN  ProvisionerCoordinator$ProvisioningStatus.lockForFullSyncWhenNoIncrementalIsUnderway(73) -  - psp_UFADdev: Cannot start FullSync of App:UFAD:UF:Groups:Services:Zoom:Service uflphi. Incremental provisioning underway since Fri Apr 26 22:53:48 EDT 2019. We'll give up and move ahead anyway in 300 seconds.

———

AND this is a poorly formatted message that comes next:  not sure what i should do to address it if anything.

2019-04-27T02:19:09+00:00 DAEMON:dev grouper-api;grouper.log;grouper_dev;daemon;2019-04-26 22:19:09,461: [FullSyncer(psp_UFADdev)-Thread] WARN  Provisioner.warnAboutCacheSizeConcerns(620) -  - Cache is very full (%.0f%%). Performance is much higher if 100.0 is large enough to hold the number provisioned groups or subjects

———

pspng config:

ldap.UFADdev.pass = XXXXXXXXXXXX
ldap.UFADdev.user = XXXXXXXXXXXXXXXXXXXXXXXXX
ldap.UFADdev.url = "ldap://ufadXXXXXXXXXXXXXXXXX.ufl.edu ldap.UFADdev.pagedResultsSize = 1000
ldap.UFADdev.timeout = 100000
ldap.UFADdev.tls = false
ldap.UFADdev.searchResultHandlers=org.ldaptive.handler.DnAttributeEntryHandler,edu.internet2.middleware.grouper.ldap.ldaptive.GrouperRangeEntryHandler
# comma-delimited list of result codes (org.ldaptive.ResultCode) to ignore, e.g. TIME_LIMIT_EXCEEDED, SIZE_LIMIT_EXCEEDED, PARTIAL_RESULTS
ldap.UFADdev.searchIgnoreResultCodes=SIZE_LIMIT_EXCEEDED,ATTRIBUTE_OR_VALUE_EXISTS

changeLog.consumer.psp_UFADdev.provisionerName = psp_UFADdev
changeLog.consumer.psp_UFADdev.class = edu.internet2.middleware.grouper.pspng.PspChangelogConsumerShim
changeLog.consumer.psp_UFADdev.type = edu.internet2.middleware.grouper.pspng.LdapGroupProvisioner
changeLog.consumer.psp_UFADdev.quartzCron = 0/15 * * * * ?
changeLog.consumer.psp_UFADdev.ldapPoolName = UFADdev
changeLog.consumer.psp_UFADdev.retryOnError = false
changeLog.consumer.psp_UFADdev.isActiveDirectory = true
changeLog.consumer.psp_UFADdev.grouperIsAuthoritative = true
changeLog.consumer.psp_UFADdev.memberAttributeName = member
changeLog.consumer.psp_UFADdev.memberAttributeValueFormat = ${ldapUser.getDn()}
changeLog.consumer.psp_UFADdev.groupSearchBaseDn = OU=Grouper,OU=Groups,OU=UF,DC=dev-ad,DC=ufl,DC=edu
changeLog.consumer.psp_UFADdev.groupCreationBaseDn = OU=Grouper,OU=Groups,OU=UF,DC=dev-ad,DC=ufl,DC=edu
changeLog.consumer.psp_UFADdev.allGroupsSearchFilter = objectclass=group
changeLog.consumer.psp_UFADdev.singleGroupSearchFilter = (&(objectclass=group)(cn=${group.name.replaceAll("(App:.+:UF:Groups:)(.+):(.*)","$3").replace(":","_")}))
changeLog.consumer.psp_UFADdev.groupSearchAttributes = cn,gidNumber,objectclass,samAccountName,name
changeLog.consumer.psp_UFADdev.groupCreationLdifTemplate = dn: ${utils.bushyDn("${group.name.replaceAll('(App:.+:UF:Groups:)(.+):(.*)','$2:'+'$3'.replace(':','_'))}","cn","ou")}||objectclass: group
changeLog.consumer.psp_UFADdev.userSearchBaseDn = OU=People,OU=UF,DC=dev-ad,DC=ufl,DC=edu
changeLog.consumer.psp_UFADdev.userSearchFilter = userPrincipalName=${subject.id}
changeLog.consumer.psp_UFADdev.userSearchAttributes = dn,cn,uid,mail,samAccountName,uidNumber,objectclass,memberOf,userPrincipalName
changeLog.consumer.psp_UFADdev.groupSearch_batchSize = 100
changeLog.consumer.psp_UFADdev.ldapSearchResultPagingSize = 1000
changelog.consumer.psp_UFADdev.grouperSubjectCacheSize = 400000
changelog.consumer.psp_UFADdev.grouperGroupCacheSize = 50000
changelog.consumer.psp_UFADdev.ldapUserCacheSize = 200000
changelog.consumer.psp_UFADdev.ldapUserCacheTime_secs = 3600
changelog.consumer.psp_UFADdev.maxValuesToChangePerOperation = 500
changeLog.consumer.psp_UFADdev.needsTargetSystemUsers = true
changeLog.consumer.psp_UFADdev.supportsEmptyGroups = true
changeLog.consumer.psp_UFADdev.createMissingUsers = true
changeLog.consumer.psp_UFADdev.userCreationBaseDn = OU=People,OU=UF,DC=dev-ad,DC=ufl,DC=edu
changeLog.consumer.psp_UFADdev.userCreationLdifTemplate = dn: CN=${subject.getId().toLowerCase().replaceAll('@ufl.edu','').replaceAll('\\.$','')}||sAMAccountName: ${subject.getId().toLowerCase().replaceAll('@ufl.edu','').replaceAll('\\.$','')}||objectclass: top||objectclass: person||objectclass: user||sn: ${subject.getId().toLowerCase().replaceAll('@ufl.edu','')}||userPrincipalName: ${subject.getId()}||displayName: ${subject.name}||cn: ${subject.getId().toLowerCase().replaceAll('@ufl.edu','')}