Grouper Users - Open Discussion List

["Bee-Lindgren, Bert" <>]

Carey,

Thanks for all these details and the work behind them!

I did change PSPNG's full-sync processing to update (non-membership) attributes and that is what is attempting this objectclass change.

I've thought a bit about how to address this... I think that special handling of objectclass or objectclass=top could get in the way later. Therefore, I think putting the values into the configuration like you did is the best approach, and I've updated the example with this.

But how to avoid needing so much diagnosis next time?... Perhaps PSPNG could notice incomplete LdifCreationTemplates by reading just-created objects and failing (or at least logging) if the new objects don't match the template? What do you think?

Thanks again,

  Bert

---

From: <> on behalf of Black, Carey M. <>
Sent: Thursday, April 25, 2019 1:27 AM
To: Grouper Users
Subject: RE: [grouper-users] PSPNG to AD question.... ( workaround )

 

I have been through a few rounds of “still getting the error… can I get more permission” but we still have not found the magic switch yet. :(

 

The issue appears to be limited to only the “Full Sync” processing.

 

INFO  LdapSystem.performLdapModify(392) -  - xxx: Performing Ldap modification: [org.ldaptive.ModifyRequest@601820118::modifyDn=cn=xxxx,ou=xxx,ou=xx,OU=xxx,OU=xxx,DC=xxx,DC=xxx,DC=xxx,DC=xxx,DC=xxx, attrMods=[[org.ldaptive.AttributeModification@219437121::attrMod=REPLACE, attribute=[objectclass[group]]]], controls=null, referralHandler=null, intermediateResponseHandlers=null]

 

Which looks like the only attribute that is attempted to be modified is “objectclass”.

Which is a silly thing to try to replace with any other value. Nor is there any reason why PSPNG should think the AD group is no longer a “group”.

It is like it just does not compare the values right.

 

 

I highly suspect an either a “real bug” in PSPNG, or a “bug” in my configuration of it. I just don’t see it yet.

 

I have also simplified my config quite a bit to try to rule out the odd stuff I was trying to do.

I think the only relevant config values are:

 

changeLog.consumer.xxxTest.groupSearchAttributes = cn,objectClass

changeLog.consumer.xxxTest.groupCreationLdifTemplate = dn: ${utils.bushyDn(group.name.replaceFirst("xxx:yyy:zzz:",""),"cn", "ou")}||cn: ${group.getExtension()}||objectclass: group

changeLog.consumer.xxxTest.groupAttributeName = memberof

changeLog.consumer.xxxTest.allGroupsSearchFilter = (objectclass=group)

changeLog.consumer.xxxTest.singleGroupSearchFilter = (&(objectclass=group)(${utils.bushyDn(group.name.replaceFirst("xxx:yyy:zzz:",""),"cn", "ou").replaceFirst(",ou=.*","")}))

 

The “replaceFirst()” is to get the DN’s in AD to not know/use the folder structure above where the integration is anchored in grouper.

In the singleGroupSearchFilter  I am using the same function to build the “cn=<name_of_group>” just for consistency. However, I then need to “cut the dn” at the first “,ou=” too. I suspect I could use “cn=${group.getExtension()}” as well, but I am trying to be as consistent as possible.

 

 

 

 

OH… I think I found it. ( It still smells like a PSPNG bug to me, but I will accept that a config change can “work around it”....)

 

 

 

 

I was staring at the object in AD with an LDAP client and it “clicked”...

                The object has TWO values for objectclass. ( top and group )

                And the “REPLACE op” is only trying to put “group” into AD.

 

                So it must be trying to “remove” top. (<-- $@$@#$%@@!!!!!!! )

 

 

The default example(for AD) .groupCreationLdifTemplate  is (on the wiki):

changeLog.consumer.pspng_activedirectory.groupCreationLdifTemplate = dn: cn=${group.name}||cn: ${group.name}||objectclass: group

 

 

However then I noticed the example for “POSIX GROUPS”: (and the only one of all of them )

changeLog.consumer.psong_posixGroup.groupCreationLdifTemplate = dn: cn=posix-${group.name}||cn: posix-${group.name}||objectclass: posixGroup||objectclass: groupOfNames||gidNumber: ${group.idIndex}

 

THERE ARE TWO “objectclass” values being set… “posixGroup” and “groupOfNames”…..

 

 

OMG… ( My forehead really took a beating try to get to this “revelation”/”understanding” of PSPNG.)

 

 

Changed (added a second “||objectclass: top” value) :

changeLog.consumer.xxx.groupCreationLdifTemplate = dn: ${utils.bushyDn(group.name.replaceFirst("xxx:yyy:zzz:",""),"cn", "ou")}||cn: ${group.getExtension()}||objectclass: group||objectclass: top

 

stop daemon

start daemon

 

start gsh.

Trigger a fully sync for the consumer…

 

provisioner_name="xxx"; // Whatever your provisioner is called in grouper_loader.properties

gs=GrouperSession.startRootSession();

full_provisioner=edu.internet2.middleware.grouper.pspng.FullSyncProvisionerFactory.getFullSyncer(provisioner_name);

full_provisioner.startFullSyncOfAllGroupsAndWaitForCompletion()

 

 

No error in the log.

 

I think that is a bug in the code. ( Or at least in the docs. )

I don’t understand how any of the example configs could work if “top” is not “implied/ignored automatically”.

So I assume that it “use to work” and now it doesn’t. So I hold to thinking it is a bug in the code.

                Maybe due to the move to the ldaptive library? Maybe some other change? But this behavior does not work with the “example configs”.

 

 

GRRRRRR….. time to get some sleep.

 

Grouper 2.3

api patches installed:        0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72, 73, 74, 75, 76, 77, 78, 79, 80, 81, 82, 83, 84, 85, 86, 87, 88, 89, 90, 91, 92, 93, 94, 95, 96, 97, 98, 99, 100, 101, 102, 103, 104, 105, 106, 107, 108, 109, 110

pspng patches installed:      0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24

 

--

Carey Matthew

 

 

From: Jeffrey Williams <>
Sent: Thursday, April 18, 2019 9:25 AM
To: Black, Carey M. <>
Cc: Grouper Users <>
Subject: Re: [grouper-users] PSPNG to AD question....

 

Confirm with your AD Team, but it sounds like they delegated the common task "Create, delete and manage groups" to your account.  Manage here means write access to the members property but I don't think much else.   Like many things Microsoft, there are several separate delegations they can use to let you read and write the other properties.  It all depends on how permissions-retentive your AD Team wants to be about it. :). 

 

Good luck!

 

 

 

 

 

 

 

On Wed, Apr 17, 2019 at 10:04 PM Black, Carey M. <> wrote:

I have *started* testing using PSPNG to create/maintain groups in AD.

 

And I have an odd error condition that I just have not sorted out....

 

My AD team is using a "least privileges" approach. So we may not have granted enough privs, but at this point I think the error that I am seeing is pointing to a PSPNG config issue more than an AD permission.... but I may be wrong.

 

 

Basically PSPNG can create groups and add users to the groups.

However, during the Full sync... it is trying to modify the group and failing.

                It is not clear to me why it is trying to modify the group.

 

What I see is this:

 

 

2019-04-17 18:08:27,071: [FullSyncer(xxxTest)-Thread] INFO  Provisioner.doFullSync(1393) -  - xxxTest-full/xxx:xxx:xxx:GMS:group6: 1 correct member subjects. Sample: ['XXXX'/'person'/'SOURCE']...

2019-04-17 18:08:27,071: [FullSyncer(xxxTest)-Thread] DEBUG LdapGroupProvisioner.updateGroupFromTemplate(258) -  - xxxTest-full: Making sure (non-membership) attributes of group are up to date: cn=gms-group6,ou=xxx,ou=xxx,dc=xxx,dc=xxx,dc=xxx,dc=xxxx,dc=xx

2019-04-17 18:08:27,146: [FullSyncer(xxxTest)-Thread] DEBUG Provisioner.evaluateJexlExpression(644) -  - Evaluated GroupTemplate Jexl _expression_: 'cn=GMS-group6'

2019-04-17 18:08:27,147: [FullSyncer(xxxTest)-Thread] DEBUG Provisioner.evaluateJexlExpression(644) -  - Evaluated GroupTemplate Jexl _expression_: 'GMS-group6'

2019-04-17 18:08:27,147: [FullSyncer(xxxTest)-Thread] DEBUG Provisioner.evaluateJexlExpression(664) -  - Evaluated entire GroupTemplate Jexl _expression_: 'dn: cn=GMS-group6

cn: GMS-group6

objectclass: group'

2019-04-17 18:08:27,147: [FullSyncer(xxxTest)-Thread] INFO  LdapProvisioner.ensureLdapOusExist(775) -  - xxxTest-full: Checking for (and creating) missing OUs in DN: cn=GMS-group6,OU=xxx,OU=xxx,DC=xxx,DC=xxx,DC=xxx,DC=xxx,DC=xx (wholeDnIsOu=false)

2019-04-17 18:08:27,148: [FullSyncer(xxxTest)-Thread] DEBUG LdapProvisioner.ensureLdapOusExist(812) -  - xxxTest-full: OU is known to exist: OU=xxx,OU=xxx,DC=xxx,DC=xxx,DC=xx,DC=xxx,DC=xx

2019-04-17 18:08:27,148: [FullSyncer(xxxTest)-Thread] INFO  LdapSystem.performLdapModify(392) -  - xxx: Performing Ldap modification: [org.ldaptive.ModifyRequest@1312181761::modifyDn=cn=GMS-group6,OU=xxx,OU=xxx,DC=xxx,DC=xxx,DC=xxx,DC=xxx,DC=xx, attrMods=[[org.ldaptive.AttributeModification@1552851056::attrMod=REPLACE, attribute=[objectclass[group]]]], controls=null, referralHandler=null, intermediateResponseHandlers=null]

2019-04-17 18:08:27,160: [FullSyncer(xxxTest)-Thread] WARN  LdapSystem.performLdapModify(407) -  - xxx: Problem while modifying ldap system based on grouper expectations. Starting to perform adaptive modifications based on data already on server: [org.ldaptive.ModifyRequest@1312181761::modifyDn=cn=GMS-group6,OU=xxx,OU=xxx,DC=xxx,DC=xxx,DC=xxx,DC=xxx,DC=xxx, attrMods=[[org.ldaptive.AttributeModification@1552851056::attrMod=REPLACE, attribute=[objectclass[group]]]], controls=null, referralHandler=null, intermediateResponseHandlers=null]: INSUFFICIENT_ACCESS_RIGHTS

...

        at org.ldaptive.provider.ProviderUtils.throwOperationException(ProviderUtils.java:55)

        at org.ldaptive.provider.jndi.JndiConnection.processNamingException(JndiConnection.java:619)

        at org.ldaptive.provider.jndi.JndiConnection.modify(JndiConnection.java:425)

        at edu.internet2.middleware.grouper.pspng.LdapSystem.performLdapModify(LdapSystem.java:397)

        at edu.internet2.middleware.grouper.pspng.LdapSystem.performLdapModify(LdapSystem.java:379)

        at edu.internet2.middleware.grouper.pspng.LdapSystem.makeLdapDataCorrect(LdapSystem.java:785)

        at edu.internet2.middleware.grouper.pspng.LdapSystem.makeLdapObjectCorrect(LdapSystem.java:743)

        at edu.internet2.middleware.grouper.pspng.LdapGroupProvisioner.updateGroupFromTemplate(LdapGroupProvisioner.java:265)

        at edu.internet2.middleware.grouper.pspng.LdapGroupProvisioner.doFullSync(LdapGroupProvisioner.java:191)

        at edu.internet2.middleware.grouper.pspng.LdapGroupProvisioner.doFullSync(LdapGroupProvisioner.java:41)

        at edu.internet2.middleware.grouper.pspng.Provisioner.doFullSync(Provisioner.java:1399)

        at edu.internet2.middleware.grouper.pspng.FullSyncProvisioner.fullSyncGroup(FullSyncProvisioner.java:660)

        at edu.internet2.middleware.grouper.pspng.FullSyncProvisioner.thread_manageFullSyncProcessing(FullSyncProvisioner.java:290)

        at edu.internet2.middleware.grouper.pspng.FullSyncProvisioner$1.run(FullSyncProvisioner.java:201)

        at java.lang.Thread.run(Thread.java:748)

Caused by: javax.naming.NoPermissionException: [LDAP: error code 50 - 00002098: SecErr: DSID-03150F93, problem 4003 (INSUFF_ACCESS_RIGHTS), data 0

^@]; remaining name 'cn=GMS-group6,OU=xxx,OU=xxx,DC=xxx,DC=xxx,DC=xxx,DC=xxx,DC=xxx'

        at com.sun.jndi.ldap.LdapCtx.mapErrorCode(LdapCtx.java:3162)

        at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:3100)

        at com.sun.jndi.ldap.LdapCtx.processReturnCode(LdapCtx.java:2891)

        at com.sun.jndi.ldap.LdapCtx.c_modifyAttributes(LdapCtx.java:1475)

        at com.sun.jndi.toolkit.ctx.ComponentDirContext.p_modifyAttributes(ComponentDirContext.java:277)

        at com.sun.jndi.toolkit.ctx.PartialCompositeDirContext.modifyAttributes(PartialCompositeDirContext.java:192)

        at org.ldaptive.provider.jndi.JndiConnection.modify(JndiConnection.java:412)

        ... 12 more

2019-04-17 18:08:27,169: [FullSyncer(xxxTest)-Thread] ERROR LdapGroupProvisioner.updateGroupFromTemplate(274) -  - xxxTest-full: Problem checking and updating group's template attributes

edu.internet2.middleware.grouper.pspng.PspException: LDAP Modification Failed

        at edu.internet2.middleware.grouper.pspng.LdapSystem.performLdapModify(LdapSystem.java:432)

        at edu.internet2.middleware.grouper.pspng.LdapSystem.performLdapModify(LdapSystem.java:379)

        at edu.internet2.middleware.grouper.pspng.LdapSystem.makeLdapDataCorrect(LdapSystem.java:785)

        at edu.internet2.middleware.grouper.pspng.LdapSystem.makeLdapObjectCorrect(LdapSystem.java:743)

        at edu.internet2.middleware.grouper.pspng.LdapGroupProvisioner.updateGroupFromTemplate(LdapGroupProvisioner.java:265)

        at edu.internet2.middleware.grouper.pspng.LdapGroupProvisioner.doFullSync(LdapGroupProvisioner.java:191)

        at edu.internet2.middleware.grouper.pspng.LdapGroupProvisioner.doFullSync(LdapGroupProvisioner.java:41)

        at edu.internet2.middleware.grouper.pspng.Provisioner.doFullSync(Provisioner.java:1399)

        at edu.internet2.middleware.grouper.pspng.FullSyncProvisioner.fullSyncGroup(FullSyncProvisioner.java:660)

       at edu.internet2.middleware.grouper.pspng.FullSyncProvisioner.thread_manageFullSyncProcessing(FullSyncProvisioner.java:290)

        at edu.internet2.middleware.grouper.pspng.FullSyncProvisioner$1.run(FullSyncProvisioner.java:201)

        at java.lang.Thread.run(Thread.java:748)

 

 

Oh.. this " Provisioner.doFullSync " starts just after I start the daemon (loader process) and keeps erroring with the above error. And requeuing the full sync... All the time... ( "Pete and repeat were sitting on the fence. Pete fell off and who was left?")

 

 

1) The second line of that log appears to show the full DN for the group but it is in all LOWER case. ( The name of the group is mixed case as is shown in on the 1st and 3rd line. )

2) The modify appears to be attempting is to change the objectclass(<--???) to the class that the group is already a part of??

3) Is there a way to turn up the logging on ldaptive to see more details about the LDAP that is being done?

 

 

If it helps here is a "scrubbed" copy of the config...

The only thing that is "strange" ( I think ) is what I am doing with the cn value.

                I am stripping part of the grouper path up to the folder tagged for the provisioner. And then any folders under it, become "-"(dash) separated values in the CN. ( Well that is the plan, and groups are being created that way...)

 

 

changeLog.consumer.xxxTest.class = edu.internet2.middleware.grouper.pspng.PspChangelogConsumerShim

changeLog.consumer.xxxTest.type = edu.internet2.middleware.grouper.pspng.LdapGroupProvisioner

changeLog.consumer.xxxTest.provisionerName = xxxTest

changeLog.consumer.xxxTest.quartzCron = 0 * * * * ?

 

changeLog.consumer.xxxTest.ldapPoolName = xxx

changeLog.consumer.xxxTest.isActiveDirectory = true

changeLog.consumer.xxxTest.memberAttributeName = member

changeLog.consumer.xxxTest.memberAttributeValueFormat = ${ldapUser.getDn()}

 

changeLog.consumer.xxxTest.groupSearchBaseDn = OU=xxx,OU=xxx,DC=xxx,DC=xxx,DC=xxx,DC=xxx,DC=xxx

changeLog.consumer.xxxTest.groupCreationBaseDn = OU=xxx,OU=xxx,DC=xxx,DC=xxx,DC=xxx,DC=xxx,DC=xxx

changeLog.consumer.xxxTest.groupSearchAttributes = cn,objectclass

 

changeLog.consumer.xxxTest.groupCreationLdifTemplate = dn: ${utils.bushyDn(group.name.replaceFirst("xxx:xxx:xxx:","").replaceAll(":","-"),"cn", "ou")}||cn: ${group.name.replaceFirst("xxx:xxx:xxx:","").replaceAll(":","-")}||objectclass: group

changeLog.consumer.xxxTest.groupAttributeName = memberof

changeLog.consumer.xxxTest.allGroupsSearchFilter = objectClass=group

changeLog.consumer.xxxTest.singleGroupSearchFilter = (&(objectClass=group)(cn=${group.name.replaceFirst("xxx:xxx:xxx:","").replaceAll(":","-")}))

changeLog.consumer.xxxTest.userSearchBaseDn = DC=xxx,DC=xxx,DC=xxx,DC=xxx,DC=xxx

changeLog.consumer.xxxTest.userSearchFilter = cn=${subject.getAttributeValue("customAttributeFromSubjectSource")}

 

 

otherJob.xxxTest_full.class = edu.internet2.middleware.grouper.pspng.FullSyncStarter

otherJob.xxxTest_full.quartzCron = 0 0 0 * * ?

 

 

 

 

Thanks for any guidance / clue bricks. 😊

 

--

Carey Matthew

 

 

--

Jeffrey Williams 

Identity Engineer
Identity & Access Services
https://its.uncg.edu