Grouper Users - Open Discussion List

[Rahul Doshi <>]

Hi Chris,

I was able to find the problem.  It turns out that member attribute stored in our LDAP environment is in uppercase where as subjectId stored in database is lowercase.  So when the calculating the deltas match on what is stored in database and what is in LDAP fails, which is causing everything to be deleted and added back.  I was able to fix the problem by simply forcing return value to be lowercase in my CmuLoaderElUtils.convertDnToSpecificValue method.

Thanks,

Rahul

From: Chris Hyzer <>
Date: Tuesday, August 7, 2012 1:20 AM
To: Rahul Doshi <>, "" <>
Subject: RE: Grouper ldap loader

Im trying to reproduce this, and Im not having much luck when I setup a test case similar to yours, when I run it, it doesnt delete and add...

If you dont mind, can you use the latest Grouper API on the 2.1 branch from subversion (or use the snapshot build link below)?

http://www.internet2.edu/grouper/release/2.1.2/grouper.api-2.1.2.tar.gz

http://www.internet2.edu/grouper/release/2.1.2/grouper.apiBinary-2.1.2.tar.gz

Change the filter so it only syncs one or two small groups instead of the full list.  e.g.

attributeAssign.getAttributeValueDelegate().assignValue(LoaderLdapUtils.grouperLoaderLdapFilterName(), "(|(cn=test:ldaptesting:test1)(cn=test:testEmptyGroup))");

Set the logging for the loader to debug in the log4j.properties:

log4j.logger.edu.internet2.middleware.grouper.app.loader = DEBUG

Cleanse anything that is sensitive (ldap server name and login name?) and please send it to me (no need to put on list)

Here is an example:

2012-08-07 00:53:09,804: [main] DEBUG GrouperLoaderConfig.retrieveLdapProfile(373) -  - LDAP config for server id: personLdap: GrouperLoaderLdapServer [batchSize=-1, countLimit=-1, driver=null, expirationTime=-1, maxPoolSize=-1, minPoolSize=-1, pass=XXXXX, pruneTimerPeriod=-1, saslAuthorizationId=, saslRealm=, timeLimit=-1, timeout=-1, tls=false, url="ldaps://server.school.edu:636/dc=school,dc=edu," user=uid=user/something.school.edu,ou=entities,dc=school,dc=edu, validateOnCheckIn=false, validateOnCheckOut=true, validatePeriodically=false, validateTimerPeriod=-1]

2012-08-07 00:53:09,807: [main] DEBUG GrouperLoaderResultset$1.callback(582) -  - Found 2 results, (6 sub-results) for serverId: personLdap, searchDn: ou=groups, filter: '(|(cn=test:testGroup)(cn=test:ldaptesting:test1)(cn=test:testEmptyGroup))', returning subject attribute: hasMember, some results: {anotherStem:groups:test:ldapTesting:test1=[netmon], anotherStem:groups:test:testGroup=[convery, ...

2012-08-07 00:53:09,817: [main] DEBUG GrouperLoaderResultset.convertToSubjectIdIfNeeded(955) -  - Converted subject id from 'netmon' to 'netmon' based on subjectExpression: '${loaderLdapElUtils.convertDnToSpecificValueTest(subjectId, 'convery', 'test:convery')}'

2012-08-07 00:53:09,823: [main] DEBUG GrouperLoaderResultset.convertToSubjectIdIfNeeded(955) -  - Converted subject id from 'convery' to '049c99123f5f45428b973062e5aae6f5' based on subjectExpression: '${loaderLdapElUtils.convertDnToSpecificValueTest(subjectId, 'convery', 'test:convery')}'

2012-08-07 00:53:09,825: [main] DEBUG GrouperLoaderResultset.convertToSubjectIdIfNeeded(955) -  - Converted subject id from 'choate' to 'choate' based on subjectExpression: '${loaderLdapElUtils.convertDnToSpecificValueTest(subjectId, 'convery', 'test:convery')}'

2012-08-07 00:53:09,826: [main] DEBUG GrouperLoaderResultset.convertToSubjectIdIfNeeded(955) -  - Converted subject id from 'mchyzer' to 'mchyzer' based on subjectExpression: '${loaderLdapElUtils.convertDnToSpecificValueTest(subjectId, 'convery', 'test:convery')}'

2012-08-07 00:53:09,828: [main] DEBUG GrouperLoaderResultset.convertToSubjectIdIfNeeded(955) -  - Converted subject id from 'bwh' to 'bwh' based on subjectExpression: '${loaderLdapElUtils.convertDnToSpecificValueTest(subjectId, 'convery', 'test:convery')}'

2012-08-07 00:53:09,829: [main] DEBUG GrouperLoaderResultset.convertToSubjectIdIfNeeded(955) -  - Converted subject id from 'harveycg' to 'harveycg' based on subjectExpression: '${loaderLdapElUtils.convertDnToSpecificValueTest(subjectId, 'convery', 'test:convery')}'

2012-08-07 00:53:09,831: [main] DEBUG GrouperLoaderType$7.runJob(862) -  - anotherStem:groupListLdapGroup: start syncing membership

2012-08-07 00:53:09,831: [main] DEBUG GrouperLoaderType.syncGroupList(1114) -  - anotherStem:groupListLdapGroup: found 6 members overall

2012-08-07 00:53:09,831: [main] DEBUG GrouperLoaderType.syncGroupList(1124) -  - anotherStem:groupListLdapGroup: syncing membership for 2 groups

2012-08-07 00:53:09,831: [main] DEBUG GrouperLoaderType.syncGroupList(1340) -  - anotherStem:groupListLdapGroup: syncing membership for anotherStem:groups:test:ldapTesting:test1 1 out of 2 groups

2012-08-07 00:53:09,872: [main] DEBUG GrouperLoaderType.syncOneGroupMembership(1926) -  - anotherStem:groups:test:ldapTesting:test1 start syncing membership

2012-08-07 00:53:09,873: [main] DEBUG GrouperLoaderType.syncOneGroupMembership(1942) -  - anotherStem:groups:test:ldapTesting:test1 syncing 1 rows

2012-08-07 00:53:09,937: [main] DEBUG GrouperLoaderType.syncOneGroupMembership(1960) -  - anotherStem:groups:test:ldapTesting:test1: saving group if necessary, result type: NO_CHANGE

2012-08-07 00:53:09,937: [main] DEBUG GrouperLoaderType.syncOneGroupMembership(2055) -  - Done assigning privilege to related groups: anotherStem:groups:test:ldapTesting:test1

2012-08-07 00:53:09,963: [main] INFO  GrouperLoaderType.syncOneGroupMembership(2301) -  - anotherStem:groups:test:ldapTesting:test1 done syncing membership, processed 1 records.  Total members: 1, inserts: 0, deletes: 0

2012-08-07 00:53:10,048: [main] DEBUG GrouperLoaderType.syncGroupList(1340) -  - anotherStem:groupListLdapGroup: syncing membership for anotherStem:groups:test:testGroup 2 out of 2 groups

2012-08-07 00:53:10,056: [main] DEBUG GrouperLoaderType.syncOneGroupMembership(1926) -  - anotherStem:groups:test:testGroup start syncing membership

2012-08-07 00:53:10,056: [main] DEBUG GrouperLoaderType.syncOneGroupMembership(1942) -  - anotherStem:groups:test:testGroup syncing 5 rows

2012-08-07 00:53:10,063: [main] DEBUG GrouperLoaderType.syncOneGroupMembership(1960) -  - anotherStem:groups:test:testGroup: saving group if necessary, result type: NO_CHANGE

2012-08-07 00:53:10,063: [main] DEBUG GrouperLoaderType.syncOneGroupMembership(2055) -  - Done assigning privilege to related groups: anotherStem:groups:test:testGroup

2012-08-07 00:53:10,107: [main] DEBUG GrouperLoaderType.syncOneGroupMembership(2149) -  - anotherStem:groups:test:testGroup will add subject to group: Penn person/bwh, 1 of 1 subjects

2012-08-07 00:53:10,107: [main] DEBUG GrouperLoaderType.syncOneGroupMembership(2182) -  - anotherStem:groups:test:testGroup will remove subject from group: jdbc/test.subject.1, 1 of 1 members

2012-08-07 00:53:10,477: [main] DEBUG GrouperLoaderType$10.callback(2217) -  - Group: anotherStem:groups:test:testGroup delete Subject id: test.subject.1, sourceId: jdbc, alreadyDeleted? false

2012-08-07 00:53:10,531: [main] DEBUG GrouperLoaderType$10.callback(2256) -  - Group: anotherStem:groups:test:testGroup add Subject id: bwh, sourceId: pennperson, alreadyAdded: false

2012-08-07 00:53:10,531: [main] INFO  GrouperLoaderType.syncOneGroupMembership(2301) -  - anotherStem:groups:test:testGroup done syncing membership, processed 6 records.  Total members: 5, inserts: 1, deletes: 1

2012-08-07 00:53:10,576: [main] DEBUG GrouperLoaderType.syncGroupList(1433) -  - anotherStem:groupListLdapGroup: done syncing membership

Thanks,

Chris

---

From: Rahul Doshi []
Sent: Thursday, July 26, 2012 4:28 PM
To: Chris Hyzer;
Subject: Re: Grouper ldap loader

Yes,  I am using LDAPGROUPLIST job and  group is being returned from the ldap filter.  I will try to use the example from the link you provided. 

Thanks,

Rahul

From: Chris Hyzer <>
Date: Thursday, July 26, 2012 4:23 PM
To: Rahul Doshi <>, "" <>
Subject: RE: Grouper ldap loader

Actually, is this an LDAP_GROUP_LIST job, and is cn=someotherstem:admins, ou=group, dc= example, dc = org one of the groups that is returns from the ldap filter (as well as a member of a group).  If so, then with the link below, you can do what you want to do.  If that member which is a group is not managed by the loader, so we can do this with a tweak to grouper or an EL tweak.  Let me know

 

Thanks,

Chris

 

From: Chris Hyzer
Sent: Thursday, July 26, 2012 4:16 PM
To: 'Rahul Doshi';
Subject: RE: Grouper ldap loader

 

I hadn’t really considered that case when creating the loader, but I think it can be done fairly easily with some EL and a java class.  It is very similar to this email, except that the group will be created perhaps in EL…

 

https://lists.internet2.edu/sympa/arc/grouper-users/2012-05/msg00026.html

 

Want me to try to send you an example?

 

Thanks,

Chris

 

 

From: [] On Behalf Of Rahul Doshi
Sent: Thursday, July 26, 2012 4:03 PM
To:
Subject: [grouper-users] Grouper ldap loader

 

Hello,

 

I am trying to use grouper loader to load all the existing groups from our ldap environment.  It seems to load the groups fine except that it is ignoring the nested group membership.  For hypothetical group below if I run the loader job,  I only see somstem:admins group created in grouper with member xyz.  What I would have expected to see is two groups somestem:admins and someotherstem:admins created in grouper and someotherstem:admins also be the member of some stem:admins.  Is there something that I am missing in my configuration? 

 

Groupname

cn=somestem:admins, ou=group, dc=example, dc=org

member=uid=xyz, ou=person,dc=example,dc=org

member=cn=someotherstem:admins, ou=group, dc= example, dc = org

 

Thanks,

Rahul