Grouper Users - Open Discussion List

["Bryan E. Wooten" <>]

Hold off looking at this.

 

We had a misunderstanding of how groups from attributes actually works.

 

Our use case was to create a group of current employees base on our uuemployee multi value attribute.

 

We had hard coded the grouperLoaderLdapGroupNameExpression to be groups:currentEmployees. We did not realize that Grouper wants to create a group for each value in the multi value attribute. Hard coding the name makes the results unpredictable.

 

When we made this setting "groups:${groupAttribute}" to grouperLoaderLdapGroupNameExpression we are getting better results (the test is still running). However, I am now getting all the extraneous groups (group for each potential value in the multi value attribute) that weren’t really part of my use case. I wish there was a way to suppress this behavior.

 

When creating groups base on sn we found the same problem as above. This is because in our LDAP some sn’s are all upper case others can be mixed case. But there is not a use case for a group based on sn.

 

Thanks,

 

Bryan

 

From: [mailto:] On Behalf Of Bryan E. Wooten
Sent: Tuesday, April 23, 2013 11:55 AM
To: Chris Hyzer;
Subject: [grouper-users] RE: I am completely confused

 

Thank you Chris.

 

I have that log4j setting. I am current working against OpenDJ because AD is very difficult to monitor incoming queries and accesses. If you have any tips besides using Server Manager I am all ears.

 

Yes our unid is unique for each person.

 

I tried using LDAP_SIMPLE to get all the members from a group we have in AD called currentEmployees. I could not get this to work. Not sure if it was an AD issue or configuration on my end. So I pointed Grouper at my OpenDJ and decided to see if I could create a group based on an attribute.  Right now I am trying to create a group base on ” sn=Nguyen “ . we have 967 in OpenDJ. When I run the loader job I only get 141 back.

 

My grouper_error.log follows.

 

-Bryan

 

2013-04-23 11:33:40,291: [main] INFO  EventLog.info(156) -  - [972deafe427d49dea90dc54be2edc626,'GrouperSystem','application'] session: start (204ms)

2013-04-23 11:33:40,440: [main] INFO  EventLog.info(156) -  - [f34cc440f1144a09af5b3ae34b10a959,'GrouperSystem','application'] session: start (13ms)

2013-04-23 11:33:40,446: [main] INFO  EventLog.info(156) -  - [41feb652413c4c518a3eb4c61ab6a0e6,'GrouperSystem','application'] session: start (3ms)

2013-04-23 11:33:42,293: [main] INFO  EventLog.info(156) -  - [d6e3b37001a94b73a67ebc8434c822ef,'GrouperSystem','application'] session: start (12ms)

2013-04-23 11:33:42,775: [main] INFO  EventLog.info(156) -  - [59a68ba73a054876a5ed9d3004f91cdb,'GrouperSystem','application'] session: start (4ms)

2013-04-23 11:33:42,788: [main] INFO  EventLog.info(156) -  - [6bb9176ad4e34d1a90477094aff4ff4e,'GrouperSystem','application'] session: start (4ms)

2013-04-23 11:33:42,795: [main] INFO  EventLog.info(156) -  - [85ffeb1eaad440fdb696064e2ee8e725,'GrouperSystem','application'] session: start (3ms)

2013-04-23 11:33:42,823: [main] INFO  EventLog.info(156) -  - [85ffeb1eaad440fdb696064e2ee8e725,'GrouperSystem','application'] add group type: 'grouperLoader' (24ms)

2013-04-23 11:34:16,831: [main] INFO  EventLog.info(156) -  - [67ae2f1681f5425f990b5e61c96acd63,'GrouperSystem','application'] session: start (11ms)

2013-04-23 11:34:46,539: [main] DEBUG GrouperLoaderConfig.retrieveLdapProfile(375) -  - LDAP config for server id: personLdap: GrouperLoaderLdapServer [batchSize=800, countLimit=600000, driver=null, expirationTime=-1, maxPoolSize=-1, minPoolSize=-1, pass=XXXXX, pruneTimerPeriod=-1, saslAuthorizationId=, saslRealm=, timeLimit=600000, timeout=-1, tls=false, url="ldap://idm-6.acs.utah.edu:389," user=cn=Directory Manager, validateOnCheckIn=false, validateOnCheckOut=true, validatePeriodically=false, validateTimerPeriod=-1]

2013-04-23 11:34:46,554: [main] WARN  AbstractLdapFactory.validate(165) -  - validate called, but no validator configured

2013-04-23 11:34:46,978: [main] DEBUG GrouperLoaderConfig.retrieveLdapProfile(375) -  - LDAP config for server id: personLdap: GrouperLoaderLdapServer [batchSize=800, countLimit=600000, driver=null, expirationTime=-1, maxPoolSize=-1, minPoolSize=-1, pass=XXXXX, pruneTimerPeriod=-1, saslAuthorizationId=, saslRealm=, timeLimit=600000, timeout=-1, tls=false, url="ldap://idm-6.acs.utah.edu:389," user=cn=Directory Manager, validateOnCheckIn=false, validateOnCheckOut=true, validatePeriodically=false, validateTimerPeriod=-1]

2013-04-23 11:34:46,993: [main] DEBUG GrouperLoaderConfig.retrieveLdapProfile(375) -  - LDAP config for server id: personLdap: GrouperLoaderLdapServer [batchSize=800, countLimit=600000, driver=null, expirationTime=-1, maxPoolSize=-1, minPoolSize=-1, pass=XXXXX, pruneTimerPeriod=-1, saslAuthorizationId=, saslRealm=, timeLimit=600000, timeout=-1, tls=false, url="ldap://idm-6.acs.utah.edu:389," user=cn=Directory Manager, validateOnCheckIn=false, validateOnCheckOut=true, validatePeriodically=false, validateTimerPeriod=-1]

2013-04-23 11:34:46,995: [main] DEBUG GrouperLoaderConfig.retrieveLdapProfile(375) -  - LDAP config for server id: personLdap: GrouperLoaderLdapServer [batchSize=800, countLimit=600000, driver=null, expirationTime=-1, maxPoolSize=-1, minPoolSize=-1, pass=XXXXX, pruneTimerPeriod=-1, saslAuthorizationId=, saslRealm=, timeLimit=600000, timeout=-1, tls=false, url="ldap://idm-6.acs.utah.edu:389," user=cn=Directory Manager, validateOnCheckIn=false, validateOnCheckOut=true, validatePeriodically=false, validateTimerPeriod=-1]

2013-04-23 11:34:46,995: [main] DEBUG GrouperLoaderConfig.retrieveLdapProfile(375) -  - LDAP config for s

server id: personLdap: GrouperLoaderLdapServer [batchSize=800, countLimit=600000, driver=null, expirationTime=-1, maxPoolSize=-1, minPoolSize=-1, pass=XXXXX, pruneTimerPeriod=-1, saslAuthorizationId=, saslRealm=, timeLimit=600000, timeout=-1, tls=false, url="ldap://idm-6.acs.utah.edu:389," user=cn=Directory Manager, validateOnCheckIn=false, validateOnCheckOut=true, validatePeriodically=false, validateTimerPeriod=-1]

2013-04-23 11:34:46,996: [main] DEBUG GrouperLoaderConfig.retrieveLdapProfile(375) -  - LDAP config for server id: personLdap: GrouperLoaderLdapServer [batchSize=800, countLimit=600000, driver=null, expirationTime=-1, maxPoolSize=-1, minPoolSize=-1, pass=XXXXX, pruneTimerPeriod=-1, saslAuthorizationId=, saslRealm=, timeLimit=600000, timeout=-1, tls=false, url="ldap://idm-6.acs.utah.edu:389," user=cn=Directory Manager, validateOnCheckIn=false, validateOnCheckOut=true, validatePeriodically=false, validateTimerPeriod=-1]

2013-04-23 11:34:46,997: [main] DEBUG GrouperLoaderConfig.retrieveLdapProfile(375) -  - LDAP config for server id: personLdap: GrouperLoaderLdapServer [batchSize=800, countLimit=600000, driver=null, expirationTime=-1, maxPoolSize=-1, minPoolSize=-1, pass=XXXXX, pruneTimerPeriod=-1, saslAuthorizationId=, saslRealm=, timeLimit=600000, timeout=-1, tls=false, url="ldap://idm-6.acs.utah.edu:389," user=cn=Directory Manager, validateOnCheckIn=false, validateOnCheckOut=true, validatePeriodically=false, validateTimerPeriod=-1]

2013-04-23 11:34:46,998: [main] DEBUG GrouperLoaderConfig.retrieveLdapProfile(375) -  - LDAP config for server id: personLdap: GrouperLoaderLdapServer [batchSize=800, countLimit=600000, driver=null, expirationTime=-1, maxPoolSize=-1, minPoolSize=-1, pass=XXXXX, pruneTimerPeriod=-1, saslAuthorizationId=, saslRealm=, timeLimit=600000, timeout=-1, tls=false, url="ldap://idm-6.acs.utah.edu:389," user=cn=Directory Manager, validateOnCheckIn=false, validateOnCheckOut=true, validatePeriodically=false, validateTimerPeriod=-1]

2013-04-23 11:34:46,999: [main] DEBUG GrouperLoaderConfig.retrieveLdapProfile(375) -  - LDAP config for server id: personLdap: GrouperLoaderLdapServer [batchSize=800, countLimit=600000, driver=null, expirationTime=-1, maxPoolSize=-1, minPoolSize=-1, pass=XXXXX, pruneTimerPeriod=-1, saslAuthorizationId=, saslRealm=, timeLimit=600000, timeout=-1, tls=false, url="ldap://idm-6.acs.utah.edu:389," user=cn=Directory Manager, validateOnCheckIn=false, validateOnCheckOut=true, validatePeriodically=false, validateTimerPeriod=-1]

2013-04-23 11:34:47,000: [main] DEBUG GrouperLoaderConfig.retrieveLdapProfile(375) -  - LDAP config for server id: personLdap: GrouperLoaderLdapServer [batchSize=800, countLimit=600000, driver=null, expirationTime=-1, maxPoolSize=-1, minPoolSize=-1, pass=XXXXX, pruneTimerPeriod=-1, saslAuthorizationId=, saslRealm=, timeLimit=600000, timeout=-1, tls=false, url="ldap://idm-6.acs.utah.edu:389," user=cn=Directory Manager, validateOnCheckIn=false, validateOnCheckOut=true, validatePeriodically=false, validateTimerPeriod=-1]

2013-04-23 11:34:47,001: [main] DEBUG GrouperLoaderConfig.retrieveLdapProfile(375) -  - LDAP config for server id: personLdap: GrouperLoaderLdapServer [batchSize=800, countLimit=600000, driver=null, expirationTime=-1, maxPoolSize=-1, minPoolSize=-1, pass=XXXXX, pruneTimerPeriod=-1, saslAuthorizationId=, saslRealm=, timeLimit=600000, timeout=-1, tls=false, url="ldap://idm-6.acs.utah.edu:389," user=cn=Directory Manager, validateOnCheckIn=false, validateOnCheckOut=true, validatePeriodically=false, validateTimerPeriod=-1]

2013-04-23 11:34:47,001: [main] DEBUG GrouperLoaderConfig.retrieveLdapProfile(375) -  - LDAP config for server id: pe

 

-          Deleted many lines at GrouperLoaderConfig.retrieveLdapProfile(375) - 

 

2013-04-23 11:34:47,662: [main] DEBUG GrouperLoaderConfig.retrieveLdapProfile(375) -  - LDAP config for server id: personLdap: GrouperLoaderLdapServer [batchSize=800, countLimit=600000, driver=null, expirationTime=-1, maxPoolSize=-1, minPoolSize=-1, pass=XXXXX, pruneTimerPeriod=-1, saslAuthorizationId=, saslRealm=, timeLimit=600000, timeout=-1, tls=false, url="ldap://idm-6.acs.utah.edu:389," user=cn=Directory Manager, validateOnCheckIn=false, validateOnCheckOut=true, validatePeriodically=false, validateTimerPeriod=-1]

2013-04-23 11:34:47,662: [main] DEBUG GrouperLoaderResultset$2.callback(888) -  - Found 1 results, (967 sub-results) for serverId: personLdap, searchDn: ou=people,o=utah.edu, filter: '(sn=nguyen)', returning subject attribute: unid, some results: {ActiveDirectory:groups:currentEmployee=[u0802045, u0802960, u0804070, u0804143, u0804371, u08044...

2013-04-23 11:34:47,664: [main] DEBUG GrouperLoaderType$8.runJob(994) -  - ActiveDirectory:groupsFromAttributesLdapGroup: start syncing membership

2013-04-23 11:34:47,664: [main] DEBUG GrouperLoaderType.syncGroupList(1114) -  - ActiveDirectory:groupsFromAttributesLdapGroup: found 141 members overall

2013-04-23 11:34:47,665: [main] DEBUG GrouperLoaderType.syncGroupList(1124) -  - ActiveDirectory:groupsFromAttributesLdapGroup: syncing membership for 1 groups

2013-04-23 11:34:47,665: [main] DEBUG GrouperLoaderType.syncGroupList(1340) -  - ActiveDirectory:groupsFromAttributesLdapGroup: syncing membership for ActiveDirectory:groups:currentEmployee 1 out of 1 groups

2013-04-23 11:34:47,670: [main] DEBUG GrouperLoaderType.syncOneGroupMembership(1926) -  - ActiveDirectory:groups:currentEmployee start syncing membership

2013-04-23 11:34:47,673: [main] DEBUG GrouperLoaderType.syncOneGroupMembership(1942) -  - ActiveDirectory:groups:currentEmployee syncing 141 rows

2013-04-23 11:34:47,693: [main] DEBUG GrouperLoaderType.syncOneGroupMembership(1960) -  - ActiveDirectory:groups:currentEmployee: saving group if necessary, result type: NO_CHANGE

2013-04-23 11:34:47,693: [main] DEBUG GrouperLoaderType.syncOneGroupMembership(2055) -  - Done assigning privilege to related groups: ActiveDirectory:groups:currentEmployee

2013-04-23 11:34:49,551: [main] INFO  GrouperLoaderType.syncOneGroupMembership(2301) -  - ActiveDirectory:groups:currentEmployee done syncing membership, processed 141 records.  Total members: 141, inserts: 0, deletes: 0

2013-04-23 11:34:49,586: [main] DEBUG GrouperLoaderType.syncGroupList(1433) -  - ActiveDirectory:groupsFromAttributesLdapGroup: done syncing membership

 

 

From: Chris Hyzer []
Sent: Tuesday, April 23, 2013 11:13 AM
To: Bryan E. Wooten;
Subject: RE: I am completely confused

 

Sorry this is frustrating, thanks for your patience.  Can you run it with this in the log4j.properties and let me know the output?

 

log4j.logger.edu.internet2.middleware.grouper.app.loader = DEBUG

 

Also, are you using the code I sent you for AD?

 

Can you try with the paging turned off and see if it is different?  J

 

I assume when you run it in your browser, all of the 811 results have a unique unid attribute…  and the unid is the grouper subject source subjectId?  I assume you are passed these types of questions J

 

I don’t know why it wouldn’t work like you specify, but if it is a filter that returns users for one group, shouldn’t it be an LDAP_SIMPLE type?

 

Thanks a lot!

 

Chris

 

 

 

From: [] On Behalf Of Bryan E. Wooten
Sent: Monday, April 22, 2013 5:32 PM
To:
Subject: [grouper-users] I am completely confused

 

I am trying to get the Grouper  LDAP_GROUPS_FROM_ATTRIBUTES to work but the results are confusing.

 

When I run the Grouper loader LDAP filter manually in my LDAP browser I get 811 results.

 

When I run the loaderRunOneJob(), I see 155 searchs. Yet over in the grouper_error.log I see this message:

 

  GrouperLoaderType.syncOneGroupMembership(2301) -  - ActiveDirectory:groups:currentEmployee done syncing membership, processed 168 records.  Total members: 168, inserts: 62, deletes: 0

 

And then when I look at the members in the UI it says there are 63 members.

 

I have DEBUG set in log4jproperties for vt-ldap, but nothing jumps out at me.

 

Does anyone have any idea how I can further trouble shoot this and understand why the Grouper group doesn’t get all the members I think it should? Something, somewhere is filtering out the results.

 

Below is my Group configuration.

 

Thanks,

 

Bryan

 

 

Attribute assignments

Owner group     Attribute name                Enabled?            Assignment values         Attribute definition        Assignment UUID

                groupsFromAttributesLdapGroup            Grouper loader LDAP     enabled                               grouperLoaderLdapDef                10635...

Metadata on assignment                             Grouper loader LDAP group attribute name        enabled                  uuemployee                grouperLoaderLdapValueDef     0b9ea...

Metadata on assignment                             Grouper loader LDAP quartz cron             enabled                  0 * 0/1 * * ?                grouperLoaderLdapValueDef     33c16...

Metadata on assignment                             Grouper loader LDAP type           enabled                  LDAP_GROUPS_FROM_ATTRIBUTES      grouperLoaderLdapValueDef     35d89...

Metadata on assignment                             Grouper loader LDAP subject _expression_              enabled                  ${subjectAttributes['unid']}        grouperLoaderLdapValueDef     4d525...

Metadata on assignment                             Grouper loader LDAP server ID  enabled                  personLdap                grouperLoaderLdapValueDef     69096...

Metadata on assignment                             Grouper loader LDAP group name _expression_    enabled                  groups:currentEmployee             grouperLoaderLdapValueDef     9243e...

Metadata on assignment                             Grouper loader LDAP subject ID type     enabled                  subjectId                grouperLoaderLdapValueDef     92bb6...

Metadata on assignment                             Grouper loader LDAP extra attributes    enabled                  unid                grouperLoaderLdapValueDef     9a5a6...

Metadata on assignment                             Grouper loader LDAP filter          enabled                  (&(uuaffiliate=afssec)(uuemployee=uuparttimeemploye))         grouperLoaderLdapValueDef     a5522...

Metadata on assignment                             Grouper loader LDAP search base DN     enabled                  ou=people,o=utah.edu                grouperLoaderLdapValueDef     ba12d...

Metadata on assignment                             Grouper loader LDAP source ID enabled                  ldap                grouperLoaderLdapValueDef     c9756...