Grouper Users - Open Discussion List

[Lynn Garrison <>]

On Mar 14, 2011, at 2:19 PM, Tom Zeller wrote:

>>        The provisioning to ldap timing will be a problem for us. We use 
>> GPFS for our all share file system and we control group access to share 
>> file systems with groups in ldap.  So all of are groups have to be 
>> provisioned to ldap.  Currently we support three types of groups - 
>> standing (department), course and user managed groups.    The standing 
>> groups are created once and changes to the groups are made once a day.  
>> Course groups are created once a semester and changes are made once a day. 
>>  The user managed groups are maintained only in ldap and changes appear as 
>> soon as they occur.   One of the reasons that we are looking at grouper is 
>> to make the changes to standing and course groups in real time.
>>        During our testing we ran into several areas of concern.   We were 
>> running ldappcng with a bulksync  and interval of 180 seconds.   On the 
>> first interval, the sync took about 45 minutes because the large group had 
>> to be provisioned.  During that interval, I added several more  small 
>> groups (2 members) to grouper.  The groups didn't appear in ldap until the 
>> large group was provisioned.  Once  the large group was in sync, 
>> provisioning of new groups and members happened very quickly.    I added 
>> one member to the large group, and  the sync was back up to taking 45 
>> minutes.  It appeared that it modified all members of the group instead of 
>> just  the one added.  At least that is what I believe that the SPML was 
>> telling me.  The large group is our faculty staff group and could 
>> potentially change multiple times a day.
>> 
>> Concerns
>>        1.  Minor change to membership appears to re-provision the group
> 
> A single membership change in grouper ("add member M to group G")
> should only result in a single corresponding ldap modify operation on
> one attribute (member) adding a single value (the member DN).
> 
> That being said, ldappcng does not provision incrementally. It (1)
> calculates how a group should be provisioned, probably searching for
> the DNs of every member, (2) queries the target ldap directory for how
> the group is currently provisioned including slurping all attributes,
> and (3) determines differences and applies the changes necessary to
> synchronize the group.
> 
> Incremental provisioning is on the roadmap, there are some design
> issues that prevent me from stating a timeline. It is the most
> important feature request for ldappcng.
> 
>>        2.  Provisioning appears to be single threaded
> 
> Yes, ldappcng is single threaded, as it depends on third-party
> libraries whose threadsafety is not under our control. Running more
> than one instance of ldappcng is probably safer, but of course, less
> efficient.
> 
> So, if your standing, course, and user managed groups exist in your
> DIT under separate OUs, for example :
> 
> ou=standing groups,dc=edu
>  cn=group 1, ...
> 
> ou=course groups, dc=edu
> cn=course group 1, ...
> 
> ou=user managed groups, dc=edu
>  cn=user managed group 1, ...
> 
> you might be able to run 3 instances of ldappcng, each provisioning an
> authoritative OU.
> 
> 
> In general, ldappcng development was aimed at configuration
> flexibility, not necessarily performance. As of right now, it employs
> no caching, which it should, probably using ehcache. However, 45
> minutes to provision a group with 32k members seems slow.
> 
> Shall we pursue how to speed up your instance of ldappcng ?
> 
> TomZ
>   
        Yes we would like to explore speeding up ldappcng.  What information 
would you like me to provide?