Grouper Users - Open Discussion List

[Joy Veronneau <>]

Hi,

I mentioned on the working group call yesterday that we have been brainstorming how to handle a group of 190K members that has been causing us some problems on the directory side.  Here's a brief explanation of the problem and some of the ideas we are kicking around. I'd be interested in your thoughts & opinions.

We will be migrating all of our current "permit" memberships into grouper when we go live. That will create several hundred new groups. Several of them are large with around 20K members, but there is one very large group called cu.alumni with 190K members.

Cu.alumni loads fine into the grouper database. The problem happens when we try to have ldappc populate the group in our directory. (We are creating the groups and populating the hasmember attribute, but not populating the corresponding ismemberof attribute for each user.) Ldappc dies at about the 90,000th hasmember entry of cu.alumni, so we came up with the idea to populate cu.alumni in our directory via just a program that uses jndi. This also croaks at about 90,000, so we load the group in chunks now, and that seems to work ok. (We turned off replication while we did the add.) We think the problem is some memory limit in jndi? but anyway...

Once we got cu.alumni loaded into the directory, we tried a sample query to see if someone is a member of the group, using a filter something like (&(cn=cu.alumni)(hasmember=jv11)). Works fine for all groups but for cu.alumni, it takes a really long time.

So...

Some reading via google indicates that the directory will indeed have problems with entries that have very large multivalued attributes. Suggestion is to use dynamic groups.

This gave us some ideas.  This particular group does not need to be maintained via the grouper ui, it might even be better if the cu.alumni group never appears in the grouper db.

Currently the cu.alumni permit is populated automatically, and we could just as easily populate something like edupersonaffiliation in the directory to indicate that someone is an alumni. Good so far...

Then we could create either a dynamic group or a filtered role based on the edupersonaffiliation value to give us a cu.alumni group in the directory...

OK... then we came up with a few requirements, here's the one that caused me to write this email:

R1.0 - All group membership queries need to be the same format as far as the search filter and results received back are concerned. Receiving a URL from the directory (because cu.alumni is a dynamic group) or receiving "nsrole" for some queries would not be acceptable.

OK, so we could create a directory plugin that would look at each search coming into the directory and reformat the query if necessary, and reformat the results if necessary, and then we could use either a dynamic group or a filtered role.

We don't have any experience with dynamic groups, filtered roles, or directory plugins, other than some cursory tests with the groups and roles.

I guess I am looking for some advice here - does this approach sound reasonable? Would you pick dynamic group over filtered role for one reason or another? Are plugins a bad idea? Got a better idea? 

Thanks!

Joy