Skip to Content.
Sympa Menu

grouper-dev - RE: [grouper-dev] RE: [grouper-users] Grouper v1.6.0 released

Subject: Grouper Developers Forum

List archive

RE: [grouper-dev] RE: [grouper-users] Grouper v1.6.0 released


Chronological Thread 
  • From: Jim Fox <>
  • To: Chris Hyzer <>
  • Cc: Tom Barton <>, "" <>, Grouper Dev <>
  • Subject: RE: [grouper-dev] RE: [grouper-users] Grouper v1.6.0 released
  • Date: Mon, 28 Jun 2010 15:37:08 -0700 (PDT)


I've not experienced this problem in our testing, and I'm a direct member of
about 100 of
our 40K+ groups. Many of those are in turn members of other groups.

Just curious, are you a wheel group member? That one is a lot faster when
checking privileges.

I'm using the grouper API, so I make a combination of
getImmediateMemberships and getEffectiveMemberships for the field 'admin'.
I expect that's similar to what you do.


1) They don't scale.

It is abhorrent to me that a few users, adding a couple of group
members to their groups, might cause a million database records to
be created. In addition this new table will slow down the addition
and deletion of members. And this is exactly where, for us, grouper
has its greatest performance issues.


FYI this is done in a daemon process in the background, and the
performance is tuned to leverage the logic in the unflattened table,
so it is definitely faster than it was in 1.4. I think it would
only cause a million database records if you have a million groups
and add someone to all (or someone to have privileges on all).
I'd be interested to know if you have a user added to an admin
group which is an admin of 10's of thousand of groups, or even
hundreds of thousands, how long that would take to propagate to
the flattened table.

The flattened table is for members of any field, isn't it? If I
have a group, say, u:fox:myfriends, and I add as a member the group,
uw:affiliation:alumni (which has 150K members) do I not add 150K
records to the flattened table -- one record for each effective
member of my group? If my group's membership was not world-read
and I wanted all myfriends to see all the others I might add the
group as a reader. Another 150K records.


3) They can't be implemented in a federated world.

Our clients know that members of groups can be of several
types: our own uwnetids; ePPNs from other sites; dns domains,
i.e. certificate CNs; Active Directory computer names; and other
groups in the registry. I can see a federated future where we
have groups with members that are groups from a remote registry,
say, UPenn. That is not so far-fetched as you might think,
and it shows just how 20th century are flattened memberships.


Im not sure how federated groups affect flattened memberships.
The flattened table is just built on a query from a view in
grouper... if the rows aren't in the view, then they wont work
with the API in unflattened mode either. If we need a resolver in
grouper to resolve external groups or dynamic groups, then other
logic will change in grouper and it wont be any different if we
use flattened or not.

Yes. My point, not so well made, is that in a federated world
membership is best calculated at the time it is required.
Anything else is likely to be out of date. I know that doesn't
apply to the flat membership table. It's just a general rule.
Maybe just my rule.



Another case if you have a lot of web service hits and the data
doesn't need to be up to date (flattened should be a few minutes
behind), then we will put a mode in where the request can go against
the flattened tables, and this will help the DB load.

This is what the subordinate directories are for.


Well, not all queries can go against subordinate
directories... when centralized permissions are used it might be
too complex of a data model for directories I believe. With groups,
getting the members of a group is easy enough. Some queries against
directories right now require web services depending on how you
deploy them. i.e. give me the groups of a user that I am allowed
to READ... does your directory handle that? mine doesn't and I
think most do not.

Actually, mine does. But that isn't exactly what I meant. We have a fastcgi application that provides a subset of our group web service API
and uses the ldap directories for a database. (It is a read-only copy of our old web service.) This responds only to program clients (simple xhtml)
and only for the members-of-group and groups-for-members searches. If we didn't already have the cgi app I would provide the same capability
through the tomcat app.


Anyways, your concerns are noted, we need to make sure performance
is acceptable and discuss this further obviously... :)

Fair enough.

Jim




Archive powered by MHonArc 2.6.16.

Top of Page