Skip to Content.
Sympa Menu

grouper-dev - [grouper-dev] RE: [grouper-users] Grouper v1.6.0 released

Subject: Grouper Developers Forum

List archive

[grouper-dev] RE: [grouper-users] Grouper v1.6.0 released

Chronological Thread 
  • From: Jim Fox <>
  • To: Chris Hyzer <>
  • Cc: Tom Barton <>, "" <>, Grouper Dev <>
  • Subject: [grouper-dev] RE: [grouper-users] Grouper v1.6.0 released
  • Date: Mon, 28 Jun 2010 12:35:04 -0700 (PDT)

So my question is: What is the consequence of running without the
change log, ignoring the flattened memberships table altogether?
Are there any core functions of grouper that are going to depend
on this table?

As Tom mentioned, the notifications are more efficient when
the external system doesn't care about why someone is a member,
just that they are a member. Also, that table is more efficient
to query. For instance, if someone is an admin of many groups,
and that person goes to the UI to in "Manage groups" mode, then
Grouper needs to calculate which groups the person can admin.
From the normal registry this has been reported too slow, but I
believe the flattened table can do it quickly (we still need to
work this part out).

I've not experienced this problem in our testing, and I'm a direct
member of about 100 of our 40K+ groups. Many of those are in turn
members of other groups.

I have several objections to flattened memberships:

1) They don't scale.

It is abhorrent to me that a few users, adding a couple of group
members to their groups, might cause a million database records to
be created. In addition this new table will slow down the addition
and deletion of members. And this is exactly where, for us, grouper
has its greatest performance issues.

2) They aren't necessary.

We export our group memberships to three places: openldap directories;
an active directory; and (coming soon) google groups. Each of these
nested group memberships. For a good reason I believe.

3) They can't be implemented in a federated world.

Our clients know that members of groups can be of several
types: our own uwnetids; ePPNs from other sites; dns domains,
i.e. certificate CNs; Active Directory computer names; and other
groups in the registry. I can see a federated future where we
have groups with members that are groups from a remote registry,
say, UPenn. That is not so far-fetched as you might think,
and it shows just how 20th century are flattened memberships.

Another case if you have a lot of web service hits and the data
doesn't need to be up to date (flattened should be a few minutes
behind), then we will put a mode in where the request can go against
the flattened tables, and this will help the DB load.

This is what the subordinate directories are for.

We find that about 90% of hits on our service are two types of queries:

- find groups a user is an effective member of

- find all members af a group

These are also the two queries that require the fastest response
time and the highest reliability. Our Shibboleth IdP, for example,
uses the first query. To provide the best support for these requests
we implement an alternate url that accesses our LDAP directories,
of which there are always at least two, rather than the registry.
Even with the web service resolving nested memberships at runtime,
the groups-for-user for me and my 100 or so groups, takes 50-70 msec.
Accessing the registry directly takes 500-1000 mesc.

A few minutes is an eternity around here. Our LDAP directories are up to date in about 5 sec.

Anyways, bottom line is, sorry this means you need more storage
space in the DB, but it would be nice to assume that people use
this feature. If you strongly disagree (and perhaps either way)
maybe we can discuss at a dev call some time.

For reasons just described I do disagree. We find the hook mechanism
much more efficient for event notification that the event log, we
already offload the most common queries to the ldap directories,
and have no other use for the flattened tables. I don't want the
overhead of maintaining them, or of running the event log.


Archive powered by MHonArc 2.6.16.

Top of Page