Skip to Content.
Sympa Menu

grouper-users - Re: [grouper-users] Replicating peculiar functionality with psp

Subject: Grouper Users - Open Discussion List

List archive

Re: [grouper-users] Replicating peculiar functionality with psp

Chronological Thread 
  • From: Colin Hudler <>
  • To: "" <>
  • Subject: Re: [grouper-users] Replicating peculiar functionality with psp
  • Date: Thu, 12 Apr 2012 11:49:15 -0500

Just to reiterate, my goal is to have a single-instance provisioner that can write groups to different and multiple LDAP container OUs. Very often there is no static group (ou), yet the group name is written to a common member attribute (dynamic groups).

**We do have the ability to change how this happens at our site**

If doing away with our custom Types and Attributes makes that possible, UChicago is very much in favor of changing. Thus far, provisioning selection has been in the hands of our users; if it becomes a server-side psp configuration, we can probably accept the cost of it becoming an admin task.

My most serious obstacle is groking PSP well enough to make it write groups into multiple containers, or none at all, without individually configuring each group (there's hundreds). Can you help with this?

My ideal is to decorate the groups and/or stems with their intended provisioning targets and parameters, but I understand if that is very hard. Our group provisioning changes at least weekly, so whatever it is would be as lite as possible. Way back when, I accomplished this with an automated ldappc config generator which created beastly configurations for hundreds of groups based on the registry Types & Attributes and wrangled several ldappc instances together. It was slow, hard to debug, and almost impossible to use for a realtime system.

On 04/12/2012 11:25 AM, Tom Zeller wrote:
I don't remember, are old-style or new-style attributes cached by
grouper ? with PIT auditing too ?

If a group with 1,000 members is added to another group, there will be
1,000 change log add_membership entries. If a provisioner needs to
query grouper for old-style or new-style attributes to determine
provisioned target identifiers, that's 1,000 "find group" queries
(cached) and then 1,000 "get old-style attribute" queries and/or 1,000
"get new-style attribute" queries, correct ?

If we decide that we need to use PIT, are the PIT "find group", "get
old-style attribute", and "get new-style attribute" queries cached too

Deleting a group whose provisioned target identifier is not solely
based on the group name will require PIT queries as well.

Maybe we should say we don't support real-time provisioning if
provisioned target identifiers are not based only on the object
(group, stem, member) name, excluding displayName.

If attribute queries are cached by grouper, then maybe deployers will
want to have a separate ehcache configuration for provisioning with
longer timeouts than the UI.


On Wed, Apr 11, 2012 at 3:29 PM, Chris
The latency I am concerned about is on the order of hours rather than
minutes, for example, patch tuesday or holiday break maintenance when
domain controllers might be down for many hours. So, I am not sure the
5 minute rule helps.
Im saying if you have 2 queries in 5 minutes, then you could get from cache.
If you query 2 hours later, the provisioning attributes would still be there,
I don't think it is a problem.

Lets discuss this at the member meeting I guess... maybe a solution is a
JSON 4k field we could pack data into, or with the new UI try to get people
off of the old-style attributes and then things would be in PIT... Im still
not sure why this is a large risk, in general the metadata about provisioning
wouldn't change too often, it seems like an edge case which we might be


-----Original Message-----

On Behalf Of Tom Zeller
Sent: Wednesday, April 11, 2012 4:15 PM

Cc: Colin Hudler
Subject: Re: [grouper-users] Replicating peculiar functionality with psp

The latency I am concerned about is on the order of hours rather than
minutes, for example, patch tuesday or holiday break maintenance when
domain controllers might be down for many hours. So, I am not sure the
5 minute rule helps.

I understand the scaling issues, and I am also unsure what to cache
for each kind of change log event. For example, for a membership
change log event, do I query for the group, member, subject, and
membership objects and cache all of them ?

The idea I am floating is for grouper to provide an attribute, e.g.
changeLogAttribute1, which if present on any object whose identifier
exists in a change log event, the values of the attribute would be
added to the change log event. So, UChicago would store the custom
ldap identifier and the target identifier in changeLogAttribute1. They
would need to use some sort of string separator, or xml, or whatever,
to concatenate two old-style attributes into a single new-style
attribute. If we have enough change log event columns available, we
could provide changeLogAttribute1, changeLogAttribute2, etc.

When I first encountered userAttribute1 or customAttribute1 in active
directory or windows live I thought they were clumsy, but at least
folks might not be surprised.

And, yeah, I think this is hacky but trying to cache a slice of the
grouper database for every change log event seems prone to performance

Thoughts ?


On Fri, Apr 6, 2012 at 6:07 PM, Chris
I think we should document that changes in provisioning attributes take 5
minutes to propagate to the provisioner... you can do a second query and
cache for 5 minutes...
Sound good? :)

I don't think we should add attributes to the change log since we would need
to add for each type of query it will get complicated to configure and what
if someone needs more attributes for various provisioners than are available.
It doesn't really scale...


-----Original Message-----

On Behalf Of Tom Zeller
Sent: Friday, April 06, 2012 12:05 AM
To: Colin Hudler

Subject: Re: [grouper-users] Replicating peculiar functionality with psp

Here is one issue worth talking about.

The current code for change log based provisioning does not handle the
custom attributes which determine whether or not a group should be
provisioned as well as the desired ldap dn.

For the first attempt at change log based provisioning, I decided to
try to provision based on change log entry data only without
additional queries. An "add membership" change log entry does not
contain the custom group attributes, so another query would be needed
to return those attributes. Not a big deal, except ... I have been
worried about phasing, meaning the potential time difference between
when a change log entry is processed and when the action producing the
change log entry occurred. I think I am alone in my concern.

A potential solution is to use PIT (point-in-time) auditing to return
the group as it was when the change log event occurred. However, if I
remember correctly, I don't think that custom attributes are included
in PIT auditing, but new attribute framework attributes are.

In general, I prefer provisioning a change log entry without
performing additional queries, meaning, a change log entry should
contain all of the data necessary for provisioning.

Maybe it could be possible to include custom attributes in change log
entries ? A change log entry has capacity for 12 strings (columns),
maybe deployers could specify additional group (or member or stem)
attributes to be returned in addition to the group (or member or stem)
name or id ?

So, rather than the provisioner querying for additional data at the
time of provisioning, perhaps grouper could include necessary
additional data in the change log entry at the time the change log
event occurred, which would alleviate my concerns regarding time

Whether or not folks use the psp for provisioning based on the change
log, I think they will face similar issues, and customizing change log
entries may help deployers regardless of their provisioning gear.

This response is probably too long already, but it might be a good
idea to standardize and perhaps provide a default configuration for
attributes used to determine whether or not objects should be
provisioned, as well as target identifiers.


On Thu, Apr 5, 2012 at 6:39 PM, Tom
Apologies for the delay in my response.

In general, I think you will be ok, meaning : you should be able to do
what you want.

It will take me a minute to parse your configuration, in the meantime,
could you help me understand :

Our static groups may not have a canonical groupDn, and often not any static
group at all.
I think you are saying that the ldap dn of a static group is user
specified, and stored as a group attribute in grouper. This seems

And, regarding :

allows the user to also specify the DOMAIN where groups are stored (we do
not always have them in the same domain as users).
What is the impact of the user specified domain ? Does each domain
require an ldap connection ?

Thanks for trying out the psp.


On Wed, Apr 4, 2012 at 12:05 PM, Colin
We have custom provisioner to LDAP. Users are allowed to add a Group Type
and attribute value that the provisioner looks for to enable the
synchronization to LDAP. One of the attributes allows the user to write the
group into one or many pre-defined LDAP containers (e.g,
ou=groups,ou=groups,ou=application,...). The attribute value is a string
containing DNs separated by semicolon.

My approach so far is the goal of psp writing exactly the same
static/dynamic groups as our custom sync. Success is measured by bulkCalc
returning whatever is the equivalent of no-op. Alternatively, I can
restructure how we use Types and Attributes. The immovable is our ability to
direct static groups using Type/Attribute for one or more LDAP containers.

I have been successful in plucking the groups in GroupDataConnector,
AllGroupNamesConnector, et al, by using GroupExactAttribute filters.
However, I wonder about the feasibility of the dynamic user-specified static
group DNs. Besides that I cannot currently conceive of the correct
psp-resolver configuration, I think this might be opposed to the design
(please let me know). Will it cause problems with the groupDn attribute? Our
static groups may not have a canonical groupDn, and often not any static
group at all. Attached is my attempt at a psp-resolver and psp.xml
configuration just for POC. I think the log message during bulkCalc "PSO
Identifier Definition 'groupDn' - Source attribute 'groupDn' does not exist"
points at least at part of my problem.

As if that isn't enough, our custom AD synchronizer works similarly, except
allows the user to also specify the DOMAIN where groups are stored (we do
not always have them in the same domain as users). Perhaps I'll tackle this
another time, but we also have user objects from a forest trust which are
resolved by the provisioner (the group-member in AD must be a specially
formatted foreign SID obtained from the other forest, not a vanilla local
user object). WHAT A MESS!

Archive powered by MHonArc 2.6.16.

Top of Page