Grouper Users - Open Discussion List

[Tom Zeller <>]

The latency I am concerned about is on the order of hours rather than
minutes, for example, patch tuesday or holiday break maintenance when
domain controllers might be down for many hours. So, I am not sure the
5 minute rule helps.

I understand the scaling issues, and I am also unsure what to cache
for each kind of change log event. For example, for a membership
change log event, do I query for the group, member, subject, and
membership objects and cache all of them ?

The idea I am floating is for grouper to provide an attribute, e.g.
changeLogAttribute1, which if present on any object whose identifier
exists in a change log event, the values of the attribute would be
added to the change log event. So, UChicago would store the custom
ldap identifier and the target identifier in changeLogAttribute1. They
would need to use some sort of string separator, or xml, or whatever,
to concatenate two old-style attributes into a single new-style
attribute. If we have enough change log event columns available, we
could provide changeLogAttribute1, changeLogAttribute2, etc.

When I first encountered userAttribute1 or customAttribute1 in active
directory or windows live I thought they were clumsy, but at least
folks might not be surprised.

And, yeah, I think this is hacky but trying to cache a slice of the
grouper database for every change log event seems prone to performance
issues.

Thoughts ?

TomZ

On Fri, Apr 6, 2012 at 6:07 PM, Chris Hyzer 
<>
 wrote:
> I think we should document that changes in provisioning attributes take 5 
> minutes to propagate to the provisioner... you can do a second query and 
> cache for 5 minutes...
> Sound good?  :)
>
> I don't think we should add attributes to the change log since we would 
> need to add for each type of query it will get complicated to configure and 
> what if someone needs more attributes for various provisioners than are 
> available.  It doesn't really scale...
>
> Thanks,
> Chris
>
> -----Original Message-----
> From: 
> 
>  
> [mailto:]
>  On Behalf Of Tom Zeller
> Sent: Friday, April 06, 2012 12:05 AM
> To: Colin Hudler
> Cc: 
> 
> Subject: Re: [grouper-users] Replicating peculiar functionality with psp
>
> Here is one issue worth talking about.
>
> The current code for change log based provisioning does not handle the
> custom attributes which determine whether or not a group should be
> provisioned as well as the desired ldap dn.
>
> For the first attempt at change log based provisioning, I decided to
> try to provision based on change log entry data only without
> additional queries. An "add membership" change log entry does not
> contain the custom group attributes, so another query would be needed
> to return those attributes. Not a big deal, except ... I have been
> worried about phasing, meaning the potential time difference between
> when a change log entry is processed and when the action producing the
> change log entry occurred. I think I am alone in my concern.
>
> A potential solution is to use PIT (point-in-time) auditing to return
> the group as it was when the change log event occurred. However, if I
> remember correctly, I don't think that custom attributes are included
> in PIT auditing, but new attribute framework attributes are.
>
> In general, I prefer provisioning a change log entry without
> performing additional queries, meaning, a change log entry should
> contain all of the data necessary for provisioning.
>
> Maybe it could be possible to include custom attributes in change log
> entries ? A change log entry has capacity for 12 strings (columns),
> maybe deployers could specify additional group (or member or stem)
> attributes to be returned in addition to the group (or member or stem)
> name or id ?
>
> So, rather than the provisioner querying for additional data at the
> time of provisioning, perhaps grouper could include necessary
> additional data in the change log entry at the time the change log
> event occurred, which would alleviate my concerns regarding time
> delays.
>
> Whether or not folks use the psp for provisioning based on the change
> log, I think they will face similar issues, and customizing change log
> entries may help deployers regardless of their provisioning gear.
>
> This response is probably too long already, but it might be a good
> idea to standardize and perhaps provide a default configuration for
> attributes used to determine whether or not objects should be
> provisioned, as well as target identifiers.
>
> TomZ
>
> On Thu, Apr 5, 2012 at 6:39 PM, Tom Zeller 
> <>
>  wrote:
>> Apologies for the delay in my response.
>>
>> In general, I think you will be ok, meaning : you should be able to do
>> what you want.
>>
>> It will take me a minute to parse your configuration, in the meantime,
>> could you help me understand  :
>>
>>> Our static groups may not have a canonical groupDn, and often not any 
>>> static
>>> group at all.
>>
>> I think you are saying that the ldap dn of a static group is user
>> specified, and stored as a group attribute in grouper. This seems
>> fine.
>>
>> And, regarding :
>>
>>> allows the user to also specify the DOMAIN where groups are stored (we do
>>> not always have them in the same domain as users).
>>
>> What is the impact of the user specified domain ? Does each domain
>> require an ldap connection ?
>>
>> Thanks for trying out the psp.
>>
>> TomZ
>>
>> On Wed, Apr 4, 2012 at 12:05 PM, Colin Hudler 
>> <>
>>  wrote:
>>> We have custom provisioner to LDAP. Users are allowed to add a Group Type
>>> and attribute value that the provisioner looks for to enable the
>>> synchronization to LDAP. One of the attributes allows the user to write 
>>> the
>>> group into one or many pre-defined LDAP containers (e.g,
>>> ou=groups,ou=groups,ou=application,...). The attribute value is a string
>>> containing DNs separated by semicolon.
>>>
>>> My approach so far is the goal of psp writing exactly the same
>>> static/dynamic groups as our custom sync. Success is measured by bulkCalc
>>> returning whatever is the equivalent of no-op. Alternatively, I can
>>> restructure how we use Types and Attributes. The immovable is our ability 
>>> to
>>> direct static groups using Type/Attribute for one or more LDAP containers.
>>>
>>> I have been successful in plucking the groups in GroupDataConnector,
>>> AllGroupNamesConnector, et al, by using GroupExactAttribute filters.
>>> However, I wonder about the feasibility of the dynamic user-specified 
>>> static
>>> group DNs. Besides that I cannot currently conceive of the correct
>>> psp-resolver configuration, I think this might be opposed to the design
>>> (please let me know). Will it cause problems with the groupDn attribute? 
>>> Our
>>> static groups may not have a canonical groupDn, and often not any static
>>> group at all. Attached is my attempt at a psp-resolver and psp.xml
>>> configuration just for POC. I think the log message during bulkCalc "PSO
>>> Identifier Definition 'groupDn' - Source attribute 'groupDn' does not 
>>> exist"
>>> points at least at part of my problem.
>>>
>>> As if that isn't enough, our custom AD synchronizer works similarly, 
>>> except
>>> allows the user to also specify the DOMAIN where groups are stored (we do
>>> not always have them in the same domain as users). Perhaps I'll tackle 
>>> this
>>> another time, but we also have user objects from a forest trust which are
>>> resolved by the provisioner (the group-member in AD must be a specially
>>> formatted foreign SID obtained from the other forest, not a vanilla local
>>> user object). WHAT A MESS!