Grouper Developers Forum

[Tom Zeller <>]

After taking a refactoring detour, I am getting back to figuring out
how to integrate (1) provisioning based on the changelog as well as
(2) optional scheduled full synchronizations (the current style of
operation).

The issue I have is that to take advantage of caching I would like the
changelog and full-sync jobs to use the same instance of the
provisioning service provider (PSP.java).

So...it seems old-fashioned but my thought is to insert changelog
entries to emulate the current full-sync style jobs. In other words,
at some scheduled time (e.g. via a grouper-loader.properties cron
statement), for every group, stem, membership, etc insert new
GROUP_UPDATE, STEM_UPDATE, MEMBERSHIP_UPDATE changelog entries even
though nothing has actually changed. This process would also be used
to bootstrap a new provisioned target.

What do you think ?

On Wed, Sep 28, 2011 at 10:34 AM, Jim Fox 
<>
 wrote:
>
> We provision our ldap directly off the change log (a variant of it) and use
> ActiveMQ to provision everything else.  We run an off line process that
> periodically compares grouper's registry with ldap and reports differences.
> So the only automatic updates to ldap are naturally in proper time order.
>
> Jim
>
>
> On Wed, 28 Sep 2011, McDermott, Michael wrote:
>
>> Date: Wed, 28 Sep 2011 02:06:24 -0700
>> From: "McDermott, Michael" 
>> <>
>> To: Tom Zeller 
>> <>
>> Cc: Grouper Dev 
>> <>
>> Subject: Re: [grouper-dev] [ldappcng] real-time & batch scheduling and
>>    clobbering avoidance ?
>>
>> Locally we are looking at solving this problem with the change log and a
>> queue for exactly this reason.  We're using a an independent, remote
>> queue, but as I read the features of ActiveMQ (Apache's JMS offering), you
>> can embed it and use it as part of an application.  To paraphrase a
>> quip about regular expressions:  'You come across a problem where it seems
>> like you can setup a simple queue in a database; now you have two
>> problems.'
>>
>>
>>
>> On Tue, Sep 27, 2011 at 10:48 PM, Tom Zeller 
>> <>
>>  wrote:
>>      How do we want to schedule real-time and batch provisioning ?
>>
>>      Currently, ldappcng has
>>
>>       -interval <seconds>
>>
>>      and
>>
>>       -intervalFullSync <seconds>
>>
>>      command line options to control how often provisioning runs are
>> performed.
>>
>>      The -interval <seconds> option looks at lastModifyTime to determine
>> if
>>      objects should be updated, and I think we will discontinue this in
>>      lieu of the changelog.
>>
>>      So, I assume, every few (configurable) seconds ldappcng will
>> provision
>>      (incrementally) based on the changelog.
>>
>>      And, I assume, deployers will want to schedule batch / full sync
>>      provisioning runs at night (or maybe more often) to make sure
>>      everything is properly provisioned.
>>
>>      However, let us assume a situation where a full sync of a large group
>>      takes a long time (e.g. 1 minute). During the full sync of the large
>>      group, membership changes might occur, which will be queued in the
>>      changelog. After the full sync of the large group is finished and we
>>      begin processing the changelog, we should not process changelog
>>      entries that were queued before the group was fully synchronized but
>>      after the full sync was started. So, I think we will need to store
>> the
>>      last update time of provisioned objects to not clobber ourselves.
>>
>>      We may also want to (optionally) schedule a full sync of a
>> provisioned
>>      object if provisioning incrementally based on the changelog
>>      experiences an error.
>>
>>      What do you think ?