Skip to Content.
Sympa Menu

grouper-dev - Re: [grouper-dev] [ldappcng] real-time & batch scheduling and clobbering avoidance ?

Subject: Grouper Developers Forum

List archive

Re: [grouper-dev] [ldappcng] real-time & batch scheduling and clobbering avoidance ?

Chronological Thread 
  • From: Tom Zeller <>
  • To: Grouper Dev <>
  • Subject: Re: [grouper-dev] [ldappcng] real-time & batch scheduling and clobbering avoidance ?
  • Date: Wed, 19 Oct 2011 11:01:55 -0500

After taking a refactoring detour, I am getting back to figuring out
how to integrate (1) provisioning based on the changelog as well as
(2) optional scheduled full synchronizations (the current style of

The issue I have is that to take advantage of caching I would like the
changelog and full-sync jobs to use the same instance of the
provisioning service provider ( seems old-fashioned but my thought is to insert changelog
entries to emulate the current full-sync style jobs. In other words,
at some scheduled time (e.g. via a cron
statement), for every group, stem, membership, etc insert new
though nothing has actually changed. This process would also be used
to bootstrap a new provisioned target.

What do you think ?

On Wed, Sep 28, 2011 at 10:34 AM, Jim Fox
> We provision our ldap directly off the change log (a variant of it) and use
> ActiveMQ to provision everything else.  We run an off line process that
> periodically compares grouper's registry with ldap and reports differences.
> So the only automatic updates to ldap are naturally in proper time order.
> Jim
> On Wed, 28 Sep 2011, McDermott, Michael wrote:
>> Date: Wed, 28 Sep 2011 02:06:24 -0700
>> From: "McDermott, Michael"
>> <>
>> To: Tom Zeller
>> <>
>> Cc: Grouper Dev
>> <>
>> Subject: Re: [grouper-dev] [ldappcng] real-time & batch scheduling and
>>    clobbering avoidance ?
>> Locally we are looking at solving this problem with the change log and a
>> queue for exactly this reason.  We're using a an independent, remote
>> queue, but as I read the features of ActiveMQ (Apache's JMS offering), you
>> can embed it and use it as part of an application.  To paraphrase a
>> quip about regular expressions:  'You come across a problem where it seems
>> like you can setup a simple queue in a database; now you have two
>> problems.'
>> On Tue, Sep 27, 2011 at 10:48 PM, Tom Zeller
>> <>
>> wrote:
>>      How do we want to schedule real-time and batch provisioning ?
>>      Currently, ldappcng has
>>       -interval <seconds>
>>      and
>>       -intervalFullSync <seconds>
>>      command line options to control how often provisioning runs are
>> performed.
>>      The -interval <seconds> option looks at lastModifyTime to determine
>> if
>>      objects should be updated, and I think we will discontinue this in
>>      lieu of the changelog.
>>      So, I assume, every few (configurable) seconds ldappcng will
>> provision
>>      (incrementally) based on the changelog.
>>      And, I assume, deployers will want to schedule batch / full sync
>>      provisioning runs at night (or maybe more often) to make sure
>>      everything is properly provisioned.
>>      However, let us assume a situation where a full sync of a large group
>>      takes a long time (e.g. 1 minute). During the full sync of the large
>>      group, membership changes might occur, which will be queued in the
>>      changelog. After the full sync of the large group is finished and we
>>      begin processing the changelog, we should not process changelog
>>      entries that were queued before the group was fully synchronized but
>>      after the full sync was started. So, I think we will need to store
>> the
>>      last update time of provisioned objects to not clobber ourselves.
>>      We may also want to (optionally) schedule a full sync of a
>> provisioned
>>      object if provisioning incrementally based on the changelog
>>      experiences an error.
>>      What do you think ?

Archive powered by MHonArc 2.6.16.

Top of Page