Skip to Content.
Sympa Menu

grouper-dev - Re: [grouper-dev] [ldappcng] real-time & batch scheduling and clobbering avoidance ?

Subject: Grouper Developers Forum

List archive

Re: [grouper-dev] [ldappcng] real-time & batch scheduling and clobbering avoidance ?

Chronological Thread 
  • From: Tom Zeller <>
  • To: Chris Hyzer <>
  • Cc: Grouper Dev <>
  • Subject: Re: [grouper-dev] [ldappcng] real-time & batch scheduling and clobbering avoidance ?
  • Date: Wed, 19 Oct 2011 11:56:46 -0500

> Do you mean inserting records into the change log table, or just making
> change log entry beans and processing them as you would off the change log?
>  For the former, you shouldn't do it since other things read that table.  
> For the latter, if you aren't sacrificing performance due to already
> knowing some of the answers that the incremental has to figure out but the
> full sync might already know, then go for it I guess.
> Note: if the incremental is running, and you need to do a full sync in the
> middle of the night, there will be some overlap.  i.e. maybe the full sync
> decides to add a member to a group, but the incremental also wants to do
> that.  Each process will need to ignore it if the member if already in the
> group, right?  Are we stopping the incremental while the full sync if
> running or will they run at the same time for a certain period of time?

Even if we stop the incremental job, once it starts again it may do
the wrong thing. For example, if during a full sync (after the correct
provisioning has been calculated from grouper) a member is deleted,
when the incremental job processes the member deletion that will be
the wrong thing.

I was thinking of inserting records into the change log table, to
preserve order of operations and to avoid overlap.

So ... is it possible to insert records in the change log table so
that they are only read by a particular consumer ? There could be lots
of records, one per every object in grouper, including memberships.

If not, and we go the bean approach, I am not sure how to avoid
overlap. We could think of some ways, timestamps, locks, ugh. The idea
I was floating was to store the timestamp of every provisioned object
upon full synchronization, and check each change log entry before
processing it to make sure that the timestamp of the change log entry
is after the full sync timestamp.

Archive powered by MHonArc 2.6.16.

Top of Page