Grouper Developers Forum

[Tom Barton <>]

Thanks Shilen. If I understand correctly, you're saying that we've 
already adapted the regular grouper_* tables to handle updates by 
multiple simultaneous processes, so don't want to mess with that. The 
new tables can be simpler in that regard because there will only be one 
process updating them.

However, I think you've also raised a concern about accuracy of 
replication between these parallel tables and the regular tables, given 
that the parallel tables are not to be touched by those multiple 
processes that update the regular tables. And of course the size of the 
database roughly doubles with parallel tables.

Is it safe to say that all issues that can arise due to multiple 
processes only arise due to management of updates that occur at nearly 
the same time? Eg, process contention like you describe, or hooks that 
need to know the state of the registry just prior to their firing.

If so, I wonder if there another approach in which a slice of the 
registry containing recent updates only is kept in some tables, allowing 
the rest of the registry to be kept in the regular tables, with some 
added columns to support PIT. Or did we already consider and reject 
something like this when designing the solution to the problem of 
multiple simultaneous updaters?

Thanks,
Tom

On 8/12/2010 9:06 AM, Shilen Patel wrote:
> Hi Tom,
>
> One of the issues with that is database constraints.  For instance, in 
> the grouper_memberships table, we have a unique key on the owner, 
> member, and field.  This ensures that immediate memberships cannot be 
> duplicated, which can otherwise happen if two processes add the same 
> membership at about the same time.  We had several jira issues on 
> that.   If we add start and end timestamps to this table, then this 
> unique key needs to be removed or changed to include other fields like 
> the start timestamp, but this will result in the same issues we had 
> before.  By the way, this concern wouldn't exist for the PIT tables 
> because there's just a single process maintaining that table.
>
> Another reason (that may not be an issue but maybe something to 
> consider) would be that these tables would become larger and that might 
> have a performance impact.
>
> Thanks!
>
> -- Shilen
>
>
> On 8/12/10 9:00 AM, Tom Barton wrote:
> > Shilen,
> >
> > I have one question for which the answer must be obvious, but I don't
> > see it right now. Why have separate tables for flattened notifications
> > and PIT, ie, why not just add columns to existing tables for start&  end
> > timestamps and active state?
> >
> > Thanks,
> > Tom
> >
> > On 8/9/2010 11:52 AM, Shilen Patel wrote:
> >   
> > > Since there were performance concerns about how we handled flattened
> > > membership notifications in 1.6 and how we were planning on handling PIT
> > > audit in 2.0, here's a new design for how we can handle both in 2.0.
> > >
> > > https://spaces.internet2.edu/display/GrouperWG/Membership+PIT+audit+and+flattened+notifications+for+v2.0 
> > >
> > >
> > >
> > > Let me know if you have any questions or concerns.
> > >
> > > Thanks!
> > >
> > > -- Shilen
> > >      
> >