Grouper Developers Forum

[Shilen Patel <>]

Tom,

So with this approach, what would be the purpose of the tables for 
recent updates?  Would new changes only go there first and then the 
daemon would put them into the regular tables?   I'm not sure I'm following.

Thanks!

-- Shilen



On 8/12/10 11:46 AM, Tom Barton wrote:
> Thanks Shilen. If I understand correctly, you're saying that we've
> already adapted the regular grouper_* tables to handle updates by
> multiple simultaneous processes, so don't want to mess with that. The
> new tables can be simpler in that regard because there will only be one
> process updating them.
>
> However, I think you've also raised a concern about accuracy of
> replication between these parallel tables and the regular tables, given
> that the parallel tables are not to be touched by those multiple
> processes that update the regular tables. And of course the size of the
> database roughly doubles with parallel tables.
>
> Is it safe to say that all issues that can arise due to multiple
> processes only arise due to management of updates that occur at nearly
> the same time? Eg, process contention like you describe, or hooks that
> need to know the state of the registry just prior to their firing.
>
> If so, I wonder if there another approach in which a slice of the
> registry containing recent updates only is kept in some tables, allowing
> the rest of the registry to be kept in the regular tables, with some
> added columns to support PIT. Or did we already consider and reject
> something like this when designing the solution to the problem of
> multiple simultaneous updaters?
>
> Thanks,
> Tom
>
> On 8/12/2010 9:06 AM, Shilen Patel wrote:
>    
> > Hi Tom,
> >
> > One of the issues with that is database constraints.  For instance, in
> > the grouper_memberships table, we have a unique key on the owner,
> > member, and field.  This ensures that immediate memberships cannot be
> > duplicated, which can otherwise happen if two processes add the same
> > membership at about the same time.  We had several jira issues on
> > that.   If we add start and end timestamps to this table, then this
> > unique key needs to be removed or changed to include other fields like
> > the start timestamp, but this will result in the same issues we had
> > before.  By the way, this concern wouldn't exist for the PIT tables
> > because there's just a single process maintaining that table.
> >
> > Another reason (that may not be an issue but maybe something to
> > consider) would be that these tables would become larger and that might
> > have a performance impact.
> >
> > Thanks!
> >
> > -- Shilen
> >
> >
> > On 8/12/10 9:00 AM, Tom Barton wrote:
> >      
> > > Shilen,
> > >
> > > I have one question for which the answer must be obvious, but I don't
> > > see it right now. Why have separate tables for flattened notifications
> > > and PIT, ie, why not just add columns to existing tables for start&   end
> > > timestamps and active state?
> > >
> > > Thanks,
> > > Tom
> > >
> > > On 8/9/2010 11:52 AM, Shilen Patel wrote:
> > >
> > >        
> > > > Since there were performance concerns about how we handled flattened
> > > > membership notifications in 1.6 and how we were planning on handling PIT
> > > > audit in 2.0, here's a new design for how we can handle both in 2.0.
> > > >
> > > > https://spaces.internet2.edu/display/GrouperWG/Membership+PIT+audit+and+flattened+notifications+for+v2.0
> > > >
> > > >
> > > >
> > > > Let me know if you have any questions or concerns.
> > > >
> > > > Thanks!
> > > >
> > > > -- Shilen
> > > >
> > > >          
> > >
> > >        
> >      
>