Grouper Developers Forum

[Chris Hyzer <>]

When is the notification going to be sent to the external system.  I hope it is pulled off a queue in the DB and processed that way, and not real-time or asynchronous from a java hook, right?  Since that is the reliability aspect.   In order for the notification to be sure it is time to notify, a commit must take place of the grouper data.  And in order for that notification to be reliably in the DB, it must be put in the grouper db, in the same transaction as the data commit.  Then it can be pulled off the audit table (and maybe a table which keeps track of the last_edited date that each notification type is up to), and sent to the external system, then that last_edited date or state keeper can be updated…

 

If we really want real time, I think we could use a hibernate session interceptor or event as a first pass…

 

http://www.hibernate.org/hib_docs/reference/en/html/events.html

 

So the hibernate session keeps track of auditing objects that are inserted in jdbc, and after the transaction is complete, it loops through them and asynchronously kicks off notifications.  If those succeed they would update the state of the table that keeps track of progress for that notification type.  Then a daemon can go through and anything 5 minutes or older can be retried (or whatever the algorithm).  Again, I think this is more complex, but we can give it a shot to achieve more speedy notifications.  The downside here is that all the notifications are not strictly queued… since multiple grouper instances are all firing the notifications that they know about, it could get out of order, and data could be inconsistent (e.g. the grouper db thinks a group exists, but that notification failed, then another grouper instance adds a member to the group and notifies.  The endpoint must know that the group doesn’t exist and trigger a failure there too…)  the more I think about it, the more I think one daemon with ordered notifications would be better.

 

Personally I think the simpler approach is to just take the current state of the data in grouper and notify (once the notification happens).  This notion of serialization and point-in-time is definitely more powerful and useful, but also more complex and might be less performant and require more storage… if that turns out to be the case, maybe it could be an option (the option being either to serialize or to just use grouper data but still have notifications/auditing available)  J

 

Thanks,

Chris

 

From: [mailto:] On Behalf Of Tom Zeller
Sent: Monday, June 02, 2008 10:11 PM
To: GW Brown, Information Systems and Computing
Cc: Chris Hyzer; Grouper Dev;
Subject: Re: [grouper-dev] changelog implementation sketch

 

On Mon, Jun 2, 2008 at 4:23 AM, GW Brown <> wrote:

I think we need to be very clear on what we are trying to achieve with auditing since it impacts greatly on our approach.

Tom B. has suggested that we use reliable (guaranteed) auditing as a way of recording changes such that these changes can be used to drive potentially less reliable notifications. Effectively, if a notification fails it might be reconstructed / queried for by a 'listener' system. So, in Java, we will have a representation of a change which can be 'serialised' to the audit database. It should then be possible to re-constitute the Java representation of the change from the data in the audit database.

 

I'm not clear on reliable vs less- or un-reliable auditing. I understand the notion of serializing a (java) change to an audit database, but I don't understand where 'guaranteed' fits in. Clues ?

 

I am also uncomfortable with the idea of using the file system to store changes largely because the API can easily be run in different JVMs on different machines which would need to mount the same file system and be properly configured.

 

And being one of those sites with multiple JVMs writing to grouper, I certainly should have realized that our data access protocol is (essentially) JDBC, hence data stored outside the database would need another way to get to it at the cost of additional complexity. Memphis' person registry has a similar setup to grouper and sometimes I wish we had written a service which wrapped (hid) the underlying database connections.

 

TomZ