Grouper Users - Open Discussion List

["Michael R. Gettes" <>]

the idea would be to only scoop up certain operations at once - stop if you hit a boundary condition.

for example:

add member, delete member changes for the same group would all be scooped up until you see a different operation.  so a rename would be different, you would stop processing the add/del, send that message and then send the rename or whatever other attribute change is needed.  everything would be processed in order in the change log.  we are thinking of putting all static group operations into one queue and single thread it the processing of that queue to avoid the problems of ordering.  as for non-group objects - like isMemberOf handling - they can be handled in parallel following the same algorithm.

/mrg

On Apr 29, 2013, at 1:12 PM, Gagné Sébastien <>

 wrote:

Hi,

Quick ideas :

How would ordering be maintained ? e.g. you don’t want to rename a group before its description is modified.

What if you have conflicting information ? e.g. two consecutive description modification of the same group, which one to keep (comes back to ordering transaction)

How would you know you have “all” the changes for that object ?

 

As for your problem, could you have two changelog consumer row ? You could have one for “last published to Active MQ” and one for “current aggregation id”, that way you restart the changelog on “last published” and start aggregating there.

 

Good luck

 

 

De :  [mailto:grouper-] De la part de Rahul Doshi
Envoyé : 29 avril 2013 11:13
À : 
Objet : [grouper-users] PSP performance

 

Hello,

 

While  jira https://bugs.internet2.edu/jira/browse/GRP-882 to improve the PSP real-time provisioning  is worked on we are looking to implement a stop gap solution to improve PSP real-time provisioning performance.   This is how we are planning to do that.  We will implement a custom change log consumer that will combine multiple updates to same object and publish it to a queue like ActiveMQ.  We will implement a client that will grab the message from the queue and perform the update as single ldap operation. 

 

One limitation we already know of this approach is we would have to continuously advance change log consumer pointer even though we are still aggregating changes in a single message to push it to ActiveMQ.  If change log consumer dies for whatever reason before we could publish the message to the queue we will lose changes and only way to recover from that would be to do bulkSync.  Are there any other issues that we could potentially  face?  Also if anyone has alternative approach to improve PSP real-time performance we would very much like to hear.

 

Thanks,

Rahul