Grouper Users - Open Discussion List

["Hyzer, Chris" <>]

I would not do this for database migrations... is there another use case for 
needing this?

If you want to sync a group in a grouper to another grouper you can use this:

https://spaces.at.internet2.edu/display/Grouper/Syncing+groups+between+group+management+systems

Anyways, beyond that, there isn't a way to do this.  You could do database 
replication though, the target would likely be readonly

-----Original Message-----
From: 

 
<>
 On Behalf Of Black, Carey M.
Sent: Monday, October 15, 2018 5:49 PM
To: 

Subject: [grouper-users] Can the ChangeLogConsumerToMessage send to a remote 
grouper instance?

My goals is to "migrate" events from one Grouper install to another. Then 
"re-play" the events into the remote grouper.

In my universe I am using grouper UI and WS services.
 NOTE: I have not yet cracked the "book" on the grouper client. ( Maybe that 
is the gap in my knowledge? )


REF1: 
https://spaces.at.internet2.edu/display/Grouper/Grouper+messaging+send+receive+change+log+adapter+example
        "Configure a message sender (change log format) in 
grouper-loader.properties"
                
edu.internet2.middleware.grouper.changeLog.ChangeLogConsumerToMessage  ( I am 
guessing this needs to be a "local queue")
        It looks like there is a class that can take a changelog and turn it 
into a "message".
        But I don't see how I can send that message to a "remote" 
grouperBuiltinMessaging system.
        If I could send to the WS "Message Receive" on Grouper2 that might be 
ideal to transport between the two. ( Especially if the "Receive" WS actually 
decodes the message and will "replay it" on the local instance too, or it 
could be smart enough to call the "right web service" from the sending code.)

REF2: 
https://spaces.at.internet2.edu/display/Grouper/Grouper+messaging+to+web+service+API
        " Hook up grouper WS to read from a queue "
        " That will cause all WS servers to listen on a queue or topic for 
messages "  ( I am guessing this needs to be a "local queue")
                But then again I am left with "how to connect it to the 
'other' grouper instances message queue?".
                
REF3: https://spaces.at.internet2.edu/display/Grouper/Grouper+Client
        Appears to be able to talk to Grouper WebServices... maybe it could 
read "Queue1 from Grouper1" and write "Queue1 on Grouper2"?



High level desired flow:
        Group1 changeLogConsumer to(-->) message queue on Grouper2 (not sure 
if REF1 can do this or not, might need REF1 and REF3?)
        And
        Grouper2 Message Queue to (-->)  Grouper2 API calls on Grouper2 ( 
REF2)


It might work like this: ( three queues, maybe just configuration to get 
there?)
        Users adds a folder on Grouper1. 
        Change log consumer creates a message on Grouper1(REF1) to "create 
folder 'newFolderName'". 
                ( Ideally, message is delivered to Grouper2 by Grouper 1. But 
might require using grouperClient REF3) to move the message from Grouper1 to 
Grouper2?)
        Message processor (REF2 on Grouper2)  reads the local queue and 
creates the 'newFolderName' on Grouper2. 

NOTE: I would be prefer to remove the queue in the middle if that could be 
done too. After all the sending ChangeLogConsumer is a queue. :) 
In this case: (two queues)
        Users adds a folder on Grouper1. 
        Change log consumer(REF1) creates a message on Grouper2 (can it do 
that?) to "create folder 'newFolderName'". 
        Message processor (REF2 on Grouper2)  reads the local queue and 
creates the 'newFolderName' on Grouper2.

NOTE: Maybe more ideal : (only one queue only on the sending side)
        Users adds a folder on Grouper1. 
        Change log consumer(some combination of Ref1 and Ref2 ?) calls WS on 
Grouper2 to "create folder 'newFolderName'".


As long as this data flow is reasonably "fast" (completed within a few 
minutes of user actions), reliable ( 100% ), and covers 100% of the Grouper 
API, then I think it would work well for the following use cases:
        ) Database migrations (across DB types)
                Likely leaves  Point in time data behind... but I think that 
could be dealt with in other "non-real time" ways for audit.
                Or if the messages could preserve the original "start dates" 
that would be ideal. Then only the "already closed" items would be left 
behind. And all active objects would be "reconstructed".
        ) Filter replicas ( where you only want some of the data to be sent. )
        ) "warm backups" / "Read only copies"

I hope I am missing just some configuration details here...

Thoughts? Clue bricks? Recipes?

Thanks in advance.

-- 
Carey Matthew