Grouper Developers Forum

["Michael R. Gettes" <>]

Thanks Tom… I do appreciate your comments.

I am unable to make the wed, noon calls.

I have expressed my comments on the spaces page about provisioning and I am working with DaveL to help in learning about AMQ.  I do take the perspective of the grouper team should include AMQ and concentrate on providing appropriate tools for getting grouper messages to AMQ, possibly in various forms.  Provisioning to LDAP, SCIM and everything else can then be handled by consumers to AMQ.  The language which the consumers are written in really shouldn’t be a concern.  The community should be developing the consumers.  It is my strong belief by pursuing this strategy the grouper team can really concentrate on grouper itself - a great tool which has no other equal and until the marketplace picks up on it - if ever.  i just want there to be absolutely no doubt in anyone’s mind of my intentions - it’s all about helping and making grouper better.

one of the key problems we have at cmu with monitoring our environment is being able to keep track of what is happening with respect to the grouper changelog consumers.  getting the data out into AMQ allows us to use AMQ tools for monitoring.  one more reason to allow for getting data out of grouper into other subsystems as rapidly as possible.

So, given some of the notes, I am very concerned about continuing to provide a grouper changelog consumer (GCC) for LDAP/AD.  Providing a GCC for Amazon SNS is a fine idea.  Designing the messages we send out to apps becomes critical and there is a note about that issue - getting enough info presented to allow for actions to be taken like tagging, message versioning and so on.

thanks for listening and for prompting this discussion further.  i do appreciate it.

/mrg

On Aug 14, 2014, at 10:29 AM, Tom Barton <> wrote:

Michael and others with deep interest in Grouper's provisioning strategy,

To keep the project's forward momentum now is the time to discuss, research, debate, and finally decide on the essential elements of provisioning work for v2.3. I strongly encourage people wishing to participate in that decision to do so on this list and on grouper-dev calls announced here. Grouper has always been shaped by the consensus of those who show up to shape it. All are welcome, and all points of view are most welcome!

Thanks,
Tom

On 8/13/2014 8:57 AM, Michael R. Gettes wrote:

I hope there will be opportunity to discuss this stuff (provisioning strategy) and A-Camp along with others who are interested in how this gets solved (BillT, Keith, ScottK, just to name a few).

thanks!

/mrg

On Aug 13, 2014, at 8:01 AM, Emily Eisbruch <> wrote:

Provisioning Strategy

https://spaces.internet2.edu/display/Grouper/Post+PSP+Provisioning

What should we have in place for provisioning for Grouper 2.3?

Tom: we had previously decided to have custom change log consumers for LDAP and for AD.

We had thought about other provisioning needs: Google, Box, Message queues, etc

Dave: A Google connector / provisioning approach (change log consumer) is being worked on in the field, with a university and help from an affiliate.  

Jim: It is important to have a mechanism to differentiate which groups are being provisioned and which are not

Dave: suggestion to use the design being used at U. Chicago, which is to decorate groups with attributes that determine which provisioners should be used for the group. (I.e. I want this group to be provisioned to LDAP)

Tom: What about message queues?

CMU has been advocating for use of Active MQ

Chris: we could make that pluggable

comment: there are issues with pushing things to message queues, and determining visibility of confidential info in the messages, and  there is a need for all  or partial encryption.  

Chris: we can follow AWS patterns for that.

Two issues: 1) Putting all messages into one queue / handler, then the messages can be pushed out to other queues.

comment: Message queue involves some latency. LDAP has to be up to date quickly. 

So this is an argument against putting everything on a message queue.

U. Washington populates Google and AD through the message bus, but does not populate the main LDAP through the message bus.

For LDAP, U Washington uses a hook that writes a temporary file, like the 1st version of the change log. U Washington uses cache to respond to membership requests, so LDAP must be up to date.

Chris:

- If you can't wait, use a hook

-if you can wait, use a change log or messaging queue

Question: can the hook write message to Amazon SQS or ActiveMQ?

A: Yes, you can do it before or after the commit happens.

Risk that if the system shuts down, the message  could get lost.

But overall it should be OK

What do we need to know to choose between hook or change log as the implementation?

Jim: most sites are happy with the way the change log is used now.

U Washington has an special situation.

Change log may be a safer approach. Less risk of systems shutting down and losing transactions

Translating ACLs can be an issue to consider, especially around Google groups.

Suggested strategy:

-Provide a change log consumer that can work w ActiveMQ and Amazon SQS

-Provide a customer change log consumer for LDAP and AD

-Make it possible to tag groups and attach attributes to groups, etc. to specify provisioning  info.

We may want to make the SCIM provisioning approach more generic, but need a working SCIM end-point.

[AI] (Dave) will document Grouper provisioning strategy on the wiki for review and discussion

https://spaces.internet2.edu/display/Grouper/Post+PSP+Provisioning