Grouper Developers Forum

[Tom Barton <>]

Scott,

We've had two epochs of provisioning work so far in the grouper project. First was the initial 
LDAPPC, which we couldn't sustain. Then came PSP (nee LDAPPCng). That strategy was founded on 
two main assumptions, or hopes. First, that the substantial slice 'n dice capabilities often 
needed to combine and transform source (group) data to make it suitable for a given target 
should occur in a common element of the architecture, and that shibboleth's attribute resolver 
was a good instance of that capability with lots of community contribution keeping it stocked 
with valuable tools. Second, that SPMLv2, though basically dead at that time, was a good 
embodiment of the types of actions that commonly occur in provisioning. Why invent something 
only slightly different? Personally, I even hoped we might revive SPML, like our colleagues did 
with the then-dead SAML when the shib project was forming.

So the idea was that most provisioning tasks should naturally factor through SPML and common 
transformation utilities, making it easier to add additional targets, maybe sources too, to the 
architecture. Alas, SPML remained dead despite effort to revive it (everyone was getting 
provisioning done, if badly, unlike federation which was then a greenfield), and moreover the 
SPML metaphor didn't always fit that well, and factoring through it did not net the hoped-for 
efficiency. Still, it works, and can be made to work better.

Now that we have a good deal of field experience with PSP, I figured it's time to reflect on 
whether we should continue to improve PSP or envision a different, third swipe at this vexing 
problem. The discussion in the minutes is the first serious such discussion we've had in the 
grouper-dev context. However it concludes, we will also think through how to bring existing 
deployments through any transition, as we have with all substantial changes to grouper. And for 
the moment, with the SURFnet/SCIM work to be started soon (which doesn't appear to benefit from 
SPML), we have a great opportunity to think alternatives through in a concrete manner. We'll see 
where that takes us.

Tom

On 7/8/2013 8:30 PM, Scott Koranda wrote:
> Hi,
>
> > PSP/provisioning Strategy
> >
> > -DaveL has been analyzing PSP strengths and challenges
> > -PSP is extensible
> > -Not fast enough for all use cases, partly due to SPML foundation
> > -There are advantages  to smaller and more efficient modules
> >
> > -Jim: U. Washington is looking into MongoDB for document store
> > -MongoDB may replace LDAP for some use cases
> > -Would need different provisioning
> > -Jim will investigate whether there is a SCIM-to-MongoDB interface
> > -MongoDB uses a JSON data structure, uses puts and gets
> >
> > -Grouper's REST API may need to be adapted, especially given the focus on REST
> > in the CIFER project.
> > -Grouper's relational backend can be a drawback for scaling.
> > -With REST becoming more common, perhaps point-to-point connectors make more
> > sense that a generalized provisioning engine like PSP
> >
> > -Currently most sites using PSP use it for provisioning to LDAP and AD systems
> > - These would both be adaptable to a good point-to-point solution
> >
> > -The provisioning engine approach (solve provisioning once with various
> > connectors) sounds better than it actually is.
> > -It was agreed that it makes sense for the Grouper project to stop investing 
> > in
> > PSP, which is a provisioning engine based on SPML
> >
> > -For the SURFnet and SCIM work that is planned, the direction is to develop
> > a change log consumers, more of a point-to-point approach
> >
> > -It's  important to ensure the point-to-point modules can work in parallel or
> > can chain together to populate several consumers
> > -parallel processing will be part of the design
> >
> > Q: Shilen: When using the change log for provisioning, what about the need
> > for bulk synchronization?
> > A: DaveL: there will also be a mass reconciliation option
> >      Grouper would need to interface with
> >         -change log consumer (incremental) approach, AND
> >         -bulk processing
> >
> > [AI] (DaveL) will start a wiki page about provisioning future.
> I would be grateful if you could include in the wiki page
> about the provisioning future a timeline and details about
> what it means to "stop investing in PSP".
>
> I am working on a number of Grouper deployments at the moment
> that rely in critical ways on Grouper -> LDAP provisioning
> using the PSP and I would like to be able to clearly explain
> what projects can expect for support and transition options
> from the Grouper development team and the community.
>
> Thanks,
>
> Scott Koranda