Grouper Developers Forum

[Jim Fox <>]

Looking forward to a Grouper with a CIFER authorization API.

Jim



TL;DR: A monolithic Grouper cannot support the CIFER API in an
enterprise.  For resilency, capacity, performance and geographical
redundancy the API must scale horizontally on commodity hardware.
Possible solutions are discussed.


Details:


The meat and potatos of a group service is its response to membership
queries.  Historically a busy Grouper installation offloaded many
of these requests onto LDAPs.  Those LDAPs, however, cannot directly
support the new CIFER authorization API, be they directly related to
groups or abstracted to permissions.  That API will be the essence
and definition of the groups service.  It will become popular.

The University of Washington's experience might be representative.
We have run a groups API for several years, opening it to any person
or organization at the university. It became popular.  A sampling of
usage from July 17, an average summer day, showed 12,159 requests
over the hour from 10 to 11am.  Roughly 70% of these requests are
for some kind of effective membership and were routed by the API
to LDAPs, specifically designed for the purpose.  These handlers
are easy to scale, impervious to failure of the registry and can
easily be scattered about the landscape.


Some way to scale the API come to mind.

A multi-master DBMS would allow a cluster of Groupers to be
synchronized by the underlying DBMS.  I think this is generally not
possible.  SQL DBMS tend to be shy of the eventual consistancy
of systems that are always available.  They have to deal with
partitioning some way or other---usually by not really supporting
multi-master.


A single-master, multi-slave DBMS might solve part of the problem.
Many DBMS support this configuration.  The API service, possbily
custom instances of grouper client, could be configured to access
the master for updates and scatter read requests amoung the slaves.
For us, at least,  this does not address the performance issue.
A typical "is user an effective member of group" takes about
400ms using the present Grouper Java API and only 40ms using the
LDAP cache.  I don't know that our service could not survive at
the slower rate.

A master DBMS with multiple nonSQL caches, e.g. LDAP, ElasticSearch, ...,
would allow the caches to respond to the most common requests.
There are new complications though.

- They would have to be always installed, as part of the
  Grouper system.

- They would have to be immediately up-to-date.  The present
  mechanism is too slow.

- They would maintain direct memberships.  There is no way I know of
  for a Grouper change log consumer (or anything else) to immediately
  update effective memberships.  Try it sometime.

- Because the cache maintain direct memberships of groups they must know
  how to resolve effective memberships and composite group memberships.