Subject: Grouper Developers Forum
RE: [grouper-dev] what will provide the CIFER API?
- From: Chris Hyzer <>
- To: Jim Fox <>, Grouper Dev <>
- Subject: RE: [grouper-dev] what will provide the CIFER API?
- Date: Tue, 10 Sep 2013 18:50:46 +0000
- Accept-language: en-US
I think LDAP or nosql, or elacticsearch (did you mean elasticache?) is
difficult to work when there is security on who is allowed to read what.
The single-master multi-slave architecture is built out in Grouper, the
client can send readonly queries to a different place than read-write,
Grouper WS can operate in readonly mode, and the databases that Grouper
supports can be setup with one-way replication.
You don't need the change log consumer for this architecture. Regarding
direct or indirect memberships, you don't have to worry about it if you use
the same database definitions. It seems like it is easy to store things as
direct-only and calculate the indirects, but the edge cases on memberships
and privileges seem to make this infeasible...
If we need to improve performance we should profile some of the operations
and see if we can improve things...
On Behalf Of Jim Fox
Sent: Tuesday, September 10, 2013 1:59 PM
To: Grouper Dev
Subject: [grouper-dev] what will provide the CIFER API?
Looking forward to a Grouper with a CIFER authorization API.
TL;DR: A monolithic Grouper cannot support the CIFER API in an
enterprise. For resilency, capacity, performance and geographical
redundancy the API must scale horizontally on commodity hardware.
Possible solutions are discussed.
The meat and potatos of a group service is its response to membership
queries. Historically a busy Grouper installation offloaded many
of these requests onto LDAPs. Those LDAPs, however, cannot directly
support the new CIFER authorization API, be they directly related to
groups or abstracted to permissions. That API will be the essence
and definition of the groups service. It will become popular.
The University of Washington's experience might be representative.
We have run a groups API for several years, opening it to any person
or organization at the university. It became popular. A sampling of
usage from July 17, an average summer day, showed 12,159 requests
over the hour from 10 to 11am. Roughly 70% of these requests are
for some kind of effective membership and were routed by the API
to LDAPs, specifically designed for the purpose. These handlers
are easy to scale, impervious to failure of the registry and can
easily be scattered about the landscape.
Some way to scale the API come to mind.
A multi-master DBMS would allow a cluster of Groupers to be
synchronized by the underlying DBMS. I think this is generally not
possible. SQL DBMS tend to be shy of the eventual consistancy
of systems that are always available. They have to deal with
partitioning some way or other---usually by not really supporting
A single-master, multi-slave DBMS might solve part of the problem.
Many DBMS support this configuration. The API service, possbily
custom instances of grouper client, could be configured to access
the master for updates and scatter read requests amoung the slaves.
For us, at least, this does not address the performance issue.
A typical "is user an effective member of group" takes about
400ms using the present Grouper Java API and only 40ms using the
LDAP cache. I don't know that our service could not survive at
the slower rate.
A master DBMS with multiple nonSQL caches, e.g. LDAP, ElasticSearch, ...,
would allow the caches to respond to the most common requests.
There are new complications though.
- They would have to be always installed, as part of the
- They would have to be immediately up-to-date. The present
mechanism is too slow.
- They would maintain direct memberships. There is no way I know of
for a Grouper change log consumer (or anything else) to immediately
update effective memberships. Try it sometime.
- Because the cache maintain direct memberships of groups they must know
how to resolve effective memberships and composite group memberships.
- [grouper-dev] what will provide the CIFER API?, Jim Fox, 09/10/2013
- RE: [grouper-dev] what will provide the CIFER API?, Chris Hyzer, 09/10/2013
Archive powered by MHonArc 2.6.16.