Subject: Grouper Developers Forum
- From: Jim Fox <>
- To: Chris Hyzer <>
- Cc: "" <>
- Subject: Re: [grouper-dev] highly available web services
- Date: Sun, 14 Jun 2009 20:09:22 -0700
These steps you outline are a lot more complicated than you let on. Why not just replicate the database?
On Jun 14, 2009, at 2:25 PM, Chris Hyzer wrote:
On the call and on the list we have been discussing highly available web services (Im thinking about reads here not writes). Some use cases:
1. Queries that aren't conducive to LDAP: e.g. if the limits are exceeded of LDAP, or maybe complex queries
2. If a deployment of grouper doesn't have LDAP, it doesn't need it
3. Queries that require logic, e.g. the attribute framework will need privilege decision point queries which require logic to get the answer
So I think web services should:
1. Slurp all data from the database into objects, and serialize them to disk.
2. For read operations, use those structures to make the decision
3. If web services are shutdown, deserialize from disk (and run recent change logs from the time it was last serialized)
4. If a certain amount of time has passed since the last slurp, slurp again. E.g. 24 hours
5. Have a thread which reads the change log every so often (e.g. every 2 minutes, configurable), update the object structures. Every so often (30 minutes), serialize to disk
6. There could be an option of configuration for if you want readonly to do this, or just hit the db, or or this if the DB is unavailable
7. Yes, if you have this mode on, it will require a lot of memory (proportional to the size of the registry)... we could swap this out for a local database or some other storage if we like, this part could even be pluggable
This way you should be able to run multiple web services instances, with a load balancer, and be highly available for readonly operations (when DB goes down)...
Anyways, I don't think any of this is all that complicated or hard or time consuming to build... also, Im not going to work on this soon, just throwing out ideas...
- highly available web services, Chris Hyzer, 06/14/2009
Archive powered by MHonArc 2.6.16.