Grouper Users - Open Discussion List

[Chris Hyzer <>]

> Regarding the 'earlier threads we wont mention': I pointed you at
> jmemcached earlier. (http://www.thimbleware.com/projects/jmemcached)
> Would you be willing/able to take a quick look at it and make a (very)
> brief assessment of how difficult it would be to introduce that? We
> have
> very good experience with memcached, though not with the java version.
> Given the setup you already have, memcached would typically do the
> query
> caching you would like, but than for the distributed set of servers. In
> my personal opinion it would be the finishing touch to allow Grouper to
> be used in HA scenario's

My thought was that the caching wouldn't be for query caching, but rather to 
put the whole registry in memory, and do all operations from that in memory 
registry.  So if the query hasn't been executed, or hasn't been executed 
recently, then it would still work.  I think I just need to do some tests to 
see who much memory you need based on how large the registry is.  Also, the 
"whole" registry wouldn't be there... I wouldn't need audit information, or 
change log (perhaps).

So in this scenario, we don't need a least-recently-used or expirable cache, 
since we need all data in the cache.  My understanding is memcache is a 
distributed cache, so one server needs to read the change log, and it 
replicates the cache to other servers.  Since grouper is generally heavy on 
reads and not writes (i.e. hundreds of writes per hour and not per minute ?) 
then I dont think it is such a burden to have all caches just read the change 
log, and reduce the number of services required.  Is there a feature I am 
missing?

As far as level of effort, why don't we start with what operations you want 
to support to start out... is it only getMembers and hasMember, or more?  I 
don't think it would be all that difficult, but I am working on some other 
time consuming enhancements at the moment...

Thanks,
Chris


> 
> regards,
> Niels
> 
> 
> 
> Chris Hyzer wrote:
> > Im interested to hear what other people are doing, so others please
> respond if you have a good setup.
> >
> > Here is what we do at Penn:
> >
> > 1. Linux lvs as a load balancer (two servers with heartbeat)
> > 2. This load balances 4 servers behind (this isn't dedicated for
> grouper, but each webapp gets its own tomcat process, e.g. grouper ui
> and grouper ws are in different tomcats... this is important)
> >   - note, the loadbalancing is very simple, sticky based on source ip
> address, which can fail for some isp's that rotate user's ip addresses,
> or users with roaming laptops, but generally it is ok for academia
> where we all have our own ip address :)  We cant easily change this
> since the apache on the app server does SSL, but this is a weakness.
> > 3. oracle active-standby synchronization, and the jdbc urls will
> auto-failover.  E.g. here is a url:
> >
> jdbc:oracle:thin:@(DESCRIPTION=(FAILOVER=on)(ADDRESS_LIST=(ADDRESS=(PRO
> TOCOL=TCP)(HOST=server1.whatever)(PORT=1521))(ADDRESS=(PROTOCOL=TCP)(HO
> ST=server2.whatever)(PORT=1522)))(CONNECT_DATA=(SERVICE_NAME=somename.w
> hatever)))
> >
> > Im not an expert about how this works, but my understanding is that
> if the active fails, there is a manual intervention, and some period of
> time (measured in minutes) to bring the standby up.  Then the apps
> should work again.
> >
> > Btw, WS doesn't require sticky sessions at the moment... though if
> you are doing some sort of authentication caching, it would be helpful,
> and in general the queries from one service might benefit from going to
> the same server since it might be caching as well.
> >
> > Note: we don't cluster among tomcats.  If one failed, users would
> have to logout and log back in.  Also, with LVS we had issues where the
> LVS thought an app server was down when it wasn't, or it thought all
> were down and shut down the app.  So we manually remove an app server
> from the cluster if it fails.  We monitor applications with nagios from
> the outside.
> >
> > In general we are happy with this setup.  Right now are using WS but
> not for mission critical stuff.  I think we will end up doing this, so
> I would like to find a better architecture for very high performance
> high availability web services (as we have been discussing in earlier
> threads, but I wont mention now :) ).
> >
> > Thanks!
> > Chris
> >
> >> -----Original Message-----
> >> From: Niels van Dijk 
> >> [mailto:]
> >> Sent: Thursday, June 18, 2009 10:37 AM
> >> To: 
> >> 
> >> Cc: Marieke de Wit; Hans Zandbelt
> >> Subject: [grouper-users] Grouper HA scenarios
> >>
> >> Hi All,
> >>
> >> We are currently exploring scenarios for setting up Grouper in a
> high
> >> available, failover setup.
> >>
> >> For the MySQL database, we plan on implementing a standard master-
> slave
> >> configuration using two mysql servers.
> >>
> >> For Groupers gui and webservices, the only requirement seems to be
> that
> >> sticky sessions need to be handled. Therefore several options seem
> to
> >> exist:
> >> 1) Clustering setup in Tomcat, as described here:
> >> http://tomcat.apache.org/tomcat-6.0-doc/cluster-howto.html
> >> 2) Use apache mod proxy in front of tomcat to handle the clustering,
> >> with sticky sessions enabled, like eg described here
> >> http://www.markround.com/archives/33-Apache-mod_proxy-balancing-
> with-
> >> PHP-sticky-sessions.html
> >> (of course using java sessionid, not php sessionid)
> >> 3) use a loadbalancer to handle the sticky sessions and just use two
> >> separate tomcat instances. Pound (http://www.apsis.ch/pound/) or a
> >> hardware LB could handle this.
> >>
> >> Did anyone set up grouper in such a way? What solution did you
> choose?
> >> And what are your experiences with that solution?
> >>
> >> thanks in advance,
> >>
> >> regards,
> >> Niels
> >>
> >>
> >>
> >> --
> >> Niels van Dijk
> >> Advanced Services
> >>
> >> T: +31 302 305 337 / M: +31 651 347 657
> >> SURFnet - PO Box 19035 - NL-3501 DA Utrecht - The Netherlands -
> >>
> >> http://www.surfnet.nl
> >> SURFnet grensverleggend netwerk voor hoger onderwijs en onderzoek
> 
> --
> Niels van Dijk
> Advanced Services
> 
> T: +31 302 305 337 / M: +31 651 347 657
> SURFnet - PO Box 19035 - NL-3501 DA Utrecht - The Netherlands -
> 
> http://www.surfnet.nl
> SURFnet grensverleggend netwerk voor hoger onderwijs en onderzoek