Skip to Content.
Sympa Menu

grouper-users - RE: [grouper-users] initial performance findings

Subject: Grouper Users - Open Discussion List

List archive

RE: [grouper-users] initial performance findings

Chronological Thread 
  • From: "caleb racey" <>
  • To: "GW Brown, Information Systems and Computing" <>, <>
  • Subject: RE: [grouper-users] initial performance findings
  • Date: Tue, 18 Sep 2007 19:32:52 +0100

Thanks for the feedback Gary

It should be noted that even with the large corpus of groups and the large
number of meberships that the ui is just about usable. A 10 second wait is
not ideal but is at the upper end of what users will put up with. This is a
good sign for future developments as improvements are liable to to bring this
figure down. The number of queries may be unsustainable for the database
server but we would have to deploy a pilot and see to confirm this. It is
also likely that in a real world deployment we would not import the full
mailing list membership into grouper. I tested with it because it was a
readily available store of genuine real world group data with which we use to
see the ui functionality and get an idea of performance. Our production
grouper deployment would likley have a smaller set of groups and would grow
organically rather than start with a big bang of groups and members.

I am a bit surprised that i can't find a method to directly get a subject's
groups given the subject id in the grouper api but i suspect i'm not fully
getting how the api works.

Apologies i won't make the conference call tommorow as i'm on holiday but
when i get back to the office i will look at the jira bug to see if we can
add anything useful to it.




From: GW Brown, Information Systems and Computing
Sent: Tue 9/18/2007 5:42 PM
To: caleb racey;

Subject: Re: [grouper-users] initial performance findings

--On 18 September 2007 16:58 +0100 caleb racey

> We do have some issues with performance to do with ui interactions.
> If I login as myself, a user with a membership of 39 groups (35 of them
> email lists), and click the "Hide Stem hierarchy and show your groups"
> button, then my 39 groups show relatively quickly. Analyses of the
> query log shows 597 select statements where issued on the database, this
> seems excessive but performs acceptably.
> The performance issues come when I decide to navigate the stem hierarchy
> to see my groups. If I navigate to ncl:webapps and click on the lists
> I get shown my 35 email list memberships after a wait of 10 seconds. The
> query log shows 21506 select statements where performed. This does not
> look particularly scalable to me, it would not take many concurrent
> users to saturate the database with queries.
I`m surprised there is such a big difference in the number of queries.
> Is this level of querying expected behaviour?
We are actively working on performance issues - Duke have reported similar
problems. On tomorrow`s conference call we will discuss a possible release
of Version 1.2.1.

There are 4 main issues:
1) Data is not always cached where it could be
2) Schema and privilege checks are occurring far more frequently than
3) The database indexes for membership data are not optimal
4) When browsing, the UI processes many more objects than should strictly
be necessary.

In CVS there is a tag - GROUPER_1_2_1_FREEZE which addresses 1-3 - it would
be interesting to see what timings and query numbers you get against that

There is an open JIRA issue - GRP-7 - which encapsulates 4. Work has been
started on this but it isn`t finished yet and may need a new approach.

GW Brown, Information Systems and Computing

Archive powered by MHonArc 2.6.16.

Top of Page