Grouper Users - Open Discussion List

["caleb racey" <>]

Thanks for the feedback  Gary
 
It should be noted that even with the large corpus of groups and the large 
number of  meberships that the ui is just about usable.  A 10 second wait is 
not ideal but is at the upper end of what users will put up with. This is a 
good sign for future developments as improvements are liable to to bring this 
figure down. The number of queries may be unsustainable for the database 
server but we would have to deploy a pilot and see to confirm this. It is 
also likely that in a real world deployment we would not import the full 
mailing list membership into grouper. I tested with it because it was a 
readily available store of genuine real world group data with which we use to 
see the ui functionality and get an idea of performance.  Our production 
grouper deployment would likley have a smaller set of groups and would grow 
organically  rather than start with a big bang of groups and members.  
 
I am a bit surprised that i can't find a method to directly get a subject's 
groups given the subject id in the grouper api but i suspect i'm not fully 
getting how the api works.
 
Apologies i won't make the conference call tommorow as i'm on holiday but 
when i get back to the office i will look at the jira bug to see if we can 
add anything useful to it. 
 
cheers
 
Cal 
 

________________________________

From: GW Brown, Information Systems and Computing 
[mailto:]
Sent: Tue 9/18/2007 5:42 PM
To: caleb racey; 

Subject: Re: [grouper-users] initial performance findings





--On 18 September 2007 16:58 +0100 caleb racey
<>
 wrote:

> We do have some issues with performance to do with ui interactions.
>
> If I login as myself, a user with a membership of 39 groups (35 of them
> email lists), and click  the "Hide Stem hierarchy and show your groups"
> button, then  my 39 groups show relatively quickly. Analyses of the
> query log shows 597 select statements where issued on the database, this
> seems excessive but performs acceptably.
>
> The performance issues come when I decide to navigate the stem hierarchy
> to see my groups. If I navigate to   ncl:webapps and click on the lists
> I get shown my 35 email list memberships after a wait of 10 seconds. The
> query log shows  21506 select statements where performed. This does not
> look particularly scalable to me, it would not take many concurrent
> users to saturate the database with queries.
I`m surprised there is such a big difference in the number of queries.
>
> Is this level of querying expected behaviour?
>
We are actively working on performance issues - Duke have reported similar
problems. On tomorrow`s conference call we will discuss a possible release
of Version 1.2.1.

There are 4 main issues:
1) Data is not always cached where it could be
2) Schema and privilege checks are occurring far more frequently than
necessary
3) The database indexes for membership data are not optimal
4) When browsing, the UI processes many more objects than should strictly
be necessary.

In CVS there is a tag - GROUPER_1_2_1_FREEZE which addresses 1-3 - it would
be interesting to see what timings and query numbers you get against that
version.

There is an open JIRA issue - GRP-7 - which encapsulates 4. Work has been
started on this but it isn`t finished yet and may need a new approach.


----------------------
GW Brown, Information Systems and Computing