Subject: Grouper Developers Forum
- From: Shilen Patel <>
- To: "GW Brown, Information Systems and Computing" <>
- Cc: Grouper Dev <>
- Subject: Re: [grouper-dev] Performance of Group Searches
- Date: Fri, 19 Oct 2007 09:22:05 -0400
GW Brown, Information Systems and Computing wrote:
--On 18 October 2007 17:24 -0400 Shilen Patel
Hello,The API checks that the user has VIEW privilege. Depending on the browse mode you are in when you search I need to do further checks - ADMIN or UPDATE for Manage groups, OPTIN for Join groups. In principle we could extend the API interfaces to pass in the privileges so the API can do all the checks - this is effectively the approach we were trying for GRP-7.
I've been working on GRP-48 which involves improving the group search
performance in the Grouper API. I've found a few ways to make some huge
performance improvements, but before I get too far into code changes and
testing, I thought I would describe what I'm doing. This is to primarily
make sure I'm not breaking any design decisions that I may not be aware
So first here are the performance results. I'll use a specific search
example using Duke's test Grouper installation. We have 3120 "ECON"
courses somewhere within the stem duke:siss:courses. Note that these
results do not use the Grouper UI.
A search for ECON at the duke stem using a non-GrouperSystem session
currently takes 134 seconds. With code changes - 22 seconds. A search
for ECON at the duke stem using a GrouperSystem session currently takes
109 seconds. With code changes - 6 seconds.
A search for ECON at the root stem using a non-GrouperSystem session
currently takes 63 seconds. With code changes - 22 seconds.
A search for ECON at the root stem using a GrouperSystem session
currently takes 39 seconds. With code changes - 6 seconds.
After the modifications, in the cases where a non-GrouperSystem session
is created, about 75 percent of the time is actually spent on privilege
checking. I haven't yet looked for performance improvements in this
area. I've also noticed that the Grouper UI also does some privilege
checking during group searches, but I don't understand why. Shouldn't
this already be taken care of in the API? Gary can you comment on this?
Since the API already does the VIEW privilege check, can that be taken out of the UI?
I wonder whether the privilege checks should be done on the final resultset after all the search filters have been resolved. Each search filter may return a lot of results, but ANDed searches may cause many to be discarded.
That's a great suggestion given how expensive the privilege checks are. I'll look into that.
Could this be worked into the query itself rather than iterating through the results and doing comparisons?
So I've made 3 primary modifications to get the performance results
1. Using ehcache and adding a new cache type in grouper.ehcache.xml,
I've adding caching to Member objects in GrouperAccessAdapter. 2. The
next modification is related to scoping the results. To determine if a
group or a stem (X) is a child of another stem (Y), the API currently
does some recursive checks up the hierarchy of X to see if Y is found.
Instead I made a modification to just check the object names. If the
name of X starts with the name of Y, then X is a child of Y.
Yes, it should be possible, however it would require changes to the API, which I'm not sure should be done in a minor release. Either way, I wouldn't expect much difference in performance. Thoughts?
- Performance of Group Searches, Shilen Patel, 10/18/2007
- Re: [grouper-dev] Performance of Group Searches, Tom Barton, 10/18/2007
- Re: [grouper-dev] Performance of Group Searches, GW Brown, Information Systems and Computing, 10/19/2007
Archive powered by MHonArc 2.6.16.