Grouper Developers Forum

[Shilen Patel <>]

GW Brown, Information Systems and Computing wrote:
> --On 18 October 2007 17:24 -0400 Shilen Patel 
> <>
>  wrote:
>
> > Hello,
> >
> > I've been working on GRP-48 which involves improving the group search
> > performance in the Grouper API.  I've found a few ways to make some huge
> > performance improvements, but before I get too far into code changes and
> > testing, I thought I would describe what I'm doing.  This is to 
> > primarily
> > make sure I'm not breaking any design decisions that I may not be aware
> > of.
> >
> > So first here are the performance results.  I'll use a specific search
> > example using Duke's test Grouper installation.  We have 3120 "ECON"
> > courses somewhere within the stem duke:siss:courses.  Note that these
> > results do not use the Grouper UI.
> >
> >
> > A search for ECON at the duke stem using a non-GrouperSystem session
> > currently takes 134 seconds.  With code changes - 22 seconds. A search
> > for ECON at the duke stem using a GrouperSystem session currently takes
> > 109 seconds.  With code changes - 6 seconds.
> > A search for ECON at the root stem using a non-GrouperSystem session
> > currently takes 63 seconds.  With code changes - 22 seconds.
> > A search for ECON at the root stem using a GrouperSystem session
> > currently takes 39 seconds.  With code changes - 6 seconds.
> >
> >
> > After the modifications, in the cases where a non-GrouperSystem session
> > is created, about 75 percent of the time is actually spent on privilege
> > checking.  I haven't yet looked for performance improvements in this
> > area.  I've also noticed that the Grouper UI also does some privilege
> > checking during group searches, but I don't understand why.  Shouldn't
> > this already be taken care of in the API?  Gary can you comment on this?
> The API checks that the user has VIEW privilege. Depending on the 
> browse mode you are in when you search I need to do further checks - 
> ADMIN or UPDATE for Manage groups, OPTIN for Join groups. In principle 
> we could extend the API interfaces to pass in the privileges so the 
> API can do all the checks - this is effectively the approach we were 
> trying for GRP-7.

Since the API already does the VIEW privilege check, can that be taken 
out of the UI?

> I wonder whether the privilege checks should be done on the final 
> resultset after all the search filters have been resolved. Each search 
> filter may return a lot of results, but ANDed searches may cause many 
> to be discarded.

That's a great suggestion given how expensive the privilege checks are.  
I'll look into that.

> > So I've made 3 primary modifications to get the performance results
> > described above.
> >
> > 1.  Using ehcache and adding a new cache type in grouper.ehcache.xml,
> > I've adding caching to Member objects in GrouperAccessAdapter.  2.  The
> > next modification is related to scoping the results.  To determine if a
> > group or a stem (X) is a child of another stem (Y), the API currently
> > does some recursive checks up the hierarchy of X to see if Y is found.
> > Instead I made a modification to just check the object names.  If the
> > name of X starts with the name of Y, then X is a child of Y.
> Could this be worked into the query itself rather than iterating 
> through the results and doing comparisons?

Yes, it should be possible, however it would require changes to the API, 
which I'm not sure should be done in a minor release.  Either way, I 
wouldn't expect much difference in performance.  Thoughts?


Thanks,

-- Shilen