Skip to Content.
Sympa Menu

grouper-dev - Re: [grouper-dev] Performance of Group Searches

Subject: Grouper Developers Forum

List archive

Re: [grouper-dev] Performance of Group Searches


Chronological Thread 
  • From: Tom Barton <>
  • To: Shilen Patel <>
  • Cc: Grouper Dev <>
  • Subject: Re: [grouper-dev] Performance of Group Searches
  • Date: Thu, 18 Oct 2007 22:26:13 -0500

Excellent. You are most definitely not insane. :-)

Shilen Patel wrote:
1. Using ehcache and adding a new cache type in grouper.ehcache.xml, I've adding caching to Member objects in GrouperAccessAdapter.

Given #3 below, how does this actually net an improvement? Inquiring minds want to know ...

2. The next modification is related to scoping the results. To determine if a group or a stem (X) is a child of another stem (Y), the API currently does some recursive checks up the hierarchy of X to see if Y is found. Instead I made a modification to just check the object names. If the name of X starts with the name of Y, then X is a child of Y.

If this makes a substantial difference in performance, it's probably worth it to bake this assumption into code and forever link names to locations. If not, it might be best to continue to evaluate this relationship structurally. Would this constrain our options when we get around to implementing support for moving and copying groups & stem hierarchies around?

3. To do the actual database search for the groups, the API currently first gets a list of all group attribute ids by doing 1 query. For the ECON example above, that would result in a list of 3120 group attribute ids. Next, the API performs 3120 queries to retrieve all of the group attribute data. Then there will be another 3120 queries to get the group data. So that's 6241 queries. Furthermore, say sometime in the future you want to call group.getName() on all of the 3120 groups, that will result in 15,000 more queries. Anyways, so I reduced all that down to 1 query that takes about 5 seconds. I've set the group attributes as a property of the group so that additional queries to get group attributes are not needed. I did not use ehcache for this, although that might be something to think about. Any thoughts on whether there will be problems if group attributes are queried and saved ahead of time like this?

When you say attributes are made a property of the group, are you referring to a hibernate construct? Is that the same for hibernate 2 & 3? Would a caching approach loose effectiveness as the number of search hits exceeds cache size?

I can't think of a downside to pre-fetching group attributes.

Tom



Archive powered by MHonArc 2.6.16.

Top of Page