Subject: Grouper Developers Forum
- From: Chris Hyzer <>
- To: Mark Weber <>
- Subject: Re: [grouper-dev] Update: Subject API Slowness
- Date: Tue, 03 Oct 2006 17:39:43 -0400
Im not experienced with Grouper (yet), but I have Hibernate tuning experience so will chime in...
We use hibernate's paging where if there are 1000 records and you display 50 on a page, it does one query to get the 50 rows (and only 50 rows in the resultset). If there are joins to other tables we handle that with "eager fetching" where for 50 rows and a one to many with another table it will be a handful more queries (NOT 50 more queries, this is the N+1 problem). Each page view should only require a handful of queries no matter how large the resultset is (unless a very complex report, then a lot of tuning is involved and maybe bypassing hibernate for jdbc (which is db independent) or using hibernate projections). But unlike what Mark described, there is no storing/refreshing of the resultset. If records are inserted as you page through, you will see those records.
If you are using a custom tag that does not support this type of paging, then you can pick another one or roll your own (what we did).
Also, its hard to tune these types of apps unless you are auditing on the DB side, so if you have a param which can dump out p6spy logs or something, which show the query and how long it took, and how many results in the resultset, that can help people out here... in grouper if you can specify what your DB driver is and give the jar, then any user can hook up p6spy to get this logging (again, not experienced enough with grouper to know for sure)
Mark Weber wrote:
Issue #1 (UI related)
One inefficiency/bug I want to point out with the UI and how it paginates result sets. If I am displaying the members of a group with a given privilege and there are 1,000 members in that group the UI displays 10/25/50/100 results at a time. However, for each new page request it re-requests the entire result set.
I think this is incorrect because new members may show up in between page requests and cause erroneous or confusing results between page requests. (i.e. they may see the same person on page 1 as they do on page 2 depending on the quantity of changes between page requests)
It's inefficient because in the case of 1000 members, at 50 results per page, it produces about 20,000 queries before I reach the last page.
If it's going to capture the entire result set it ought to store it, sort it and paginate it. And it should also allow for the user to "refresh" the results (thus implying the results are point-in-time)...IMHO.
Issue #2 (Back-end)
My trials led me to Hibernate's front door and our DBA is out this week so we're having trouble getting some hard numbers. But something looks askew. I thought it might be something with Oracle or something 10g specific?? I analyzed the tables tried adding different indexes etc... nothing made a dent. Next week we will be able to see specific activity for a given connection alongside the SQL it's attempting to execute.
So, in the meantime, I decided to switch out back-ends and use MySQL on localhost. Page loads for displaying privilegee's (50 members) went from 124 seconds down to 23 seconds. Better, but not yet acceptable. The fact that it's on localhost is probably the reason for the speed increase.
I'm going to try creating different groups with different permissions to see if I can spot something obvious. Maybe it's just having trouble displaying a group where every member has every privilege??
- Update: Subject API Slowness, Mark Weber, 10/03/2006
- Re: [grouper-dev] Update: Subject API Slowness, Chris Hyzer, 10/03/2006
- Re: [grouper-dev] Update: Subject API Slowness, Chad La Joie, 10/03/2006
- Re: [grouper-dev] Update: Subject API Slowness, GW Brown, Information Systems and Computing, 10/04/2006
Archive powered by MHonArc 2.6.16.