Skip to Content.
Sympa Menu

grouper-dev - Re: [grouper-dev] Update: Subject API Slowness

Subject: Grouper Developers Forum

List archive

Re: [grouper-dev] Update: Subject API Slowness

Chronological Thread 
  • From: "GW Brown, Information Systems and Computing" <>
  • To: Mark Weber <>,
  • Subject: Re: [grouper-dev] Update: Subject API Slowness
  • Date: Wed, 04 Oct 2006 09:02:01 +0100

Hi Mark,

Some responses below.


--On 03 October 2006 16:19 -0500 Mark Weber <> wrote:

Issue #1 (UI related)
One inefficiency/bug I want to point out with the UI and how it paginates
result sets. If I am displaying the members of a group with a given
privilege and there are 1,000 members in that group the UI displays
10/25/50/100 results at a time.
For info, you can control the default and available paging sizes in
pager.pagesize.selection=10 25 50 100

However, for each new page request it
re-requests the entire result set.
Early on I raised paging as something the Subject and Grouper APIs should offer. At the time there were more important fish to fry and it was decided to put off any work in this area until we had more experience with how the APIs were working. At the time there was also talk about configuring limits on how many results could be returned from a Subject API search, though that would not work for Group memberships.

I think this is incorrect because new members may show up in between page
requests and cause erroneous or confusing results between page requests.
(i.e. they may see the same person on page 1 as they do on page 2
depending on the quantity of changes between page requests)
This type of issue is well understood in the database world (see <>). I'm not sure we want to address this at the database level. How often it would be an issue is hard to tell. In principle, a group has a lastModified flag, so the UI could warn a user if a group has changed. It would then be up to the user how they reacted - carry on paging, or start over
It's inefficient because in the case of 1000 members, at 50 results per
page, it produces about 20,000 queries before I reach the last page.
That does seem a lot. The UI generally tries to only make the required subjects available to the JSP layer - so it does not 'require' that the API instantiates all subjects. We should definitely look at optimising this area.

If it's going to capture the entire result set it ought to store it, sort
it and paginate it. And it should also allow for the user to "refresh"
the results (thus implying the results are point-in-time)...IMHO.
I'm not sure this is clear cut, because whether instantiating and 'storing' large numbers of objects 'pays for itself' may depend on how often you use the resulting cache. How often will some one page a long membership list or set of search results and for what reason - i.e. are there better ways of achieving the task at hand e.g. if you want to see if a person is a member of a group you can see which groups an individual is a member of from the Subject Summary page. If you want to print a list of members, do a mail merge etc, it would make sense to export the entire membership in Excel (or similar) format.

Issue #2 (Back-end)
My trials led me to Hibernate's front door and our DBA is out this week
so we're having trouble getting some hard numbers. But something looks
askew. I thought it might be something with Oracle or something 10g
specific?? I analyzed the tables tried adding different indexes etc...
nothing made a dent. Next week we will be able to see specific activity
for a given connection alongside the SQL it's attempting to execute.

So, in the meantime, I decided to switch out back-ends and use MySQL on
localhost. Page loads for displaying privilegee's (50 members) went from
124 seconds down to 23 seconds. Better, but not yet acceptable. The fact
that it's on localhost is probably the reason for the speed increase.

I'm going to try creating different groups with different permissions to
see if I can spot something obvious. Maybe it's just having trouble
displaying a group where every member has every privilege??

.Mark G. Weber----------------------Division of Information Technology.

University Of Wisconsin - Madison|
| 608/263.8945 Room 3159 Computer Science|

GW Brown, Information Systems and Computing

Archive powered by MHonArc 2.6.16.

Top of Page