Grouper Users - Open Discussion List

[Chris Hyzer <>]

I think another common use case is finding a subject in a group.  i.e. we 
have a subject picker that wants to do a subject search (in one source) where 
the subject is an employee.  This takes a long time since it has to do the 
search of 500k subjects, then for the results, see which ones are employees 
(which is batched, but still takes a while).  Since I am a jdbc source guy, I 
would agree that adding more capabilities into the subject source would be 
nice (could solve this problem).  However, when searching/sorting over 
multiple sources, it seems more difficult even if the all the sources are 
jdbc ones... so that is why keeping a sort string in the member table (for 
all subjects in all sources which are members of groups) seems like a good 
thing to do.  If we kept a sort string in the member table, then we could 
also keep a lowered search string, then we only need to do that instead of 
give the JDBC sources extra features over JNDI.  And if we do that, we might 
as well keep a name/description so if they become unresolvable, we can still 
show who they are (were).  Though I don't see keeping other attributes in 
there (full replication) :)

Thanks,
Chris

-----Original Message-----
From: Jim Fox 
[mailto:]
 
Sent: Sunday, October 03, 2010 12:20 PM
To: GW Brown, Information Systems and Computing
Cc: Chris Hyzer; Peter DiCamillo; 

Subject: Re: [grouper-users] Grouper 1.6.1 UI membership list sorting



It is for this reason more than any other that we cache all our subjects in 
tables in grouper's database.  We can use simple database joins to allow the 
DBMS do all the selecting and sorting, by subject id or name or whatever.  

It seems obvious enough that caching a subject sort string is not much easier 
than caching the entire subject record.  Possibly version 2.0 could require, 
or at least optimize for, local caches of subjects.   

Jim

On Oct 3, 2010, at 3:49 AM, GW Brown, Information Systems and Computing wrote:

> The problem is that the API is now doing the paging - but it has no means 
> to sort the data because, as Chris says, we don't store subject attributes. 
> API paging was added because even without sorting, processing very large 
> memberships would bring the UI to a grinding halt. So, unfortunately the 
> comparator.sort.limit has become redundant.
> 
> Going forward I think there are a coupe of options.
> 
> 1) The UI checks how many results there will be and if under 
> comparator.sort.limit sets this as the pagesize for the API.
> 2) The API stores a sort String for subjects so it can add an order by 
> clause to its query. Some mechanism would be needed to refresh sort Strings.
> 
> I can have a look at option 1, but we have talked about option 2 before so 
> I'll wait and see what the consensus is. Either way it is likely to be 2.0 
> before we resolve it.
> 
> Gary
> 
> --On 03 October 2010 01:59 -0400 Chris Hyzer 
> <>
>  wrote:
> 
>> I can reproduce that.  The problem is that subject data is stored
>> externally to Grouper (in sources), so if there are 50000 subjects in a
>> group, that is 50000 subject lookups (assuming none in cache), which
>> doesn't scale.  Granted if you had 356 with 250 on a page, that should
>> sort.  :)   What use case do you have?  Trying to see if someone is in a
>> group?  Just curious, what is the workaround?  Anyways, Im sure Gary will
>> weigh in here as well...
>> 
>> Thanks,
>> Chris
>> 
>> -----Original Message-----
>> From: 
>> 
>> [mailto:]
>>  On Behalf Of Peter DiCamillo
>> Sent: Saturday, October 02, 2010 11:14 PM
>> To: 
>> 
>> Subject: [grouper-users] Grouper 1.6.1 UI membership list sorting
>> 
>> I'm having a problem where membership lists I view in the full Grouper
>> UI are not sorted as I would expect.  When the size of the membership
>> list does not exceed the page size the list is sorted.  But when more
>> than one page is needed, the list on each page is not sorted.  The value
>> of comparator.sort.limit in media.properties is 50000, larger than the
>> size of any of our groups, because even if it's slow, we always want
>> sorted lists.
>> 
>> For example, I viewed a list with 2 direct members and 356 indirect
>> members.  When I viewed the indirect members or all members with a page
>> size of 500 the list was sorted.  But with a page size of 250 the lists
>> on the pages were not sorted.
>> 
>> I didn't install Grouper 1.3, but we may have had a local fix for this.
>>  But in any case, it appears that since 1.3 the code that for handling
>> pages and sorting has changed
>> 
>> Peter
>> 
> 
> 
> 
> ----------------------
> GW Brown, Information Systems and Computing
>