Grouper Developers Forum

["GW Brown, Information Systems and Computing" <>]

Hi Chris,

Comments below.

Gary

--On 24 February 2009 23:22 -0500 Chris Hyzer 
<>
 wrote:

> Hey,
>
>
>
> I know I have brought this up before and got shot down, but I have a
> little different spin on it this time, so hear me out.
>
>
>
> I got a request to add a user to be a reader on a 60k member group.  So I
> go to the UI, pull up the group, pull up the readers, and click the
> button to assign to more users.  Then I wait, and wait, and wait, then I
> get an error.
>
>
>
> https://bugs.internet2.edu/jira/browse/GRP-234
>
>
>
> Apparently it will look at members (60k), and show the first 50 (or so)
> so you can pick the readers from the members.  This is not an easy task,
> since to get the first fifty based on a subject attribute, you need to go
> get all 60k, then put all the subject attributes together, sort, and chop
> off the list.
Browsing 60K members in the UI is never likely to be satisfactory. Do you 
have a sort limit set - comparator.sort.limit - in media.properties?

If there were an immediate membership count on the group object updated 
when  memberships are changed, the UI and other clients could adapt their 
behaviour and try not to do 'silly' things like list 60K of members.

> The particular error here was because some of the subjects
> were unresolvable, and it wasn't handled correctly (I think it
> shouldn't fail if it has unresolvable subjects).
Yes we should fix that - either exclude it from the results or have 
attribute requests return 'unresolvable'
> So yes, I need to
> delete my unresolvable memberships, and I can also assign privs to
> grouper with unresolvable members via GSH…
>
>
>
> But I got to thinking, what if there were a periodic job which did the
> unresolvable report (the daily report can already do this), and while
> doing that job, when it was resolving subjects, it just stashed a local
> copy of the attributes.
Sounds like the sort of thing Signet does. Could get quite big. What about 
having an extra Subject method - getSortString? The value would be stashed 
in the Member table - along with the time it was last verified (then you 
can configure the value to be verified and updated if the time of retrieval 
is more than, say, an hour since the last check)
> Then, under normal operations, it would go to
> the source for the attributes.  But for things like queries where sorting
> and paging are needed
We don't currently sort in the query but if we stored the sort string we 
could do that for Membership lists rather than have the UI do it. Add some 
paging methods and the ability to filter by Source and we could do the 
current functionality without ever having to retrieve more than a page of 
results
> (or optionally if the source is unavailable, or if
> the subject is unresolvable?) grouper could use the local information.
> So things could get a little out of sync… if someone changes their name
> in the source system, grouper will sort and page based on the old data,
> but the new data will appear on the screen.  Maybe grouper could also
> update the info when a subject is looked up and the data is different or
> something.
If you had a sort string that would likely be informative.
> Grouper could also optionally index this stuff in a search engine so
> subject queries based on substring of description (although not real
> time) would be superfast.   http://www.hibernate.org/410.html  This way
> the source would really only need to be optimized for ID lookups, not
> text searches…
I like the idea of indexing grouper attributes this way and adding to the 
current query filters. Not sure about Subjects. My take would be that 
Sources ought to be able to do the searching, sorting and paging you need. 
We can extend the Subject API to take advantage of such features, or 
sources can emulate them, but we shouldn't have Grouper specific stuff.
> Anyways, just a thought…
>
>
>
> Thanks!
>
> Chris



----------------------
GW Brown, Information Systems and Computing