Grouper Users - Open Discussion List

[Shilen Patel <>]

We discussed this on the dev call and didn’t come up with any other
approaches.  So if you do try out the composite idea, please let us know
how it goes.

Note that because adding the composite group flattens out the memberships,
you’d end up having ~830 more membership records.  But that’s a lot less
than the 30+ million group sets you should be saving. :)

Thanks!

- Shilen

On 4/8/15, 8:08 AM, "Shilen Patel" 
<>
 wrote:

>Right, it’s the nested groups that are causing all the group sets.  It
>occurs to me that if the group that is being assigned the 95K read privs
>(etc:course_readers?) is made into a composite group (e.g. union or
>complement with an empty group), then that should significantly reduce the
>group sets because composite groups are essentially flattened in the
>database.  You should then only have 1 additional group set per read priv
>assignment (instead of ~352).  I’m trying to think if there would be any
>negative impact of doing this in your case, but none come to mind at the
>moment.  Of course, making the switch would be quite expensive since you
>would be undoing all the group set assignments.  I don’t know if you have
>a separate environment where you can test this approach first (if you
>decide to proceed with it)?
>
>Aside from the alternatives that you already mentioned (increasing disk
>space, reducing indexes, using EveryEntity instead), does Chris or anyone
>else have any other ideas?
>
>Thanks!
>
>- Shilen
>
>On 4/7/15, 6:28 PM, "Robert Bradley" 
><>
> wrote:
>
>>On 07/04/15 21:38, Shilen Patel wrote:
>>> Can you describe your structure a little bit more?  E.g. Do you have
>>>one
>>> group with ~1200 subjects (non-group subjects) and that group is
>>>assigned
>>> (or going to be assigned) read privilege to ~95,000 other groups?  And
>>> with a subset of that currently done, you¹re seeing 34 million
>>>additional
>>> rows in grouper_group_set?   I would only expect 95K additional rows in
>>> grouper_group_set unless the 1200 group has non-person members.  If you
>>> look at all the direct and indirect members of that group, how many are
>>> people vs groups?
>>>
>>
>>I shall try!  By the sound of it though, our very heavy use of nested
>>groups is possibly the cause.
>>
>>The Oxford overall Grouper structure at the moment is to use two main
>>stems: a "course" stem, which contains a hierarchy of course groups, and
>>an "organisation" stem, which contains the organisational structure.  In
>>practice, this latter stem is mostly split into "colleges" and
>>"university" stems.  Each department or college has its own set of IT
>>staff and administrators, which are represented by Grouper groups, and
>>these groups are then members of higher-level groups (e.g. all
>>humanities IT staff or all college IT staff).  In the general case, the
>>membership of these groups will be defined by the departments or
>>colleges in question, making manual population of the top-level groups
>>harder.  To make things extra-interesting, several users are in multiple
>>groups, managing many different departments or sub-departments.
>>
>>Thanks to all our subgroups and indirection, the 1180 or so members of
>>our "All university IT staff" group turn out to be ~350 subgroups at
>>various levels of indirection + ~830 "actual" (non g:gsa) subjects.
>>
>>The background to this particular problem is that as part of our pilot
>>rollout and testing, we'd like these IT staff group members to create
>>personalized test groups for their departments and colleges based on the
>>organizational and course groups we have added.  To avoid accidentally
>>leaking group membership data to students and other departments, our
>>users only have VIEW privileges on groups by default.  We then grant
>>READ access to organisational groups to a local "admins" group for each
>>college or department.
>>
>>For the courses stem, we decided to take a less granular approach for
>>our initial tests.  I created a single group on our test system called
>>etc:course_readers with our "All university IT staff" group as the sole
>>member for now.  This was then assigned READ access to all the course
>>stem groups via the inheritance rule described on the wiki.  I don't
>>know if that matters though, since I expect that using the Web Services
>>or gsh to assign the privileges one group at a time would have a similar
>>effect.
>>
>>-- 
>>Dr Robert Bradley
>>Identity and Access Management, IT Services, University of Oxford
>>
>