Grouper Users - Open Discussion List

[Shilen Patel <>]

Can you describe your structure a little bit more?  E.g. Do you have one
group with ~1200 subjects (non-group subjects) and that group is assigned
(or going to be assigned) read privilege to ~95,000 other groups?  And
with a subset of that currently done, you¹re seeing 34 million additional
rows in grouper_group_set?   I would only expect 95K additional rows in
grouper_group_set unless the 1200 group has non-person members.  If you
look at all the direct and indirect members of that group, how many are
people vs groups?

Thanks!

- Shilen

On 4/7/15, 12:48 PM, "Robert Bradley" 
<>
 wrote:

>On 01/04/15 18:03, Robert Bradley wrote:
>> On 01/04/15 17:25, Shilen Patel wrote:
>>> Do you need privileges to be in the change log?  If not, then set
>>> changeLog.includeFlattenedPrivileges=false in
>>>grouper-loader.properties.
>>>
>> 
>> I don't think that we need privilege changes in the change log at all
>> for the moment, so that is worth trying.  I had tried just including the
>> non-flattened privileges instead, but that seemed to make little
>> difference to the increase in changelog size.  (That surprised me, since
>> I'd expect a thousandth of the addPrivilege events I saw in that case.)
>> 
>
>Setting changeLog.includeFlattenedPrivileges=false and
>changeLog.includeNonFlattenedPrivileges=false successfully managed to
>keep the addPrivilege events out after all, so thanks.  Unfortunately,
>we then ran into size issues with our grouper_group_set and
>grouper_pit_group_set tables, which I think are less avoidable.
>
>Using the query "select count(*) as readers_count from grouper_group_set
>as gs inner join grouper_fields as f on gs.field_id=f.id where
>f.name='readers';" as a guide, we currently seem to be about 30% of the
>way through our privilege assignment.  (~1200 users * ~95000 groups =
>~114M assignments, with 34M currently in the table.)  At present,
>grouper_group_set is 18 GB and grouper_pit_group_set is 15 GB, with
>about 50 GB of indices for each table.  I would expect that these will
>grow to about 60 GB and 50 GB respectively with up to 150 GB of index
>per table.
>
>My main question here is how much space does the Grouper database
>typically occupy, and do these numbers make sense?  I can't see anything
>obvious that would suggest we're wasting space (outside of indexes
>maybe), so perhaps we simply need a lot more disk space than the 155 GB
>we have allocated to Postgres.  On the other hand, we are granting a lot
>of users read access to a lot of groups.  Perhaps we're simply using
>Grouper in an atypical manner?  As I recall, the default setup grants
>EveryEntity global READ privileges, which would reduce the overheads a
>lot.
>
>
>-- 
>Dr Robert Bradley
>Identity and Access Management, IT Services, University of Oxford
>