Grouper Users - Open Discussion List

[Robert Bradley <>]

On 01/04/15 18:03, Robert Bradley wrote:
> On 01/04/15 17:25, Shilen Patel wrote:
>> Do you need privileges to be in the change log?  If not, then set
>> changeLog.includeFlattenedPrivileges=false in grouper-loader.properties.
>>
> 
> I don't think that we need privilege changes in the change log at all
> for the moment, so that is worth trying.  I had tried just including the
> non-flattened privileges instead, but that seemed to make little
> difference to the increase in changelog size.  (That surprised me, since
> I'd expect a thousandth of the addPrivilege events I saw in that case.)
> 

Setting changeLog.includeFlattenedPrivileges=false and
changeLog.includeNonFlattenedPrivileges=false successfully managed to
keep the addPrivilege events out after all, so thanks.  Unfortunately,
we then ran into size issues with our grouper_group_set and
grouper_pit_group_set tables, which I think are less avoidable.

Using the query "select count(*) as readers_count from grouper_group_set
as gs inner join grouper_fields as f on gs.field_id=f.id where
f.name='readers';" as a guide, we currently seem to be about 30% of the
way through our privilege assignment.  (~1200 users * ~95000 groups =
~114M assignments, with 34M currently in the table.)  At present,
grouper_group_set is 18 GB and grouper_pit_group_set is 15 GB, with
about 50 GB of indices for each table.  I would expect that these will
grow to about 60 GB and 50 GB respectively with up to 150 GB of index
per table.

My main question here is how much space does the Grouper database
typically occupy, and do these numbers make sense?  I can't see anything
obvious that would suggest we're wasting space (outside of indexes
maybe), so perhaps we simply need a lot more disk space than the 155 GB
we have allocated to Postgres.  On the other hand, we are granting a lot
of users read access to a lot of groups.  Perhaps we're simply using
Grouper in an atypical manner?  As I recall, the default setup grants
EveryEntity global READ privileges, which would reduce the overheads a lot.


-- 
Dr Robert Bradley
Identity and Access Management, IT Services, University of Oxford

Attachment: signature.asc
Description: OpenPGP digital signature