Grouper Users - Open Discussion List

[Shilen Patel <>]

Hello,

Our DBAs have added an index that significantly improves performance for 
group and stem deletes in our environment.  However, the stem deletes 
are still much slower than what we saw in Grouper 1.1.  We added an 
index on the grouper_memberships table for the columns owner_id, 
list_name, list_type, and mship_type. 

The grouper_event.log file now shows the following:

Add stem:

2007-06-18 12:05:43,942: 
[d60a6558-0a2e-4b5e-a3d5-f826537d91a7,'GrouperSystem','application'] add 
stem: 'duke:test:stem02' (30ms)

Delete stem:

2007-06-18 12:06:05,837: 
[db54619a-ac2e-4a90-994c-174d0a2e778c,'GrouperSystem','application'] 
revoke naming priv: stem='duke:test:stem02' priv='create' (15ms)
2007-06-18 12:06:05,879: 
[db54619a-ac2e-4a90-994c-174d0a2e778c,'GrouperSystem','application'] 
revoke naming priv: stem='duke:test:stem02' priv='stem' (42ms)
2007-06-18 12:06:05,888: 
[d60a6558-0a2e-4b5e-a3d5-f826537d91a7,'GrouperSystem','application'] 
delete stem: 'duke:test:stem02' (8207ms)


Add group:

2007-06-18 12:06:59,165: 
[d60a6558-0a2e-4b5e-a3d5-f826537d91a7,'GrouperSystem','application'] add 
group: 'duke:test:group02' (173ms)
2007-06-18 12:06:59,324: 
[d60a6558-0a2e-4b5e-a3d5-f826537d91a7,'GrouperSystem','application'] 
grant access priv: group='duke:test:group02' priv='read' 
subject='GrouperAll'/'application'/'g:isa' (38ms)
2007-06-18 12:06:59,379: 
[d60a6558-0a2e-4b5e-a3d5-f826537d91a7,'GrouperSystem','application'] 
grant access priv: group='duke:test:group02' priv='view' 
subject='GrouperAll'/'application'/'g:isa' (55ms)

Delete group:

2007-06-18 12:07:02,656: 
[db54619a-ac2e-4a90-994c-174d0a2e778c,'GrouperSystem','application'] 
revoke access priv: group='duke:test:group02' priv='admin' (43ms)
2007-06-18 12:07:02,677: 
[db54619a-ac2e-4a90-994c-174d0a2e778c,'GrouperSystem','application'] 
revoke access priv: group='duke:test:group02' priv='optin' (21ms)
2007-06-18 12:07:02,699: 
[db54619a-ac2e-4a90-994c-174d0a2e778c,'GrouperSystem','application'] 
revoke access priv: group='duke:test:group02' priv='optout' (21ms)
2007-06-18 12:07:02,746: 
[db54619a-ac2e-4a90-994c-174d0a2e778c,'GrouperSystem','application'] 
revoke access priv: group='duke:test:group02' priv='read' (47ms)
2007-06-18 12:07:02,767: 
[db54619a-ac2e-4a90-994c-174d0a2e778c,'GrouperSystem','application'] 
revoke access priv: group='duke:test:group02' priv='update' (21ms)
2007-06-18 12:07:02,823: 
[db54619a-ac2e-4a90-994c-174d0a2e778c,'GrouperSystem','application'] 
revoke access priv: group='duke:test:group02' priv='view' (55ms)
2007-06-18 12:07:02,862: 
[d60a6558-0a2e-4b5e-a3d5-f826537d91a7,'GrouperSystem','application'] 
delete group: 'duke:test:group02' (250ms)


We'll continue to look for further improvements.

Thanks,

-- Shilen



Shilen Patel wrote:
> Hello,
>
> I've imported our production Grouper 1.1 data into a test instance 
> with 1.2rc3.  I've noticed improvements while doing adds, but the 
> performance for deletes is very poor.
>
> Our environment has 100,000 entries in the grouper_groups table, 1 
> million entries in the grouper_memberships table, and 90,000 entries 
> in the grouper_stems table.  We have an admin group that has admin 
> privileges to about 30,000 groups.  There are 2 members of the admin 
> group.  GrouperAll also has read and view privileges to these groups.
>
>
> The following are some examples of the performance we're seeing from 
> the grouper_event.log file.  Notice the time it takes to revoke 
> privileges.
>
> Add stem:
>
> 2007-06-15 12:02:33,312: 
> [764fe553-0e6e-45cc-b7b5-05636a15594a,'GrouperSystem','application'] 
> add stem: 'duke:test:stem02' (54ms)
>
> Delete stem:
>
> 2007-06-15 12:03:23,068: 
> [ef91783d-20e6-4602-b520-c7fe2ca2ddb4,'GrouperSystem','application'] 
> revoke naming priv: stem='duke:test:stem02' priv='create' (36ms)
> 2007-06-15 12:03:39,033: 
> [ef91783d-20e6-4602-b520-c7fe2ca2ddb4,'GrouperSystem','application'] 
> revoke naming priv: stem='duke:test:stem02' priv='stem' (15965ms)
> 2007-06-15 12:03:39,066: 
> [764fe553-0e6e-45cc-b7b5-05636a15594a,'GrouperSystem','application'] 
> delete stem: 'duke:test:stem02' (25787ms)
>
>
> Add group:
>
> 2007-06-15 12:06:16,310: 
> [764fe553-0e6e-45cc-b7b5-05636a15594a,'GrouperSystem','application'] 
> add group: 'duke:test:group02' (143ms)
>
> Delete group:
>
> 2007-06-15 12:08:35,443: 
> [ef91783d-20e6-4602-b520-c7fe2ca2ddb4,'GrouperSystem','application'] 
> revoke access priv: group='duke:test:group02' priv='admin' (27875ms)
> 2007-06-15 12:08:35,476: 
> [ef91783d-20e6-4602-b520-c7fe2ca2ddb4,'GrouperSystem','application'] 
> revoke access priv: group='duke:test:group02' priv='optin' (33ms)
> 2007-06-15 12:08:35,501: 
> [ef91783d-20e6-4602-b520-c7fe2ca2ddb4,'GrouperSystem','application'] 
> revoke access priv: group='duke:test:group02' priv='optout' (25ms)
> 2007-06-15 12:08:54,764: 
> [ef91783d-20e6-4602-b520-c7fe2ca2ddb4,'GrouperSystem','application'] 
> revoke access priv: group='duke:test:group02' priv='read' (19263ms)
> 2007-06-15 12:08:54,814: 
> [ef91783d-20e6-4602-b520-c7fe2ca2ddb4,'GrouperSystem','application'] 
> revoke access priv: group='duke:test:group02' priv='update' (50ms)
> 2007-06-15 12:09:10,469: 
> [ef91783d-20e6-4602-b520-c7fe2ca2ddb4,'GrouperSystem','application'] 
> revoke access priv: group='duke:test:group02' priv='view' (15655ms)
> 2007-06-15 12:09:46,399: 
> [764fe553-0e6e-45cc-b7b5-05636a15594a,'GrouperSystem','application'] 
> delete group: 'duke:test:group02' (98832ms)
>
>
>
> Any ideas on what the problem might be?
>
> Thanks,
>
> -- Shilen