Grouper Users - Open Discussion List

[Chris Hyzer <>]

You could try Jim Fox's vt-ldap source which has better performance.  I think 
the link for it is here:

http://staff.washington.edu/fox/grouper/dist/

How are you loading your group into Grouper?  Grouper loader, WS, GSH, API?
15 minutes for 30k members is 30 members/sec which is pretty good I think.

I cant really comment on ldappcng performance.

For loading into grouper though, it has to do a bunch of work, so even if it 
does a subject query for each row, the order of magnitude of the operation 
overall will still be the same if you remove the subject query.  I would 
assume it would be similar for ldappcng as well.  However, you will be better 
off if you can load by specifying the subjectId and sourceId so it goes to 
one source with one query to resolve.  If you only specify a netId, and call 
idOrIdentifier, and not sourceId, that will query each source (~4?), with two 
queries (8 total).  The loader is pretty efficient about this.

Generally Grouper is designed assuming there is not a lot of membership 
churn, right?  I.e. you will have your 30k students in a group, and each day, 
you might add or remove a couple hundred, and then a few days a year, you 
have +- 5k.  Those loads will take a while as you see for the few days where 
the new students are entered.  

Do you think these performance numbers will be a problem for you?

Btw, I don't think it will change your performance that much, but I think 
that a JDBC source has opportunities in the future to have some performance 
improvements and feature improvements since it could bulk load subjects and 
page/sort better, but Im not an ldap person, so maybe we could do a similar 
thing there too.  At Penn we just the jdbc2 source...

Thanks,
Chris

-----Original Message-----
From: 

 
[mailto:]
 On Behalf Of Lynn Garrison
Sent: Friday, March 11, 2011 11:02 AM
To: 

Subject: [grouper-users] ldap source vs jdbc source

        Our test environment at Penn State is configure with an ldap source; 
oracle for the gsa.  We recently executed a test load  with our largest group 
- ~32k faculty/staff members.  We loaded all the members to grouper and then 
provisioned them to lpad using ldappcng.  We executed the test several times.

        The load into grouper executed in 15 to 18 minutes.  
        The provision to ldap with ldappcng executed in 45 to 50 minutes.  

        We are using the source api and ldappcng version 1.6.3.
Questions:

        1.  Are these reasonable times?   
        2.  Would we see an improvement in the ldappcng execution time if we 
were using a jdbc source?  
 

        We are looking at replacing the current mechanism for managing groups 
- 66000+ groups, ~32k members in the largest group.  One of the requirements 
is that all groups be provision to ldap and available for use as soon as they 
are created.