Grouper Users - Open Discussion List

[David Langenberg <>]

On Mar 21, 2013, at 9:38 AM, Tim Darby <>

 wrote:

I have one large group of 314,000 people and no group members and it takes over 2 hours to sync it. I'll describe what I see happening during bulksync of this group:

Grouper 2.1.3, LDAP is both the subject source and the target.

1. PSP performs 314,000 LDAP searches like this (uaId is our subject identifier):

access.20130321-053828:[21/Mar/2013:05:38:26 -0700] conn=49972860 op=117424 SRCH base="ou=people,dc=eds,dc=arizona,dc=edu" scope=2 filter="(&(uaId=104197441616)(objectClass=person))" attrs="cn uaId cn

2

. Then PSP performs the same 314,000 LDAP searches again:

access.20130321-053828:[21/Mar/2013:05:56:22 -0700] conn=49972861 op=218402 SRCH base="ou=people,dc=eds,dc=arizona,dc=edu" scope=2 filter="(&(uaId=104197441616)(objectClass=person))" attrs="cn uaId cn

Steps 1 and 2 took a total of 36 minutes.  First question is, why does PSP search all the subjects twice?

Because in your setup LDAP is both the source and the target.  If an RDBMS was the source and LDAP the target, you'd see it load up all the source identifiers from the RDBMS and then do the same thing for the target so that it can do the comparisons.   

3. PSP does a search on the group:

access.20130321-060831:[21/Mar/2013:06:09:40 -0700] conn=49972860 op=286509 SRCH base="cn=arizona.edu:community:edupersonaffiliations:former-student,ou=grouper,ou=groups,dc=eds,dc=arizona,dc=edu" scope=0 filter="(objectClass=*)" attrs="objectClass cn description isMemberOf hasMember objectClass"

4. One hour and 43 minutes later, PSP performs a MOD on the group:

access:[21/Mar/2013:07:52:06 -0700] conn=49972862 op=286508 MOD dn="cn=arizona.edu:community:edupersonaffiliations:former-student,ou=grouper,ou=groups,dc=eds,dc=arizona,dc=edu"

Why does step 4 take so long?

Step 4 involves comparing the source against each PSO and the attributes therein.  You can see how that'd vastly increase the amount of computations and time necessary to accomplish the operation.  

Dave