Grouper Users - Open Discussion List

[Jeffrey Crawford <>]

Humm, actually what you are mentioning is making me a little nervous, We started to integrate with our AD in addition to our LDAP and our full sync went from just under 30 min, to 7 hours. I just chalked it up to some misconfiguration on the AD side and we haven't really debugged that yet since it's not going to production for a while.

We did check that the lookup attribute was indexed, is AD just slow via it's LDAP interface? I did seem to respond quickly in batches but then take a few second pause every so often, however the pauses really add up over time on a large number of users.

UW, I'm sure you guys did a lot of grouper to AD discovery.

Jeffrey E. Crawford
Enterprise Service Team

    ^         ^

   / \  ^    / \    ^

  /   \/ \  /   \  / \

 /        \/     \/   \

/                      \

You have been assigned this mountain to prove to others that it *can* be moved.

On Fri, Aug 18, 2017 at 7:51 AM, Dave Churchley <> wrote:

Thanks for the tips, Robert. I'd like to a pose a follow-up question to anyone in the community (particularly anyone using PSP to provision to AD) about expectations.

Looking in grouper_error.log suggests that each PspChangeLogConsumer.processChangeLogEntry takes about 2 seconds "Elapsed time" and, when we tried it, a bulkSync took about a fortnight. What kind of performance should we expect to be able to achieve?

I appreciate this is probably dependent upon environment. We're running Grouper 2.2.2 and, from Grouper Report, we have:
memberships:           969,308
groups:                16,068
members:               79,796
folders:               950

Thanks
Dave


>-----Original Message-----
>From: [mailto:
>] On Behalf Of Robert Bradley
>Sent: 16 August 2017 17:48
>To:
>Subject: Re: [grouper-users] Large number of changes and provisioning
>

>-----BEGIN PGP SIGNED MESSAGE-----
>Hash: SHA256
>
>On 16/08/17 16:52, Dave Churchley wrote:
>> Seems like a great idea to me. I hope you don’t mind me asking an
>> off-topic (but definitely related) question…
>>
>> We’ve suffered from similar backlogs with PSP when there have been
>> large membership changes. The first thing that struck me about your
>> solution, however, was that you said your bulkSync takes half an
>> hour. For us it takes about 2 weeks! For this reason, we never do a
>> bulkSync and rely on PSP incremental provisioning to AD so when we
>> get a backlog we just have to live with it.
>>
>> We’re on v2.2 using PSP to provision to AD. Since we started using
>> PSP we’ve found it to be quite slow but, under normal load, it
>> copes pretty well and updates AD in about a minute. We do
>> occasionally suffer provisioning backlogs though. We know there’s a
>> big one on the horizon at the end of the month, as our academic
>> year switches over. Last year we had a backlog for about 2 weeks.
>>
>> Does anyone have any suggestions that might help us out? We’ve
>> recently been testing PSPNG on v2.3 but it doesn’t appear to be
>> quite production-ready for our needs yet.
>>
>
>Things I'd suggest from personal experience are:
>
>1. Run the bulk sync after setting the following in
>grouper-loader.properties:
>
>changeLog.psp.fullSync.omitDiffResponses = true
>changeLog.psp.fullSync.omitSyncResponses = true
>
>and setting logSpml="false" in psp-services.xml (within the <Service
>... /> tags).  These shouldn't directly affect performance, but does
>reduce memory pressure and the GC overheads caused by it.  If you can
>increase the JVM -Xmx and -Xms parameters fairly easily, I'd recommend
>doing that as well.
>
>2.
>
>Increasing caching in ehcache.xml (both TTL and number of entries) can
>help a lot by reducing round trips to your subject sources (AD?).
>
>I'd also check the LDAP indexing of any subject or provisioned
>attributes.  In OpenLDAP terms, you'd ideally want "pres,eq" indexes
>for attributes that aren't used by the Subject API search, and
>"pres,sub,eq" indexes for those that are searchable.  I'm assuming
>that AD has similar index controls, but lack the experience with AD to
>comment.
>
>
>- --
>Dr Robert Bradley
>Identity and Access Management Team, IT Services, University of Oxford
>-----BEGIN PGP SIGNATURE-----
>
>iQIzBAEBCAAdFiEEgF3NFfO9FqlA+ME+lGGnynav474FAlmUd0MACgkQlGGnyn
>av
>476j/A//VMRBAAtvGxh5aC0CrBbSvuWZ8hbX5xZ3fDs81GB7eY3wZjdRMlyBmc
>0f
>fva60ZExEcJA8RXqcbJp1+DOBqwbwRylwRTU2gKClsaq/j7ULZ3G5JX/PBxP14sz
>APT+vjAE9rSLijiEOGrGS9mxdIo9sNQM0UX46A3oEOuKC3s3cF2M3QEA3LV0iW
>dW
>D4M6PxqGcY/+luvZNcV2habvO3J/evg5Sv38U2S8XDxGV2yJCsNwSaeJDYniB/S
>X
>wdtobiyhYVyuhVUT0GYuJxKN8gnU7fcV1Ok9sTH4sjM7EpaFj/BqI11B3w4Cd3w
>d
>Xhg3HuJv22hMKMDiKT8vjre4nf2temU1kwTTlsX4XEj2OJfjiSVB+BdhA3jgV3HG
>GyPBUnI6mFMzB286h9JMIJivmz/mi/8gO5uCO3DuhPerh10q2dwdzKKR9/O9Px
>sn
>ZK5z9U+zgjUB1iFyoG6z0gW+lKuzUO7yE34qBgxeQ5k/Calg1AmP6pPh1IYQCBtd
>dEPSLtN1DHnVz9CLgAoLgNB/PpSU+WWY4mcYFXuJjwbBCNfaZwWTQCUgHey
>GNSZN
>3J2ugFFpoFALWw9tK/4yD4o/gZcPw5rOWfH7Ai+QNHxNStsHw3mMAhOdfjNN
>fRdf
>dGdiq157rdFMAx0HNO/IAYNeVf9GdNBb3SNDgUW11hQp0PKaBGA=
>=+CVF
>-----END PGP SIGNATURE-----