grouper-dev - [grouper-dev] speedup by caching in the ldap source adapter ?
Subject: Grouper Developers Forum
List archive
- From: Tom Zeller <>
- To: Grouper Dev <>
- Subject: [grouper-dev] speedup by caching in the ldap source adapter ?
- Date: Thu, 17 May 2012 18:43:57 -0500
When provisioning an ldap target, it seems that querying for members
takes the most time, especially when provisioning the "everyone"
group. Even if the source resolver has a large cache, at least one
ldap search is performed for every grouper membership.
To avoid the performance penalty and scaling issues with
one-ldap-search-per-membership, I added a simple cache to the ldap
source adapter, and this cache is warmed by slurping the target ldap
directory at startup.
In other words, rather than n "(uid=...)" searches, just do one
"(uid=*)" search.
I hesitate to give numbers, but on my local box, provisioning a group
with 100k members takes approximately 25 minutes (not hours !) without
the warmed-up cache. With the warmed-up cache, provisioning a group
with 100k members takes more like 5 minutes.
Another option is to make sure that all subject attributes needed for
provisioning are written to the grouper_members table, and avoid ldap
member searches entirely.
What do you think ?
TomZ
- [grouper-dev] speedup by caching in the ldap source adapter ?, Tom Zeller, 05/17/2012
- Re: [grouper-dev] speedup by caching in the ldap source adapter ?, Michael R. Gettes, 05/17/2012
- RE: [grouper-dev] speedup by caching in the ldap source adapter ?, Chris Hyzer, 05/18/2012
- Re: [grouper-dev] speedup by caching in the ldap source adapter ?, Michael R. Gettes, 05/18/2012
- RE: [grouper-dev] speedup by caching in the ldap source adapter ?, Chris Hyzer, 05/18/2012
- Re: [grouper-dev] speedup by caching in the ldap source adapter ?, Michael R. Gettes, 05/18/2012
- RE: [grouper-dev] speedup by caching in the ldap source adapter ?, Chris Hyzer, 05/18/2012
- Re: [grouper-dev] speedup by caching in the ldap source adapter ?, Michael R. Gettes, 05/18/2012
- Re: [grouper-dev] speedup by caching in the ldap source adapter ?, Gary Chapman, 05/18/2012
- Re: [grouper-dev] speedup by caching in the ldap source adapter ?, Tom Zeller, 05/18/2012
- RE: [grouper-dev] speedup by caching in the ldap source adapter ?, Chris Hyzer, 05/18/2012
- RE: [grouper-dev] speedup by caching in the ldap source adapter ?, Chris Hyzer, 05/18/2012
- Re: [grouper-dev] speedup by caching in the ldap source adapter ?, Michael R. Gettes, 05/17/2012
Archive powered by MHonArc 2.6.16.