Skip to Content.
Sympa Menu

grouper-dev - Re: [grouper-dev] Subject API Slowness

Subject: Grouper Developers Forum

List archive

Re: [grouper-dev] Subject API Slowness

Chronological Thread 
  • From: Mark Weber <>
  • To: Tom Barton <>
  • Cc:
  • Subject: Re: [grouper-dev] Subject API Slowness
  • Date: Mon, 02 Oct 2006 16:49:26 -0500

FYI, I rebuilt the Subject API with JNDI Connection Pooling turned on and that had quite a significant effect on the xml-import. Loading the 1000, it went from 6 min down to 1 min 50 sec.

However, I still suspect that there is something j'ed up elsewhere with our config because the UI is still extremely slow in places. (When I attempt to display the members of the group with a specific privilege)

I included the diff's in case others want to the Subject API with CP turned on.

I haven't done any real testing with this change except to load 1000 subjects into grouper and verify that it's *not* opening and closing a connection for each query. So... mileage may vary. Also, I think you need jdk1.4+ to do the connection pooling.


Mark Weber wrote:
Here is a break down of the relevant pieces of our environments and other info.

(Note: I haven't run again with Ellen's suggestion of allocating more memory to the jvm, but will do that asap)

Development Environment:
Oracle 10g (4-CPU IBM AIX)

Subject Store:
Sun ONE Directory Server v5.2 (UltraSPARC III solaris9)

jvm, tomcat5.5, (2.4Ghz p4, Ubuntu 6.06)

BEA WebLogic 9.2 (2-way UltraSPARC III, solaris9)

We have roughly 200,000 subjects in the subject store for test. We will likely have 250,000 subjects on a full production load. For now we are loading small basic groups that are very shallow.

Loading 2 groups of a 1000 each for emp's and students.


In this batch, we are giving every entry, every privilege (admin, read, view, update, optin, optout) We intend to mix things up and try deeply nested groups, fewer privileges etc. But first, we want to solve the slowness issues.




LDAP Indexes:

cn: Equality, Presence, SubString
WiscEduPVI: Equality, Presence


read, compare, search, selfwrite, write

Tom Barton wrote:

Although the jndi source adapter does not use connection pooling, your statement of problem makes me wonder if other issues are also at play here, ie, that these absurdly long times may be the result of more than just lack of connection pooling.

Some particulars below...

Mark Weber wrote:
We've started doing some initial load testing and found that it's pretty slow. Our initial estimates put a full load of our population into grouper, via the xml-import facility, would take about 8 days.

Could you give some basic facts & stats - what back-end database, how large its overall allocation, about how many "large" groups & "small" groups in the load, how deeply nested, how many employ non-default privileges.

I think I've narrowed it down to the Subject API. Specifically the Subject API configured using JNDI/LDAP adapter. For Instance, the UI takes 74 seconds to load the " All Groups - Current subjects with [xyz] privilege" page. It displays the first 50 members and creates a connection, bind, search, result, unbind, and disconnect for each person on the page. Same with the xml-import.

This is far from the fraction of a second I'm accustomed to seeing it take the jndi source adapter to display 50 subjects. I suspect there's another factor at play in this example, or perhaps your ldap server configuration (ACLs, authentication mechanism, other unknown operational params) might produce substantially different connection setup and query response times compared to the rather vanilla config of the openldap server I use for prototyping.

So, is there some configuration option that allows for connection pooling within the subject API when it's using LDAP?

Not in this version.

This is an assumption, but if we used the jdbc source adapter instead, would it use connection pooling and thereby eliminate the problem altogether?

I think it does indeed use connection pooling.


.Mark G. Weber----------------------Division of Information Technology.
University Of Wisconsin - Madison|
|608/263.8945 Room 3159 Computer Science|
RCS file:
retrieving revision 1.5
diff -u -8 -p -r1.5
--- 27 Jun 2006 22:33:51 -0000 1.5
+++ 2 Oct 2006 21:28:06 -0000
@@ -207,16 +207,17 @@ public class JNDISourceAdapter
* Setup environment.
* @throws SourceUnavailableException
protected void setupEnvironment(Properties props)
throws SourceUnavailableException {

props.getProperty("INITIAL_CONTEXT_FACTORY") );
props.getProperty("PROVIDER_URL") );
+ environment.put("com.sun.jndi.ldap.connect.pool",
props.getProperty("POOL_CONNECTIONS") );
props.getProperty("SECURITY_PRINCIPAL") );

if (props.getProperty("SECURITY_PROTOCOL")!= null) {
environment.put(Context.SECURITY_PROTOCOL, "ssl");
Context context = null;
try {
Index: sources.xml
RCS file: /home/cvs/i2mi/subject/conf/sources.xml,v
retrieving revision 1.5
diff -u -8 -p -r1.5 sources.xml
--- sources.xml 8 Jun 2006 00:13:01 -0000 1.5
+++ sources.xml 2 Oct 2006 21:30:23 -0000
@@ -138,16 +138,20 @@ $Id: sources.xml,v 1.5 2006/06/08 00:13:
+ <param-name>POOL_CONNECTIONS</param-name>
+ <param-value>true</param-value>
+ </init-param>
+ <init-param>

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature

Archive powered by MHonArc 2.6.16.

Top of Page