Grouper Developers Forum

[Mark Weber <>]

Here is a break down of the relevant pieces of our environments and 
other info.

(Note: I haven't run again with Ellen's suggestion of allocating more 
memory to the jvm, but will do that asap)


Development Environment:
------------------------
Back-End:
  Oracle 10g (4-CPU IBM AIX)

Subject Store:
  Sun ONE Directory Server v5.2 (UltraSPARC III solaris9)

Grouper(tomcat):
  jvm 1.5.0.06, tomcat5.5, (2.4Ghz p4, Ubuntu 6.06)

Grouper(WebLogic):
  BEA WebLogic 9.2 (2-way UltraSPARC III, solaris9)


We have roughly 200,000 subjects in the subject store for test.  We will 
likely have 250,000 subjects on a full production load.  For now we are 
loading small basic groups that are very shallow.

Loading 2 groups of a 1000 each for emp's and students.

uwmsn:[employees1k]
uwmsn:[students1k]

In this batch, we are giving every entry, every privilege (admin, read, 
view, update, optin, optout)  We intend to mix things up and try deeply 
nested groups, fewer privileges etc.  But first, we want to solve the 
slowness issues.

sources.xml
------------
<init-param>
  <param-name>SubjectID_AttributeType</param-name>
  <param-value>wiscEduPVI</param-value>
</init-param>

<init-param>
  <param-name>Name_AttributeType</param-name>
  <param-value>cn</param-value>
</init-param>

<init-param>
  <param-name>Description_AttributeType</param-name>
  <param-value>cn</param-value>
</init-param>


LDAP Indexes:
-------------

cn:          Equality, Presence, SubString
WiscEduPVI:  Equality, Presence


LDAP ACI's:
------------
uid=gapi,ou=apps,o=isp:

Rights:
  read, compare, search, selfwrite, write
Targets:
  ou=people,o=wisctest.wisc.edu,o=isp



Tom Barton wrote:
> Mark,
>
> Although the jndi source adapter does not use connection pooling, your 
> statement of problem makes me wonder if other issues are also at play 
> here, ie, that these absurdly long times may be the result of more than 
> just lack of connection pooling.
>
> Some particulars below...
>
> Mark Weber wrote:
> > We've started doing some initial load testing and found that it's 
> > pretty slow.  Our initial estimates put a full load of our population 
> > into grouper, via the xml-import facility, would take about 8 days.
>
> Could you give some basic facts & stats - what back-end database, how 
> large its overall allocation, about how many "large" groups & "small" 
> groups in the load, how deeply nested, how many employ non-default 
> privileges.
>
> > I think I've narrowed it down to the Subject API.  Specifically the 
> > Subject API configured using JNDI/LDAP adapter.  For Instance, the UI 
> > takes 74 seconds to load the " All Groups   - Current subjects with 
> > [xyz] privilege" page.  It displays the first 50 members and creates a 
> > connection, bind, search, result, unbind, and disconnect for each 
> > person on the page.  Same with the xml-import.
>
> This is far from the fraction of a second I'm accustomed to seeing it 
> take the jndi source adapter to display 50 subjects. I suspect there's 
> another factor at play in this example, or perhaps your ldap server 
> configuration (ACLs, authentication mechanism, other unknown operational 
> params) might produce substantially different connection setup and query 
> response times compared to the rather vanilla config of the openldap 
> server I use for prototyping.
>
> > So, is there some configuration option that allows for connection 
> > pooling within the subject API when it's using LDAP?
>
> Not in this version.
>
> > This is an assumption, but if we used the jdbc source adapter instead, 
> > would it use connection pooling and thereby eliminate the problem 
> > altogether?
>
> I think it does indeed use connection pooling.
>
> Tom

--
.Mark G. Weber----------------------Division of Information Technology.
|
            University Of Wisconsin - Madison|
|608/263.8945                               Room 3159 Computer Science|
`---------------------------------------------------------------------'

Attachment: smime.p7s
Description: S/MIME Cryptographic Signature