Skip to Content.
Sympa Menu

grouper-users - Re: [grouper-users] ldappc, problems during large scale groups replication

Subject: Grouper Users - Open Discussion List

List archive

Re: [grouper-users] ldappc, problems during large scale groups replication

Chronological Thread 
  • From:
  • To: Grouper Users <>
  • Subject: Re: [grouper-users] ldappc, problems during large scale groups replication
  • Date: Thu, 7 Jun 2007 10:46:52 -0400

At 3:23 PM -0500 6/5/07, Tom Barton wrote:
I'm fairly sure this is a problem with the JNDISourceAdapter. I think you can cause a similar failure by trying to do an xml-export of a large group. At least I have done so, with the error message clearly identifying the culprit. I think that some pool of connections it uses gets depleted " Address already in use: connect", possibly because the connection pooling strategy isn't tuned to needing so many ldap queries to perform a single logical operation. Or maybe the ldap server isn't able to process them fast enough to keep the pool from becoming full of pending queries, and the pool manager can't deal with that.

I also note that replacing that JNDI source with a JDBC one containing the same subjects worked around the problem I encountered. Which further implicates the JNDI source adapter as the culprit.

I tried one other trick -- I removed the -groups parameter from the ldappc command line (but left -memberships), so ldappc would only be maintaining the isMemberOf values on the user objects. Still got the connection closed msgs, during the run.

We're now thinking of trying Tom's idea -- create a table within oracle with one user per row, and use that as a source instead of ldap. That should get rid of -- literally -- hundreds of thousands of ldap queries...

So, what do I need to do?

1) can I just build the oracle table (including as columns all the attributes that Grouper will retrieve, including the "key" value for each person), edit the sources.xml to replace the ldap source with this new table? And run ldappc again? I would build the oracle table by exporting an ldif from the existing ldap server, and then loading that data into oracle.

2) or do I need to create the oracle table, and then reload MACE Grouper from scratch, using the oracle table as a source?

3) what can I get away with (minimally) as columns in the oracle table? I currently have this in my sources.xml file:

///Attributes you would like to display when doing a search

brownBRUID is the key value. I'm concerned about DN... ldappc is going to need that value when it populates hasMember on the group objects... do I need to add that to another column in oracle? But, if I have to use step 2 above, how does the DN get generated?

4) What else should I be worried about?


Archive powered by MHonArc 2.6.16.

Top of Page