Grouper Users - Open Discussion List

[Chris Hyzer <>]

TomZ feel free to chime in...

I did a loader job from our ldap which returned 30k subjects.  I didn't have 
the source configured, but I saw that many results come back from ldap.

Which type of job is this?  I tried LDAP_SIMPLE

I set this in the log4j.properties:

log4j.logger.edu.internet2.middleware.grouper.ldap = DEBUG

I saw this in the logs:

2012-04-03 00:35:13,908: [DefaultQuartzScheduler_Worker-1] DEBUG 
LdapSession$1.callback(274) -  - Found 30650 results for serverId: 
personLdap, searchDn: ou=whatever, filter: '(cn=someFilter:whatever)', 
returning attribute: hasMember, some results: ArrayList size: 30650: [0]: 
whatever

Do you see that or is it 500?

It is in the LdapSession class, doing this:

            searchResultIterator = ldap.search(searchDn,
                searchFilterObject, searchControls);

I would think the problem is in vt-ldap (or one of its configs), or the ldap 
server setting.  TomZ do you have some help here?  :)

Thanks,
chris


-----Original Message-----
From: Jeff McCullough 
[mailto:]
 
Sent: Monday, April 02, 2012 3:13 PM
To: Chris Hyzer
Cc: Tom Zeller; 
;
 CalNet Administration
Subject: Re: [grouper-users] Grouper LDAP loader - large result sets

I should add that the subject ids in the group are the first 500 (of the 
total 76119) records returned using ldapsearch. -Jeff

On Apr 2, 2012, at 12:02 PM, Jeff McCullough wrote:

> Hi Chris,
> 
> I'm using gsh. The possible result set is 76119, and the group created is 
> of size 500.
> 
> Jeff
> 
> ----------------
> gsh script from your examples
> -----------------
> grouperSession = GrouperSession.startRootSession();
> group = new 
> GroupSave(grouperSession).assignName("someStem:myLdapGroup").assignCreateParentStemsIfNotExist(true).save();
> attributeAssign = 
> group.getAttributeDelegate().assignAttribute(LoaderLdapUtils.grouperLoaderLdapAttributeDefName()).getAttributeAssign();
> attributeAssign.getAttributeValueDelegate().assignValue(LoaderLdapUtils.grouperLoaderLdapTypeName(),
>  "LDAP_SIMPLE");
> attributeAssign.getAttributeValueDelegate().assignValue(LoaderLdapUtils.grouperLoaderLdapFilterName(),
>  "(&(objectclass=person)(uid=*))");
> attributeAssign.getAttributeValueDelegate().assignValue(LoaderLdapUtils.grouperLoaderLdapQuartzCronName(),
>  "0 * * * * ?");
> attributeAssign.getAttributeValueDelegate().assignValue(LoaderLdapUtils.grouperLoaderLdapServerIdName(),
>  "ucbLdap");
> attributeAssign.getAttributeValueDelegate().assignValue(LoaderLdapUtils.grouperLoaderLdapSubjectAttributeName(),
>  "uid");
> attributeAssign.getAttributeValueDelegate().assignValue(LoaderLdapUtils.grouperLoaderLdapSubjectIdTypeName(),
>  "subjectIdentifier");
> group = GroupFinder.findByName(grouperSession, "someStem:myLdapGroup");
> loaderRunOneJob(group);
> 
> 
> On Mar 31, 2012, at 8:06 PM, Chris Hyzer wrote:
> 
>> What type of ldap loader job is it?  How big is the resultset supposed to 
>> be and how big is the group which it is not the right size?
>> 
>> thanks,
>> chris
>> ________________________________________
>> From: 
>> 
>>  
>> []
>>  on behalf of Tom Zeller 
>> []
>> Sent: Saturday, March 31, 2012 11:52 AM
>> To: Jeff McCullough
>> Cc: 
>> ;
>>  CalNet Administration
>> Subject: Re: [grouper-users] Grouper LDAP loader - large result sets
>> 
>> Are you receiving size or time limit error messages ? What makes you think 
>> jndi is trying to page ?
>> 
>> I will have to look, but I thought the grouper ldap loader wrapped 
>> vt-ldap, which itself wraps jndi, so there might be knobs we can tweak.
>> 
>> TomZ
>> 
>> On Mar 31, 2012, at 1:38 AM, Jeff McCullough 
>> <>
>>  wrote:
>> 
>>> 
>>> I'm trying out grouper's LDAP loader. I'm finding that when I'm doing a 
>>> large result query that I'm only getting a nibble of the total possible 
>>> results back. Our LDAP server, Oracle DSEE, doesn't support paging, 
>>> though it is a fairly common LDAP server feature for dealing with large 
>>> data result sets. It is looking like JDNI is trying to page despite both 
>>> the bind I'm using and the server can return a very large data result set 
>>> using ldapsearch. Does anyone have experience with this, and have 
>>> suggestions about how to let JDNI/Grouper retrieve the entire result set 
>>> in one pass?
>>> 
>>> Thanks,
>>> Jeff
>