Grouper Users - Open Discussion List

[Jeff McCullough <>]

I see the debug line, and it reports ArrayList size: 500.

Jeff



On Apr 2, 2012, at 9:43 PM, Chris Hyzer 
<>
 wrote:

> TomZ feel free to chime in...
> 
> I did a loader job from our ldap which returned 30k subjects.  I didn't 
> have the source configured, but I saw that many results come back from ldap.
> 
> Which type of job is this?  I tried LDAP_SIMPLE
> 
> I set this in the log4j.properties:
> 
> log4j.logger.edu.internet2.middleware.grouper.ldap = DEBUG
> 
> I saw this in the logs:
> 
> 2012-04-03 00:35:13,908: [DefaultQuartzScheduler_Worker-1] DEBUG 
> LdapSession$1.callback(274) -  - Found 30650 results for serverId: 
> personLdap, searchDn: ou=whatever, filter: '(cn=someFilter:whatever)', 
> returning attribute: hasMember, some results: ArrayList size: 30650: [0]: 
> whatever
> 
> Do you see that or is it 500?
> 
> It is in the LdapSession class, doing this:
> 
>            searchResultIterator = ldap.search(searchDn,
>                searchFilterObject, searchControls);
> 
> I would think the problem is in vt-ldap (or one of its configs), or the 
> ldap server setting.  TomZ do you have some help here?  :)
> 
> Thanks,
> chris
> 
> 
> -----Original Message-----
> From: Jeff McCullough 
> [mailto:]
>  
> Sent: Monday, April 02, 2012 3:13 PM
> To: Chris Hyzer
> Cc: Tom Zeller; 
> ;
>  CalNet Administration
> Subject: Re: [grouper-users] Grouper LDAP loader - large result sets
> 
> I should add that the subject ids in the group are the first 500 (of the 
> total 76119) records returned using ldapsearch. -Jeff
> 
> On Apr 2, 2012, at 12:02 PM, Jeff McCullough wrote:
> 
>> Hi Chris,
>> 
>> I'm using gsh. The possible result set is 76119, and the group created is 
>> of size 500.
>> 
>> Jeff
>> 
>> ----------------
>> gsh script from your examples
>> -----------------
>> grouperSession = GrouperSession.startRootSession();
>> group = new 
>> GroupSave(grouperSession).assignName("someStem:myLdapGroup").assignCreateParentStemsIfNotExist(true).save();
>> attributeAssign = 
>> group.getAttributeDelegate().assignAttribute(LoaderLdapUtils.grouperLoaderLdapAttributeDefName()).getAttributeAssign();
>> attributeAssign.getAttributeValueDelegate().assignValue(LoaderLdapUtils.grouperLoaderLdapTypeName(),
>>  "LDAP_SIMPLE");
>> attributeAssign.getAttributeValueDelegate().assignValue(LoaderLdapUtils.grouperLoaderLdapFilterName(),
>>  "(&(objectclass=person)(uid=*))");
>> attributeAssign.getAttributeValueDelegate().assignValue(LoaderLdapUtils.grouperLoaderLdapQuartzCronName(),
>>  "0 * * * * ?");
>> attributeAssign.getAttributeValueDelegate().assignValue(LoaderLdapUtils.grouperLoaderLdapServerIdName(),
>>  "ucbLdap");
>> attributeAssign.getAttributeValueDelegate().assignValue(LoaderLdapUtils.grouperLoaderLdapSubjectAttributeName(),
>>  "uid");
>> attributeAssign.getAttributeValueDelegate().assignValue(LoaderLdapUtils.grouperLoaderLdapSubjectIdTypeName(),
>>  "subjectIdentifier");
>> group = GroupFinder.findByName(grouperSession, "someStem:myLdapGroup");
>> loaderRunOneJob(group);
>> 
>> 
>> On Mar 31, 2012, at 8:06 PM, Chris Hyzer wrote:
>> 
>>> What type of ldap loader job is it?  How big is the resultset supposed to 
>>> be and how big is the group which it is not the right size?
>>> 
>>> thanks,
>>> chris
>>> ________________________________________
>>> From: 
>>> 
>>>  
>>> []
>>>  on behalf of Tom Zeller 
>>> []
>>> Sent: Saturday, March 31, 2012 11:52 AM
>>> To: Jeff McCullough
>>> Cc: 
>>> ;
>>>  CalNet Administration
>>> Subject: Re: [grouper-users] Grouper LDAP loader - large result sets
>>> 
>>> Are you receiving size or time limit error messages ? What makes you 
>>> think jndi is trying to page ?
>>> 
>>> I will have to look, but I thought the grouper ldap loader wrapped 
>>> vt-ldap, which itself wraps jndi, so there might be knobs we can tweak.
>>> 
>>> TomZ
>>> 
>>> On Mar 31, 2012, at 1:38 AM, Jeff McCullough 
>>> <>
>>>  wrote:
>>> 
>>>> 
>>>> I'm trying out grouper's LDAP loader. I'm finding that when I'm doing a 
>>>> large result query that I'm only getting a nibble of the total possible 
>>>> results back. Our LDAP server, Oracle DSEE, doesn't support paging, 
>>>> though it is a fairly common LDAP server feature for dealing with large 
>>>> data result sets. It is looking like JDNI is trying to page despite both 
>>>> the bind I'm using and the server can return a very large data result 
>>>> set using ldapsearch. Does anyone have experience with this, and have 
>>>> suggestions about how to let JDNI/Grouper retrieve the entire result set 
>>>> in one pass?
>>>> 
>>>> Thanks,
>>>> Jeff
>> 
>