Grouper Users - Open Discussion List

[Jeff McCullough <>]

I did some further investigation of the LDAP server logs, and was able to 
track the issue back to the grouper-loader.properties file. There was an 
error there, so the loader was defaulting to an anonymous bind. The 
sources.xml tag delimiting of the same information was fine, so the other 
lookups were working. Thank you for your time on this.

Jeff


On Apr 2, 2012, at 11:35 PM, Jeff McCullough wrote:

> I see the debug line, and it reports ArrayList size: 500.
> 
> Jeff
> 
> 
> 
> On Apr 2, 2012, at 9:43 PM, Chris Hyzer 
> <>
>  wrote:
> 
>> TomZ feel free to chime in...
>> 
>> I did a loader job from our ldap which returned 30k subjects.  I didn't 
>> have the source configured, but I saw that many results come back from 
>> ldap.
>> 
>> Which type of job is this?  I tried LDAP_SIMPLE
>> 
>> I set this in the log4j.properties:
>> 
>> log4j.logger.edu.internet2.middleware.grouper.ldap = DEBUG
>> 
>> I saw this in the logs:
>> 
>> 2012-04-03 00:35:13,908: [DefaultQuartzScheduler_Worker-1] DEBUG 
>> LdapSession$1.callback(274) -  - Found 30650 results for serverId: 
>> personLdap, searchDn: ou=whatever, filter: '(cn=someFilter:whatever)', 
>> returning attribute: hasMember, some results: ArrayList size: 30650: [0]: 
>> whatever
>> 
>> Do you see that or is it 500?
>> 
>> It is in the LdapSession class, doing this:
>> 
>>           searchResultIterator = ldap.search(searchDn,
>>               searchFilterObject, searchControls);
>> 
>> I would think the problem is in vt-ldap (or one of its configs), or the 
>> ldap server setting.  TomZ do you have some help here?  :)
>> 
>> Thanks,
>> chris
>> 
>> 
>> -----Original Message-----
>> From: Jeff McCullough 
>> [mailto:]
>>  
>> Sent: Monday, April 02, 2012 3:13 PM
>> To: Chris Hyzer
>> Cc: Tom Zeller; 
>> ;
>>  CalNet Administration
>> Subject: Re: [grouper-users] Grouper LDAP loader - large result sets
>> 
>> I should add that the subject ids in the group are the first 500 (of the 
>> total 76119) records returned using ldapsearch. -Jeff
>> 
>> On Apr 2, 2012, at 12:02 PM, Jeff McCullough wrote:
>> 
>>> Hi Chris,
>>> 
>>> I'm using gsh. The possible result set is 76119, and the group created is 
>>> of size 500.
>>> 
>>> Jeff
>>> 
>>> ----------------
>>> gsh script from your examples
>>> -----------------
>>> grouperSession = GrouperSession.startRootSession();
>>> group = new 
>>> GroupSave(grouperSession).assignName("someStem:myLdapGroup").assignCreateParentStemsIfNotExist(true).save();
>>> attributeAssign = 
>>> group.getAttributeDelegate().assignAttribute(LoaderLdapUtils.grouperLoaderLdapAttributeDefName()).getAttributeAssign();
>>> attributeAssign.getAttributeValueDelegate().assignValue(LoaderLdapUtils.grouperLoaderLdapTypeName(),
>>>  "LDAP_SIMPLE");
>>> attributeAssign.getAttributeValueDelegate().assignValue(LoaderLdapUtils.grouperLoaderLdapFilterName(),
>>>  "(&(objectclass=person)(uid=*))");
>>> attributeAssign.getAttributeValueDelegate().assignValue(LoaderLdapUtils.grouperLoaderLdapQuartzCronName(),
>>>  "0 * * * * ?");
>>> attributeAssign.getAttributeValueDelegate().assignValue(LoaderLdapUtils.grouperLoaderLdapServerIdName(),
>>>  "ucbLdap");
>>> attributeAssign.getAttributeValueDelegate().assignValue(LoaderLdapUtils.grouperLoaderLdapSubjectAttributeName(),
>>>  "uid");
>>> attributeAssign.getAttributeValueDelegate().assignValue(LoaderLdapUtils.grouperLoaderLdapSubjectIdTypeName(),
>>>  "subjectIdentifier");
>>> group = GroupFinder.findByName(grouperSession, "someStem:myLdapGroup");
>>> loaderRunOneJob(group);
>>> 
>>> 
>>> On Mar 31, 2012, at 8:06 PM, Chris Hyzer wrote:
>>> 
>>>> What type of ldap loader job is it?  How big is the resultset supposed 
>>>> to be and how big is the group which it is not the right size?
>>>> 
>>>> thanks,
>>>> chris
>>>> ________________________________________
>>>> From: 
>>>> 
>>>>  
>>>> []
>>>>  on behalf of Tom Zeller 
>>>> []
>>>> Sent: Saturday, March 31, 2012 11:52 AM
>>>> To: Jeff McCullough
>>>> Cc: 
>>>> ;
>>>>  CalNet Administration
>>>> Subject: Re: [grouper-users] Grouper LDAP loader - large result sets
>>>> 
>>>> Are you receiving size or time limit error messages ? What makes you 
>>>> think jndi is trying to page ?
>>>> 
>>>> I will have to look, but I thought the grouper ldap loader wrapped 
>>>> vt-ldap, which itself wraps jndi, so there might be knobs we can tweak.
>>>> 
>>>> TomZ
>>>> 
>>>> On Mar 31, 2012, at 1:38 AM, Jeff McCullough 
>>>> <>
>>>>  wrote:
>>>> 
>>>>> 
>>>>> I'm trying out grouper's LDAP loader. I'm finding that when I'm doing a 
>>>>> large result query that I'm only getting a nibble of the total possible 
>>>>> results back. Our LDAP server, Oracle DSEE, doesn't support paging, 
>>>>> though it is a fairly common LDAP server feature for dealing with large 
>>>>> data result sets. It is looking like JDNI is trying to page despite 
>>>>> both the bind I'm using and the server can return a very large data 
>>>>> result set using ldapsearch. Does anyone have experience with this, and 
>>>>> have suggestions about how to let JDNI/Grouper retrieve the entire 
>>>>> result set in one pass?
>>>>> 
>>>>> Thanks,
>>>>> Jeff
>>> 
>>