Grouper Users - Open Discussion List

[Dominique Petitpierre <>]

Hello,

thank you for your reply:

On 22.09.20 14:19, Black, Carey M. wrote:

Dominique,

“
INFO  LdapSystem.performLdapRead(688) -  - Active Directory: Searching with Ldap RangeEntryHandler

DEBUG AbstractOperation.execute(122) -  - execute request=[org.ldaptive.SearchRequest@260620851::baseDn=cn=testgroup,ou=workflow1,ou=bpm-oracle,ou=bpm-poc,ou=application,ou=student,ou=grouper,ou=groups,ou=_unige,dc=isis-klif,dc=unige,dc=ch, searchFilter=[org.ldaptive.SearchFilter@-253519907::filter=objectclass=*, parameters={}], returnAttributes=[member], searchScope=OBJECT, timeLimit=0, sizeLimit=0, derefAliases=null, typesOnly=false, binaryAttributes=null, sortBehavior=UNORDERED, searchEntryHandlers=[org.ldaptive.ad.handler.RangeEntryHandler@17257],... 

“

“sizeLimit=0,"  ß I believe that is wrong for AD.  AD will only return a fixed set at a time and if the LDAP client does not get pages of data then the results will “stop at that AD configured limit”. ( Which appear to be: “lDAPAdminLimits: MaxValRange=1500" )

    You are confusing two limits:
    the maximum number of entries (here groups) to return for one LDAP
    search (lDAPAdminLimits: MaxPageSize=1000),
      needed for entries paging.
      and the maximum number of values for an attribute (here members)
      to return in one result entry (lDAPAdminLimits: MaxValRange=1500), needed
        for attribute values paging.
        and actually entry paging was set (cf. log text "Using ldap
        search-result paging").

In the log line that you cite, entry paging has no point since the scope is OBJECT (aka base), because performLdapRead takes a dn as argument .
Moreover, performLdapRead does work properly for attribute paging:  the point of providing that log line was to show that the searchEntryHandlers was correctly set to RangeEntryHandler by that method, RangeEntryHandler is the ldaptive class that does the attribute paging.

 

Try setting:  ldapSearchResultPagingSize   ( I think the previous post about this may have quoted the wrong java.properties key name….. )
Also verify ( and/or set )   searchResultPagingEnabled = TRUE   ( It should default that way for AD, but if you are seeing “sizeLimit=0” then … that just does not look like it is set to me. )

    Before I looked at the code, while I was still playing with
    configurations I did try that: no change. In the example provided
    only one entry is returned by the search: it cannot be affected by
    that parameter.
    And it cannot change something since the problem is not about
    entries that are not paged but about the member attribute values
    that are not paged.

Just a guess… but I think that is the source of your issue.

Again, could you please look closer at the evidence I provided:
- the method performLdapRead that sets searchEntryHandlers to RangeEntryHandler works properly.
- the method performLdapSearchRequest, although it logs that it does, eventually fails to set searchEntryHandlers to RangeEntryHandler, and thus does not work properly if attribute paging would be necessary because, well, the handler is missing!

 

 

 

And I may be wrong about this:

                However, the LDAP library changed at some point in time.

                I think that change made the “extra handlers” not needed after an upgrade of the LDAP client from “vtldap to ldaptive”.

    That is an interesting lead; in my setting it uses ldaptive 1.1.2
    distributed with Grouper 2.4.0.
    But then why would performLdapRead
    and performLdapSearchRequest bother to set RangeEntryHandler as a
    EntryHandler if the target directory is an Active Directory?
    And why would it fail to handle attribute paging if it was not
    necessary. By the way RangeEntryHandler is a class provided by
    ldaptive.

>  So maybe that explains what you are seeing
>           as a “bug” in the ldap code?
>       
      
      I not only see it in the performLdapSearchRequest method code but
      I see that ldaptive does not get the right handler and that in the
      result (execute response line) the member attribute has no values
      but there is a spurious attribute named "member;range=0-1499" with
      lots of values. (in my message the message is truncated with
      ellipsis because there is 1500 distinguished names in that line).

So:
- please look closer at the evidence I provided in https://lists.internet2.edu/sympa/arc/grouper-users/2020-09/msg00042.html , or  if you have the appropriate context, do the test I described in my last message (https://lists.internet2.edu/sympa/arc/grouper-users/2020-09/msg00050.html).

Also, I am curious:
- do you  see ENTRY_ALREADY_EXISTS WARN level log messages in your logs during full sync of large groups (and not small ones, <= 1500 members)?
- If not, what is the log level for edu.internet2.middleware.grouper.pspng  and PSPNG patch version?

Regards.

-- 
Mr Dominique Petitpierre, user=Dominique.Petitpierre domain=unige.ch
IT Division, University of Geneva, Switzerland