Grouper Users - Open Discussion List

[Shilen Patel <>]

We’ve seen the issues you’re describing before and validating at an interval that’s less than the F5 timeout has seemed to have avoided the issue in other places.  You may want to try that as a possible quick workaround.  You can either add those two properties or if you’re on a recent 2.5 container, you can configure it using the external systems UI screen.  If you continue to see exceptions, it might be helpful to see the full stacktrace.

 

- Shilen

 

From: <> on behalf of Baron Fujimoto <>
Reply-To: Baron Fujimoto <>
Date: Friday, June 4, 2021 at 7:10 PM
To: Shilen Patel <>
Cc: Grouper Users <>
Subject: Re: [grouper-users] LDAP UnboundID bug? (hardcoded JNDI reference)

 

Our timeout issue has a couple of necessary conditions:

- at least one of either our application or LDAP hosts behind anF5 load balancer

- JDK version ≥ 8u231

 

We had a support case open w/ Oracle re this, but after months of back and forth, they ultimately determined that the JDK was acting according to spec and that it was the LDAP libraries and/or applications that were not handling results properly.

 

In Grouper we see this manifest as failed subject lookups due to closed LDAP connections. These are representative log excerpts:

 

=====

[https-jsse-nio-8443-exec-10] ERROR LdapSourceAdapter.getLdapResultsHelper(594) - < ... > - Ldap Exception: Problem with ldap conection: personLdap,

Error querying ldap server id: personLdap, searchDn: dc=hawaii,dc=edu, filter: '(& (uid=foobar) (objectclass=uhEduPerson))', returning attributes: cn, uhuuid, uid, sn, givenname

java.lang.RuntimeException: Problem with ldap conection: personLdap,

Error querying ldap server id: personLdap, searchDn: dc=hawaii,dc=edu, filter: '(& (uid=foobar) (objectclass=uhEduPerson))', returning attributes: cn, uhuuid, uid, sn, givenname

[...]

Caused by: [org.ldaptive.LdapException@1471483139::resultCode=null, matchedDn=null, responseControls=null, referralURLs=null, messageId=-1, message=javax.naming.NamingException: LDAP connection has been closed; remaining name 'dc=hawaii,dc=edu', providerException=javax.naming.Nam

ingException: LDAP connection has been closed; remaining name 'dc=hawaii,dc=edu']

[...]

Caused by: javax.naming.NamingException: LDAP connection has been closed; remaining name 'dc=hawaii,dc=edu'

[https-jsse-nio-8443-exec-10] WARN  WsSubjectLookup.retrieveSubjectIfNeeded(328) - < ... > - WsSubjectLookup[

  subjectFindResult=SUCCESS,subjectIdentifier=foobar]

edu.internet2.middleware.subject.SourceUnavailableException: Ldap Exception: Problem with ldap conection: personLdap,

Error querying ldap server id: personLdap, searchDn: dc=hawaii,dc=edu, filter: '(& (uid=foobar) (objectclass=uhEduPerson))', returning attributes: cn, uhuuid, uid, sn, givenname

[...]

Caused by: java.lang.RuntimeException: Problem with ldap conection: personLdap,

Error querying ldap server id: personLdap, searchDn: dc=hawaii,dc=edu, filter: '(& (uid=foobar) (objectclass=uhEduPerson))', returning attributes: cn, uhuuid, uid, sn, givenname

Caused by: [org.ldaptive.LdapException@1471483139::resultCode=null, matchedDn=null, responseControls=null, referralURLs=null, messageId=-1, message=javax.naming.NamingException: LDAP connection has been closed; remaining name 'dc=hawaii,dc=edu', providerException=javax.naming.NamingException: LDAP connection has been closed; remaining name 'dc=hawaii,dc=edu']

[...]

Caused by: javax.naming.NamingException: LDAP connection has been closed; remaining name 'dc=hawaii,dc=edu'

=====

 

Our interpretation of what is happening is that the F5 is dropping the LDAP connection, but this is not detected up on the Grouper end. The pool validator apparently does not check before attempting to use the expected LDAP connection, nor does it retry upon the resulting connection error.

 

Reviewing our grouper-loader.properties, it doesn't look like we have any ldap.personValidate* properties set. Although some of the properties appear to be fairly self-descriptive, I haven't found a good reference for the properties in the Grouper wiki. Did I overlook it somewhere? For example, do we need to set ldap.personLdap.customizePooling=true if we want to experiment with the other dap.personValidate* properties?

 

I'm also subscribed to the incommon-grouper slack channel, but I confess to being a little ambivalent about it. On the one hand, folks there are very helpful and it often seems more responsive than the mailing list. On the other hand, slack is also more ephemeral and requires real-time monitoring and doesn't seem as useful for researching what may have gone before like with a mailing list archive (though I am no sort of power user, so I could well be overlooking some slack feature that would help with these issues).

 

On Fri, Jun 4, 2021 at 1:47 AM Shilen Patel <> wrote:

We talked before about upgrading to the latest ldaptive version, which has its own LDAP implementation.  Or using UnboundID directly.  I think the plan was to change this in 2.6. 

 

By the way, what timeout issue are you having currently?  What pool validator settings do you have?  I use the following (to validate every 5 minutes) to avoid timeout issues.

 

ldap.personLdap.validatePeriodically = true

ldap.personLdap.validateTimerPeriod = 300000

 

Are you on Slack?

 

- Shilen 

 

From: <> on behalf of Baron Fujimoto <>
Reply-To: Baron Fujimoto <>
Date: Thursday, June 3, 2021 at 8:50 PM
To: Grouper Users <>
Subject: [grouper-users] LDAP UnboundID bug? (hardcoded JNDI reference)

 

We were trying to use an UnboundID config to work around an F5/JDK/LDAP timeout bug, but we think we've encountered a bug in Grouper that prevents us from using this workaround.

 

When trying UnboundID, Grouper starts up with no issue, but later when LDAP is used it fails at this line (as determined by a consultants we're working with):

<https://github.com/Internet2/grouper/blob/1b0995ca8e7882a04ae8a4429a7e0cd846f21417/grouper/src/grouper/edu/internet2/middleware/grouper/ldap/ldaptive/LdaptiveSessionImpl.java#L267>

 

The source appears to be hard-coded as a JndiProviderConfig and isn't adaptable to UnboundID providers such as org.ldaptive.provider.unboundid.UnboundIDProvider. The consultants suggest that the Grouper devs would need to build in functionality to support both or drop JNDI support and switch these items to UnboundID.

 

--

Baron Fujimoto <> :: UH Information Technology Services
minutas cantorum, minutas balorum, minutas carboratum desendus pantorum

 

--

Baron Fujimoto <> :: UH Information Technology Services
minutas cantorum, minutas balorum, minutas carboratum desendus pantorum