Skip to Content.
Sympa Menu

grouper-users - Re: [Ext] Re: [grouper-users] More PSP-NG: custom user attributes in userSearchFilter

Subject: Grouper Users - Open Discussion List

List archive

Re: [Ext] Re: [grouper-users] More PSP-NG: custom user attributes in userSearchFilter

Chronological Thread 
  • From: "Gettes, Michael" <>
  • To: "Coleman, Erik C" <>
  • Cc: Chris Hyzer <>, Bryan Wooten <>, "" <>
  • Subject: Re: [Ext] Re: [grouper-users] More PSP-NG: custom user attributes in userSearchFilter
  • Date: Tue, 5 Jun 2018 17:12:41 +0000
  • Accept-language: en-US
  • Ironport-phdr: 9a23:K4D9nh9+oNy/Df9uRHKM819IXTAuvvDOBiVQ1KB+0+4XIJqq85mqBkHD//Il1AaPAd2Graocw8Pt8InYEVQa5piAtH1QOLdtbDQizfssogo7HcSeAlf6JvO5JwYzHcBFSUM3tyrjaRsdF8nxfUDdrWOv5jAOBBr/KRB1JuPoEYLOksi7ze+/94HTbglSmDaxfa55IQmrownWqsQYm5ZpJLwryhvOrHtIeuBWyn1tKFmOgRvy5dq+8YB6/ShItP0v68BPUaPhf6QlVrNYFygpM3o05MLwqxbOSxaE62YGXWUXlhpIBBXF7A3/U5zsvCb2qvZx1S+HNsDtU7s6RSqt4LtqSB/wiScIKTg58H3MisdtiK5XuQ+tqwBjz4LRZoyaM+dwfr7GfdMCW2VOQtpRWSJGAoO5dYQPDuwBNvtco4Tyo1YCqB2zDhSuCuzy0D9Fnn/407Mn3eQ9Hw/I3wIgENAJvnTVsNr6O7wfXvqpwKnN1zjDb+9a1DX75YPVch4hu/aMXbdofMbfyEcgCR/FjkmOpoz+Jz+ey+MDs2mH4OpgT+2glWonpw9sqTWoyccjlJXJhp4LxVDf7ih53Yg1JdmiREFlfNGkDYJduieHPIV4RcMiRntnuCc8yrAep5G7eDUKyIgmxx7bcPyLaZOI4hX7WOaeIDd4mHJleK+kiBqo8EigzfXwVtGu0FZNqSpFnMHMtncM1xzV9siLUOdy/kCk2TqX1gDT7uVEIUEylarcKp4h2aQ8mYQOvkTeBiP2mFn2jLOOdkk99OWn8fzoba39pp+BLIB0iB/xPbgymsClAOQ3LxACX2yc+eSz273s41f5TK9UgfIrj6nUto3RK8cDpqOhHgNZzIUu5wyhAzu40tkUh3sKIV1fdB6Zj4XkNEnCLO3kAfuljVmhkS1ny+3CM7DiGJnBM2bPnK/vfblj6ENQ1A8+wNVc6p9QFr0OPPf+VVPqu9PCDhI0NhK4zuT8B9lj044RRHmADrGbPa7UrFSG/PggI/OWa48QoDv9K+Yq5/rpjXIhgl8deLWp3YMNZHClEPRmJ1+VbmTxjdccCWsKvww+Q/L2iFCaTDJfeXW/U7gi6j0lCo+qF4XOS4+3jLCf3Cq2EYVaZmVcBVCNFXfoeZ+EW/AJaC+KLc9ujCAEVaS/RI86zhyuqQH6y759IuXK5yIYqIrv1MJp6O3LiREy6Tt0AtyS02GXSGF0g3sISCEs3KxmvEx90UmM0bJjg/FDEdxT5uhJUhshNZLC1eB6CtbyWh7fcdeTTlapXMmmDS8rQt0v3tAOfhU1J9L3xDvE2yevCrlR35GRBZ982eiUi3X6LO58z3LH0q4glB8rTtYZZkO8gasqvSfeG5LEiAHRsqehda0R2GSFoGiO0XaDp2lFXgo2XKnYCyNMLnDKpMj0sxuRB4SlDq4qZ04ckMM=

FWIW - something like

docker container cp /dev/null test-compose_daemon_1:/opt/grouper/grouper.apiBinary/conf/sources.xml

Has the same effect of nullifying the sources.xml - with the TIER grouper, at the moment, amq jar files may also be getting in the way and nullifying the amq jar (see your error message) also fixes that problem.

The TIER team is working with the grouper devs to fix a problem related to not including the amq jar since the TIER reference implementation is for rabbitmq this would fix other problems related to PSPNG in the TIER grouper package.

To everyone else - this all may seem like gibberish.  In the end, as we work through these TIER grouper packaging issues - there will be a capability to deploy grouper demonstrating how easy it is to get things set up to demonstrate and use.  This work will benefit everyone in the long run.  It’s good stuff - the grouper devs and the TIER teams are doing some great work behind the scenes.

I hope this helps.


On Jun 5, 2018, at 12:40 PM, Coleman, Erik C <> wrote:

OK I just figured this out.  There appears to be some sort of conflict with and sources.xml within the TIER Grouper package that basically made my subject sources disappear from the loader’s perspective. I had this problem with the UI implementation as well.  If sources.xml exists, it appears that my subject sources defined in do not get read in, and I get a warning (not an error) of them both existing. I scoured the logs and could not find the subject sources being configured.
My solution for now is to add a Dockerfile RUN command to delete the sources.xml file from the image at build time. Once deleted, the daemon is seeing the subject sources and can resolve the subjects, and thus I am not getting the JEXL errors anymore.
From:  <> On Behalf Of Coleman, Erik C
Sent: Tuesday, June 5, 2018 9:48 AM
To: Hyzer, Chris <>; Bryan Wooten <>; Gettes, Michael <>
Subject: RE: [Ext] Re: [grouper-users] More PSP-NG: custom user attributes in userSearchFilter
Here is my sanitized configuration:
ldap.uofildap.ldapUrl = ldap://
ldap.uofildap.user = authman-subject-bind
ldap.uofildap.bindDn = CN=authman-subject-bind,OU=CITES-IDM service accounts,OU=CITES-IDM Service Access,OU=CITES-IDM,OU=CITES-Services,OU=CITES,OU=Urbana,DC=ad,DC=uillinois,DC=edu
ldap.uofildap.pass = **************
ldap.uofildap.bindCredential = *****************
#optional, if you are using tls, set this to true.  Generally you will not be using an SSL URL to use TLS...
ldap.uofildap.tls = true
# PSP-NG Configuration
changeLog.psp.fullSync.quartzCron = 0 4 * * * ?
changeLog.psp.fullSync.runAtStartup = true
changeLog.psp.fullSync.class = edu.internet2.middleware.grouper.pspng.FullSyncStarter
changeLog.consumer.uofi_urbana.class = edu.internet2.middleware.grouper.pspng.PspChangelogConsumerShim
changeLog.consumer.uofi_urbana.type = edu.internet2.middleware.grouper.pspng.LdapGroupProvisioner
changeLog.consumer.uofi_urbana.quartzCron = 0 * * * * ?
changeLog.consumer.uofi_urbana.ldapPoolName = uofildap
changeLog.consumer.uofi_urbana.isActiveDirectory = true
changeLog.consumer.uofi_urbana.memberAttributeName = member
changeLog.consumer.uofi_urbana.memberAttributeValueFormat = ${ldapUser.getDn()}
changeLog.consumer.uofi_urbana.groupSearchBaseDn = ou=AuthMan,ou=CITES-Services,ou=CITES,ou=Urbana,dc=ad,dc=uillinois,dc=edu
changeLog.consumer.uofi_urbana.allGroupsSearchFilter = objectclass=group
changeLog.consumer.uofi_urbana.singleGroupSearchFilter = (&(objectclass=group)(cn=${}))
changeLog.consumer.uofi_urbana.groupCreationLdifTemplate = dn: cn=${}||cn: ${}||objectclass: group
changeLog.consumer.uofi_urbana.groupCreationBaseDn = ou=AuthMan,ou=CITES-Services,ou=CITES,ou=Urbana,dc=ad,dc=uillinois,dc=edu
changeLog.consumer.uofi_urbana.needsTargetSystemUsers = true
changeLog.consumer.uofi_urbana.userSearchBaseDn = ou=people,dc=ad,dc=uillinois,dc=edu
changeLog.consumer.uofi_urbana.userSearchFilter = samAccountName=${subject.getAttributeValue('samaccountname')}
changeLog.consumer.uofi_urbana.userSearchAttributes = cn,distinguishedName,uiucEduUIN,displayName,samAccountName,objectclass
Thanks for any tips!
From: Hyzer, Chris <> 
Sent: Tuesday, June 5, 2018 8:45 AM
To: Coleman, Erik C <>; Bryan Wooten <>; Gettes, Michael <>
Subject: RE: [Ext] Re: [grouper-users] More PSP-NG: custom user attributes in userSearchFilter
Did you post your sanitized config for the pspng?
From:  [] On Behalf Of Coleman, Erik C
Sent: Monday, June 04, 2018 11:56 PM
To: Hyzer, Chris <>; Bryan Wooten <>; Gettes, Michael <>
Subject: RE: [Ext] Re: [grouper-users] More PSP-NG: custom user attributes in userSearchFilter
To answer Bryan’s first question, I am using the LdapGroupProvisioner, as this is exclusively creating groups.
I am still stuck, I tried a variety of JEXL syntax and it still won’t evaluate samAccountName. I couldn’t find any notes that said we set something up special before.  I even looked up the various methods and classes here: and they seem to be valid.
Chris, are you inferring that I need to define a hash map or something to pre-define the attribute so that it can be used by PSPNG? I’m still not clear what classes or methods are valid in the context of, and the pspng provisioner in particular.
From: Hyzer, Chris <> 
Sent: Monday, June 4, 2018 9:39 PM
To: Bryan Wooten <>; Gettes, Michael <>
Cc: ; Coleman, Erik C <>
Subject: RE: [Ext] Re: [grouper-users] More PSP-NG: custom user attributes in userSearchFilter
I added this to the GSH wiki… if someone has the variables and types for the pspng to make this more realistic please let us know, or maybe add to pspng wiki…

_expression_ language testing


Set this in = DEBUG
Run GSH:
gsh 0% GrouperSession grouperSession = GrouperSession.startRootSession();
gsh 1% Group group = GroupFinder.findByName(grouperSession, "apps:loader");
gsh 2% Map variableMap =  new HashMap();
gsh 3% variableMap.put("theGroup", group);
gsh 4% String result = GrouperUtil.substituteExpressionLanguage("Name: ${}", variableMap);
gsh 5% result
Name: apps:loader
This is the log entry:
2018-06-04 22:32:58,197: [main] DEBUG GrouperUtil.substituteExpressionLanguage(9416) -  - Subsituting EL: 'Name: ${}', and with env vars: theGroup, grouperUtil with result: 'Name: apps:loader'



From:  [] On Behalf Of Bryan Wooten
Sent: Monday, June 04, 2018 9:18 PM
To: Gettes, Michael <>; Bryan Wooten <>
Cc: ; 
Subject: Re: [Ext] Re: [grouper-users] More PSP-NG: custom user attributes in userSearchFilter
I am a recovering Java programmer. I swear if I never see a null pointer exception again, I will die a happy man…
But I can still backtrace a Java stack dump in a log file for a clue.
I tried to read the Jexl docs from Apache and Grouper and was immediately lost…
I would ask readers to share their Jexl strings…
So I turned to my tried and true Java IDE (Netbeans / shoot me / not Eclipse or Jetbrains) and did some simple Java string parsing. I may have included some grouper jars files and some Apache Jexl jars…
Garbage in / Garbage out stuff.
“Co-programming” with a very competent co-worker (AD expert and knows more than enough about Linux/ shell).
Honestly our biggest set-back was that Loader Diagnostics for LDAP quit working (SQL works fine)? On our to-do / figure out list. I made a post re: this issue to no avail. :(
BTW, if anyone is interested, we figured out how to do a SQL loader job against a CSV file (don’t ask why data feeds are still so out-dated).
From: "Gettes, Michael" <>
Date: Monday, June 4, 2018 at 6:42 PM
To: Bryan Wooten <>
Cc: "" <>, "" <>
Subject: Re: [Ext] Re: [grouper-users] More PSP-NG: custom user attributes in userSearchFilter
Ok, this makes more sense.  I read your reply a couple of days ago and I just didn’t get it.
Maybe you can answer the next question I had - I need to write some jexl that will alter variables.  How would I test this without actually invoking grouper loader and PSPNG?  Seems awfully expensive to try something and then fire up the loader to see if it works.  I’d like to construct some variable values and then run the jexl over it to see if it was modified correctly.  I’m not a Java geek so this may be easy but I just need the magical incantation to make it happen.


On Jun 4, 2018, at 8:13 PM, Bryan Wooten <> wrote:
Last week my co-work and I spent most of a day trying to figure out Jexl.
The wiki examples were trivial. But we had a use case to load AD groups into Grouper keeping the AD hierarchy. (cn=group1, ou=groups, ou=dept1, ou=majorOrg,dc=school,dc=edu) where all the ou’s become stems/folders and the cn is the group name).
Using a basic LDAP filter like (&(objectclass=group)(cn=hospital*))
So the wiki provided no complete list of methods available to the Java class LoaderLdap.ElUtils.
So we ended searching for the source (LoaderLdap.ElUtils) on Git. And found the exact method needed. It performed as expected! Victory.
Grouper can be made very functional via configuration only, if you know the subtleties of the syntax and relations to the various property files.
I know submissions to the wiki are encouraged, but sometimes I feel a grouper-documentation list could be a benefit. But I have had bad ideas before.
From: <> on behalf of "Gettes, Michael" <>
Date: Monday, June 4, 2018 at 3:46 PM
To: Paul Engle <>
Cc: "" <>, "" <>
Subject: [Ext] Re: [grouper-users] More PSP-NG: custom user attributes in userSearchFilter
WARNING: Stop. Think. Read. This is an external email.

+1 - this is part of where the documentation is lacking a little bit and what led to the questions I had about JEXL in grouper and how it is used.


On Jun 4, 2018, at 5:16 PM, Paul Engle <> wrote:
  Is this in an LdapGroupProvisioner or an LdapAttributeProvisioner? In the group provisioner, anytime I've needed something from the user entry, I've referenced it as ${ldapUser.getStringValue('foo')}.  I've only used the ${subject} reference in the attribute provisioner. I don't know if that makes a difference or not, or under what contexts each of those objects is available.
Paul Engle
Office of Information Technology
On Mon, Jun 4, 2018 at 3:44 PM Coleman, Erik C <> wrote:
As a follow-up to my two subject sources scenario, I’m having trouble with custom subject attributes. I now have PSP-NG turned on, and it successfully syncs groups that are set with the provision_to attribute, but when it comes time to add members, I’m getting this error in the logs:
grouper-api;grouper_error.log;2018-06-04 19:36:59,091: [uofi_urbana-FullSync-Thread] ERROR Provisioner.evaluateJexlExpression(556) - - Jexl _expression_ samAccountName=${subject.getAttributeValue("samAccountName")} could not be evaluated for subject ''650000001'/'person'/'uofinetid'/null' and group 'null/null' which used variableMap '{userSearchBaseDn=ou=people,dc=ad,dc=uillinois,dc=edu, provisionerType=LdapGroupProvisioner, groupCreationBaseDn=ou=AuthMan,ou=CITES-Services,ou=CITES,ou=Urbana,dc=ad,dc=uillinois,dc=edu, , subject='650000001'/'person'/'uofinetid', provisionerName=uofi_urbana, groupSearchBaseDn=ou=AuthMan,ou=CITES-Services,ou=CITES,ou=Urbana,dc=ad,dc=uillinois,dc=edu}'
Here’s what I have set in
changeLog.consumer.uofi_urbana.userSearchFilter = samAccountName=${subject.getAttributeValue("samAccountName")}
This used to work in our older “pre-TIER” test environment. I’ve considered alternatives:
samAccountName=${}  -- won’t work because we specify subject id using our own uiucEduUIN attribute as specified in the
uiucEduUIN=${} – won’t work because one of my subject sources doesn’t use uiucEduUIN for subject id, it uses samAccountName.
Switching to use samAccountName for subject id won’t work, because we have too many people changing netids quite often.
A look at my personal subject record in Grouper appears to have everything it’s trying to evaluate:
Unique ID:
Coleman, Erik C
Coleman, Erik C
uiuceduuin:        650000001
dn:        CN=ecc,ou=People,dc=ad,dc=uillinois,dc=edu
displayname:     Coleman, Erik C
department:      Technology Services
distinguishedname:              CN=ecc,OU=People,DC=ad,DC=uillinois,DC=edu
samaccountname:          ecc
Member ID:       6fe2e751a0e14e41b896ee6cb8e23e02
Source ID:          uofinetid
Source name:    UOFI AD People
Why is the JEXL _expression_ getting an error? What’s the right JEXL syntax for referencing a “custom” attribute in a subject?
Erik Coleman
University of Illinois at Urbana-Champaign

Archive powered by MHonArc 2.6.19.

Top of Page