Grouper Users - Open Discussion List

["Gettes, Michael" <>]

FWIW - something like

docker container cp /dev/null test-compose_daemon_1:/opt/grouper/grouper.apiBinary/conf/sources.xml

Has the same effect of nullifying the sources.xml - with the TIER grouper, at the moment, amq jar files may also be getting in the way and nullifying the amq jar (see your error message) also fixes that problem.

The TIER team is working with the grouper devs to fix a problem related to not including the amq jar since the TIER reference implementation is for rabbitmq this would fix other problems related to PSPNG in the TIER grouper package.

To everyone else - this all may seem like gibberish.  In the end, as we work through these TIER grouper packaging issues - there will be a capability to deploy grouper demonstrating how easy it is to get things set up to demonstrate and use.  This work will benefit everyone in the long run.  It’s good stuff - the grouper devs and the TIER teams are doing some great work behind the scenes.

I hope this helps.

/mrg

On Jun 5, 2018, at 12:40 PM, Coleman, Erik C <> wrote:

OK I just figured this out.  There appears to be some sort of conflict with subject.properties and sources.xml within the TIER Grouper package that basically made my subject sources disappear from the loader’s perspective. I had this problem with the UI implementation as well.  If sources.xml exists, it appears that my subject sources defined in subject.properties do not get read in, and I get a warning (not an error) of them both existing. I scoured the logs and could not find the subject sources being configured.

 

My solution for now is to add a Dockerfile RUN command to delete the sources.xml file from the image at build time. Once deleted, the daemon is seeing the subject sources and can resolve the subjects, and thus I am not getting the JEXL errors anymore.

 

-Erik

 

 

 

From:  <> On Behalf Of Coleman, Erik C
Sent: Tuesday, June 5, 2018 9:48 AM
To: Hyzer, Chris <>; Bryan Wooten <>; Gettes, Michael <>
Cc: 
Subject: RE: [Ext] Re: [grouper-users] More PSP-NG: custom user attributes in userSearchFilter

 

Here is my sanitized configuration:

 

ldap.uofildap.url = "ldap://ourldapurl.illinois.edu:389/dc=ad,dc=uillinois,dc=edu

ldap.uofildap.ldapUrl = ldap://ourldapurl.illinois.edu:389

 

ldap.uofildap.user = authman-subject-bind

ldap.uofildap.bindDn = CN=authman-subject-bind,OU=CITES-IDM service accounts,OU=CITES-IDM Service Access,OU=CITES-IDM,OU=CITES-Services,OU=CITES,OU=Urbana,DC=ad,DC=uillinois,DC=edu

 

ldap.uofildap.pass = **************

ldap.uofildap.bindCredential = *****************

 

#optional, if you are using tls, set this to true.  Generally you will not be using an SSL URL to use TLS...

ldap.uofildap.tls = true

 

#####################################

# PSP-NG Configuration

#####################################

changeLog.psp.fullSync.quartzCron = 0 4 * * * ?

changeLog.psp.fullSync.runAtStartup = true

changeLog.psp.fullSync.class = edu.internet2.middleware.grouper.pspng.FullSyncStarter

 

changeLog.consumer.uofi_urbana.class = edu.internet2.middleware.grouper.pspng.PspChangelogConsumerShim

changeLog.consumer.uofi_urbana.type = edu.internet2.middleware.grouper.pspng.LdapGroupProvisioner

changeLog.consumer.uofi_urbana.quartzCron = 0 * * * * ?

changeLog.consumer.uofi_urbana.ldapPoolName = uofildap

changeLog.consumer.uofi_urbana.isActiveDirectory = true

changeLog.consumer.uofi_urbana.memberAttributeName = member

changeLog.consumer.uofi_urbana.memberAttributeValueFormat = ${ldapUser.getDn()}

changeLog.consumer.uofi_urbana.groupSearchBaseDn = ou=AuthMan,ou=CITES-Services,ou=CITES,ou=Urbana,dc=ad,dc=uillinois,dc=edu

changeLog.consumer.uofi_urbana.allGroupsSearchFilter = objectclass=group

changeLog.consumer.uofi_urbana.singleGroupSearchFilter = (&(objectclass=group)(cn=${group.name}))

changeLog.consumer.uofi_urbana.groupCreationLdifTemplate = dn: cn=${group.name}||cn: ${group.name}||objectclass: group

changeLog.consumer.uofi_urbana.groupCreationBaseDn = ou=AuthMan,ou=CITES-Services,ou=CITES,ou=Urbana,dc=ad,dc=uillinois,dc=edu

changeLog.consumer.uofi_urbana.needsTargetSystemUsers = true

changeLog.consumer.uofi_urbana.userSearchBaseDn = ou=people,dc=ad,dc=uillinois,dc=edu

changeLog.consumer.uofi_urbana.userSearchFilter = samAccountName=${subject.getAttributeValue('samaccountname')}

changeLog.consumer.uofi_urbana.userSearchAttributes = cn,distinguishedName,uiucEduUIN,displayName,samAccountName,objectclass

 

Thanks for any tips!

 

-Erik

 

 

From: Hyzer, Chris <> 
Sent: Tuesday, June 5, 2018 8:45 AM
To: Coleman, Erik C <>; Bryan Wooten <>; Gettes, Michael <>
Cc: 
Subject: RE: [Ext] Re: [grouper-users] More PSP-NG: custom user attributes in userSearchFilter

 

Did you post your sanitized config for the pspng?

 

From:  [] On Behalf Of Coleman, Erik C
Sent: Monday, June 04, 2018 11:56 PM
To: Hyzer, Chris <>; Bryan Wooten <>; Gettes, Michael <>
Cc: 
Subject: RE: [Ext] Re: [grouper-users] More PSP-NG: custom user attributes in userSearchFilter

 

To answer Bryan’s first question, I am using the LdapGroupProvisioner, as this is exclusively creating groups.

 

I am still stuck, I tried a variety of JEXL syntax and it still won’t evaluate samAccountName. I couldn’t find any notes that said we set something up special before.  I even looked up the various methods and classes here: https://internet2.github.io/grouper/master/grouper-parent/apidocs/ and they seem to be valid.

 

Chris, are you inferring that I need to define a hash map or something to pre-define the attribute so that it can be used by PSPNG? I’m still not clear what classes or methods are valid in the context of grouper-loader.properties, and the pspng provisioner in particular.

 

-Erik

 

 

From: Hyzer, Chris <> 
Sent: Monday, June 4, 2018 9:39 PM
To: Bryan Wooten <>; Gettes, Michael <>
Cc: ; Coleman, Erik C <>
Subject: RE: [Ext] Re: [grouper-users] More PSP-NG: custom user attributes in userSearchFilter

 

I added this to the GSH wiki… if someone has the variables and types for the pspng to make this more realistic please let us know, or maybe add to pspng wiki…

_expression_ language testing

 

Set this in log4j.properties

 

 

log4j.logger.edu.internet2.middleware.grouper.util.GrouperUtil = DEBUG

 

 

Run GSH:

 

 

gsh 0% GrouperSession grouperSession = GrouperSession.startRootSession();

gsh 1% Group group = GroupFinder.findByName(grouperSession, "apps:loader");

gsh 2% Map variableMap =  new HashMap();

gsh 3% variableMap.put("theGroup", group);

gsh 4% String result = GrouperUtil.substituteExpressionLanguage("Name: ${theGroup.name}", variableMap);

gsh 5% result

Name: apps:loader

 

This is the log entry:

 

2018-06-04 22:32:58,197: [main] DEBUG GrouperUtil.substituteExpressionLanguage(9416) -  - Subsituting EL: 'Name: ${theGroup.name}', and with env vars: theGroup, grouperUtil with result: 'Name: apps:loader'

 

 

 

 

From:  [] On Behalf Of Bryan Wooten
Sent: Monday, June 04, 2018 9:18 PM
To: Gettes, Michael <>; Bryan Wooten <>
Cc: ; 
Subject: Re: [Ext] Re: [grouper-users] More PSP-NG: custom user attributes in userSearchFilter

 

I am a recovering Java programmer. I swear if I never see a null pointer exception again, I will die a happy man…

 

But I can still backtrace a Java stack dump in a log file for a clue.

 

I tried to read the Jexl docs from Apache and Grouper and was immediately lost…

 

I would ask readers to share their Jexl strings…

 

So I turned to my tried and true Java IDE (Netbeans / shoot me / not Eclipse or Jetbrains) and did some simple Java string parsing. I may have included some grouper jars files and some Apache Jexl jars…

 

Garbage in / Garbage out stuff.

 

“Co-programming” with a very competent co-worker (AD expert and knows more than enough about Linux/ shell).

 

Honestly our biggest set-back was that Loader Diagnostics for LDAP quit working (SQL works fine)? On our to-do / figure out list. I made a post re: this issue to no avail. :(

 

BTW, if anyone is interested, we figured out how to do a SQL loader job against a CSV file (don’t ask why data feeds are still so out-dated).

 

-Bryan

 

From: "Gettes, Michael" <>
Date: Monday, June 4, 2018 at 6:42 PM
To: Bryan Wooten <>
Cc: "" <>, "" <>
Subject: Re: [Ext] Re: [grouper-users] More PSP-NG: custom user attributes in userSearchFilter

 

Ok, this makes more sense.  I read your reply a couple of days ago and I just didn’t get it.

 

Maybe you can answer the next question I had - I need to write some jexl that will alter variables.  How would I test this without actually invoking grouper loader and PSPNG?  Seems awfully expensive to try something and then fire up the loader to see if it works.  I’d like to construct some variable values and then run the jexl over it to see if it was modified correctly.  I’m not a Java geek so this may be easy but I just need the magical incantation to make it happen.

 

Ideas?

 

Thanks!

 

/mrg

 

On Jun 4, 2018, at 8:13 PM, Bryan Wooten <> wrote:

 

Last week my co-work and I spent most of a day trying to figure out Jexl.

 

The wiki examples were trivial. But we had a use case to load AD groups into Grouper keeping the AD hierarchy. (cn=group1, ou=groups, ou=dept1, ou=majorOrg,dc=school,dc=edu) where all the ou’s become stems/folders and the cn is the group name).

 

Using a basic LDAP filter like (&(objectclass=group)(cn=hospital*))

 

So the wiki provided no complete list of methods available to the Java class LoaderLdap.ElUtils.

 

So we ended searching for the source (LoaderLdap.ElUtils) on Git. And found the exact method needed. It performed as expected! Victory.

 

Grouper can be made very functional via configuration only, if you know the subtleties of the syntax and relations to the various property files.

 

I know submissions to the wiki are encouraged, but sometimes I feel a grouper-documentation list could be a benefit. But I have had bad ideas before.

 

-Bryan

 

From: <> on behalf of "Gettes, Michael" <>
Date: Monday, June 4, 2018 at 3:46 PM
To: Paul Engle <>
Cc: "" <>, "" <>
Subject: [Ext] Re: [grouper-users] More PSP-NG: custom user attributes in userSearchFilter

 

******************************************************
WARNING: Stop. Think. Read. This is an external email.
******************************************************

+1 - this is part of where the documentation is lacking a little bit and what led to the questions I had about JEXL in grouper and how it is used.

 

/mrg

 

On Jun 4, 2018, at 5:16 PM, Paul Engle <> wrote:

 

Erik,

  Is this in an LdapGroupProvisioner or an LdapAttributeProvisioner? In the group provisioner, anytime I've needed something from the user entry, I've referenced it as ${ldapUser.getStringValue('foo')}.  I've only used the ${subject} reference in the attribute provisioner. I don't know if that makes a difference or not, or under what contexts each of those objects is available.

 

  -paul

 

-- 

Paul Engle

Office of Information Technology

713-348-4702

 

 

On Mon, Jun 4, 2018 at 3:44 PM Coleman, Erik C <> wrote:

Hello,

 

As a follow-up to my two subject sources scenario, I’m having trouble with custom subject attributes. I now have PSP-NG turned on, and it successfully syncs groups that are set with the provision_to attribute, but when it comes time to add members, I’m getting this error in the logs:

 

grouper-api;grouper_error.log;2018-06-04 19:36:59,091: [uofi_urbana-FullSync-Thread] ERROR Provisioner.evaluateJexlExpression(556) - - Jexl _expression_ samAccountName=${subject.getAttributeValue("samAccountName")} could not be evaluated for subject ''650000001'/'person'/'uofinetid'/null' and group 'null/null' which used variableMap '{userSearchBaseDn=ou=people,dc=ad,dc=uillinois,dc=edu, provisionerType=LdapGroupProvisioner, groupCreationBaseDn=ou=AuthMan,ou=CITES-Services,ou=CITES,ou=Urbana,dc=ad,dc=uillinois,dc=edu, , subject='650000001'/'person'/'uofinetid', provisionerName=uofi_urbana, groupSearchBaseDn=ou=AuthMan,ou=CITES-Services,ou=CITES,ou=Urbana,dc=ad,dc=uillinois,dc=edu}'

 

Here’s what I have set in grouper-loader.properties:

 

changeLog.consumer.uofi_urbana.userSearchFilter = samAccountName=${subject.getAttributeValue("samAccountName")}

 

This used to work in our older “pre-TIER” test environment. I’ve considered alternatives:

 

samAccountName=${subject.id}  -- won’t work because we specify subject id using our own uiucEduUIN attribute as specified in the subject.properties.

 

uiucEduUIN=${subject.id} – won’t work because one of my subject sources doesn’t use uiucEduUIN for subject id, it uses samAccountName.

 

Switching to use samAccountName for subject id won’t work, because we have too many people changing netids quite often.

 

A look at my personal subject record in Grouper appears to have everything it’s trying to evaluate:

 

Unique ID:

650000001

 

Name:

Coleman, Erik C

 

Description:

Coleman, Erik C

 

uiuceduuin:        650000001

dn:        CN=ecc,ou=People,dc=ad,dc=uillinois,dc=edu

displayname:     Coleman, Erik C

department:      Technology Services

distinguishedname:              CN=ecc,OU=People,DC=ad,DC=uillinois,DC=edu

samaccountname:          ecc

Member ID:       6fe2e751a0e14e41b896ee6cb8e23e02

Source ID:          uofinetid

Source name:    UOFI AD People

 

Why is the JEXL _expression_ getting an error? What’s the right JEXL syntax for referencing a “custom” attribute in a subject?

 

Thanks!

 

Erik Coleman

University of Illinois at Urbana-Champaign