Grouper Users - Open Discussion List

[Shaun Koh <>]

Hi Bert (and Sean),

 

Thanks, I believe this is linked to the renaming issue I posted a few weeks ago as it occurs when PSPNG attempts to process the group create event. -- fingers crossed this patch will do the trick

 

In the interim, we’ve configured a trigger in the database to increase the LAST_SEQUENCE_PROCESSED whenever a new error is inserted into the GROUPER_LOADER_LOG for the PSPNG jobs. – not ideal but prevents provisioning blocks

 

Best Regards,

Shaun K.

 

From: [mailto:] On Behalf Of Bee-Lindgren, Bert
Sent: Wednesday, 1 February 2017 7:39 a.m.
To: Sean Mason; Marwan Shaher;
Subject: Re: [grouper-users] PSPNG ChangeLog issue where group is delete, or re-named

 

Thank you for the details. I've created GRP-1479 from them.

 

The problem with the Create-Delete-Recreate-etc processes are that PSPNG is handling a Group-Creation event with whatever information was in Grouper, which is different and insufficient and a bad idea if the group has since been deleted.

 

PSPNG is already "smart" enough to look for deleted in groups in various other event types, but not in the Group-Creation events. I've put the necessary checks into place, and I'll package up a patch after this change passes the unit tests this afternoon.

 

This was somehow easier for me to find than the renaming problems, which I'm continuing to investigate. Of course, it is remotely possible that the patch from GRP-1479 will help that as well.

 

 

 

---

From: <> on behalf of Sean Mason <>
Sent: Tuesday, January 31, 2017 9:24 AM
To: Marwan Shaher;
Subject: RE: [grouper-users] PSPNG ChangeLog issue where group is delete, or re-named

 

I've interpreted things differently, or perhaps am working under a different set of circumstances.  The AD group had not been created before the Grouper group was deleted in my environment.  It appears to me that it never had a chance to be created.  Instead, a Grouper Group was created, a few members added, and then the group was quickly deleted, which created about 5 changelog entries yet to be processed.

The PSPNG provisioner comes along and attempts to provision the new group based on those changelog entries, and continue over the next several changelog entries to add members, and then eventually delete the group.  Except when it attempts to resolve group information from grouper itself in that first "create" changelog entry, null is returned since that group isn't in Grouper anymore, and the JEXL scripts to resolve the LDAP group choke on that 'null' Group that is returned.

For example, you noted the JEXL _expression_: (&(objectclass=group)(cn=${group.extension}))
In my environment ${group.extension} throws a NullPointerException because 'group' is 'null', I assume because it no longer exists since it was deleted before the ChangeLog transaction was attempted.

I'm not certain if the internals of PSPNG should take care of this situation internally, or perhaps there is some configuration or more involved JEXL should handle this particular situation.

-----Original Message-----
From: [] On Behalf Of Marwan Shaher
Sent: Friday, January 27, 2017 6:14 PM
To: ; Sean Mason <>
Subject: Re: [grouper-users] PSPNG ChangeLog issue where group is delete, or re-named

Sean,
I'm trying to troubleshoot a somewhat similar issue in our dev environment. I think I've narrowed it down to the "singleGroupSearchFilter" parameter and how it works with the logic in the edu/internet2/middleware/grouper/pspng/GrouperGroupInfo.java code. I have the singleGroupSearchFilter parameter in grouper-loader.properties set to
(&(objectclass=group)(cn=${group.extension}))
That deletes the groups from AD if they were deleted in the Grouper UI, as long as the groups didn't have any subjects added to them at all.
I've also tried it with
(&(objectclass=group)(cn=${grouperUtil.extensionFromName(name)}))
But, that doesn't seem to delete the groups whether they had subjects added to them at any point in time, or not.

It seems to me though from your log post that the singleGroupSearchFilter parameter is not set to an ldap filter ? If I'm reading it correctly, the parameter is set to "dn:
cn=${grouperUtil.extensionFromName(name)} sAMAccountName:
${grouperUtil.extensionFromName(name)} cn:
${grouperUtil.extensionFromName(name)} objectclass: group " ?

Thanks,

- Marwan






On 01/23/2017 09:28 AM, Sean Mason wrote:
> Hi There,
>
>
>
> I have a pilot instance of Grouper running, with PSPNG provisioning
> group memberships to active directory for an academic department.
>
>
>
> I'm having an issue with the ChangeLog consumer tripping over groups
> that were created, then deleted or renamed.
>
> When the consumer gets to the transaction in the log, the result is:
>
>
>
> 2017-01-23 11:02:04,488: [DefaultQuartzScheduler_Worker-4] INFO
> LdapGroupProvisioner.createGroup(251) -  - Creating LDAP group for
> GrouperGroup: null
>
>
>
> (which makes sense to me, since the group no longer exists).
>
> However, this throws a wrench in the JEXL evaluation:
>
>
>
> 2017-01-23 11:02:04,495: [DefaultQuartzScheduler_Worker-4] ERROR
> Provisioner.evaluateJexlExpression(523) -  - Jexl _expression_ dn:
> cn=${grouperUtil.extensionFromName(name)}
>
> sAMAccountName: ${grouperUtil.extensionFromName(name)}
>
> cn: ${grouperUtil.extensionFromName(name)}
>
> objectclass: group could not be evaluated for subject 'null/null' and
> group 'null/null' which used variableMap
> '{userSearchBaseDn=OU=people,DC=...,
> provisionerType=LdapGroupProvisioner,
> groupCreationBaseDn=OU=Grouper,OU=...,
> ,
> provisionerName=pspng_nexus, groupSearchBaseDn=OU=Grouper,...'
>
> 2017-01-23 11:02:04,497: [DefaultQuartzScheduler_Worker-4] ERROR
> ChangeLogHelper.processRecords(255) -  - Error
>
> java.lang.NullPointerException
>
>         at java.io.StringReader.<init>(StringReader.java:50)
>
>         at
> edu.internet2.middleware.grouper.pspng.LdapGroupProvisioner.createGrou
> p(LdapGroupProvisioner.java:258)
>
>         at
> edu.internet2.middleware.grouper.pspng.LdapGroupProvisioner.createGrou
> p(LdapGroupProvisioner.java:54)
>
>         at
> edu.internet2.middleware.grouper.pspng.Provisioner.provisionItem(Provi
> sioner.java:887)
>
>         at
> edu.internet2.middleware.grouper.pspng.Provisioner.provisionBatchOfIte
> ms(Provisioner.java:1299)
>
>         at
> edu.internet2.middleware.grouper.pspng.PspChangelogConsumerShim.proces
> sChangeLogEntries(PspChangelogConsumerShim.java:71)
>
>         at
> edu.internet2.middleware.grouper.changeLog.ChangeLogHelper.processReco
> rds(ChangeLogHelper.java:245)
>
>         at
> edu.internet2.middleware.grouper.app.loader.GrouperLoaderType$4.runJob
> (GrouperLoaderType.java:629)
>
>         at
> edu.internet2.middleware.grouper.app.loader.GrouperLoaderJob.runJob(Gr
> ouperLoaderJob.java:416)
>
>         at
> edu.internet2.middleware.grouper.app.loader.GrouperLoaderJob.execute(G
> rouperLoaderJob.java:318)
>
>         at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
>
>         at
> org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.ja
> va:573)
>
>
>
> Then:
>
> 2017-01-23 11:02:04,501: [DefaultQuartzScheduler_Worker-4] ERROR
> ChangeLogHelper.processRecords(286) -  - Did not get all the way
> through the batch! -1 != 2884196
>
>
>
> Which puts us in a state where the same transactions are tried again
> and again without moving forward.  I tried setting "retryOnError" to
> false, but that did not appear to change anything.  I suspect that
> setting may only affect "catchable" errors?  In any case, even if we
> get past this transaction, there are a number right after representing
> memberships being added to the group that could also be tried, and
> will likely fail...
>
>
>
> Have I missed a configuration that will allow the PSPNG to skip over
> these types of entries, or at the very least, move on from this error?
>
> I am on the latest patches of both the API, and PSPNG, and willing to
> experiment to get this going.
>
>
>
> Thanks,
>
> Sean.
>