Grouper Developers Forum

["Redman, Chad" <>]

 

 

Black, Carey M. wrote:

 

> I wonder if the Loader Job is doing the removals of the members before it deletes the group due to having zero members at the end of the job? ( I have not looked at the code. )

 

It's definitely the structure of the Group.delete() method. That method deletes some associated records, then calls GrouperDAOFactory.getFactory().getGroup().delete(). That ends up in Hib3GroupDAO which does some other deletes, before calling byObject.delete(g) as the last step. It's the last step that calls the hook, so plenty of things are happening in an uncommitted transaction before that point.

 

> BTW: You did not report Grouper Version and Patch level. ( How do you expect a developer to help you if you can’t tell them what code you are running!? ??? LOL. 😊 )

 

Fair point! Since this is the dev list, the version is the tip of the master branch :) Originally this was found at our institution's installation which is a few patches behind, but I confirmed with the current code before posting.

 

 

>                 “sync process” ( PSP? PSPNG? Something else? )

>                                 You were not completely clear on these details, but assuming that the sync jobs are operating on a Change Log Consumer based sync process.

>                                 If so, then there is a delay between when the Memberships are removed and when the “events” are processed from those removals. ( Depends on how often/when you > run those jobs. )

 

It's a group list loader, and the like string is set so that any groups that stop being found in the query get deleted.

 

> B) I am guessing that the current DB transaction state is being used (“AKA: not dirty read, but consistent with the current transaction. So the remove members SQL ‘has been processed’ already as far as the Grouper API inside that DB transactions is concerned. “ Which appears to be Chris’s thoughts too.)

 

I think you're both right. I can confirm Chris' solution of a second transaction works to solve this, so I will go with it. I can look at the wiki and see if there is anything I can add. The hook articles are mostly old and in varying states.

 

There is also a Membership hook which might have been more straightforward. But that will get called much more frequently than a group delete hook which will actually not fire very often. So it would some degree of a performance hit.

 

-Chad

 

 

 

 

From: Black, Carey M. [mailto:]
Sent: Thursday, May 30, 2019 11:43 PM
To: Hyzer, Chris <>; Redman, Chad <>;
Subject: RE: How to add a group preDelete hook to check a group's memberships?

 

Chad,

 

By all means do what Chris said. Well, assuming your DB does not support dirty reads. If it does, then I am not sure that would help. ( YMMV )

 

 

The setup:

                Group A is managed by a loader job and it is a member of another group (Group B) that is being synced out to some other destination.

                Sometimes Group A gets all members removed ( by the loader job) and the group is deleted. ( By the loader job)

 

 

I wonder if the Loader Job is doing the removals of the members before it deletes the group due to having zero members at the end of the job? ( I have not looked at the code. )

 

 

 

Not a definitive answer, but some additional thoughts from the peanut gallery. 😊

BTW: You did not report Grouper Version and Patch level. ( How do you expect a developer to help you if you can’t tell them what code you are running!? ??? LOL. 😊 )

 

 

For refence the docs say this:

                https://spaces.at.internet2.edu/display/Grouper/Hooks :

                                “The pre is before the sql statement is sent to the DB, and the post is after, though both are before the SQL commit.”

                                Which apparently is not true. Hum…. Bug? ( in the docs? Or in the code? Hum…)

 

                “sync process” ( PSP? PSPNG? Something else? )

                                You were not completely clear on these details, but assuming that the sync jobs are operating on a Change Log Consumer based sync process.

                                If so, then there is a delay between when the Memberships are removed and when the “events” are processed from those removals. ( Depends on how often/when you run those jobs. )

 

 

The point of interest is really the “SQL Commit” point.  If you can get the attribute removed before the commit then all really should be fine. But that may not be 100% required either.

A)     It may be possible for you to reach into the PIT data as part of the preDelete processing. ( But I am not 100% sure of that. I think it could work, but I am not sure. )

B)      I am guessing that the current DB transaction state is being used (“AKA: not dirty read, but consistent with the current transaction. So the remove members SQL ‘has been processed’ already as far as the Grouper API inside that DB transactions is concerned. “ Which appears to be Chris’s thoughts too.)

 

 

 

And, I said all of that to say this. ( Yes this idea is a race condition for sure, but….)

                Given that the daemon ( via : changeLog.changeLogTempToChangeLog.quartz.cron ) needs to convert the change_log_temp to change_log (real events). Then the “sync process” needs to be triggered ( via… another schedule?? ). There should be some “delay” ( a minute, or two?) between the Membership removal in grouper and the CLC processing the events. I would expect that as long as the attribute is removed from the “sync group” before the “sync runs” that the membership events would be IGNORED because “at sync time” the group of the memberships would not qualify to be processed. Well unless the sync process uses the PIT data and it includes attribute assignments at the time of the delete. ( If it does that, then I think you are not able to do what you are trying to do.)

                MAYBE, if we are talking about PSPNG….

                Maybe you could add the “etc:pspng:do_not_provision_to” attribute to the sync group to get PSPNG to “just stop processing” the group?

 

                So you may even be able to code it as a PostDelete method and “get away with it” before the CLC runs.

 

 

 

 

Chris,

 

RE: “Weird that the hook is called after deleting memberships. “

 

I think something in the design of the system “removes all of the dependences” before the object itself is “deleted”. ( And I think this is part of the issue that Chad is seeing. )

 

 

A recent example with attributes assigned to attributes.

 

Attr1 is assigned to a group (or a stem or whatever).

  Attr2 is assigned to the assignment of Attr1 to the group(or a stem or whatever).

 

                I observed that in the attributeValue hook when Attr1 was deleted by a user.

                                The delete of the meta assignments went through the hooks first.

                                Then the attribute that was deleted came through the hooks.

                I was surprised by that order.

                I was *more* surprise that the user could delete an object indirectly that they did not have permission to delete.

                                And that “illegal delete” was processed  by the preDelete hook before the object they asked to delete too!?

                                I was more hung up on the permissions than the order, but the order did strike me as “not ideal” as well.

 

                I would have much preferred to catch a PreDelete on Attr1 to be able to prevent the delete of Attr2. But that appears to not currently be possible.

 

 

So the observed order in the system is this.

 

User “deletes Attr1”.

 

// value is removed from Attr2

AttributeAssignValueHookPrint.attributeAssignPreDelete() for Attr2

AttributeAssignValueHookPrint.attributeAssignPostDelete() for Attr2

 

// assignment of Attr2 is removed

AttributeAssignHookPrint.attributeAssignPreDelete() for Attr2

AttributeAssignHookPrint.attributeAssignPostDelete() for Attr2

 

//value is removed from Attr1

AttributeAssignValueHookPrint.attributeAssignPreDelete() for Attr1

AttributeAssignValueHookPrint.attributeAssignPostDelete() for Attr1

 

// assignment of Attr1 is removed

AttributeAssignHookPrint.attributeAssignPreDelete() for Attr1

AttributeAssignHookPrint.attributeAssignPostDelete() for Attr1

 

I suspect the same kind of “order issue” is part of Chad’s problem here too.

 

 

 

FWIW: It would be more ideal (IMHO) if the does were correct ( pre before sql to DB) and the order was the following:

(Obviously not how the system is currently working. And it would be a major change to do it this way.)

 

// user fired a “delete attribute assignment” that is where this chain should start.

AttributeAssignHookPrint.attributeAssignPreDelete() for Attr1

// next since there is a value it should be removed

AttributeAssignValueHookPrint.attributeAssignPreDelete() for Attr1

// since there are meta assignments they need removed before the outer attribute assignment

AttributeAssignHookPrint.attributeAssignPreDelete() for Attr2

// if they have values they need removed too

AttributeAssignValueHookPrint.attributeAssignPreDelete() for Attr2

 

// then walk back up the stack

AttributeAssignValueHookPrint.attributeAssignPostDelete() for Attr2

AttributeAssignHookPrint.attributeAssignPostDelete() for Attr2

AttributeAssignValueHookPrint.attributeAssignPostDelete() for Attr1

AttributeAssignHookPrint.attributeAssignPostDelete() for Attr1

 

HTH.

--

Carey Matthew

 

From: <> On Behalf Of Hyzer, Chris
Sent: Thursday, May 30, 2019 3:05 PM
To: Redman, Chad <>;
Subject: RE: [grouper-dev] How to add a group preDelete hook to check a group's memberships?

 

Weird that the hook is called after deleting memberships.  In any case, would this work in there?

 

It opens a new transaction and reads the group 😊

 

    final Group group = null;   ß-- set this to your group, need final var

   

    // the tx type of READ_WRITE_NEW gives you a new tx…   

    Set<Member> members = (Set<Member>)HibernateSession.callbackHibernateSession(GrouperTransactionType.READ_WRITE_NEW, AuditControl.WILL_NOT_AUDIT, new HibernateHandler() {

     

      public Object callback(HibernateHandlerBean hibernateHandlerBean)

          throws GrouperDAOException {

        return group.getMembers();

      }

    });

 

 

From: <> On Behalf Of Redman, Chad
Sent: Thursday, May 30, 2019 12:00 PM
To:
Subject: [grouper-dev] How to add a group preDelete hook to check a group's memberships?

 

I am working on a hook where, if a group is deleted but is a member of another group in a certain folder, delete the sync attribute in that target group before it can sync the emptied out memberships. The source groups are from a loader, so they can go out of scope and get deleted at any point, so it's not normally a manual delete. Maybe there is something built in to do this, but I thought it would work with a hook.

 

I've spent a number of hours trying to track down why a simple membership query works fine to find a group's memberships, all except where it is used in a preDelete hook. After boosting the db trace levels, I can see that the group.delete() is deleting from memberships and group_set (without committing) before calling onPreDelete(). That explains why the membership query looks different within the hook.

 

Any ideas on the approach I should take to this? Maybe this function already exists elsewhere?

 

Thanks

-Chad