Grouper Users - Open Discussion List

[Akki Kumar <>]

Hi Shilen,

Thank you for the quick reply.

1) Just to clarify, did you run /tmp/grouperInstaller/grouper.apiBinary-2.2.2/bin/findbadmemberships.gsh ? 

A) No, I ran the Rollup Hierarchy loader first, and then findBadMemberships() command. 

2) If so, are you saying the problem is still there even before you run the loader again? 

A) The problem shows up after I ran the Rollup Hierarchy loader.

3) And if you run findBadMemberships() again, what does it say?

A) GSH shows below message.

gsh 27%

gsh 28% findBadMemberships();

Checking Composite Memberships

Checking Group Sets

Incomplete group set hierarchy (GSH script will attempt to delete and recreate it): owner groupId=a722042adfca4e74b645334c338c1a01, owner group name=College:Departments:Information Tech:Networking Dept - All, member groupId=8698f36071c74c45a6c849497f532f25, member group name=College:Departments:Information Tech:Networking Dept:Networking Apps.

Incomplete group set hierarchy (GSH script will attempt to delete and recreate it): owner groupId=8fc21aad397342f9ae7d68d7fdefbfe3, owner group name=College:Departments:Information Tech:Networking Dept - All, member groupId=be3d9a00601244c182396cbe1edaeff9, member group name=College:Departments:Information Tech:Networking Dept:Networking Research.

Checking memberships where member is a deleted group

Checking GrouperAll

Writing file: /tmp/grouperInstaller/grouper.apiBinary-2.2.2/bin/findbadmemberships.gsh

findBadMemberships completed successfully

gsh 29%

gsh 30%

Questions:

1) I noticed, some sub-group members shows up in the grand-parent groups(A->C), but some members do not.  I'm guessing something to do with Incomplete group set hierarchy (as per above logs). What does incomplete group set hierarchy mean?

2) Is there a way to run loader sequentially, first sub-group (B->C) and then parent groups (A->B)?

"I'm guessing what's happening is that you're running loader jobs in multiple threads and two different threads are adding the two immediate group sets at about the same time in different database transactions and therefore don't realize that an effective one should be created."

Thank you,

Akki

On Mon, Feb 20, 2017 at 10:28 AM, Shilen Patel <> wrote:

Just to clarify, did you run /tmp/grouperInstaller/grouper.apiBinary-2.2.2/bin/findbadmemberships.gsh ?  If so, are you saying the problem is still there even before you run the loader again?  And if you run findBadMemberships() again, what does it say?

Regarding group sets, say you have 3 groups – A, B, C.  B is a member of A and C is a member of B.  The group set table would hold the immediate relationships (A->B and B->C) but it would also hold the effective relationship (A->C).  That's done so it can compute effective memberships quickly.

I'm guessing what's happening is that you're running loader jobs in multiple threads and two different threads are adding the two immediate group sets at about the same time in different database transactions and therefore don't realize that an effective one should be created.

Thanks!

- Shilen

From: Akki Kumar <>
Date: Monday, February 20, 2017 at 10:19 AM
To: Shilen Patel <>
Cc: "Hyzer, Chris" <>, Stephen A Sazama <>, "" <>

Subject: Re: [grouper-users] Re: Hierarchical Loader Bug

Thank you for the prompt reply Shilen! When I ran findBadMemberships() command, gsh showed below logs. Even after deleting and re-adding members (by gsh), users still didn't show up in the group (College:Departments:Information Tech:Networking Dept - All). What does incomplete group set hierarchy mean?

gsh 27%

gsh 28% findBadMemberships();

Checking Composite Memberships

Checking Group Sets

Incomplete group set hierarchy (GSH script will attempt to delete and recreate it): owner groupId=a722042adfca4e74b645334c338c1a01, owner group name=College:Departments:Information Tech:Networking Dept - All, member groupId=8698f36071c74c45a6c849497f532f25, member group name=College:Departments:Information Tech:Networking Dept:Networking Apps.

Incomplete group set hierarchy (GSH script will attempt to delete and recreate it): owner groupId=8fc21aad397342f9ae7d68d7fdefbfe3, owner group name=College:Departments:Information Tech:Networking Dept - All, member groupId=be3d9a00601244c182396cbe1edaeff9, member group name=College:Departments:Information Tech:Networking Dept:Networking Research.

Checking memberships where member is a deleted group

Checking GrouperAll

Writing file: /tmp/grouperInstaller/grouper.apiBinary-2.2.2/bin/findbadmemberships.gsh

findBadMemberships completed successfully

gsh 29%

gsh 30%

Thank you,

Akki

On Fri, Feb 17, 2017 at 11:10 AM, Shilen Patel <> wrote:

When you have this problem, what does this output (again from GSH)?

gsh 1% findBadMemberships()

Thanks!

- Shilen

From: <Hyzer>, Chris <>
Date: Tuesday, February 14, 2017 at 1:06 AM
To: Stephen A Sazama <>, "" <>
Cc: Akki Kumar <>
Subject: RE: [grouper-users] Re: Hierarchical Loader Bug

When you have the problem, does running this from GSH help?

 

new edu.internet2.middleware.grouper.misc.AddMissingGroupSets().addAllMissingGroupSets();

 

Shilen, you have other ideas?

 

Thanks

Chris

 

From: Stephen A Sazama []
Sent: Monday, February 13, 2017 4:39 PM
To: Hyzer, Chris <>;
Cc: Akki Kumar <>
Subject: Re: [grouper-users] Re: Hierarchical Loader Bug

 

Hi,

 

I work with Akki. What we're trying to do with the second loader is assign all sub-department groups as members of their parent departments (creating rollup groups). So if Group A is the parent department of Group B, and Group B is the parent department of Group C, then Group A has one direct member (B) and B has one direct member (C). They also have some other members that are people from LDAP. The behavior we're seeing is that in this scenario:

- Group B shows up as a direct member of A (correct).

- Group B's people members show up as indirect members of A (correct).

- Group C shows up as an indirect member of A (correct).

- Group C's people members do NOT show up as members of A (unexpected).

 

Departmental groups with a depth of 2 or more do not have their LDAP subjects added as indirect members of the top-level parent department, even though the group containing them is an indirect member. We were wondering if anyone else has seen this behavior. Are there any cases in Grouper where a group's members are not supposed to be inherited by groups of which it is a member, or any kind of caching we should look out for? Apparently, if we remove the membership linking A to B/C and then re-run the loader, the members are all added correctly.

 

Thanks again,

Stephen Sazama

 

On Fri, Feb 10, 2017 at 10:53 AM, Akki Kumar <> wrote:

Hi Chris,

 

I have attached below screenshots (https://drive.google.com/drive/folders/0BwgGnZC7vA-6eFVUMThjUzNWc0k):

• Departmental Loader - Configuration
• Rollup Loader - Configuration
• Departmental.jpg – Showing folders and group’s structure.
• Hierarchy 1.jpg – Showing Networking Apps group with one member added to it.
• Hierarchy 2.jpg – Showing Networking Research group with one member added to it.
• Hierarchy 3.jpg – Showing Networking Dept - All group is linked directly to the subgroups (Networking Apps & Networking Research) and members are added indirectly to it.
• Hierarchy 4.jpg – Showing Information Tech - All group is linked directly to the subgroups (Networking Dept - All) and members are not added to the group (BUG).

 

Issue: As shown in the Hierarchy 4.jpg screenshot, in the top level group (Information Tech – All), the subgroups are linked, but the members from the subgroups are not shown (after loader run). However, after manually deleting the Networking Dept – All group from the top level group, and re-running the loader, the members are shown in the top level group (Refer to screenshot Manually Delete.jpg). Not sure why grouper loader only links all members after manually deleting the group from the top level. Any guidance is truly appreciated.

 

Please let me know if you need more details.

 

Thank you,

Akki

 

On Thu, Feb 9, 2017 at 4:06 PM, Hyzer, Chris <> wrote:

I understand you have a problem but I cant really understand what it is by your email.  Can you send along the loader configuration and be a little more explicit with screenshots etc?

 

Thanks

Chris

 

 

From: Akki Kumar [mailto:]
Sent: Thursday, February 09, 2017 3:21 PM
To: Hyzer, Chris <>;
Subject: Hierarchical Loader Bug

 

Hi Chris,

 

I created Organization Hierarchical groups using two loaders:

• Departmental Hierarchy
• Rollup Hierarchy

 

When I ran the Rollup Hierarchy loader, and it link group as described below (screenshot):

1)  All groups under Networking Dept folder are linked to the Networking Dept – All group, and it shows all linked group members under the Networking Dept – All. (Expected Behavior)

2)  All groups under Networking Dept – All group are futher linked to the Information Tech – All group, but it didn’t show members of the sub-groups (such as Networking Dept group). (Bug)

Screenshot link: https://drive.google.com/drive/folders/0BwgGnZC7vA-6eFVUMThjUzNWc0k

 

 

However, during testing, I noticed that when I manually delete the Networking Dept – All group from the Information Tech – All group and then re-run the Rollup Hierarchies loader, it would link the Networking Dept – All groups and also show all members of its sub-groups. As per logs, the loader is running successfully and there are no errors.

 

We are baffled by the loader behavior and checking to see if there are any solution for the bug. (We are using Grouper 2.2.2)

 

 

Thank you,

Akki