Grouper Users - Open Discussion List

[Shilen Patel <>]

API patch 75 includes a modification of bad membership finder that
attempts to find/delete duplicates.  If you want to try it out, let me
know if that solves the problem.

It's currently a test patch so for the installer to install it, you would
need a grouper.installer.properties file with the following:

download.server.url = http://software.internet2.edu/grouper
grouper.version = 2.3.0
grouperInstaller.useTestPatches = true


And run the installer like this:

java -cp .:grouperInstaller.jar
edu.internet2.middleware.grouperInstaller.GrouperInstaller


Then run bad membership finder like this (and it should show duplicates):

$ ./bin/gsh.sh -findBadMemberships -all

...

Checking Composite Memberships
Checking Group Sets
Duplicate group set: owner=f2759edf3c1c4026acf91a0483b94567,
member=f2759edf3c1c4026acf91a0483b94567,
field=e9d258aae2e148aaa3598546e861fab7.
Duplicate group set: owner=f2759edf3c1c4026acf91a0483b94567,
member=f2759edf3c1c4026acf91a0483b94567,
field=e9d258aae2e148aaa3598546e861fab7.
Duplicate group set: owner=7c4184c4cdf6446b8ec1834178bba5c3,
member=7c4184c4cdf6446b8ec1834178bba5c3,
field=e9d258aae2e148aaa3598546e861fab7.
Duplicate group set: owner=80d972c5a0714c119c871381cbdfbbba,
member=80d972c5a0714c119c871381cbdfbbba,
field=e9d258aae2e148aaa3598546e861fab7.
Checking memberships where member is a deleted group
Checking GrouperAll

Once it is fixed per the instructions that it would output, also sync pit
tables just to make sure the duplicates there (if any) are taken care of.

groovy:000>     new
edu.internet2.middleware.grouper.misc.SyncPITTables().syncAllPITTables();


Thanks!


- Shilen




On 8/4/17, 7:35 AM, "Shilen Patel" 
<>
 wrote:

>>> What is grouper_group_sets for?
>
>It's used to efficiently calculate effective memberships.
>
>>> How did I get duplicates?  Something from the 2.1->2.3 upgrade?
>
>Possibly.  The upgrade does add some group sets, but I can't see how it
>would have added duplicates unless you were running multiple instances of
>the upgrade at the same time.  There is a create_time column on the group
>set table so that may tell you when the duplicates were added.  Also, what
>fields do the field_id columns correspond to in the duplicates?
>
>>> How do I get rid of them?
>
>Hopefully, for each set of duplicates, no more than 1 has a foreign key to
>another group set.  If that's the case, it should be safe to just delete
>one row without a foreign key.  Would a patch help?
>
>- Shilen
>
>On 8/3/17, 6:46 PM, "Jim Fox" 
><>
> wrote:
>
>>
>>I do seem to have duplicates.  In this case there are two entries that
>>differ only in parent_id.
>>
>>What is grouper_group_sets for?
>>How did I get duplicates?  Something from the 2.1->2.3 upgrade?
>>How do I get rid of them?
>>
>>Thanks,
>>
>>Jim
>>
>>
>>On Thu, 3 Aug 2017, Shilen Patel wrote:
>>
>>> Date: Thu, 3 Aug 2017 15:32:01
>>> From: Shilen Patel 
>>> <>
>>> To: Jim Fox 
>>> <>
>>> Cc: 
>>> ""
>>>  
>>> <>
>>> Subject: Re: [grouper-users] unable to add group to group field (2.3)
>>> 
>>> Looks like you may have duplicates in your group set table.  I don't
>>>think bad membership finder catches that right now.  I can add it there,
>>>but
>>> you can verify by running this I think:
>>> 
>>> select owner_group_id, member_group_id, field_id from grouper_group_set
>>>where depth='0' and owner_group_id is not null and member_group_id is
>>>not
>>> null and field_id is not null group by owner_group_id, member_group_id,
>>>field_id having count(*) > 1
>>> 
>>> Thanks!
>>> 
>>> - Shilen
>>> 
>>> On 8/3/17, 6:13 PM, "Jim Fox" 
>>> <>
>>>  wrote:
>>> 
>>> 
>>> This is experimenting with grouper api 2.3
>>> 
>>> I am trying to add a group as a field member (not membership).  I get
>>>an
>>> exception: (sorry for the messay post)
>>> 
>>> I am able to add non-group subjects.
>>> 
>>> Something to do with group sets?
>>> 
>>> Jim
>>> 
>>> ------------------------------------------------------
>>> Aug 03, 2017 2:59:11 PM org.apache.catalina.core.StandardWrapperValve
>>>invoke
>>> SEVERE: Servlet.service() for servlet [gws] in context with path
>>>[/gws_grouper] threw exception [Request processing failed; nested
>>>exception
>>> is java.lang.RuntimeException:
>>>edu.internet2.middleware.grouper.internal.dao.GrouperDAOException:
>>>Problem in HibernateSession:
>>> HibernateSession (6c4c2a54): notNew, notReadonly, READ_WRITE_NEW,
>>>activeTransaction, session (5bd96f94),
>>> Exception in uniqueResult: (class
>>>edu.internet2.middleware.grouper.group.GroupSet), ByHqlStatic, query:
>>>'select gs from GroupSet as gs where
>>> gs.ownerGroupId = :id and memberGroupId = :id and fieldId = :field and
>>>depth='0'', cacheable: true, cacheRegion:
>>> 
>>>edu.internet2.middleware.grouper.internal.dao.hib3.Hib3GroupSetDAO.FindS
>>>e
>>>lfGroup, tx type: null, tx type: nullBind var[0]: 'Param (class
>>> java.lang.String): 'id'->'881fef48f9e32bedf88b8fe5d96ed250'Bind var[1]:
>>>'Param (class java.lang.String):
>>> 'field'->'0094f3965ca2419286d3c4c3c9a1fb4e', ,
>>> Exception in save: edu.internet2.middleware.grouper.Membership,
>>>edu.internet2.middleware.grouper.hibernate.ByObject@44744e2e,
>>> Problem in HibernateSession: HibernateSession (6e7b95d1): notNew,
>>>notReadonly, READ_WRITE_NEW, activeTransaction, session (5bd96f94),
>>> Exception in save: edu.internet2.middleware.grouper.Membership,
>>>ByObjectStatic, query: ', cacheable: null, cacheRegion: null,
>>>entityName:
>>> ImmediateMembershipEntry, tx type: null,
>>> , group name: u:fox:spud97, subject: Subject groupName: u:fox:spud97,
>>>sourceId: g:gsa, field: authorized-senders,
>>> Problem in HibernateSession: HibernateSession (29610a0e): new,
>>>notReadonly, READ_WRITE_NEW, notActiveTransaction, session (5bd96f94)]
>>>with
>>> root cause
>>> org.hibernate.NonUniqueResultException: query did not return a unique
>>>result: 2
>>>          at 
>>>org.hibernate.internal.AbstractQueryImpl.uniqueElement(AbstractQueryImpl
>>>.
>>>java:971)
>>>          at 
>>>org.hibernate.internal.AbstractQueryImpl.uniqueResult(AbstractQueryImpl.
>>>j
>>>ava:960)
>>>          at 
>>>edu.internet2.middleware.grouper.hibernate.ByHql.uniqueResult(ByHql.java
>>>:
>>>282)
>>>          at 
>>>edu.internet2.middleware.grouper.hibernate.ByHqlStatic$1.callback(ByHqlS
>>>t
>>>atic.java:348)
>>>          at 
>>>edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibe
>>>r
>>>nateSession(HibernateSession.java:700)
>>>          at 
>>>edu.internet2.middleware.grouper.hibernate.ByHqlStatic.uniqueResult(ByHq
>>>l
>>>Static.java:339)
>>>          at 
>>>edu.internet2.middleware.grouper.internal.dao.hib3.Hib3GroupSetDAO.findS
>>>e
>>>lfGroup(Hib3GroupSetDAO.java:356)
>>>          at 
>>>edu.internet2.middleware.grouper.Membership.processPostMembershipSaveOrU
>>>p
>>>date(Membership.java:2135)
>>>          at 
>>>edu.internet2.middleware.grouper.Membership.onPostSave(Membership.java:1
>>>9
>>>97)
>>>          at 
>>>edu.internet2.middleware.grouper.hibernate.ByObject.save(ByObject.java:2
>>>2
>>>4)
>>>          at 
>>>edu.internet2.middleware.grouper.hibernate.ByObjectStatic$7.callback(ByO
>>>b
>>>jectStatic.java:494)
>>>          at 
>>>edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibe
>>>r
>>>nateSession(HibernateSession.java:700)
>>>          at 
>>>edu.internet2.middleware.grouper.hibernate.ByObjectStatic.save(ByObjectS
>>>t
>>>atic.java:481)
>>>          at 
>>>edu.internet2.middleware.grouper.internal.dao.hib3.Hib3MembershipDAO.sav
>>>e
>>>(Hib3MembershipDAO.java:2053)
>>>          at 
>>>edu.internet2.middleware.grouper.Membership.internal_addImmediateMembers
>>>h
>>>ip(Membership.java:1246)
>>>          at 
>>>edu.internet2.middleware.grouper.Group$4.callback(Group.java:1392)
>>>          at 
>>>edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibe
>>>r
>>>nateSession(HibernateSession.java:700)
>>>          at 
>>>edu.internet2.middleware.grouper.Group.internal_addMember(Group.java:136
>>>4
>>>)
>>>          at 
>>>edu.internet2.middleware.grouper.Group.addMember(Group.java:938)
>>>          at 
>>>edu.internet2.middleware.grouper.Group.addMember(Group.java:899)
>>>          at 
>>>edu.washington.gws.group.grouper.GrouperGroupFactory.updateFieldMembers(
>>>G
>>>rouperGroupFactory.java:848)
>>>    ...
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>>> 
>