Grouper Users - Open Discussion List

[Colin Hudler <>]

Greetings,

Today I am troubleshooting a multi-symptom problem with grouper 1.6.2 
running on oracle 11g.  I know of nothing that happened or changed in 
the last two days to cause this (we've been running smoothly until now). 
 Sorry for vomiting these stack traces here; I hope it helps someone to 
see the direction of the problem.  I also hope that the problems are 
related, because they all showed up around the same time.  When browsing 
to a particular group-of-groups in the UI this error is logged:

2010-12-03 13:21:17,695: [TP-Processor23] ERROR 
GrouperCapableAction.execute(?) - < chudler 
485E91D737D48700834238215E9B8B65-0110 840a2034d00a41608b9ec2436cd5c142 
21882177X ucmcdb > - java.lang.RuntimeException: Too many results: 42672
        at 
edu.internet2.middleware.grouper.internal.dao.hib3.Hib3MembershipDAO.findAllByGroupOwnerOptions(Hib3MembershipDAO.java:630)
        at 
edu.internet2.middleware.grouper.MembershipFinder.findMemberships(MembershipFinder.java:81)
        at 
edu.internet2.middleware.grouper.ui.actions.PopulateGroupMembersAction.grouperExecute(Unknown 
Source)
        at 
edu.internet2.middleware.grouper.ui.actions.GrouperCapableAction$1.callback(Unknown 
Source)
        at
<snip>
org.apache.tomcat.util.threads.ThreadPool$ControlRunnable.run(ThreadPool.java:685)
        at java.lang.Thread.run(Thread.java:619)

The web interface simply shows the generic error page.  Additionally, 
our daily report from grouper-loader generates a different stack trace 
(an oracle error!) while it is syncing flat memberships table:

edu.internet2.middleware.grouper.misc.GrouperReportException: Problem 
generating daily report
	at 
edu.internet2.middleware.grouper.misc.GrouperReport.runReport(GrouperReport.java:400)
	at 
edu.internet2.middleware.grouper.app.loader.GrouperLoaderType$2.runJob(GrouperLoaderType.java:215)
	at 
edu.internet2.middleware.grouper.app.loader.GrouperLoaderJob.runJob(GrouperLoaderJob.java:344)
	at 
edu.internet2.middleware.grouper.app.loader.GrouperLoaderJob.execute(GrouperLoaderJob.java:247)
        at org.quartz.core.JobRunShell.run(JobRunShell.java:202)
	at 
org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:529)
Caused by: 
edu.internet2.middleware.grouper.internal.dao.GrouperDAOException: 
Problem in HibernateSession: HibernateSession: isNew: true, isReadonly: 
true, grouperTransactionType: READONLY_NEW,
Exception in list: (class edu.internet2.middleware.grouper.Membership), 
ByHqlStatic, query: 'select ms from MembershipEntry ms where 
ms.enabledDb='T' and not exists (select 1 from FlatMembership flatMship 
    where flatMship.ownerId=ms.ownerId     and 
flatMship.memberId=ms.memberUuid and flatMship.fieldId=ms.fieldId) and 
not exists (select 1 from ChangeLogEntryTemp temp, ChangeLogType type 
  where temp.string06 = ms.ownerGroupId and temp.string09=ms.fieldId 
  and type.actionName='addMembership' and 
type.changeLogCategory='membership' and type.id=temp.changeLogTypeId) 
and not exists (select 1 from ChangeLogEntryTemp temp, ChangeLogType 
type     where temp.string07 = ms.ownerId and temp.string10=ms.fieldId 
   and type.actionName='addPrivilege' and 
type.changeLogCategory='privilege' and type.id=temp.changeLogTypeId) and 
not exists (select 1 from ChangeLogEntryTemp temp, ChangeLogType type, 
Member m, MembershipEntry ms2     where temp.string09=:fieldId and 
type.actionName='addMembership' and type.changeLogCategory='membership' 
and type.id=temp.changeLogTypeId     and m.subjectIdDb=temp.string06 and 
m.subjectSourceIdDb='g:gsa'     and ms2.memberUuid = m.uuid     and 
ms.ownerId=ms2.ownerId     and ms.fieldId = ms2.fieldId and 
ms2.enabledDb='T') order by ms.immediateMembershipId, ms.groupSetId', 
cacheable: false, cacheRegion: 
edu.internet2.middleware.grouper.internal.dao.hib3.Hib3FlatMembershipDAO.FindMissingFlatMemberships, 
tx type: null, options: QueryOptions: queryPaging: pageSize: 100000, 
pageNumberOnIndexed: 1, , tx type: nullBind var[0]: 'Param (class 
java.lang.String): 'fieldId'->'8007cdc65bc24e928c13ab7a9ea3fae2',
	at 
edu.internet2.middleware.grouper.hibernate.HibernateSession._internal_hibernateSessionCatch(HibernateSession.java:424)
	at 
edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession(HibernateSession.java:510)
	at 
edu.internet2.middleware.grouper.hibernate.ByHqlStatic.list(ByHqlStatic.java:346)
	at 
edu.internet2.middleware.grouper.hibernate.ByHqlStatic.listSet(ByHqlStatic.java:395)
	at 
edu.internet2.middleware.grouper.internal.dao.hib3.Hib3FlatMembershipDAO.findMissingFlatMemberships(Hib3FlatMembershipDAO.java:288)
	at 
edu.internet2.middleware.grouper.misc.SyncFlatTables.addMissingFlatMemberships(SyncFlatTables.java:341)
	at 
edu.internet2.middleware.grouper.misc.SyncFlatTables.syncAllFlatTables(SyncFlatTables.java:158)
	at 
edu.internet2.middleware.grouper.misc.GrouperReport.runReport(GrouperReport.java:195)
        ... 5 more
Caused by: org.hibernate.exception.SQLGrammarException: could not 
update: 
[MembershipEntry#component[uuid,immediateMembershipId,groupSetId]{groupSetId=55d3a30ada5040febdd7ac5ddc504ae5, 
immediateMembershipId=00554d01d4eb451e93305b898f81adf3, 
uuid=00554d01d4eb451e93305b898f81adf3:55d3a30ada5040febdd7ac5ddc504ae5}]
	at 
org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:67)
	at 
org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:43)
	at 
org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:2430)
	at 
org.hibernate.persister.entity.AbstractEntityPersister.updateOrInsert(AbstractEntityPersister.java:2312)
	at 
org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:2612)
	at 
org.hibernate.action.EntityUpdateAction.execute(EntityUpdateAction.java:96)
<snip>
edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession(HibernateSession.java:500)
        ... 11 more
Caused by: java.sql.SQLException: ORA-01779: cannot modify a column 
which maps to a non key-preserved table

        at oracle.jdbc.driver.T4CTTIoer.processError(T4CTTIoer.java:439)


I can easily duplicate this with gsh: "new 
SyncFlatTables().sendNotifications(false).syncAllFlatTables();", but 
debugging logs haven't revealed much of use.  Finally, "gsh 
-findbadmemberships -all" fails with:

org/hibernate/engine/TwoPhaseLoad.java:104:in `initializeEntity': 
org.hibernate.AssertionFailure: possible non-threadsafe access to the 
session, (NativeException)
Problem in HibernateSession: HibernateSession: isNew: false, isReadonly: 
false, grouperTransactionType: READ_WRITE_NEW,
Exception in list: (class [Ljava.lang.Object;), ByHqlStatic, query: 
'select distinct ms, m from MembershipEntry as ms, Member as m, Field as 
field where       ms.ownerGroupId   = :owner            and 
ms.memberUuid  = :member           and  ms.fieldId = :fuuid and 
ms.memberUuid  = m.uuid     and ms.enabledDb = 'T'', cacheable: false, 
cacheRegion: 
edu.internet2.middleware.grouper.internal.dao.hib3.Hib3MembershipDAO.FindAllByGroupOwnerAndMemberAndField, 
tx type: null, tx type: nullBind var[0]: 'Param (class 
java.lang.String): 'owner'->'7a246c38-b60e-4b11-96e6-67683fe82237', Bind 
var[1]: 'Param (class java.lang.String): 
'member'->'89fc30292ed14ea091d5e792c24038e8'Bind var[2]: 'Param (class 
java.lang.String): 'fuuid'->'8007cdc65bc24e928c13ab7a9ea3fae2', ,
Exception in delete: edu.internet2.middleware.grouper.Membership, 
edu.internet2.middleware.grouper.hibernate.ByObject@6f65bb15,
Problem in HibernateSession: HibernateSession: isNew: false, isReadonly: 
false, grouperTransactionType: READ_WRITE_NEW,
Exception in delete: edu.internet2.middleware.grouper.Membership, 
ByObjectStatic, query: ', cacheable: null, cacheRegion: null, 
entityName: ImmediateMembershipEntry, tx type: null,
Problem in HibernateSession: HibernateSession: isNew: false, isReadonly: 
false, grouperTransactionType: READ_WRITE_NEW,
Problem in HibernateSession: HibernateSession: isNew: true, isReadonly: 
false, grouperTransactionType: READ_WRITE_NEW
	from org/hibernate/loader/Loader.java:854:in 
`initializeEntitiesAndCollections'
        from org/hibernate/loader/Loader.java:729:in `doQuery'
	from org/hibernate/loader/Loader.java:236:in 
`doQueryAndInitializeNonLazyCollections'
        from org/hibernate/loader/Loader.java:2213:in `doList'
        from org/hibernate/loader/Loader.java:2104:in `listIgnoreQueryCache'
        from org/hibernate/loader/Loader.java:2099:in `list'
        from org/hibernate/loader/hql/QueryLoader.java:378:in `list'
        from org/hibernate/hql/ast/QueryTranslatorImpl.java:338:in `list'
         ... 35 levels...
        from run.rb:50
        from run.rb:40:in `each'
        from run.rb:40

Several of our own loader jobs and scripts (using the API) generate the 
same error.