Skip to Content.
Sympa Menu

grouper-users - Re: [grouper-users] grouper PSP errors

Subject: Grouper Users - Open Discussion List

List archive

Re: [grouper-users] grouper PSP errors


Chronological Thread 
  • From: Rahul Doshi <>
  • To: Gagné Sébastien <>
  • Cc: "" <>
  • Subject: Re: [grouper-users] grouper PSP errors
  • Date: Mon, 18 Mar 2013 19:52:29 +0000
  • Accept-language: en-US
  • Authentication-results: sfpop-ironport05.merit.edu; dkim=neutral (message not signed) header.i=none

Hi Sébastien,

Thanks for the suggestions.  I was able to fix the problem by following steps suggested by Shilen.

Rahul

From: Gagné Sébastien <>
Date: Monday, March 18, 2013 3:47 PM
To: Rahul Doshi <>, Shilen Patel <>
Cc: "" <>
Subject: RE: [grouper-users] grouper PSP errors

Have you tried turning it off and on ?  (“it” being the Daemon)

 

If it didn’t work a quick(and drastic) solution would be :

-          Shutdown your Grouper Loader

-          Flush either GROUPER_CHANGE_LOG or GROUPER_CHANGELOG_TEMP table

 

This depends on what data you want to keep. If the PSP provisioning completed all the current changelog I would flush the CHANGE_LOG table so the new temp changelog can be brought in.

 

There might be better solutions and this depends if it’s in production or not.

 

PS. Even if you have multiple servers running Grouper, only one of them must have the Daemon running.

 

 

De : [] De la part de Rahul Doshi
Envoyé : 18 mars 2013 14:09
À : Shilen Patel
Cc :
Objet : Re: [grouper-users] grouper PSP errors

 

Yes, we are still having the problem.  I just went back in the logs to find the first occurrence of the problem so that it might help to know better what triggered the issue.  Current errors are below.

 

2013-03-18 14:05:50,061: [DefaultQuartzScheduler_Worker-5] ERROR GrouperLoaderJob.runJob(406) -  - Error on job: CHANGE_LOG_changeLogTempToChangeLog

edu.internet2.middleware.grouper.internal.dao.GrouperDAOException: Problem in HibernateSession: HibernateSession: isNew: false, isReadonly: false, grouperTransactionType: READ_WRITE_NEW,

Exception in uniqueResult: (class edu.internet2.middleware.grouper.pit.PITMember), ByHqlStatic, query: 'select pitMember from PITMember as pitMember where pitMember.sourceId = :id and activeDb = 'T'', cacheable: true, cacheRegion: edu.internet2.middleware.grouper.internal.dao.hib3.Hib3PITMemberDAO.FindBySourceIdActive, tx type: null, tx type: nullBind var[0]: 'Param (class java.lang.String): 'id'->'07058251fa4b46cba6cab0ecbe9a6d0d', ,

Problem in HibernateSession: HibernateSession: isNew: true, isReadonly: false, grouperTransactionType: READ_WRITE_NEW

        at edu.internet2.middleware.grouper.hibernate.HibernateSession._internal_hibernateSessionCatch(HibernateSession.java:495)

        at edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession(HibernateSession.java:581)

        at edu.internet2.middleware.grouper.hibernate.ByHqlStatic.uniqueResult(ByHqlStatic.java:323)

        at edu.internet2.middleware.grouper.internal.dao.hib3.Hib3PITMemberDAO.findBySourceIdActive(Hib3PITMemberDAO.java:70)

        at edu.internet2.middleware.grouper.changeLog.ChangeLogTempToEntity.processMembershipAdd(ChangeLogTempToEntity.java:708)

        at edu.internet2.middleware.grouper.changeLog.ChangeLogTempToEntity.access$1700(ChangeLogTempToEntity.java:59)

        at edu.internet2.middleware.grouper.changeLog.ChangeLogTempToEntity$1.callback(ChangeLogTempToEntity.java:135)

        at edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession(HibernateSession.java:571)

        at edu.internet2.middleware.grouper.changeLog.ChangeLogTempToEntity.convertRecords(ChangeLogTempToEntity.java:91)

        at edu.internet2.middleware.grouper.app.loader.GrouperLoaderType$4.runJob(GrouperLoaderType.java:550)

        at edu.internet2.middleware.grouper.app.loader.GrouperLoaderJob.runJob(GrouperLoaderJob.java:398)

        at edu.internet2.middleware.grouper.app.loader.GrouperLoaderJob.execute(GrouperLoaderJob.java:300)

        at org.quartz.core.JobRunShell.run(JobRunShell.java:202)

        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:529)

Caused by: org.hibernate.NonUniqueResultException: query did not return a unique result: 3

        at org.hibernate.impl.AbstractQueryImpl.uniqueElement(AbstractQueryImpl.java:899)

        at org.hibernate.impl.AbstractQueryImpl.uniqueResult(AbstractQueryImpl.java:890)

        at edu.internet2.middleware.grouper.hibernate.ByHql.uniqueResult(ByHql.java:257)

        at edu.internet2.middleware.grouper.hibernate.ByHqlStatic$1.callback(ByHqlStatic.java:332)

        at edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession(HibernateSession.java:571)

        ... 12 more

2013-03-18 14:05:50,066: [DefaultQuartzScheduler_Worker-5] ERROR GrouperLoaderJob.execute(303) -  - Error running up job

edu.internet2.middleware.grouper.internal.dao.GrouperDAOException: Problem in HibernateSession: HibernateSession: isNew: false, isReadonly: false, grouperTransactionType: READ_WRITE_NEW,

Exception in uniqueResult: (class edu.internet2.middleware.grouper.pit.PITMember), ByHqlStatic, query: 'select pitMember from PITMember as pitMember where pitMember.sourceId = :id and activeDb = 'T'', cacheable: true, cacheRegion: edu.internet2.middleware.grouper.internal.dao.hib3.Hib3PITMemberDAO.FindBySourceIdActive, tx type: null, tx type: nullBind var[0]: 'Param (class java.lang.String): 'id'->'07058251fa4b46cba6cab0ecbe9a6d0d', ,

Problem in HibernateSession: HibernateSession: isNew: true, isReadonly: false, grouperTransactionType: READ_WRITE_NEW

        at edu.internet2.middleware.grouper.hibernate.HibernateSession._internal_hibernateSessionCatch(HibernateSession.java:495)

        at edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession(HibernateSession.java:581)

        at edu.internet2.middleware.grouper.hibernate.ByHqlStatic.uniqueResult(ByHqlStatic.java:323)

        at edu.internet2.middleware.grouper.internal.dao.hib3.Hib3PITMemberDAO.findBySourceIdActive(Hib3PITMemberDAO.java:70)

        at edu.internet2.middleware.grouper.changeLog.ChangeLogTempToEntity.processMembershipAdd(ChangeLogTempToEntity.java:708)

        at edu.internet2.middleware.grouper.changeLog.ChangeLogTempToEntity.access$1700(ChangeLogTempToEntity.java:59)

        at edu.internet2.middleware.grouper.changeLog.ChangeLogTempToEntity$1.callback(ChangeLogTempToEntity.java:135)

        at edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession(HibernateSession.java:571)

        at edu.internet2.middleware.grouper.changeLog.ChangeLogTempToEntity.convertRecords(ChangeLogTempToEntity.java:91)

        at edu.internet2.middleware.grouper.app.loader.GrouperLoaderType$4.runJob(GrouperLoaderType.java:550)

        at edu.internet2.middleware.grouper.app.loader.GrouperLoaderJob.runJob(GrouperLoaderJob.java:398)

        at edu.internet2.middleware.grouper.app.loader.GrouperLoaderJob.execute(GrouperLoaderJob.java:300)

        at org.quartz.core.JobRunShell.run(JobRunShell.java:202)

        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:529)

Caused by: org.hibernate.NonUniqueResultException: query did not return a unique result: 3

        at org.hibernate.impl.AbstractQueryImpl.uniqueElement(AbstractQueryImpl.java:899)

        at org.hibernate.impl.AbstractQueryImpl.uniqueResult(AbstractQueryImpl.java:890)

        at edu.internet2.middleware.grouper.hibernate.ByHql.uniqueResult(ByHql.java:257)

        at edu.internet2.middleware.grouper.hibernate.ByHqlStatic$1.callback(ByHqlStatic.java:332)

        at edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession(HibernateSession.java:571)

        ... 12 more

 

Thanks,

Rahul

 

From: Shilen Patel <>
Date: Monday, March 18, 2013 2:05 PM
To: Rahul Doshi <>
Cc: "" <>
Subject: Re: [grouper-users] grouper PSP errors

 

The time in the logs you gave are from last week.  Are you continuing to see the exact same errors appear every minute?  Can you send your latest logs?  You may have some duplicate point in time entries that need to be deleted.  The issue was fixed in 2.1.3.

 

Thanks!

 

-- Shilen

 

From: Rahul Doshi <>
Date: Mon, 18 Mar 2013 17:52:52 +0000
To: Shilen Patel <>
Cc: "" <>
Subject: Re: [grouper-users] grouper PSP errors

 

We are running grouper 2.1.2 and I am quite sure we don't have multiple grouper daemons running.

 

Thanks,

Rahul

 

From: Shilen Patel <>
Date: Monday, March 18, 2013 1:38 PM
To: Rahul Doshi <>
Cc: "" <>
Subject: Re: [grouper-users] grouper PSP errors

 

Do you have multiple instances of the Grouper daemon running?  What version of Grouper do you have?

 

Thanks!

 

-- Shilen

 

From: Rahul Doshi <>
Date: Mon, 18 Mar 2013 17:32:34 +0000
To: "" <>
Subject: [grouper-users] grouper PSP errors

 

Hello,

 

We are having some database errors which is causing PSP to not update the groups correctly. Does anyone has any suggestions how I can avoid this error in the future and what is the best way to recover from this?

 

2013-03-11 09:09:54,094: [DefaultQuartzScheduler_Worker-9] ERROR JDBCExceptionReporter.logExceptions(234) -  - Duplicate entry '947053' for key 'PRIMARY'

2013-03-11 09:09:54,096: [DefaultQuartzScheduler_Worker-9] ERROR GrouperLoaderJob.runJob(406) -  - Error on job: CHANGE_LOG_changeLogTempToChangeLog

edu.internet2.middleware.grouper.internal.dao.GrouperDAOException: Problem in HibernateSession: HibernateSession: isNew: false, isReadonly: false, grouperTransactionType: READ_WRITE_NEW,

Exception in save: edu.internet2.middleware.grouper.changeLog.ChangeLogEntry, ByObjectStatic, query: ', cacheable: null, cacheRegion: null, entityName: ChangeLogEntryEntity, tx type: null,

Problem in HibernateSession: HibernateSession: isNew: true, isReadonly: false, grouperTransactionType: READ_WRITE_NEW

        at edu.internet2.middleware.grouper.hibernate.HibernateSession._internal_hibernateSessionCatch(HibernateSession.java:495)

        at edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession(HibernateSession.java:581)

        at edu.internet2.middleware.grouper.hibernate.ByObjectStatic.save(ByObjectStatic.java:468)

        at edu.internet2.middleware.grouper.internal.dao.hib3.Hib3ChangeLogEntryDAO.save(Hib3ChangeLogEntryDAO.java:47)

        at edu.internet2.middleware.grouper.changeLog.ChangeLogEntry.save(ChangeLogEntry.java:356)

        at edu.internet2.middleware.grouper.changeLog.ChangeLogTempToEntity$1.callback(ChangeLogTempToEntity.java:196)

        at edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession(HibernateSession.java:571)

        at edu.internet2.middleware.grouper.changeLog.ChangeLogTempToEntity.convertRecords(ChangeLogTempToEntity.java:91)

        at edu.internet2.middleware.grouper.app.loader.GrouperLoaderType$4.runJob(GrouperLoaderType.java:550)

        at edu.internet2.middleware.grouper.app.loader.GrouperLoaderJob.runJob(GrouperLoaderJob.java:398)

        at edu.internet2.middleware.grouper.app.loader.GrouperLoaderJob.execute(GrouperLoaderJob.java:300)

        at org.quartz.core.JobRunShell.run(JobRunShell.java:202)

        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:529)

Caused by: org.hibernate.exception.ConstraintViolationException: Could not execute JDBC batch update,

Exception in save: edu.internet2.middleware.grouper.changeLog.ChangeLogEntry,

        at org.hibernate.exception.SQLStateConverter.convert(SQLStateConverter.java:96)

        at org.hibernate.exception.JDBCExceptionHelper.convert(JDBCExceptionHelper.java:66)

        at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:275)

        at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:268)

        at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:184)

        at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321)

        at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:51)

        at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1216)

        at edu.internet2.middleware.grouper.hibernate.ByObject.save(ByObject.java:214)

        at edu.internet2.middleware.grouper.hibernate.ByObjectStatic$7.callback(ByObjectStatic.java:481)

        at edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession(HibernateSession.java:571)

        ... 11 more

Caused by: java.sql.BatchUpdateException: Duplicate entry '947053' for key 'PRIMARY'

        at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:2024)

        at com.mysql.jdbc.PreparedStatement.executeBatch(PreparedStatement.java:1449)

        at com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeBatch(NewProxyPreparedStatement.java:1723)

        at org.hibernate.jdbc.BatchingBatcher.doExecuteBatch(BatchingBatcher.java:70)

        at org.hibernate.jdbc.AbstractBatcher.executeBatch(AbstractBatcher.java:268)

        ... 19 more

Caused by: com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: Duplicate entry '947053' for key 'PRIMARY'

        at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native Method)

        at sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:57)

        at sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)

        at java.lang.reflect.Constructor.newInstance(Constructor.java:532)

        at com.mysql.jdbc.Util.handleNewInstance(Util.java:411)

        at com.mysql.jdbc.Util.getInstance(Util.java:386)

        at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:1039)

        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3597)

        at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3529)

        at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:1990)

        at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2151)

        at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2625)

        at com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:2119)

        at com.mysql.jdbc.PreparedStatement.executeUpdate(PreparedStatement.java:2415)

        at com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1976)

        ... 23 more

 

2013-03-11 09:10:00,043: [DefaultQuartzScheduler_Worker-1] ERROR GrouperLoaderType$4.runJob(596) -  - Problem with change log consumer: psp

edu.internet2.middleware.grouper.exception.GrouperStaleObjectStateException: Problem in HibernateSession: HibernateSession: isNew: true, isReadonly: false, grouperTransactionType: READ_WRITE_NEW

        at edu.internet2.middleware.grouper.hibernate.HibernateSession._internal_hibernateSessionCatch(HibernateSession.java:491)

        at edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession(HibernateSession.java:581)

        at edu.internet2.middleware.grouper.hibernate.ByObjectStatic.saveOrUpdate(ByObjectStatic.java:354)

        at edu.internet2.middleware.grouper.internal.dao.hib3.Hib3ChangeLogConsumerDAO.saveOrUpdate(Hib3ChangeLogConsumerDAO.java:41)

        at edu.internet2.middleware.grouper.changeLog.ChangeLogHelper.processRecords(ChangeLogHelper.java:258)

        at edu.internet2.middleware.grouper.app.loader.GrouperLoaderType$4.runJob(GrouperLoaderType.java:589)

        at edu.internet2.middleware.grouper.app.loader.GrouperLoaderJob.runJob(GrouperLoaderJob.java:398)

        at edu.internet2.middleware.grouper.app.loader.GrouperLoaderJob.execute(GrouperLoaderJob.java:300)

        at org.quartz.core.JobRunShell.run(JobRunShell.java:202)

        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:529)

Caused by: org.hibernate.StaleObjectStateException: Row was updated or deleted by another transaction (or unsaved-value mapping was incorrect): [edu.internet2.middleware.grouper.changeLog.ChangeLogConsumer#8be8a00baea240eaae513780a4325886],

Exception in save: edu.internet2.middleware.grouper.changeLog.ChangeLogConsumer, : [edu.internet2.middleware.grouper.changeLog.ChangeLogConsumer#8be8a00baea240eaae513780a4325886]

        at org.hibernate.persister.entity.AbstractEntityPersister.check(AbstractEntityPersister.java:1950)

        at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:2594)

        at org.hibernate.persister.entity.AbstractEntityPersister.updateOrInsert(AbstractEntityPersister.java:2494)

        at org.hibernate.persister.entity.AbstractEntityPersister.update(AbstractEntityPersister.java:2821)

        at org.hibernate.action.EntityUpdateAction.execute(EntityUpdateAction.java:113)

        at org.hibernate.engine.ActionQueue.execute(ActionQueue.java:273)

        at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:265)

        at org.hibernate.engine.ActionQueue.executeActions(ActionQueue.java:185)

        at org.hibernate.event.def.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:321)

        at org.hibernate.event.def.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:51)

        at org.hibernate.impl.SessionImpl.flush(SessionImpl.java:1216)

        at edu.internet2.middleware.grouper.hibernate.ByObject.saveOrUpdate(ByObject.java:377)

        at edu.internet2.middleware.grouper.hibernate.ByObjectStatic$5.callback(ByObjectStatic.java:367)

        at edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession(HibernateSession.java:571)

        ... 8 more

 

2013-03-11 09:10:50,047: [DefaultQuartzScheduler_Worker-6] ERROR GrouperLoaderJob.runJob(406) -  - Error on job: CHANGE_LOG_changeLogTempToChangeLog

edu.internet2.middleware.grouper.internal.dao.GrouperDAOException: Problem in HibernateSession: HibernateSession: isNew: false, isReadonly: false, grouperTransactionType: READ_WRITE_NEW,

Exception in uniqueResult: (class edu.internet2.middleware.grouper.pit.PITMember), ByHqlStatic, query: 'select pitMember from PITMember as pitMember where pitMember.sourceId = :id and activeDb = 'T'', cacheable: true, cacheRegion: edu.internet2.middleware.grouper.internal.dao.hib3.Hib3PITMemberDAO.FindBySourceIdActive, tx type: null, tx type: nullBind var[0]: 

'Param (class java.lang.String): 'id'->'07058251fa4b46cba6cab0ecbe9a6d0d', ,

Problem in HibernateSession: HibernateSession: isNew: true, isReadonly: false, grouperTransactionType: READ_WRITE_NEW

        at edu.internet2.middleware.grouper.hibernate.HibernateSession._internal_hibernateSessionCatch(HibernateSession.java:495)

        at edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession(HibernateSession.java:581)

        at edu.internet2.middleware.grouper.hibernate.ByHqlStatic.uniqueResult(ByHqlStatic.java:323)

        at edu.internet2.middleware.grouper.internal.dao.hib3.Hib3PITMemberDAO.findBySourceIdActive(Hib3PITMemberDAO.java:70)

        at edu.internet2.middleware.grouper.changeLog.ChangeLogTempToEntity.processMembershipAdd(ChangeLogTempToEntity.java:708)

        at edu.internet2.middleware.grouper.changeLog.ChangeLogTempToEntity.access$1700(ChangeLogTempToEntity.java:59)

        at edu.internet2.middleware.grouper.changeLog.ChangeLogTempToEntity$1.callback(ChangeLogTempToEntity.java:135)

        at edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession(HibernateSession.java:571)

        at edu.internet2.middleware.grouper.changeLog.ChangeLogTempToEntity.convertRecords(ChangeLogTempToEntity.java:91)

        at edu.internet2.middleware.grouper.app.loader.GrouperLoaderType$4.runJob(GrouperLoaderType.java:550)

        at edu.internet2.middleware.grouper.app.loader.GrouperLoaderJob.runJob(GrouperLoaderJob.java:398)

        at edu.internet2.middleware.grouper.app.loader.GrouperLoaderJob.execute(GrouperLoaderJob.java:300)

        at org.quartz.core.JobRunShell.run(JobRunShell.java:202)

        at org.quartz.simpl.SimpleThreadPool$WorkerThread.run(SimpleThreadPool.java:529)

Caused by: org.hibernate.NonUniqueResultException: query did not return a unique result: 3

        at org.hibernate.impl.AbstractQueryImpl.uniqueElement(AbstractQueryImpl.java:899)

        at org.hibernate.impl.AbstractQueryImpl.uniqueResult(AbstractQueryImpl.java:890)

        at edu.internet2.middleware.grouper.hibernate.ByHql.uniqueResult(ByHql.java:257)

        at edu.internet2.middleware.grouper.hibernate.ByHqlStatic$1.callback(ByHqlStatic.java:332)

        at edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession(HibernateSession.java:571)

        ... 12 more

 

Thanks,

Rahul




Archive powered by MHonArc 2.6.16.

Top of Page