Grouper Users - Open Discussion List

[Shilen Patel <>]

Is an instance of the daemon running while doing this?  Can you make sure all 
daemon instances are off, grab the max sequence_number from 
grouper_change_log_entry, run the sync again, and if it errors again grab the 
max sequence number again along with the error message and send all that?

Thanks!

- Shilen

On 4/14/18, 12:50 AM, "Black, Carey M." 
<>
 wrote:

    Shilen,
    
    I thought I had sent it all.... did the gsh layer eat part of it?
    
    I ran it with the explicit catch and I do see a lot more output than 
before. Hum...
    
    Here it is...
    
    "
    groovy:000> try { new 
edu.internet2.middleware.grouper.misc.SyncPITTables().syncAllPITTables(); } 
catch (Exception e) { e.printStackTrace(); }
    Number of entries in grouper_change_log_entry_temp: 0
    
    
    Searching for missing active point in time fields
    Found 0 missing active point in time fields
    
    
    Searching for missing active point in time members
    Found 0 missing active point in time members
    
    
    Searching for missing active point in time stems
    Found 0 missing active point in time stems
    
    
    Searching for missing active point in time groups
    Found 0 missing active point in time groups
    
    
    Searching for missing active point in time role sets
    Found 0 missing active point in time role sets
    
    
    Searching for missing active point in time attribute defs
    Found 0 missing active point in time attribute defs
    
    
    Searching for missing active point in time attribute def names
    Found 0 missing active point in time attribute def names
    
    
    Searching for missing active point in time attribute def name sets
    Found 0 missing active point in time attribute def name sets
    
    
    Searching for missing active point in time actions
    Found 0 missing active point in time actions
    
    
    Searching for missing active point in time action sets
    Found 0 missing active point in time action sets
    
    
    Searching for missing active point in time group sets
    Found 0 missing active point in time group sets
    
    
    Searching for missing active point in time memberships
    Found 590904 missing active point in time memberships
    edu.internet2.middleware.grouper.internal.dao.GrouperDAOException: 
Problem in HibernateSession: HibernateSession (2d9e5730): notNew, 
notReadonly, READ_WRITE_NEW, activeTransaction, session (7321cd6c),
    Exception in save: 
edu.internet2.middleware.grouper.changeLog.ChangeLogEntry, ByObjectStatic, 
query: ', cacheable: null, cacheRegion: null, entityName: 
ChangeLogEntryEntity, tx type: null,
    Exception in save: edu.internet2.middleware.grouper.pit.PITMembership, 
edu.internet2.middleware.grouper.hibernate.ByObject@682b99de,
    Problem in HibernateSession: HibernateSession (eaab6fe): new, 
notReadonly, READ_WRITE_NEW, notActiveTransaction, session (7321cd6c),
    Exception in saveOrUpdate: 
edu.internet2.middleware.grouper.pit.PITMembership, ByObjectStatic, query: ', 
cacheable: null, cacheRegion: null, entityName: null, tx type: null
            at 
edu.internet2.middleware.grouper.hibernate.HibernateSession._internal_hibernateSessionCatch(HibernateSession.java:588)
            at 
edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession(HibernateSession.java:710)
            at 
edu.internet2.middleware.grouper.hibernate.ByObjectStatic.saveBatch(ByObjectStatic.java:535)
            at 
edu.internet2.middleware.grouper.internal.dao.hib3.Hib3ChangeLogEntryDAO.saveBatch(Hib3ChangeLogEntryDAO.java:61)
            at 
edu.internet2.middleware.grouper.pit.PITMembership.onPreSave(PITMembership.java:517)
            at 
edu.internet2.middleware.grouper.hibernate.ByObject.saveOrUpdate(ByObject.java:375)
            at 
edu.internet2.middleware.grouper.hibernate.ByObjectStatic$5.callback(ByObjectStatic.java:376)
            at 
edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession(HibernateSession.java:700)
            at 
edu.internet2.middleware.grouper.hibernate.ByObjectStatic.saveOrUpdate(ByObjectStatic.java:363)
            at 
edu.internet2.middleware.grouper.internal.dao.hib3.Hib3PITMembershipDAO.saveOrUpdate(Hib3PITMembershipDAO.java:41)
            at 
edu.internet2.middleware.grouper.pit.PITMembership.save(PITMembership.java:267)
            at 
edu.internet2.middleware.grouper.misc.SyncPITTables.processMissingActivePITMemberships(SyncPITTables.java:328)
            at 
edu.internet2.middleware.grouper.misc.SyncPITTables.syncAllPITTables(SyncPITTables.java:236)
            at 
edu.internet2.middleware.grouper.misc.SyncPITTables$syncAllPITTables.call(Unknown
 Source)
            at 
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
            at 
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
            at 
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)
            at groovysh_evaluate.run(groovysh_evaluate:3)
            at groovysh_evaluate$run.call(Unknown Source)
            at 
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
            at groovysh_evaluate$run.call(Unknown Source)
            at 
org.codehaus.groovy.tools.shell.Interpreter.evaluate(Interpreter.groovy:70)
            at 
org.codehaus.groovy.tools.shell.Evaluator$evaluate.call(Unknown Source)
            at 
org.codehaus.groovy.tools.shell.Groovysh.execute(Groovysh.groovy:191)
            at 
org.codehaus.groovy.tools.shell.Shell.leftShift(Shell.groovy:122)
            at org.codehaus.groovy.tools.shell.Shell$leftShift$0.call(Unknown 
Source)
            at 
org.codehaus.groovy.tools.shell.ShellRunner.work(ShellRunner.groovy:95)
            at 
org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$work(InteractiveShellRunner.groovy)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
            at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:498)
            at 
org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
            at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
            at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
            at 
org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132)
            at 
org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:152)
            at 
org.codehaus.groovy.tools.shell.InteractiveShellRunner.work(InteractiveShellRunner.groovy:124)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
            at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:498)
            at 
org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210)
            at 
org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.callCurrent(PogoMetaMethodSite.java:59)
            at 
org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:158)
            at 
org.codehaus.groovy.tools.shell.ShellRunner.run(ShellRunner.groovy:59)
            at 
org.codehaus.groovy.tools.shell.InteractiveShellRunner.super$2$run(InteractiveShellRunner.groovy)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
            at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:498)
            at 
org.codehaus.groovy.reflection.CachedMethod.invoke(CachedMethod.java:93)
            at groovy.lang.MetaMethod.doMethodInvoke(MetaMethod.java:325)
            at groovy.lang.MetaClassImpl.invokeMethod(MetaClassImpl.java:1213)
            at 
org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuperN(ScriptBytecodeAdapter.java:132)
            at 
org.codehaus.groovy.runtime.ScriptBytecodeAdapter.invokeMethodOnSuper0(ScriptBytecodeAdapter.java:152)
            at 
org.codehaus.groovy.tools.shell.InteractiveShellRunner.run(InteractiveShellRunner.groovy:83)
            at java_lang_Runnable$run.call(Unknown Source)
            at 
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
            at 
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
            at 
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:117)
            at 
org.codehaus.groovy.tools.shell.Groovysh.run(Groovysh.groovy:596)
            at 
org.codehaus.groovy.tools.shell.Groovysh$run$0.callCurrent(Unknown Source)
            at 
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCallCurrent(CallSiteArray.java:52)
            at 
org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:154)
            at 
org.codehaus.groovy.runtime.callsite.AbstractCallSite.callCurrent(AbstractCallSite.java:166)
            at 
org.codehaus.groovy.tools.shell.Groovysh.run(Groovysh.groovy:561)
            at org.codehaus.groovy.tools.shell.Groovysh$run.call(Unknown 
Source)
            at 
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
            at 
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
            at 
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133)
            at 
org.codehaus.groovy.tools.shell.Main.startGroovysh(Main.groovy:186)
            at sun.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
            at 
sun.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:62)
            at 
sun.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
            at java.lang.reflect.Method.invoke(Method.java:498)
            at 
org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite$PogoCachedMethodSiteNoUnwrapNoCoerce.invoke(PogoMetaMethodSite.java:210)
            at 
org.codehaus.groovy.runtime.callsite.PogoMetaMethodSite.call(PogoMetaMethodSite.java:71)
            at 
org.codehaus.groovy.runtime.callsite.CallSiteArray.defaultCall(CallSiteArray.java:48)
            at 
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:113)
            at 
org.codehaus.groovy.runtime.callsite.AbstractCallSite.call(AbstractCallSite.java:133)
            at org.codehaus.groovy.tools.shell.Main.main(Main.groovy:152)
            at 
edu.internet2.middleware.grouper.app.gsh.GrouperShell.grouperShellHelper(GrouperShell.java:274)
            at 
edu.internet2.middleware.grouper.app.gsh.GrouperShell.main(GrouperShell.java:162)
            at 
edu.internet2.middleware.grouper.app.gsh.GrouperShellWrapper.main(GrouperShellWrapper.java:31)
    Caused by: org.hibernate.exception.ConstraintViolationException: could 
not execute batch,
    Exception in save: 
edu.internet2.middleware.grouper.changeLog.ChangeLogEntry, 
edu.internet2.middleware.grouper.hibernate.ByObject@5609b30d
            at 
org.hibernate.exception.internal.SQLStateConversionDelegate.convert(SQLStateConversionDelegate.java:112)
            at 
org.hibernate.exception.internal.StandardSQLExceptionConverter.convert(StandardSQLExceptionConverter.java:42)
            at 
org.hibernate.engine.jdbc.spi.SqlExceptionHelper.convert(SqlExceptionHelper.java:109)
            at 
org.hibernate.engine.jdbc.batch.internal.BatchingBatch.performExecution(BatchingBatch.java:119)
            at 
org.hibernate.engine.jdbc.batch.internal.BatchingBatch.doExecuteBatch(BatchingBatch.java:97)
            at 
org.hibernate.engine.jdbc.batch.internal.AbstractBatchImpl.execute(AbstractBatchImpl.java:147)
            at 
org.hibernate.engine.jdbc.internal.JdbcCoordinatorImpl.executeBatch(JdbcCoordinatorImpl.java:214)
            at 
org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:589)
            at 
org.hibernate.engine.spi.ActionQueue.executeActions(ActionQueue.java:434)
            at 
org.hibernate.event.internal.AbstractFlushingEventListener.performExecutions(AbstractFlushingEventListener.java:337)
            at 
org.hibernate.event.internal.DefaultFlushEventListener.onFlush(DefaultFlushEventListener.java:39)
            at org.hibernate.internal.SessionImpl.flush(SessionImpl.java:1224)
            at 
edu.internet2.middleware.grouper.hibernate.ByObject.saveBatch(ByObject.java:278)
            at 
edu.internet2.middleware.grouper.hibernate.ByObjectStatic$8.callback(ByObjectStatic.java:548)
            at 
edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession(HibernateSession.java:700)
            ... 83 more
    Caused by: java.sql.BatchUpdateException: Duplicate entry '46447346' for 
key 'PRIMARY'
            at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 
Method)
            at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
            at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
            at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
            at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
            at com.mysql.jdbc.Util.getInstance(Util.java:387)
            at 
com.mysql.jdbc.SQLError.createBatchUpdateException(SQLError.java:1154)
            at 
com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1773)
            at 
com.mysql.jdbc.PreparedStatement.executeBatchInternal(PreparedStatement.java:1257)
            at 
com.mysql.jdbc.StatementImpl.executeBatch(StatementImpl.java:959)
            at 
com.mchange.v2.c3p0.impl.NewProxyPreparedStatement.executeBatch(NewProxyPreparedStatement.java:2544)
            at 
org.hibernate.engine.jdbc.batch.internal.BatchingBatch.performExecution(BatchingBatch.java:110)
            ... 94 more
    Caused by: 
com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: 
Duplicate entry '46447346' for key 'PRIMARY'
            at sun.reflect.NativeConstructorAccessorImpl.newInstance0(Native 
Method)
            at 
sun.reflect.NativeConstructorAccessorImpl.newInstance(NativeConstructorAccessorImpl.java:62)
            at 
sun.reflect.DelegatingConstructorAccessorImpl.newInstance(DelegatingConstructorAccessorImpl.java:45)
            at java.lang.reflect.Constructor.newInstance(Constructor.java:423)
            at com.mysql.jdbc.Util.handleNewInstance(Util.java:404)
            at com.mysql.jdbc.Util.getInstance(Util.java:387)
            at com.mysql.jdbc.SQLError.createSQLException(SQLError.java:932)
            at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3878)
            at com.mysql.jdbc.MysqlIO.checkErrorPacket(MysqlIO.java:3814)
            at com.mysql.jdbc.MysqlIO.sendCommand(MysqlIO.java:2478)
            at com.mysql.jdbc.MysqlIO.sqlQueryDirect(MysqlIO.java:2625)
            at com.mysql.jdbc.ConnectionImpl.execSQL(ConnectionImpl.java:2551)
            at 
com.mysql.jdbc.PreparedStatement.executeInternal(PreparedStatement.java:1861)
            at 
com.mysql.jdbc.PreparedStatement.executeUpdateInternal(PreparedStatement.java:2073)
            at 
com.mysql.jdbc.PreparedStatement.executeBatchSerially(PreparedStatement.java:1751)
            ... 98 more
    ===> null
    "
    
    I know I did not see this before...
    -->  " Caused by: 
com.mysql.jdbc.exceptions.jdbc4.MySQLIntegrityConstraintViolationException: 
Duplicate entry '46447346' for key 'PRIMARY' "
    
    -- 
    Carey Matthew 

    
    
    -----Original Message-----
    From: Shilen Patel 
<>
 
    Sent: Friday, April 13, 2018 8:06 PM
    To: Black, Carey M. 
<>
    Cc: 

 Mailing List 
<>
    Subject: Re: [grouper-users] syncAllPITTables ... does not fix all of the 
things it finds... bombs before finishing...
    
    Can you surround that by a try/catch so we get the full exception?
    
    try { new 
edu.internet2.middleware.grouper.misc.SyncPITTables().syncAllPITTables(); } 
catch (Exception e) { e.printStackTrace(); }
    
    Thanks!
    
    - Shilen
    
    On 4/13/18, 6:50 PM, 
"
 on behalf of Black, Carey M." 
<
 on behalf of 
>
 wrote:
    
        I was doing some "maintenance checks" and was surprised to find that 
when I ran " new 
edu.internet2.middleware.grouper.misc.SyncPITTables().syncAllPITTables();" 
Things were found to be missing.
        
        Some of the items appeared to be corrected up to a point during the 
run of the syncAllPITTables(). 
        
        However, the process bombs out:
        "
        Searching for missing active point in time memberships
        Found 592949 missing active point in time memberships
        ERROR 
edu.internet2.middleware.grouper.internal.dao.GrouperDAOException:
        Problem in HibernateSession: HibernateSession (6df09e64): notNew, 
notReadonly, READ_WRITE_NEW, activeTransaction, session (186514ee),
        Exception in save: 
edu.internet2.middleware.grouper.changeLog.ChangeLogEntry, ByObjectStatic, 
query: ', cacheable: null, cacheRegion: null, entityName: 
ChangeLogEntryEntity, tx type: null,
        Exception in save: 
edu.internet2.middleware.grouper.pit.PITMembership, 
edu.internet2.middleware.grouper.hibernate.ByObject@7adb263a,
        Problem in HibernateSession: HibernateSession (57da5aba): new, 
notReadonly, READ_WRITE_NEW, notActiveTransaction, session (186514ee),
        Exception in saveOrUpdate: 
edu.internet2.middleware.grouper.pit.PITMembership, ByObjectStatic, query: ', 
cacheable: null, cacheRegion: null, entityName: null, tx type: null
                at 
edu.internet2.middleware.grouper.hibernate.HibernateSession._internal_hibernateSessionCatch
 (HibernateSession.java:588)
                at 
edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession
 (HibernateSession.java:710)
                at 
edu.internet2.middleware.grouper.hibernate.ByObjectStatic.saveBatch 
(ByObjectStatic.java:535)
                at 
edu.internet2.middleware.grouper.internal.dao.hib3.Hib3ChangeLogEntryDAO.saveBatch
 (Hib3ChangeLogEntryDAO.java:61)
                at 
edu.internet2.middleware.grouper.pit.PITMembership.onPreSave 
(PITMembership.java:517)
                at 
edu.internet2.middleware.grouper.hibernate.ByObject.saveOrUpdate 
(ByObject.java:375)
                at 
edu.internet2.middleware.grouper.hibernate.ByObjectStatic$5.callback 
(ByObjectStatic.java:376)
                at 
edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession
 (HibernateSession.java:700)
                at 
edu.internet2.middleware.grouper.hibernate.ByObjectStatic.saveOrUpdate 
(ByObjectStatic.java:363)
                at 
edu.internet2.middleware.grouper.internal.dao.hib3.Hib3PITMembershipDAO.saveOrUpdate
 (Hib3PITMembershipDAO.java:41)
                at edu.internet2.middleware.grouper.pit.PITMembership.save 
(PITMembership.java:267)
                at 
edu.internet2.middleware.grouper.misc.SyncPITTables.processMissingActivePITMemberships
 (SyncPITTables.java:328)
                at 
edu.internet2.middleware.grouper.misc.SyncPITTables.syncAllPITTables 
(SyncPITTables.java:236)
                at 
edu.internet2.middleware.grouper.misc.SyncPITTables$syncAllPITTables.call 
(Unknown Source)
        "
        
        OK... 592k is a fairly large set....
        
        I tracked the stacktrace back to lines 280(+ ish) of 
edu.internet2.middleware.grouper.misc.SyncPITTables.java .
        
        And since I see the log output of " Found 592949 missing active point 
in time memberships"... I appear to be able to execute through line 286...
        Which is just before a for loop. :)
        
        So... to try to figure out what is going on I dropped into gsh to see 
what I could see with a manual iteration of the code. :)
        
        // root sesson
        rsess = GrouperSession.startRootSession();
        // find the missing memberships
        mships = 
GrouperDAOFactory.getFactory().getPITMembership().findMissingActivePITMemberships();
        ///.... A long time passes.....
        /// And a lot of output happens. (I am guessing 592949 worth of a 
"Set<Membership>" :) ) 
        ///     OK... :)
        mships.size()
        // ===> 590930
        //
        //    hum.... not exactly the same count as when it failed... 
        //    maybe it fixed some before it failed?
        //
        // Get the iterator... for the for loop and manually try to do the 
first one returned....
        Iter = mships.iterator();
        // make sure it has something
        iter.hasNext();
        //===> true
        // get the first member
        mship = iter.next();
        //===> 
Membership[createTime=1522953142643,creatorUuid=10e2f04137854100be0a0b6688200b9a,depth=0,listName=members,listType=list,memberUuid=d4e035ab57514286928ec105e34c4edb,groupId=8a9982154ecc432089b2a5d3ade41a36,type=immediate]
        // get the pitField
        pitField = 
GrouperDAOFactory.getFactory().getPITField().findBySourceIdActive(mship.getFieldId(),
 true)
        // get the pitMember
        pitMember = 
GrouperDAOFactory.getFactory().getPITMember().findBySourceIdActive(mship.getMemberUuid(),
 true);
        // set a new UUID for the membership
        
pitMembership.setId(edu.internet2.middleware.grouper.internal.util.GrouperUuid.getUuid());
        //===> null
        ///  ^^^^^^ null ??? Is that right????
        pitMembership.getId();
        //===> 17bd3c3339d44bb6bf87aa947ad99e1e
        // oh.. it has one.. it just did not get echoed... ok... moving on...
        pitMembership.setMemberId(pitMember.getId());
        // ===> null
        pitMembership.getMemberId();
        // ===> 0bb0d58dee9249268a7ac31962ab6c73
        pitMembership.setFieldId(pitField.getId());
        // ===> null
        pitMembership.getFieldId();
        // ===> f5a893ef1a764c039f8c8286a47e8488
        pitMembership.setActiveDb("T");
        // ===> null
        pitMembership.getActiveDb();
        //===> T
        pitMembership.setStartTimeDb(System.currentTimeMillis() * 1000);
        // ===> null
        pitMembership.getStartTimeDb();
        // ===> 1523657695155000
        
pitMembership.setOwnerGroupId(GrouperDAOFactory.getFactory().getPITGroup().findBySourceIdActive(mship.getOwnerGroupId(),
 true).getId());
        // ===> null
        pitMembership.getOwnerGroupId();
        // ===> 8e465160bb8a4541b4b466da3bc3788b
        
pitMembership.setOwnerGroupId(GrouperDAOFactory.getFactory().getPITGroup().findBySourceIdActive(mship.getOwnerGroupId(),
 true).getId());
        //===> null
        mship.getContextId()
        //===> 88d48142c2574bb9b01c5f61eb9b9876
        !GrouperUtil.isEmpty(mship.getContextId())
        //===> true
        pitMembership.setContextId(mship.getContextId());
        //===> null
        pitMembership.getContextId();
        //===> 88d48142c2574bb9b01c5f61eb9b9876
        includeFlattenedMemberships = 
GrouperLoaderConfig.retrieveConfig().propertyValueBoolean("changeLog.includeFlattenedMemberships",
 true);
        //===> true
        includeFlattenedPrivileges = 
GrouperLoaderConfig.retrieveConfig().propertyValueBoolean("changeLog.includeFlattenedPrivileges",
 true);
        // ===> true
        
pitMembership.setFlatMembershipNotificationsOnSaveOrUpdate(includeFlattenedMemberships);
        // ===> null
        
pitMembership.setFlatPrivilegeNotificationsOnSaveOrUpdate(includeFlattenedPrivileges);
        // ===> null
        includeRolesWithPermissionChanges = 
GrouperLoaderConfig.retrieveConfig().propertyValueBoolean("changeLog.includeRolesWithPermissionChanges",
 false);
        //===> false
        groovy:000> 
pitMembership.setNotificationsForRolesWithPermissionChangesOnSaveOrUpdate(includeRolesWithPermissionChanges);
        //===> null
        pitMembership.save();
        
        // and boom .. .on the first one out of the set .   :)
        "
        ERROR 
edu.internet2.middleware.grouper.internal.dao.GrouperDAOException:
        Problem in HibernateSession: HibernateSession (75413bc6): new, 
notReadonly, READ_WRITE_NEW, notActiveTransaction, session (77b76a1b),
        Exception in saveOrUpdate: 
edu.internet2.middleware.grouper.pit.PITMembership, ByObjectStatic, query: ', 
cacheable: null, cacheRegion: null, entityName: null, tx type: null
                at 
edu.internet2.middleware.grouper.hibernate.HibernateSession._internal_hibernateSessionCatch
 (HibernateSession.java:588)
                at 
edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession
 (HibernateSession.java:710)
                at 
edu.internet2.middleware.grouper.hibernate.ByObjectStatic.saveOrUpdate 
(ByObjectStatic.java:363)
                at 
edu.internet2.middleware.grouper.internal.dao.hib3.Hib3PITMembershipDAO.saveOrUpdate
 (Hib3PITMembershipDAO.java:41)
                at edu.internet2.middleware.grouper.pit.PITMembership.save 
(PITMembership.java:267)
                at 
edu.internet2.middleware.grouper.pit.PITMembership$save$12.call (Unknown 
Source)
        "
        
        
        I suspect that the root source of all of these are related to some 
large groups being loaded by the Loader process and things not "always 
working out".
        However, the "fix it" process (AKA: syncAllPITTables()) fails.
        
        Any ideas? 
        Is this addressed in a later patch? Than I am on? ( 
grouper_v2_3_0_api_patch_79.state = applied  ... Yea.. I have a few to 
install yet....)
        
        Thanks in advance.
        
        -- 
        Carey Matthew