Grouper Users - Open Discussion List

[Shilen Patel <>]

Can you surround that by a try/catch so we get the full exception?

try { new 
edu.internet2.middleware.grouper.misc.SyncPITTables().syncAllPITTables(); } 
catch (Exception e) { e.printStackTrace(); }

Thanks!

- Shilen

On 4/13/18, 6:50 PM, 
"
 on behalf of Black, Carey M." 
<
 on behalf of 
>
 wrote:

    I was doing some "maintenance checks" and was surprised to find that when 
I ran " new 
edu.internet2.middleware.grouper.misc.SyncPITTables().syncAllPITTables();" 
Things were found to be missing.
    
    Some of the items appeared to be corrected up to a point during the run 
of the syncAllPITTables(). 
    
    However, the process bombs out:
    "
    Searching for missing active point in time memberships
    Found 592949 missing active point in time memberships
    ERROR edu.internet2.middleware.grouper.internal.dao.GrouperDAOException:
    Problem in HibernateSession: HibernateSession (6df09e64): notNew, 
notReadonly, READ_WRITE_NEW, activeTransaction, session (186514ee),
    Exception in save: 
edu.internet2.middleware.grouper.changeLog.ChangeLogEntry, ByObjectStatic, 
query: ', cacheable: null, cacheRegion: null, entityName: 
ChangeLogEntryEntity, tx type: null,
    Exception in save: edu.internet2.middleware.grouper.pit.PITMembership, 
edu.internet2.middleware.grouper.hibernate.ByObject@7adb263a,
    Problem in HibernateSession: HibernateSession (57da5aba): new, 
notReadonly, READ_WRITE_NEW, notActiveTransaction, session (186514ee),
    Exception in saveOrUpdate: 
edu.internet2.middleware.grouper.pit.PITMembership, ByObjectStatic, query: ', 
cacheable: null, cacheRegion: null, entityName: null, tx type: null
            at 
edu.internet2.middleware.grouper.hibernate.HibernateSession._internal_hibernateSessionCatch
 (HibernateSession.java:588)
            at 
edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession
 (HibernateSession.java:710)
            at 
edu.internet2.middleware.grouper.hibernate.ByObjectStatic.saveBatch 
(ByObjectStatic.java:535)
            at 
edu.internet2.middleware.grouper.internal.dao.hib3.Hib3ChangeLogEntryDAO.saveBatch
 (Hib3ChangeLogEntryDAO.java:61)
            at edu.internet2.middleware.grouper.pit.PITMembership.onPreSave 
(PITMembership.java:517)
            at 
edu.internet2.middleware.grouper.hibernate.ByObject.saveOrUpdate 
(ByObject.java:375)
            at 
edu.internet2.middleware.grouper.hibernate.ByObjectStatic$5.callback 
(ByObjectStatic.java:376)
            at 
edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession
 (HibernateSession.java:700)
            at 
edu.internet2.middleware.grouper.hibernate.ByObjectStatic.saveOrUpdate 
(ByObjectStatic.java:363)
            at 
edu.internet2.middleware.grouper.internal.dao.hib3.Hib3PITMembershipDAO.saveOrUpdate
 (Hib3PITMembershipDAO.java:41)
            at edu.internet2.middleware.grouper.pit.PITMembership.save 
(PITMembership.java:267)
            at 
edu.internet2.middleware.grouper.misc.SyncPITTables.processMissingActivePITMemberships
 (SyncPITTables.java:328)
            at 
edu.internet2.middleware.grouper.misc.SyncPITTables.syncAllPITTables 
(SyncPITTables.java:236)
            at 
edu.internet2.middleware.grouper.misc.SyncPITTables$syncAllPITTables.call 
(Unknown Source)
    "
    
    OK... 592k is a fairly large set....
    
    I tracked the stacktrace back to lines 280(+ ish) of 
edu.internet2.middleware.grouper.misc.SyncPITTables.java .
    
    And since I see the log output of " Found 592949 missing active point in 
time memberships"... I appear to be able to execute through line 286...
    Which is just before a for loop. :)
    
    So... to try to figure out what is going on I dropped into gsh to see 
what I could see with a manual iteration of the code. :)
    
    // root sesson
    rsess = GrouperSession.startRootSession();
    // find the missing memberships
    mships = 
GrouperDAOFactory.getFactory().getPITMembership().findMissingActivePITMemberships();
    ///.... A long time passes.....
    /// And a lot of output happens. (I am guessing 592949 worth of a 
"Set<Membership>" :) ) 
    /// OK... :)
    mships.size()
    // ===> 590930
    //
    //    hum.... not exactly the same count as when it failed... 
    //    maybe it fixed some before it failed?
    //
    // Get the iterator... for the for loop and manually try to do the first 
one returned....
    Iter = mships.iterator();
    // make sure it has something
    iter.hasNext();
    //===> true
    // get the first member
    mship = iter.next();
    //===> 
Membership[createTime=1522953142643,creatorUuid=10e2f04137854100be0a0b6688200b9a,depth=0,listName=members,listType=list,memberUuid=d4e035ab57514286928ec105e34c4edb,groupId=8a9982154ecc432089b2a5d3ade41a36,type=immediate]
    // get the pitField
    pitField = 
GrouperDAOFactory.getFactory().getPITField().findBySourceIdActive(mship.getFieldId(),
 true)
    // get the pitMember
    pitMember = 
GrouperDAOFactory.getFactory().getPITMember().findBySourceIdActive(mship.getMemberUuid(),
 true);
    // set a new UUID for the membership
    
pitMembership.setId(edu.internet2.middleware.grouper.internal.util.GrouperUuid.getUuid());
    //===> null
    ///  ^^^^^^ null ??? Is that right????
    pitMembership.getId();
    //===> 17bd3c3339d44bb6bf87aa947ad99e1e
    // oh.. it has one.. it just did not get echoed... ok... moving on...
    pitMembership.setMemberId(pitMember.getId());
    // ===> null
    pitMembership.getMemberId();
    // ===> 0bb0d58dee9249268a7ac31962ab6c73
    pitMembership.setFieldId(pitField.getId());
    // ===> null
    pitMembership.getFieldId();
    // ===> f5a893ef1a764c039f8c8286a47e8488
    pitMembership.setActiveDb("T");
    // ===> null
    pitMembership.getActiveDb();
    //===> T
    pitMembership.setStartTimeDb(System.currentTimeMillis() * 1000);
    // ===> null
    pitMembership.getStartTimeDb();
    // ===> 1523657695155000
    
pitMembership.setOwnerGroupId(GrouperDAOFactory.getFactory().getPITGroup().findBySourceIdActive(mship.getOwnerGroupId(),
 true).getId());
    // ===> null
    pitMembership.getOwnerGroupId();
    // ===> 8e465160bb8a4541b4b466da3bc3788b
    
pitMembership.setOwnerGroupId(GrouperDAOFactory.getFactory().getPITGroup().findBySourceIdActive(mship.getOwnerGroupId(),
 true).getId());
    //===> null
    mship.getContextId()
    //===> 88d48142c2574bb9b01c5f61eb9b9876
    !GrouperUtil.isEmpty(mship.getContextId())
    //===> true
    pitMembership.setContextId(mship.getContextId());
    //===> null
    pitMembership.getContextId();
    //===> 88d48142c2574bb9b01c5f61eb9b9876
    includeFlattenedMemberships = 
GrouperLoaderConfig.retrieveConfig().propertyValueBoolean("changeLog.includeFlattenedMemberships",
 true);
    //===> true
    includeFlattenedPrivileges = 
GrouperLoaderConfig.retrieveConfig().propertyValueBoolean("changeLog.includeFlattenedPrivileges",
 true);
    // ===> true
    
pitMembership.setFlatMembershipNotificationsOnSaveOrUpdate(includeFlattenedMemberships);
    // ===> null
    
pitMembership.setFlatPrivilegeNotificationsOnSaveOrUpdate(includeFlattenedPrivileges);
    // ===> null
    includeRolesWithPermissionChanges = 
GrouperLoaderConfig.retrieveConfig().propertyValueBoolean("changeLog.includeRolesWithPermissionChanges",
 false);
    //===> false
    groovy:000> 
pitMembership.setNotificationsForRolesWithPermissionChangesOnSaveOrUpdate(includeRolesWithPermissionChanges);
    //===> null
    pitMembership.save();
    
    // and boom .. .on the first one out of the set .   :)
    "
    ERROR edu.internet2.middleware.grouper.internal.dao.GrouperDAOException:
    Problem in HibernateSession: HibernateSession (75413bc6): new, 
notReadonly, READ_WRITE_NEW, notActiveTransaction, session (77b76a1b),
    Exception in saveOrUpdate: 
edu.internet2.middleware.grouper.pit.PITMembership, ByObjectStatic, query: ', 
cacheable: null, cacheRegion: null, entityName: null, tx type: null
            at 
edu.internet2.middleware.grouper.hibernate.HibernateSession._internal_hibernateSessionCatch
 (HibernateSession.java:588)
            at 
edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession
 (HibernateSession.java:710)
            at 
edu.internet2.middleware.grouper.hibernate.ByObjectStatic.saveOrUpdate 
(ByObjectStatic.java:363)
            at 
edu.internet2.middleware.grouper.internal.dao.hib3.Hib3PITMembershipDAO.saveOrUpdate
 (Hib3PITMembershipDAO.java:41)
            at edu.internet2.middleware.grouper.pit.PITMembership.save 
(PITMembership.java:267)
            at 
edu.internet2.middleware.grouper.pit.PITMembership$save$12.call (Unknown 
Source)
    "
    
    
    I suspect that the root source of all of these are related to some large 
groups being loaded by the Loader process and things not "always working out".
    However, the "fix it" process (AKA: syncAllPITTables()) fails.
    
    Any ideas? 
    Is this addressed in a later patch? Than I am on? ( 
grouper_v2_3_0_api_patch_79.state = applied  ... Yea.. I have a few to 
install yet....)
    
    Thanks in advance.
    
    -- 
    Carey Matthew