Grouper Users - Open Discussion List

["Bryan E. Wooten" <>]

Hmmm,

 

Shutdown the loader daemon and re-ran the syncAllPITTables() command and get this:

 

gsh 2%  new edu.internet2.middleware.grouper.misc.SyncPITTables().syncAllPITTables()

 

 

Searching for missing active point in time fields

Found 0 missing active point in time fields

 

 

Searching for missing active point in time members

Found 0 missing active point in time members

 

 

Searching for missing active point in time stems

Found 0 missing active point in time stems

 

 

Searching for missing active point in time groups

Found 0 missing active point in time groups

 

 

Searching for missing active point in time role sets

Found 0 missing active point in time role sets

 

 

Searching for missing active point in time attribute defs

Found 0 missing active point in time attribute defs

 

 

Searching for missing active point in time attribute def names

Found 0 missing active point in time attribute def names

 

 

Searching for missing active point in time attribute def name sets

Found 0 missing active point in time attribute def name sets

 

 

Searching for missing active point in time actions

Found 0 missing active point in time actions

 

 

Searching for missing active point in time action sets

Found 0 missing active point in time action sets

 

 

Searching for missing active point in time group sets

Found 0 missing active point in time group sets

 

 

Searching for missing active point in time memberships

Found 218873 missing active point in time memberships

// Error: unable to evaluate command: Sourced file: inline evaluation of: `` new edu.internet2.middleware.grouper.misc.SyncPITTables().syncAllPITTables();'' : Method Invocation syncAllPITTables

// See error log for full stacktrace

// caused by: java.lang.RuntimeException:

// Active PITGroup with sourceId=c07a606ca3da4146b8ecf85265a52222 not found

gsh 3%

 

We are early in development, should I just re-init my DB and start from scratch? Or is there way out of this mess?

 

-Bryan

 

From: Shilen Patel [mailto:]
Sent: Friday, May 10, 2013 8:08 AM
To: Bryan E. Wooten
Cc:
Subject: Re: [grouper-users] Out of memory during loader start with full sync = true

 

 

 

From: "Bryan E. Wooten" <>
Date: Friday, May 10, 2013 9:59 AM
To: "" <>
Subject: [grouper-users] Out of memory during loader start with full sync = true

 

We renamed and moved some folders and groups. So I decided that a full sync was in order. I set all my Load LDAP groups cron entry to run next Wednesday and made the same schedule for the PSP change log consumer.

 

While it was running (this can take a long while with many groups having over 10k members, some 100k) I started to notice this exception (it occurred well over 100 times):

java.lang.RuntimeException: Active PITGroup with sourceId=6fe2f5c77a7b47509e86ead620ed2c2d not found,

Problem in HibernateSession: HibernateSession: isNew: true, isReadonly: false, grouperTransactionType: READ_WRITE_NEW

        at edu.internet2.middleware.grouper.internal.dao.hib3.Hib3PITGroupDAO.findBySourceIdActive(Hib3PITGroupDAO.java:100)

        at edu.internet2.middleware.grouper.changeLog.ChangeLogTempToEntity.processGroupUpdate(ChangeLogTempToEntity.java:286)

 

 

 

In a previous email, did you say you had deleted 300k change log entries?  Were some or all of them in the temp table?  If so, that might explain this error.  Try running this:

 

gsh 0% new edu.internet2.middleware.grouper.misc.SyncPITTables().syncAllPITTables()

 

I don't think this is related to your memory error though.

 

 

And then after about an hour running I got an out of memory error so I stopped the loader.

 

I see in the gsh start script this:

 

if [ "$MEM_MAX" = "" ]; then

MEM_MAX=750m

Fi

 

I guess I need to set this in setenv.sh?

 

But do I also need to make these settings in grouper-loader.properties:

 

# Omit diff responses from bulk response to conserve memory.

# changeLog.psp.fullSync.omitDiffResponses = true

 

# Omit sync responses from bulk response to conserve memory.

# changeLog.psp.fullSync.omitSyncResponses = true

 

 

Do you have a stack trace for the memory error?  I think setting those 2 properties would help.

 

Thanks!

 

-- Shilen