Grouper Users - Open Discussion List

[Shilen Patel <>]

From: "Bryan E. Wooten" <>
Date: Friday, May 10, 2013 9:59 AM
To: "" <>
Subject: [grouper-users] Out of memory during loader start with full sync = true

We renamed and moved some folders and groups. So I decided that a full sync was in order. I set all my Load LDAP groups cron entry to run next Wednesday and made the same schedule for the PSP change log consumer.

 

While it was running (this can take a long while with many groups having over 10k members, some 100k) I started to notice this exception (it occurred well over 100 times):

java.lang.RuntimeException: Active PITGroup with sourceId=6fe2f5c77a7b47509e86ead620ed2c2d not found,

Problem in HibernateSession: HibernateSession: isNew: true, isReadonly: false, grouperTransactionType: READ_WRITE_NEW

        at edu.internet2.middleware.grouper.internal.dao.hib3.Hib3PITGroupDAO.findBySourceIdActive(Hib3PITGroupDAO.java:100)

        at edu.internet2.middleware.grouper.changeLog.ChangeLogTempToEntity.processGroupUpdate(ChangeLogTempToEntity.java:286)

 

In a previous email, did you say you had deleted 300k change log entries?  Were some or all of them in the temp table?  If so, that might explain this error.  Try running this:

gsh 0% new edu.internet2.middleware.grouper.misc.SyncPITTables().syncAllPITTables()

I don't think this is related to your memory error though.

And then after about an hour running I got an out of memory error so I stopped the loader.

 

I see in the gsh start script this:

 

if [ "$MEM_MAX" = "" ]; then

MEM_MAX=750m

Fi

 

I guess I need to set this in setenv.sh?

 

But do I also need to make these settings in grouper-loader.properties:

 

# Omit diff responses from bulk response to conserve memory.

# changeLog.psp.fullSync.omitDiffResponses = true

 

# Omit sync responses from bulk response to conserve memory.

# changeLog.psp.fullSync.omitSyncResponses = true

 

Do you have a stack trace for the memory error?  I think setting those 2 properties would help.

Thanks!

-- Shilen