Skip to Content.
Sympa Menu

grouper-users - [grouper-users] GC overhead limit exceeded on 2.3.0 + patches

Subject: Grouper Users - Open Discussion List

List archive

[grouper-users] GC overhead limit exceeded on 2.3.0 + patches


Chronological Thread 
  • From: Michael R Gettes <>
  • To: " Mailing List" <>
  • Subject: [grouper-users] GC overhead limit exceeded on 2.3.0 + patches
  • Date: Mon, 9 Jan 2017 16:40:35 -0500
  • Ironport-phdr: 9a23:mzz44hNGBTg5XYa/kWEl6mtUPXoX/o7sNwtQ0KIMzox0IvX8rarrMEGX3/hxlliBBdydsKMYzbCK+Pm9ACQp2tWoiDg6aptCVhsI2409vjcLJ4q7M3D9N+PgdCcgHc5PBxdP9nC/NlVJSo6lPwWB6nK94iQPFRrhKAF7Ovr6GpLIj8Swyuu+54Dfbx9GiTe5br5+Ngu6oAreusULgYZuN7s6xwfUrHdPZ+lY335jK0iJnxb76Mew/Zpj/DpVtvk86cNOUrj0crohQ7BAAzsoL2465MvwtRneVgSP/WcTUn8XkhVTHQfI6gzxU4rrvSv7sup93zSaPdHzQLspVzmu87tnRRn1gyoBKjU38nzYitZogaxbvB2vqBNwzZPXbo+bKvRwebjQfc8DRWpEQspRVzBNDp26YoASD+QBJ+FYr4zlqlUJrBuxGQmsC/nxxT9Mm3T4wbAx3eU/HgHawAwvAckOu2nTodXwLqcSXu64wLXTwDXbdPNWxC3x55bVfRA8uPyBW697f8TWyUkqDQzFj1OQpJTqPzOP0eQNrnCb4PR6Ve61kWEnsxlxriCzxscskInGmJgaxU3e+SVj3YY1I8W0SEFhYd6jDZtQrz+VN5FqTcw4R2FovDw1xaYCuZGlZiQF1I4oxxrba/CffYiI/w7jVPuLIThmmHJqYqm/hxWo/Ui61+38UM+520tJoCpditTAq24B2h/J5sSaV/dw/V2t1SuA2g3S8O1IPE45mKvdJpU82LA/jIATvl7GHiLumEX5kquWdkI89+i08eTrfKvqppmBO4NpkA7yKL4hmtehDuQkKAQOR3Ob+fmm2LL94EL5Xa1GjvwwkqbHrJDXPdkXqrC6DgNPz4ov9hWyAy273NkXnHQLNk9JdRyGgoTxPlHBOvH4DfOxg1S2lzdrwujLPrj7ApXCIHXPjKzsfa5m605B0gY8081Q549MBrEbPP3zQlPxtMDfDhIhPAy02eHnCMhl1owAQ2KDG7KZMLjMvl+S/eIiOO2MZI4OuDbhMPgp+eTijX4/mV8BY6apx50XZ26kHvh4OUmWf2fjgspSWVsN60AxVuv3kFCYFCNIamypd6M6+jwhDo+6V8HOSp3lpbWA2Ca9EdVyZ3taQBjYHm3vap2JQbIRcy+IOedglCAJT76sV9Vn2B2z4lzU0b1ie8/V8S0RuJar7pBa6vHPmAp6oTluAJ/F+3mWUid5kn5eFGx+57x2vUEokgTL6qN/mfENTdE=

Hi,

We run in a 12GB VM with -Xmx4096m on the loader and we are not using any
swap space and 2.3.0 + patches (applied in december, i think).

After a long running loader job we see the error at the bottom of this mail.
When we get this error, remaining loader jobs go bad and we have to restart
the loader. We are going to add memory. But, should we be using a specific
GC algorithm or taking some other steps for a large environment? We
currently have about 8M memberships (14M loader memberships) over 80K groups.
We have only been in production since June. I am seeking any
advice/guidance on what to do cuz I fear we are going to see this problem
again. We have a number of loader jobs managing groups of 100K to 230K. We
do run concurrent jobs on multiple servers (currently 2 cores for 2 loader
job servers about to go to 4 cores).

I should also note: while we run with 4096m the loader nightly report shows
heapsize as 3GB.
heapSize: 3 GB
heapMaxSize: 3 GB
heapSizeFree: 1 GB
version: 2.3.0, build date: 2016/04/16 05:37:21, env: PRODUCTION
os.name: Linux
os.arch: amd64
os.version: 2.6.32-642.6.2.el6.x86_64
java.version: 1.7.0_121
java.vendor: Oracle Corporation

Thank you!

/mrg

2017-01-09 08:56:48,254: [pool-1-thread-6229] ERROR
GrouperLoaderType.syncGroupLogicForOneGroup(1634) - - Error in job for
group: Course:2171:UP:AA:044532:all
edu.internet2.middleware.grouper.internal.dao.GrouperDAOException: Problem in
HibernateSession: null
at
edu.internet2.middleware.grouper.hibernate.HibernateSession._internal_hibernateSessionCatch(HibernateSession.java:604)
at
edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession(HibernateSession.java:710)
at
edu.internet2.middleware.grouper.app.loader.db.Hib3GrouperLoaderLog.store(Hib3GrouperLoaderLog.java:730)
at
edu.internet2.middleware.grouper.app.loader.GrouperLoaderType.syncGroupLogicForOneGroup(GrouperLoaderType.java:1621)
at
edu.internet2.middleware.grouper.app.loader.GrouperLoaderType.access$200(GrouperLoaderType.java:118)
at
edu.internet2.middleware.grouper.app.loader.GrouperLoaderType$11.callLogic(GrouperLoaderType.java:1542)
at
edu.internet2.middleware.grouper.app.loader.GrouperLoaderType$11.callLogic(GrouperLoaderType.java:1538)
at
edu.internet2.middleware.grouper.util.GrouperCallable$1.callback(GrouperCallable.java:164)
at
edu.internet2.middleware.grouper.GrouperSession.callbackGrouperSession(GrouperSession.java:974)
at
edu.internet2.middleware.grouper.util.GrouperCallable.callLogicWithSessionIfExists(GrouperCallable.java:161)
at
edu.internet2.middleware.grouper.util.GrouperCallable.call(GrouperCallable.java:132)
at java.util.concurrent.FutureTask.run(FutureTask.java:262)
at
java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1145)
at
java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:615)
at java.lang.Thread.run(Thread.java:745)
Caused by: java.lang.OutOfMemoryError: GC overhead limit exceeded
at java.util.Arrays.copyOf(Arrays.java:2367)
at
java.lang.AbstractStringBuilder.ensureCapacityInternal(AbstractStringBuilder.java:117)
at
java.lang.AbstractStringBuilder.append(AbstractStringBuilder.java:562)
at java.lang.StringBuffer.append(StringBuffer.java:320)
at java.io.StringWriter.write(StringWriter.java:94)
at
edu.internet2.middleware.grouperClient.config.ConfigPropertiesCascadeCommonUtils.copy(ConfigPropertiesCascadeCommonUtils.java:5088)
at
edu.internet2.middleware.grouperClient.config.ConfigPropertiesCascadeCommonUtils.copy(ConfigPropertiesCascadeCommonUtils.java:5073)
at
edu.internet2.middleware.grouperClient.config.ConfigPropertiesCascadeCommonUtils.toString(ConfigPropertiesCascadeCommonUtils.java:5056)
at
edu.internet2.middleware.grouperClient.config.ConfigPropertiesCascadeBase$ConfigFile.retrieveContents(ConfigPropertiesCascadeBase.java:636)
at
edu.internet2.middleware.grouperClient.config.ConfigPropertiesCascadeBase.filesNeedReloadingBasedOnContents(ConfigPropertiesCascadeBase.java:984)
at
edu.internet2.middleware.grouperClient.config.ConfigPropertiesCascadeBase.retrieveFromConfigFileOrCache(ConfigPropertiesCascadeBase.java:926)
at
edu.internet2.middleware.grouperClient.config.ConfigPropertiesCascadeBase.retrieveConfig(ConfigPropertiesCascadeBase.java:89)
at
edu.internet2.middleware.grouper.cfg.GrouperConfig.retrieveConfig(GrouperConfig.java:179)
at
edu.internet2.middleware.grouper.hibernate.HibernateSession.isReadonlyMode(HibernateSession.java:118)
at
edu.internet2.middleware.grouper.hibernate.HibernateSession.<init>(HibernateSession.java:181)
at
edu.internet2.middleware.grouper.hibernate.HibernateSession._internal_hibernateSession(HibernateSession.java:467)
at
edu.internet2.middleware.grouper.hibernate.HibernateSession.callbackHibernateSession(HibernateSession.java:659)
... 13 more




Archive powered by MHonArc 2.6.19.

Top of Page