Subject: Grouper Users - Open Discussion List
- From: "Misagh Moayyed" <>
- To: <>
- Subject: RE: [grouper-users] RE: grouper web service crash
- Date: Thu, 16 Apr 2015 09:00:26 -0700 (MST)
On #3, Grouper uses a patched hibernate jar. If you are pulling jars from anywhere else other than the grouper distro, it may cause this problem. The jar you pulled down is not the same thing that grouper uses.
1. That is a superfluous warning, you can ignore it, or you can patch the WS manually and recompile so it doesn’t happen anymore
2. I don’t know why you wouldn’t be able to get connections from the database. If the DBA can monitor how many connections from where are being used, maybe it can point to a problem? Maybe you can change logging from c3p0? Maybe put that in its own file which is rotated and cant fill up the filesystem (it’s a LOT of output). Maybe you need to increase the max in c3p0 to 50 instead of 16. Maybe you need more app servers.
log4j.logger.com.mchange = DEBUG
3. Regarding this: Caused by: java.lang.NoSuchFieldError: INSTANCE, When I google that it is a jar problem. Ive never seen it before. Was the whole stack in the email? Since you have not used the Grouper jars but have pulled them in from maven, maybe something isn’t correct? Nothing jumps out at me when I glance at the list (looking for duplicates). Maybe you could try building Grouper WS without maven (the Grouper ant file) and see if it helps? J Note, improving jar management is on our roadmap for v2.3
We have been experiencing a lot of problems where grouper web service is crashing almost once or twice every week.
Eventually we get the problem where grouper is unable to checkout jdbc connections.
I tried to read the code at HibernateSession.java:571 and am unsure if the problem arises because connections aren’t getting closed; one of the typical error we see in the logs is:
2015-04-13 20:04:32,656: [TP-Processor19] WARN WsStemLookup.retrieveStemIfNeeded(208) - < etc:wsusers:grouper-wsuser-ursa - 220.127.116.11 > - Invalid query: WsStemLookup[
2015-04-13 20:04:32,661: [pool-1-thread-750] ERROR Member$2.call(3845) - - Error updating member attributes:
edu.internet2.middleware.grouper.internal.dao.GrouperDAOException: Problem in HibernateSession: HibernateSession: isNew: true, isReadonly: false, grouperTransactionType: READ_WRITE_NEW
Caused by: java.lang.NoSuchFieldError: INSTANCE
I think add more memory.
There are limits in the config files for max number of records retrieved, that might help.
Hmmm, not sure why not check out JDBC connections, maybe need to change a database setting or a config settings to have a larger pool?
We had an incident where grouper web service crashed. Tomcat logs showed that there was an out of heap memory exception.
We suspect that this was caused because an application was retrieving 10’s of thousands of groups information. Is there any way we
could have configured grouper to fail more gracefully?
Secondly, the same application had typos in some web service requests and we saw invalid query being logged. Later on, grouper crashed
again because it could not check out a jdbc connection from the pool — had to restart tomcat again.
- [grouper-users] Re: grouper web service crash, Kediyal, Prashant, 04/14/2015
Archive powered by MHonArc 2.6.16.