Grouper Users - Open Discussion List

["Hyzer, Chris" <>]

We fixed this in august and it was supposed to be in a patch but it appears a 
patch was never made.  Sorry about that.

https://bugs.internet2.edu/jira/browse/GRP-1365

I just made one (patch 30).  Yes, now that loader jobs are threaded, it uses 
more connections, finishes faster.

Michael / Scott (not an "Office" reference) or anyone, if you think the 
params in the default need to be changed let me know

https://github.com/Internet2/grouper/blob/GROUPER_2_3_BRANCH/grouper/conf/grouper.hibernate.base.properties

Thanks,
Chris

-----Original Message-----
From: 

 
[mailto:]
 On Behalf Of Michael R. Gettes
Sent: Tuesday, November 01, 2016 7:35 PM
To: Scott Koranda 
<>
Cc: grouper-users 
<>
Subject: Re: [grouper-users] exhausting hibernate JDBC connection pool

Scott, while I can’t answer the why questions you ask I can share my recipe 
after spending some time with Shilen at TechX and pushed me to really figure 
this out.  Here is what we use to manage about 64K course groups for last, 
current and next semester.  I run 2 loader jobs every morning and processing 
takes about 1 hour on a 2 CPU VM with 12G (we run loader jobs on 2 instances 
of Grouper against an Oracle DB on SSD disk).  I hope this helps.

hibernate.c3p0.max_size = 40
hibernate.c3p0.min_size = 5
hibernate.c3p0.idle_test_period = 300
hibernate.c3p0.timeout = 1800
hibernate.c3p0.max_statements = 50
hibernate.c3p0.acquireIncrement = 1
hibernate.c3p0.checkoutTimeout=60000

/mrg

> On Nov 1, 2016, at 17:33, Scott Koranda 
> <>
>  wrote:
> 
> Hi,
> 
> I am helping manage a large Grouper deployment that manages a
> number of groups for a campus, including groups that represent
> course offerings ("course groups").
> 
> November 1 marks a "transition" from one set of course groups
> to another. The result is a large amount of work for 3 loader
> jobs that manage the course groups.
> 
> In the past earlier versions of Grouper, all running on the
> same VM (no changes in the available resources have happened),
> have ridden out this transition without issue.
> 
> Today, however, using Grouper version 2.3 was different. We
> saw that the loader consumed all of the available JDBC
> connections. The default was set in
> grouper.hibernate.base.properties:
> 
> hibernate.c3p0.max_size = 16
> 
> This caused a number of loader jobs, including maintenance
> jobs like CHANGE_LOG_changeLogTempToChangeLog, to be starved.
> 
> Stopping the loader and changing to
> 
> hibernate.c3p0.max_size = 32
> 
> made the issue go away.
> 
> My question is what might be different in Grouper version 2.3
> that caused all of the JDBC connections to be consumed where
> previously they were not? 
> 
> I am wondering if code has changed that causes the Grouper
> loader to be more "aggressive" in how it processes "a lot of
> work"?
> 
> I also note that we did not see this issue on the dev system.
> That VM, however, has less CPU cores and memory available to
> it, so I am wondering if that constrained the loader in some
> way and prevented it from scaling up and exhausting the JDBC
> connections?
> 
> I appreciate any input the Grouper team has.
> 
> Thanks,
> 
> Scott K