Grouper Users - Open Discussion List

[Siju Jacob <>]

Hi Team,

     We are using grouper 2.3 with latest updates. We are trying to scale our grouper to enterprise level. We are facing some performance issue due to my limited knowledge on grouper.

    While trying to run the loaders with below configuration its taking couple of days for a loader with 3000 groups and 300k members to load.

 

    We are running on JDK 1.7 , Linux system with X86 Large - 4vCPU 16GB Mem and Centos, Oracle 11g database.

 

    I am having little trouble in understanding below configuration (groupThreadPoolSize) and the connection pool size (hibernate.c3p0.max_size = 100).

 

    Based on the below configuration, if there is 10 loaders with more than 1000 groups and 200k members each, running at the same time going to spawn 20 threads in total or is it going to start 10*20=200 threads in total?

 

     I believe each thread is going to use one connection from the connection pool. Please correct me if I am wrong.

 

     If hibernate.c3p0.max_size = 100 and all 100 connections is going to be used by loaders there won’t be any connection left for grouper UI?

 

     Please let me know what will be the ideal setting for the below configurations based on your experience for the enterprise level grouper with above mentioned load.

 

Any advice or guidance will be of great help and would be greatly appreciated..!

 

 

-à grouper-loader.base.properties

# number of threads in the loader threadpool.  Only this number of jobs can run at once

# jobs which are on deck will block, or will fail if the blocking timeout occurs

# a job is running if it is loading (not just scheduled)

loader.thread.pool.size=10

 

# if should use threads in the loader for add/remove member

loader.use.membershipThreads=true

 

# number of threads to use for each group job (not shared among jobs)

loader.membershipThreadPoolSize=10

 

# if should use threads in the loader for each group in a group of groups

loader.use.groupThreads=true

 

# number of threads to use for each list of groups job (not shared among jobs)

loader.groupThreadPoolSize=20

 

 

-àgrouper.hibernate.base.properties

 

hibernate.c3p0.max_size = 100

 

Thanks,

Siju