Skip to Content.
Sympa Menu

grouper-users - [grouper-users] threaded loader jobs

Subject: Grouper Users - Open Discussion List

List archive

[grouper-users] threaded loader jobs

Chronological Thread 
  • From: Chris Hyzer <>
  • To: "" <>
  • Subject: [grouper-users] threaded loader jobs
  • Date: Mon, 3 Nov 2014 14:49:43 +0000
  • Accept-language: en-US

I added threads to loader jobs in two places:  when processing a list of groups, each group will get its own thread, and in a group, each membership add or delete will get its own thread.


This seems to speed things up by a factor of 4.


When loading 200 groups with 10 members each, the old loader took 41.4 seconds.


With threads on memberships (wont see a big gain in this case since it is only 10 members), it took: 34.5 seconds.


With threads on groups, it took 15.5 seconds.


With threads on groups and memberships it took: 11.2 seconds.


This was on my dev PC with mysql.  Maybe with a production database the gains will be more dramatic.


These four settings control the threading in the


# if should use threads in the loader for add/remove member



# number of threads to use for each group job (not shared among jobs)



# if should use threads in the loader for each group in a group of groups



# number of threads to use for each list of groups job (not shared among jobs)




This is sort of experimental, but I think we should default to on for 2.2.1 since it helps a lot and if it is a problem we can turn it off easily.




  • [grouper-users] threaded loader jobs, Chris Hyzer, 11/03/2014

Archive powered by MHonArc 2.6.16.

Top of Page