Skip to Content.
Sympa Menu

grouper-users - Re: [grouper-users] Re: [JIRA] (GRP-1381) Use member table subject identifier in loader jobs to improve performance

Subject: Grouper Users - Open Discussion List

List archive

Re: [grouper-users] Re: [JIRA] (GRP-1381) Use member table subject identifier in loader jobs to improve performance


Chronological Thread 
  • From: Jared Hoffman <>
  • To: Shilen Patel <>
  • Cc: "" <>
  • Subject: Re: [grouper-users] Re: [JIRA] (GRP-1381) Use member table subject identifier in loader jobs to improve performance
  • Date: Fri, 14 Oct 2016 16:40:30 -0400
  • Ironport-phdr: 9a23:JWgXuB8fsRqvEP9uRHKM819IXTAuvvDOBiVQ1KB+2u4cTK2v8tzYMVDF4r011RmSAtWdtqkP0reempujcFJDyK7JiGoFfp1IWk1NouQttCtkPvS4D1bmJuXhdS0wEZcKflZk+3amLRodQ56mNBX660e/5j8KGxj5KRE9ZqGsQtaT3IyL0LWJ+5zXbgMAvDOhbLc6eA62rAHcsOEanIhvbKs9102ajGFPfrF0zHllKRqzmBjy4862tMpj9Cpevvsn9eZDXKPreKJ+QLBFWmd1e1sp7dHm4EGQBTCE4WERByBPykJF

Wonderful, thank you! I can confirm similar levels of improvement with our data. 

For 1820 members, 320 seconds down to 4.5 seconds.
For 146 members, 26 seconds down to < 1 second.

I also tested this with the "run loader process to sync group" in the UI. The patch appears to only affect the scheduled cron tasks. I did configure the subjectIdentifierAttribute0 in the sources.xml for the UI and API.






On Fri, Oct 14, 2016 at 12:13 PM, Shilen Patel <> wrote:
I've added a new patch for 2.3 that significantly improves the performance of the loader in cases where you're using a subject identifier instead of subject id.   I did a simple test of re-running a loader job for a group with 2936 members where the loader job used subject identifier. Before it was taking 90-100 seconds. Now it takes less than 1 second.

It requires that you configure the sources.xml file to save the subject identifier into grouper's grouper_member table.  See https://bugs.internet2.edu/jira/browse/GRP-1381.

Thanks!

- Shilen

On 10/5/16, 12:16 PM, "Shilen Patel (JIRA)" <> wrote:

Shilen Patel created GRP-1381:
---------------------------------

             Summary: Use member table subject identifier in loader jobs to improve performance
                 Key: GRP-1381
                 URL: https://bugs.internet2.edu/jira/browse/GRP-1381
             Project: Grouper
          Issue Type: Bug
      Security Level: Standard (Standard bug, may impact functionality but does not represent a security vulnerability)
          Components: grouperLoader
            Reporter: Shilen Patel
            Assignee: Shilen Patel
             Fix For: 2.3.1, 2.3.0.patch






--
This message was sent by Atlassian JIRA
(v6.4.11#64026)




--
Jared Hoffman  Associate Director for Enterprise Infrastructure
Kenyon College LBIS    740.427.5948




Archive powered by MHonArc 2.6.19.

Top of Page