Skip to Content.
Sympa Menu

grouper-users - Re: [grouper-users] Optimized group provisionning in grouper database

Subject: Grouper Users - Open Discussion List

List archive

Re: [grouper-users] Optimized group provisionning in grouper database


Chronological Thread 
  • From: Shilen Patel <>
  • To: "" <>, "Hyzer, Chris" <>
  • Cc: "" <>
  • Subject: Re: [grouper-users] Optimized group provisionning in grouper database
  • Date: Fri, 16 Mar 2018 19:38:09 +0000
  • Accept-language: en-US
  • Authentication-results: oit.duke.edu; spf=none
  • Ironport-phdr: 9a23:MmaGox9+1MPOvv9uRHKM819IXTAuvvDOBiVQ1KB+0+0QIJqq85mqBkHD//Il1AaPAd2Araocw8Pt8InYEVQa5piAtH1QOLdtbDQizfssogo7HcSeAlf6JvO5JwYzHcBFSUM3tyrjaRsdF8nxfUDdrWOv5jAOBBr/KRB1JuPoEYLOksi7ze+/94HdbglSmDaxfa55IQmrownWqsQYm5ZpJLwryhvOrHtIeuBWyn1tKFmOgRvy5dq+8YB6/ShItP0v68BPUaPhf6QlVrNYFygpM3o05MLwqxbOSxaE62YGXWUXlhpIBBXF7A3/U5zsvCb2qvZx1S+HNsDwULs6Wymt771zRRHolikJKiI5/m/UhMx+jq1UrhOhqABwzYHbe4yVKOFxfqbBcd8GX2dMXMBcXDFBDIOmaIsPCvIMM/tYr4n8o1sBsx2+DhS2Cuz11j9HmHv21rA93uQ6EQHG3QggEMwIsHnPsdr1KaQSXv6vwKbW0DrDcupb1DHg44bGdRAhpOuDXbN2ccfJxkkvDQTFjkiIqYP7JTOay/4Cs2+B4+piT+2vlXQopxlwojip2scjlJPJhoQLxVDY6yl12pg6KsClSEJjed6kCpxQtyaUN4tuRcMiQmdouDogxrIavp67eTAGyJQ7xx7Dc/yHaJGE7xz5W+mJOTh4i3NleK6/hxav6kes0PHzVs6x0FlSoSpFk8HMumkT2xPO8MiIVOdy/ka71jmXyQ/T8f1LIUctmabFNpEhzaQwlp0OsUTbACD2glz6gLGLeUUh/ein9vnnYrH4qZ+HMYJ/lwLwMrw2l8CiAek0LBICUmaB9eih17Du+Vf1TKtEg/A2iqXVrZ7XKMUBqqO5AgJZyIUu5wy/AjqkyNgYk2cIIVFZdB+CioXpNVLDLfPiAvq6nVijjjNmyvDDM7H/BpjBMHrOnbLhcLtz60Ncxgw+xs1F6Z1OELEOOvfzV1fxtNPGCh85NBS5w+P9B9V80oMSQG2PA6GAPK/OtVKE/OUvI+iQZIMPpTn9Nvsl6OD0gXAnhFAdYLGl3ZoRaH+mAvRrO1iVbHryjtsbH2oGpBcyQe/uhVGYTzJeaXa/U7o55j4hCYKmCYnDRpqqgLyExCq7BZ1WZmFCCl+SF3flbJiLW+oPaC2MOMNujCELVaW5R487yR6urBP6y6ZgLufM9S0Yr5Xj1MVt5+LNjxE+7CF0D9+D3GGXVGF5hWcIRz4t3KBjukxx1E2P0al+g/xEC9NT/fVJXRkmNZLC1eB1Fc39Wh+SNuuOHRykX9GnDDd3QcgwydgIfU95F/24gxvC1mykGfVdw76WAJ059OTTxXn7IMBjwnLK/LMshFgvBMVVYynuzI5u5QXJQ8bik1+YjOziIaEX3D/f+X2rzHGF+lxAXQh2F6jJQCZMSFHRqIHC5kLGQrnmMb07Pw8JndWCK69HbPXlkFZLAvrvJYKNMCqKh26sCEPQlfu3Z43wdjBF0Q==
  • Spamdiagnosticmetadata: NSPM
  • Spamdiagnosticoutput: 1:99

After you upgraded to 2.3, did you analyze your tables?

https://spaces.internet2.edu/pages/viewpage.action?pageId=14517958#APIBuilding&Configuration-AnalyzingTablestoImproveQueryPerformance

What database do you use? How long does it take to run the create method in
your GrouperDAO once on average? Also, just wondering, have you tried
running multiple threads to allow creations to happen in parallel?

Thanks!

- Shilen

On 3/16/18, 5:50 AM,
"
on behalf of Julien Gribonvald"
<
on behalf of
>
wrote:

I'm using directly the grouper-api, and it uses api generic methods.
The project is here :
https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_GIP-2DRECIA_esco-2Dgrouper-2Dmodules_&d=DwIDaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=sWqutME58phurE0oO57Icg&m=ZtuV_vgGx5WjIRe39F0QbEfwJYcxAMFvJZbn2_tzxpg&s=Nb0iZ22adxZreTvg-8Rvy2lXQ07_IuONWAJo-sEDBLo&e=
And the main class used is here :

https://urldefense.proofpoint.com/v2/url?u=https-3A__github.com_GIP-2DRECIA_esco-2Dgrouper-2Dmodules_blob_master_esco-2Dgrouper-2Dsarapis-2DalimManager_src_main_java_org_esco_grouper_dao_GrouperDAO.java&d=DwIDaQ&c=imBPVzF25OnBgGmVOlcsiEgHoG1i6YHLR0Sj_gZ4adc&r=sWqutME58phurE0oO57Icg&m=ZtuV_vgGx5WjIRe39F0QbEfwJYcxAMFvJZbn2_tzxpg&s=SDKio10_-xuExwLipGPXqd9ch-lWEwC5eUPNzr4mTLo&e=

This project provide a template system to create folder and groups with
permission and to provision membership depending on user's informations
provided.

Thanks,
Julien

Le 15/03/2018 à 19:12, Hyzer, Chris a écrit :
> Are you using GSH or the loader?
>
> Can you send a portion of a script that takes a while and we can take a
look?
>
> Manipulating the database directly would probably result in
corruption...
>
> Thanks
> Chris
>
> -----Original Message-----
> From:


[mailto:]
On Behalf Of Julien Gribonvald
> Sent: Thursday, March 15, 2018 2:07 PM
> To:

> Subject: [grouper-users] Optimized group provisionning in grouper
database
>
> Hi,
>
> I would like to know if there are somes optimized process to be able to
> manage groups and folders, permissions on theses groups and the
> membership provisionning from a batch ?
>
> We just moved in production from grouper 1.6.3 to grouper 2.3.0 and I
> used for a long time the grouper api for this process, but with the
> version 2.3 when you have to manager around 40K groups by automatization
> it takes really too long time (the main time is lost on groups and
> folders creation with privilegies) ! Hopefully this kind of management
> (such number of groups to update) appear only few times during the year,
> but we would need something really more fast for such process.
>
> I watched on other grouper tools, but all are using the same "slow"
> grouper-api methods for managing groups, after my problems is that i
> don't know how I can create groups and folders more directly into the
> grouper dabatase, so for me it's really dangerous to do that, or maybe ?
>
> So I would like to know if there could be more optimization, like
> disabling ldap request on subject api (has we are provisionning ldap
> user's account at the same time), and other stuff that could permit to
> manage a such number of groups in few/some hours instead of several
days ?
>
>
> Thanks for your help.
>

--
Julien Gribonvald





Archive powered by MHonArc 2.6.19.

Top of Page