grouper-users - RE: [grouper-users] loader FIFO
Subject: Grouper Users - Open Discussion List
List archive
- From: "Black, Carey M." <>
- To: "" <>
- Cc: "Hyzer, Chris" <>, "" <>
- Subject: RE: [grouper-users] loader FIFO
- Date: Tue, 11 Dec 2018 15:06:08 +0000
- Accept-language: en-US
- Authentication-results: spf=pass (sender IP is 128.146.138.8) smtp.mailfrom=osu.edu; internet2.edu; dkim=pass (signature was verified) header.d=osu.edu;internet2.edu; dmarc=pass action=none header.from=osu.edu;
- Authentication-results-original: spf=none (sender IP is ) ;
- Ironport-phdr: 9a23:QzfosBxokCKyc9fXCy+O+j09IxM/srCxBDY+r6Qd2u4VIJqq85mqBkHD//Il1AaPAd2Lraocw8Pt8InYEVQa5piAtH1QOLdtbDQizfssogo7HcSeAlf6JvO5JwYzHcBFSUM3tyrjaRsdF8nxfUDdrWOv5jAOBBr/KRB1JuPoEYLOksi7ze+/94HQbglSmDaxfa55IQmrownWqsQYm5ZpJLwryhvOrHtIeuBWyn1tKFmOgRvy5dq+8YB6/ShItP0v68BPUaPhf6QlVrNYFygpM3o05MLwqxbOSxaE62YGXWUXlhpIBBXF7A3/U5zsvCb2qvZx1S+HNsDwULs6Wymt771zRRHolikJKiI5/m/UhMx+jq1boQ6uqBNkzoHOfI2ZKOBzcr/Bcd4YQ2dKQ8ZfVzZGAoO5d4YDAfcMPeJCoIbhpFsCtQa1CBe2C+P11DBJhmL906og3+gvDArL2xcvH9YUv3vIr9X5LroeXf67waTTyjXDdO5W1S3j54fVbxAsuPeBVq9zf8rJ0UQjCRnKgkmNpYHgIj+Zy/kBvm2V7+dvSe6jl2sqqw9vrTWg2sghj4zEi4AJxl3E7Sl13oc4KcOkREN+fdKpFYZcuDuHO4dqR84vQWNluCgmxbADupO2cycHxZQiyhPRd/OIbo2F7Q/9W+mLJTp1gXBod6+/ihuy6USgxPPzW8qo3FtPqydJjMTAuW0P2hHW5MWLVPtw8V2k1DuP1Q3e6v1ILEU7mKXBL5Mu2Lg9nYcJv0vZBC/5gkD2gbeWdko6/uio7PzqbK36q5GbK4N5hRjyPLk2lMCiGOg4NRMBUHaB9eS7yb3j4Vb2QLJXjv0wj6bVqojaJd4cpq6lHQBazpoj6xe4DzegytgYmmQHLE5BeBKAiIjpOEvCL+z/Dfe6m1iskTFryO7aPrD5HJnCNGTPnKrkcLpg9kJQ1Q8+zd5Q6p5IFr0MJfzzVVH+udPECxI0Ng65z/rnBdll04MRQ2OPAquXMKPItl+I4/oiLPKIZIAPpDn9KOIp6+PwgXI3glIderKp3YYNZH+mA/tmPl2VYWDwjdcZDWcKog0+QfTliF2YVj5Tem6yULwm5j0iFYKmFpnMRpq2gLGa2Ce7H4ZWZnxdClySC3vodoOEW+sSZyKIJM9ujCAEWaa7R4A/yB6uqVyy970yZOPS9jcVnY/oz9Mz6uHO31lm/Dp4Fcm1zmqWRCd5kn5eFBEs26Uq62Z5w1yAleBTiuZVBJR2obkBBg0+PJXfibUgUPj1QR+Hc9uUHgX1Cu66CC08G4pii+QFZFxwTo341ELKwjaqDrkJlreCGJ0z9OfG0mPsI9pmkieehrI5gQw+S9BUfS29i6hz+hKbJraBkl7Rzvz6M/1AgmiUrzfFnSvr3glDVRJoF6DMXHQRfEzT+NPi+wXPQ6L9QbUhLgZbz8OecO1HZsC6xVlFRfK2IN3FeCrxgGa/Ax+U26mBJJXjYC0D0T/cBkkJn0F2nz6GOAEyCz3nrzfZFyEoGF7yMELq7eRkrn6nFAk5wxzZJ0Fk3qC+rwYcnufUQvgP37UC7SEmrTg8HFu009/MTdSaoA80e75BJ94x/QRK
- Spamdiagnosticmetadata: NSPM
- Spamdiagnosticoutput: 1:99
Andre, Maybe you are using the wrong tools in Grouper to achieve your goals? My personal philosophy ( IMHO and subject to debate by the list
J ) : Change Log Consumers (CLC) should be for outbound flow of changes from Grouper to other systems. Note: You can also loop the CLC’s through a messaging systems for this too. Hooks are more on the “interactive UI” ( but not strictly interactive the work can be done via async to the UI threads as well) activities for the system. Which leads me to these question: What are your CLC’s doing that you would expect to see in the UI? Should those things only made in Grouper after some external system state is changed? Maybe that order should be changed to allow the user to “do all of their work” and “let the system process the CLC” later? Additionally another “random” (yet maybe related) question:
What RDMBS are you using?
Are your DBA’s seeing any long running queries? ( If so, what are they?) And note: You should always have at least one loader/daemon running.
If it is not running all of the time, then there will be “a lot of work” to catch up on when it starts and that then will be an avalanche of CLC activities too. ( And some things that are done
with Grouper rules will also not be enforced regularly either. ) -- Carey Matthew From: <>
On Behalf Of Hyzer, Chris The change log starts as a single queue as id’s get assigned and point in time and flattened memberships are calculated. The UI will still work though. If you have a recent grouper then you can/should run multiple loaders. We have 2.3 at penn and run 5 loader processes. Might be overkill, but just as a datapoint Thanks Chris From: <>
On Behalf Of Andre Daniels Are all grouper changesets put into a single queue? It appears that new changes are starved while while large jobs are processing. Is there a way to keep the UI responsive while a sync or some other large job is being processed? Similarly,
is it possible to run 2 loaders? I was cautioned not to, but I would like to have a loader that starts up periodically and runs a sync. Thanks, Andre Andre Daniels Sr. Developer/Security Analyst University of California Santa Cruz (831) 459-1980 |
- [grouper-users] loader FIFO, Andre Daniels, 12/10/2018
- RE: [grouper-users] loader FIFO, Hyzer, Chris, 12/10/2018
- RE: [grouper-users] loader FIFO, Black, Carey M., 12/11/2018
- Re: [grouper-users] loader FIFO, Andre Daniels, 12/16/2018
- Re: [grouper-users] loader FIFO, Bee-Lindgren, Bert, 12/19/2018
- Re: [grouper-users] loader FIFO, Andre Daniels, 12/16/2018
- Re: [grouper-users] loader FIFO, Jeffrey Williams, 12/17/2018
- Re: [grouper-users] loader FIFO, Andre Daniels, 12/17/2018
- RE: [grouper-users] loader FIFO, Hyzer, Chris, 12/18/2018
- RE: [grouper-users] loader FIFO, Black, Carey M., 12/11/2018
- RE: [grouper-users] loader FIFO, Hyzer, Chris, 12/10/2018
Archive powered by MHonArc 2.6.19.