Skip to Content.
Sympa Menu

grouper-users - RE: [grouper-users] Grouper patching I don't fully understand

Subject: Grouper Users - Open Discussion List

List archive

RE: [grouper-users] Grouper patching I don't fully understand


Chronological Thread 
  • From: "Hyzer, Chris" <>
  • To: "Black, Carey M." <>, Olivier Salaün <>, "" <>
  • Subject: RE: [grouper-users] Grouper patching I don't fully understand
  • Date: Wed, 22 Jan 2020 20:44:09 +0000
  • Arc-authentication-results: i=1; mx.microsoft.com 1; spf=pass smtp.mailfrom=isc.upenn.edu; dmarc=pass action=none header.from=isc.upenn.edu; dkim=pass header.d=isc.upenn.edu; arc=none
  • Arc-message-signature: i=1; a=rsa-sha256; c=relaxed/relaxed; d=microsoft.com; s=arcselector9901; h=From:Date:Subject:Message-ID:Content-Type:MIME-Version:X-MS-Exchange-SenderADCheck; bh=sBals/G/U9U6IoXmF2eJ4FJXA/0KV85BMc+M4uSXk1o=; b=BhXhNKh5iqVHW/vxVmDFh6Skt1DVB1aYMunjrz6OXTZg0rQ7ifBbXH4xFqWv50WFcWROcPuIsAk3N6yuwv02IaqBFmcHDG6mFH+grc0l4hWhIvjBbX+Vog9bEErc1qlQwlWnnnJHHegXwFqKHcyBQiAHVfp9cHx7Vc9miqb5iRx2jVjWdcU4kLVLOOb9MNb4MYItYQunFUrCo2+N3Umv99CG92/axHcuW+/muIEHhzaS6MgMpC6pUCoBYe1PWDJ7cxlz60YTQAh32OYlgYRm5qLjXNMuBGYfQT1G7HVd+UNhA4WmUpijLpoeNxoNuuaWkCQMg5YomjAZmdA53jsnGg==
  • Arc-seal: i=1; a=rsa-sha256; s=arcselector9901; d=microsoft.com; cv=none; b=l9bqnYCjuhrbfSPRDvz4p8KKNtxV1YZgWnXRXBikUDlAjZUTPAS5D5w8NbtPCjm0bRi+YJikkg7ezQyksjN8ZR4hne4pfBlpof62su2uegnB5vGDLhmvZpza0K0lMlifArBvTtDW4awDNVPlIwZZXZlxp4ElXp2bCz26IOD2+e3R1QhGZ8rNqmenEg9x0qACPXUXs2AQzev82pM1bZXziP14uCwrfIjpHV9/EFfp5JGrHCCJ7CuHyW9aahkudqzM0wfiGkafm+/snzdQNMsowKgc3BUBQ1i75CJfiHlU1dE0myfjKuwBMm4E52Pvrx/JxiPX0UAiQHeUT1FTTnTn0g==

Wow, thanks so much for this thorough response (Carey) Matt.

 

But “Sadly” and “modernized”????  ouch! 😊   Its glorious, and will save the grouper support team and deployers a ton of time and reduce frustration.  Its going to be great!  😊

 

We wont really be supporting patches/tarballs in 2.4 anymore (you should migrate to container) and patches/tarballs wont exist in 2.5 (you must move to container).  The link Matt referenced below (container maturity level 0) shows how to easily run the 2.4 container on a unix server without having any container infrastructure or orchestration.  We are trying to make the barrier of containers reduced or eliminated for people who still have fears/uncertainties/doubts.  😊  

 

If anyone has concerns/suggestions about this approach please let me know asap so we can include those requirements in the 2.5 deployment/install/upgrade design.  We don’t want to alienate anyone.  😊

 

Thanks!

Chris

 

From: <> On Behalf Of Black, Carey M.
Sent: Wednesday, January 22, 2020 10:03 AM
To: Olivier Salaün <>;
Subject: RE: [grouper-users] Grouper patching I don't fully understand

 

Olivier,

 

Sadly (IMHO), the Grouper “install process” is being “modernized” into the use of containers.

                If you are on v2.4 it is strongly encouraged that you move to using a container based deployment model instead of a “local install”.

 

REF: https://github.internet2.edu/docker/grouper

                Of note is the https://hub.docker.com/r/tier/grouper/tags link. ( This is currently the place to get the ITAP containers from. )

REF: https://spaces.at.internet2.edu/display/ITAP/InCommon+Trusted+Access+Platform+Release

REF: https://github.internet2.edu/docker/grouper/blob/2.4.0-80-u51-w10-p11-20191118/README.md 

                Note latest URL/version will be slightly different all the time. (These are links to a github branch that is specific to a combination of “API”, “UI”, “WS”, and “PSPNG” versions. )

                Note: This is really for docs/info more than “code”. You likely don’t need to “compile” or “install” to “deploy using a container”.

 

REF: https://spaces.at.internet2.edu/display/Grouper/Install+the+Grouper+2.4+container+with+maturity+level+0

                This might be a good reference guide for starting where you are and moving towards the container model.

 

 

 

 

However, ( back to your comments about the current grouperInstaller)

 

I see your confusion about the “local install” process. Here are a few things that took me a while to wrap my head around.

                I try to forget about the parts and focus on the “install locations”.

                                A UI “install location” has the UI, and the API. ( and maybe local customizations )

                                A WS “install location” has the WS, and the API. ( and maybe local customizations )

                                A daemon ( AKA: api ) “install location” only has the API. ( and maybe local customizations )

                                Etc… etc… ( and maybe local customizations )

 

                Each location needs to be “patched” and/or “upgraded” independently by the grouperInstaller. Because when you run it, you point it at “a single location”. So when you use only the UI, then you only have to patch the UI location(s).

                But you never *only* have the UI. You always need a daemon/API to have a full system too. ( WS is optional, PSPNG optional, etc…, etc… )

                So you also always have to patch more than one location.

 

                So you need to have a set of config files for each *location* and run the installer per location to get everything up to date.

                However, there is this clause “It will apply and manage the API patches as well.”. So when you patch the “UI” (location) the grouperinstaller knows to also patch the API that exists in that location. This helps to avoid needing to run the patches on the same location more than once. (Assuming it all works the first time. :) )

 

 

For what it is worth:

The container model is very different.

                The container has “all the standard files” in “well known locations”. You pull the right container and you get all of the “install locations” in it, and patched to a known level in that one step. You need to add your config ( and maybe local customizations ) to get it to be “your Grouper install”. ( And there are very “docker” specific choices to make about how you add what you want added. But it ultimately becomes a script/process. ) The “maturity level 0” link is likely a way to plan/transition into a container model.

 

Since you are already on v2.4 really all you would need to do is be able to spin up the container with your config ( and maybe local customizations ).

                So look at the Daemon, UI, (maybe WS? ) links on that page for the “basic how to”.

 

                Daemon: https://spaces.at.internet2.edu/display/Grouper/Install+the+Grouper+2.4+container+with+maturity+level+0#InstalltheGrouper2.4containerwithmaturitylevel0-Daemon

                UI:    https://spaces.at.internet2.edu/display/Grouper/Install+the+Grouper+2.4+container+with+maturity+level+0#InstalltheGrouper2.4containerwithmaturitylevel0-UI

                WS:  https://spaces.at.internet2.edu/display/Grouper/Install+the+Grouper+2.4+container+with+maturity+level+0#InstalltheGrouper2.4containerwithmaturitylevel0-WS

 

 

Hope that helps.

 

--

Carey Matthew

 

From: <> On Behalf Of Olivier Salaün
Sent: Wednesday, January 22, 2020 9:09 AM
To:
Subject: [grouper-users] Grouper patching I don't fully understand

 

Hello,

We are moving to Grouper 2.4.0 on a Linux server and while applying latest patches I got troubles applying patches to the grouper API. I think I understand the process now, but I want to make sure I apply patches the right way. Also since the documentation was confusing (for me), maybe this email might be helpful for others.

We run the grouper installer for patching as follows :

java -cp .:grouper.installer-2.4.0/grouperInstaller.jar edu.internet2.middleware.grouperInstaller.GrouperInstaller

The grouper.installer.properties file includes the following entries :

  • grouperInstaller.default.installOrUpgrade = patch
  • grouperInstaller.default.appToUpgrade = api
  • grouperInstaller.autorun.grouperWhereInstalled = /data/webapps/grouper-test.univ-rennes1.fr/application/grouper.ui-2.4.0/dist/grouper

https://spaces.at.internet2.edu/display/Grouper/Grouper+patching states that:

Note, if you are patching the UI or WS for example, you do not need to patch the API for the UI/WS, you just need to run the patcher for the UI or WS.  It will apply and manage the API patches as well.

Therefore I assumed running Grouper installer in patch mode on grouper.ui-2.4.0/dist/grouper/ would also patch the Grouper API used by gsh.sh (GROUPER_HOME=/data/webapps/grouper-test.univ-rennes1.fr/application/grouper.apiBinary-2.4.0), but it didn't.

################ Checking patch grouper_v2_4_0_api_patch_89
...
Applying file: /data/webapps/grouper-test.univ-rennes1.fr/application/grouper.ui-2.4.0/dist/grouper/WEB-INF/classes/edu/internet2/middleware/grouper/app/deprovisioning/GrouperDeprovisioningOverallConfiguration.class
Applying file: /data/webapps/grouper-test.univ-rennes1.fr/application/grouper.ui-2.4.0/dist/grouper/WEB-INF/classes/edu/internet2/middleware/grouper/app/deprovisioning/GrouperDeprovisioningOverallConfiguration.java
Patch successfully applied: grouper_v2_4_0_api_patch_89
 - added to end of property file: grouper_v2_4_0_api_patch_89.state = applied

The Grouper installer patched code in the grouper.ui-2.4.0/dist/grouper/WEB-INF/classes/ hierarchy, which is a copy of the grouper.apiBinary-2.4.0/conf/edu/ hierarchy.

I indeed realized $GROUPER_HOME/bin/gsh.sh did not have the latest patches applied :

gsh.sh
Detected Grouper directory structure 'api' (valid is api or webapp)
Using GROUPER_HOME:           /data/webapps/grouper-test.univ-rennes1.fr/application/grouper.apiBinary-2.4.0
Using GROUPER_CONF:           /data/webapps/grouper-test.univ-rennes1.fr/application/grouper.apiBinary-2.4.0/conf
...
api patches installed:        0, 1, 2, 3, 4, 5, 6, 7, 8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20, 21, 22, 23, 24, 25, 26, 27, 28, 29, 30, 31, 32, 33, 34, 35, 36, 37, 38, 39, 40, 41, 42, 43, 44, 45, 46, 47, 48, 49, 50, 51, 52, 53, 54, 55, 56, 57, 58, 59, 60, 61, 62, 63, 64, 65, 66, 67, 68, 69, 70, 71, 72

I found out I have 5 versions of gsh.sh in my Grouper hierarchy:

  1. grouper.apiBinary-2.4.0/bin/gsh.sh : the gsh.sh version used by our GSH scripts,
  2. grouper.ui-2.4.0/dist/grouper/WEB-INF/bin/gsh.sh : associated with API code used by Grouper UI
  3. grouper.ui-2.4.0/webapp/WEB-INF/bin/gsh.sh : source code for (2)
  4. grouper.ws-2.4.0/grouper-ws/build/dist/grouper-ws/WEB-INF/bin/gsh.sh : associated with API code used by Grouper WS
  5. grouper.ws-2.4.0/grouper-ws/webapp/WEB-INF/bin/gsh.sh : source code for (4)

I had to run the grouper installer again to patch the API located in the grouper.apiBinary-2.4.0/ hierarchy and now gsh.sh is up to patch 89 :-)

Can you confirm that I need to run the patching process twice to update both UI and API code?

Any way to make Grouper install a single version of the Grouper API code (for the sake of maintenability and comprehensibility)?

-- 
Olivier Salaün
DSI / pôle SI / équipe SNUM
Tel : 02 23 23 74 54



Archive powered by MHonArc 2.6.19.

Top of Page