Grouper Users - Open Discussion List

["Hyzer, Chris" <>]

They don’t allow underscores in headers by default?  Ugh.  Nice work tracking it down!

 

From: Darren Boss [mailto:]
Sent: Friday, September 22, 2017 3:38 PM
To: Hyzer, Chris <>;
Subject: Re: [grouper-users] New UI problems with reverse proxy

 

Finally figured it out. If you deploy using an nginx ingress controller under Kubernetes or if you more generically use nginx as a reverse proxy in front of grouper ui, you need to enable allow underscores in headers otherwise you are in for a world of pain.

 

In Kubernetes the configmap I used for nginx configuration is:

apiVersion: v1                                 

kind: ConfigMap                                

metadata:                                      

  name: nginx-custom-configuration             

  labels:                                      

    k8s-app: nginx-ingress-controller          

  namespace: kube-system                       

data:                                          

  proxy-connect-timeout: "10"                  

  proxy-read-timeout: "120"                    

  proxy-send-timeout: "120"                    

  enable-underscores-in-headers: "True"

 

See https://github.com/kubernetes/ingress/tree/master/examples/customization/custom-configuration/nginx.

 

Thanks for your help Chris, I wouldn't have caught the patch issue if you hadn't responded. Hopefully this helps others in the future.

 

On Fri, Sep 22, 2017 at 12:34 PM Darren Boss <> wrote:

Another oops on my end and was not using patched ui but am now.

 

New behavior, after a couple of reloads I end up at the grouper/grouperExternal/public/UiV2Public.index?operation=UiV2Public.postIndex&function=UiV2Public.error&code=csrf&OWASP_CSRFTOKEN=<TOKEN HERE> and see the following message:

Maybe your session timed out and you need to start again. This should not happen under normal operation. CSRF error.

 

On Fri, Sep 22, 2017 at 12:08 PM Darren Boss <> wrote:

Sorry, I sent this too soon. My ingress controller ip address changed so I'm using a regex now for the ip and now I'm back to the issue with the ui infinitely reloading but now I should be running a patched version of the UI.

 

On Fri, Sep 22, 2017 at 11:48 AM Darren Boss <> wrote:

The Unicon dockerfiles have been changed and it appears to fix the issue with patch application:

https://github.com/Unicon/grouper-dockerized/commit/436dc56e432eea2eced7a6066756c6089609249b

 

I don't know and haven't checked to see if the Docker images have been rebuilt in Dockerhub, I'm going to build these myself anyway. Redeployed and now UI behaves differently, I'm back to getting "There was an error with your request." on every click. It appears that I'm getting a mix of http and https in the UI and my browser is blocking access to non-secure requests. Tomcat is not running with TLS certs, this is being handled by the nginx proxy in Kubernetes. What config changes would I need to make to give the grouper ui use https for all requests or will I need to rewrite the urls at a layer outside of grouper?

 

I can see in the installation instructions that if using the interactive installer it's asked by the installer:
Go here for the Grouper UI (change hostname if on different host): http://localhost:8080/grouper/

I need a way to tweak this after the fact or to pass in this information in the installer, possibly in the grouper-ui.properties file? I see that properties can be set via environment vars, this is super useful for containerized deployments, even more important when deploying under environments like Kubernetes so I'm glad to see that.

 

On Wed, Sep 20, 2017 at 10:35 AM Darren Boss <> wrote:

That is certainly helping. I've also raised an issue on the Unicon repo about the patches not being applied (https://github.com/Unicon/grouper-dockerized/issues/11). Travelling at the moment but I'll see about redeploying grouper now that I have images with patches applied and if there is any change in behaviour when accessing the grouper ui.

 

May have found the TIER Dockerfile for grouper and the related grouper.installer.properties at https://github.internet2.edu/docker/grouper/blob/1cb4b767002f7d1f7aa790f25d70edd660eae659/container_files/etc/grouper.installer.properties. I had trouble building their images since they require you to manually uncomment the lines around download of the JDK and the method they use to download the JDK need updating before it will work properly but I think I've got it building now.

 

On Wed, Sep 20, 2017 at 10:12 AM Hyzer, Chris <> wrote:

There is a file that keeps track of which patches are installed.  patch status properties.  It seems like the patches are in the container, but the file that keeps track of those patches is not updated?  So some patches appear to be there but grouper doesn’t know it so dependent patches would not be installed either.

 

Would be cleanest to make sure that file is up to date in the image.  You could also try putting this in the grouper.installer.properties:

 

# autorun, Problem installing patch since this patch file: <filename> is not the same as what the patch expects:

# Do you want to force install this patch (t|f)? [f]:

grouperInstaller.autorun.forceInstallPatch = t

 

Im looking at the release notes and I don’t think your issue is fixed in a patch, but definitely getting them installed is the first step:

 

https://spaces.internet2.edu/display/Grouper/v2.3+Release+Notes

 

Thanks

Chris

 

From: Darren Boss [mailto:]
Sent: Tuesday, September 19, 2017 9:07 AM

To: Hyzer, Chris <>;

Subject: Re: [grouper-users] New UI problems with reverse proxy

 

Some discussion occured just between Chris and myself. It does appear that not all patches are getting applied when building the Grouper UI Docker image. The Dockerfile does the war build and applies patching using this grouper.installer-ui.properties file:

 

grouperInstaller.autorun.useDefaultsAsMuchAsAvailable = true

grouperInstaller.autorun.continueAfterDeleteUiWorkDirectory = true

grouperInstaller.autorun.actionEgInstallUpgradePatch = patch

grouperInstaller.autorun.tarballDirectory = /tmp/grp-ui

grouperInstaller.autorun.appToUpgrade = ui

grouperInstaller.autorun.grouperWhereInstalled = /opt/apache-tomcat-6.0.44/webapps/grouper/

grouperInstaller.autorun.patchAction = install

grouper.version = 2.3.0

download.server.url = "http://software.internet2.edu/grouper

 

Right off the bat there is a problem appling a the first api patch:

Downloading from URL: http://software.internet2.edu/grouper/release/2.3.0/patches/grouper_v2_3_0_api_patch_0.tar.gz to file: /tmp/grp-ui/patches/grouper_v2_3_0_api_patch_0.tar.Cannot apply patch since this patch file:

  /opt/apache-tomcat-6.0.44/webapps/grouper/WEB-INF/classes/edu/internet2/middleware/grouper/util/GrouperUtil$2.class

  /tmp/grp-ui/patches/grouper_v2_3_0_api_patch_0/new/classes/edu/internet2/middleware/grouper/util/GrouperUtil$2.class

  is supposed to be new, but it already exists:

  /opt/apache-tomcat-6.0.44/webapps/grouper/WEB-INF/classes/edu/internet2/middleware/grouper/util/GrouperUtil$2.class

Problem applying patch since this file:

  /opt/apache-tomcat-6.0.44/webapps/grouper/WEB-INF/classes/edu/internet2/middleware/grouper/util/GrouperUtil$MaskingThread.class

  should not exist yet

  Do you want to force install this patch (t|f)? [f]: 

<using default which is blank due to grouperInstaller.autorun.useDefaultsAsMuchAsAvailable and grouperInstaller.autorun.forceInstallPatch>: 

Cannot apply patch since this patch file:

  /tmp/grp-ui/patches/grouper_v2_3_0_api_patch_0/new/classes/edu/internet2/middleware/grouper/util/GrouperUtil$MaskingThread.class

  is supposed to be new, but it already exists:

  /opt/apache-tomcat-6.0.44/webapps/grouper/WEB-INF/classes/edu/internet2/middleware/grouper/util/GrouperUtil$MaskingThread.class

Problem applying patch since this file:

  /opt/apache-tomcat-6.0.44/webapps/grouper/WEB-INF/classes/edu/internet2/middleware/grouper/util/GrouperUtil.class

  should not exist yet

  Do you want to force install this patch (t|f)? [f]: 

<using default which is blank due to grouperInstaller.autorun.useDefaultsAsMuchAsAvailable and grouperInstaller.autorun.forceInstallPatch>: 

Cannot apply patch since this patch file:

  /tmp/grp-ui/patches/grouper_v2_3_0_api_patch_0/new/classes/edu/internet2/middleware/grouper/util/GrouperUtil.class

  is supposed to be new, but it already exists:

  /opt/apache-tomcat-6.0.44/webapps/grouper/WEB-INF/classes/edu/internet2/middleware/grouper/util/GrouperUtil.class

Problem applying patch since this file:

  /opt/apache-tomcat-6.0.44/webapps/grouper/WEB-INF/classes/edu/internet2/middleware/grouper/util/GrouperUtil.java

  should not exist yet

  Do you want to force install this patch (t|f)? [f]: 

<using default which is blank due to grouperInstaller.autorun.useDefaultsAsMuchAsAvailable and grouperInstaller.autorun.forceInstallPatch>: 

Cannot apply patch since this patch file:

  /tmp/grp-ui/patches/grouper_v2_3_0_api_patch_0/new/classes/edu/internet2/middleware/grouper/util/GrouperUtil.java

  is supposed to be new, but it already exists:

  /opt/apache-tomcat-6.0.44/webapps/grouper/WEB-INF/classes/edu/internet2/middleware/grouper/util/GrouperUtil.java

Problem applying patch since this file:

  /opt/apache-tomcat-6.0.44/webapps/grouper/WEB-INF/classes/edu/internet2/middleware/grouper/util/GrouperUtil$DaemonThreadFactory.class

  should not exist yet

  Do you want to force install this patch (t|f)? [f]: 

<using default which is blank due to grouperInstaller.autorun.useDefaultsAsMuchAsAvailable and grouperInstaller.autorun.forceInstallPatch>: 

Cannot apply patch since this patch file:

  /tmp/grp-ui/patches/grouper_v2_3_0_api_patch_0/new/classes/edu/internet2/middleware/grouper/util/GrouperUtil$DaemonThreadFactory.class

  is supposed to be new, but it already exists:

  /opt/apache-tomcat-6.0.44/webapps/grouper/WEB-INF/classes/edu/internet2/middleware/grouper/util/GrouperUtil$DaemonThreadFactory.class

 - added to end of property file: grouper_v2_3_0_api_patch_0.state = error

 

Does someone understand what might be happening? Are the patches being applied but detected as errors because of the force installation and then subsequent patches not being applied because it's detected that previous dependent patches have not applied?

 

 

On Mon, Sep 18, 2017 at 10:13 AM Hyzer, Chris <> wrote:

> Full disclousure, I'm running Grouper in containers under Kubernetes with an nginx ingress

> controller which is terminating TLS at the ingress. When I use "kubectl port-forward pod-name"

> and access the ui over localhost:8080 I have no issues in accessing the interface.

 

So you are saying it works when you are not load balancing?  But it doesn’t work with load balancing?  Is the load balancing sticky by cookie?  Does it work if it is load balancing with one node in the pool?

 

What the X-Grouper-path response http header coming back from the server?

 

Are you sure there are no errors in any of the logs?

 

Thanks

Chris

 

 

From: [mailto:] On Behalf Of Darren Boss
Sent: Monday, September 18, 2017 9:58 AM
To:
Subject: [grouper-users] New UI problems with reverse proxy

 

Question about the New UI and OWASP csrfguard and accessing the UI behind a nginx reverse proxy.

 

Took me a little while to figure out the additions to make to Tomcat to fix the major issue of accessing the UI after which the admin ui works. This is a common solution when running tomcat behind a reverse proxy. I have another tweak for Jetty that accomplished the same.

 

<Valve className="org.apache.catalina.valves.RemoteIpValve"

        internalProxies="my.nginx.proxy.address"

        remoteIpHeader="x-forwarded-for"

        requestAttributesEnabled="true"

        protocolHeader="x-forwarded-proto"

        protocolHeaderHttpsValue="https"/>

 

Now when I click on any link in the new ui I end up in an infinite redirect loop with &csrfExtraParam=xyz being added to the existing url on every redirect. I've tried to search through the mailing list to find a resolution. I know there are lots of deployments behind reverse proxies so this should be a solved issue.

 

Full disclousure, I'm running Grouper in containers under Kubernetes with an nginx ingress controller which is terminating TLS at the ingress. When I use "kubectl port-forward pod-name" and access the ui over localhost:8080 I have no issues in accessing the interface.

 

I'm also incredably new to deploying grouper but had been using the deployment at Duke when I was a staff member there and I believe their deployment was running under OpenShift so probably a similar deployment to what I'm doing now.

--

Darren Boss

Senior Programmer/Analyst
Programmeur-analyste principal

(o) 416.228.1234 x 230

(c) 919.525.0083

155 University Ave, Suite 302 Toronto, ON M5H 3B7
www.computecanada.ca / www.calculcanada.ca 
@ComputeCanada 

--

Darren Boss

Senior Programmer/Analyst
Programmeur-analyste principal

(o) 416.228.1234 x 230

(c) 919.525.0083

155 University Ave, Suite 302 Toronto, ON M5H 3B7
www.computecanada.ca / www.calculcanada.ca 
@ComputeCanada 

--

Darren Boss

Senior Programmer/Analyst
Programmeur-analyste principal

(o) 416.228.1234 x 230

(c) 919.525.0083

155 University Ave, Suite 302 Toronto, ON M5H 3B7
www.computecanada.ca / www.calculcanada.ca 
@ComputeCanada 

--

Darren Boss

Senior Programmer/Analyst
Programmeur-analyste principal

(o) 416.228.1234 x 230

(c) 919.525.0083

155 University Ave, Suite 302 Toronto, ON M5H 3B7
www.computecanada.ca / www.calculcanada.ca 
@ComputeCanada 

--

Darren Boss

Senior Programmer/Analyst
Programmeur-analyste principal

(o) 416.228.1234 x 230

(c) 919.525.0083

155 University Ave, Suite 302 Toronto, ON M5H 3B7
www.computecanada.ca / www.calculcanada.ca 
@ComputeCanada 

--

Darren Boss

Senior Programmer/Analyst
Programmeur-analyste principal

(o) 416.228.1234 x 230

(c) 919.525.0083

155 University Ave, Suite 302 Toronto, ON M5H 3B7
www.computecanada.ca / www.calculcanada.ca 
@ComputeCanada 

--

Darren Boss

Senior Programmer/Analyst
Programmeur-analyste principal

(o) 416.228.1234 x 230

(c) 919.525.0083

155 University Ave, Suite 302 Toronto, ON M5H 3B7
www.computecanada.ca / www.calculcanada.ca 
@ComputeCanada