grouper-study

[Jack Stewart <>]

Everyone,

Again, this is a great discussion!

I guess I've stepping around it this question, but I'll ask it directly: when will the Grouper containers be delivered by TIER in such a way that the configuration files and/or individual configuration parameters can easily be converted to secrets?

I've been trying to make the point that if we have to tear into every one of the Grouper configuration files and convert them over to our secrets management system, even if it's only during major upgrades, that's a lot of work.

I recently heard that the Grouper containers would be made to be more "Kubernetes friendly" soon.  Is this what was meant?

Thanks, Jack

On Mon, May 7, 2018 at 12:21 PM, John Gasper <> wrote:

I’m back… There’s one more option that I just learned about. It requires a more recent version of Docker Engine and be using Swarm mode, but Docker Secrets and Configs supports templating. You can basically set the file to something like this:

 

hibernate.connection.password = {{ secret “grouper_db_password” }}

 

You save the config file as a Docker config, but do it like so:

docker config create --template-driver golang grouper.hibernate.properties ./grouper.hibernate.properties

 

This tells Docker that before it publishes the config file into the running container, grouper.hibernate.properties, needs to run through the golang merge/template process.

 

Now save the password string as a Docker Secret named grouper_db_password.

 

Then when you create your service you just map your secrets like normal and the password will be provided in the config.

 

Unfortunately the functionality is so new that there isn’t a lot of documentation on it, but this blog entry does talk about it some more: https://blog.sunekeller.dk/docker-18-03-config-and-secret-templating/

John Gasper
IAM Consultant
Unicon, Inc.
PGP/GPG Key: 0xbafee3ef

From: <> on behalf of John Gasper <>
Date: Tuesday, May 1, 2018 at 3:46 PM
To: John Schrader <>, Jack Stewart <>

Cc: "Hyzer, Chris" <>, Christopher Hubing <>, "" <>, csp study grouper <>
Subject: Re: [grouper-users] Containerized Grouper and Secrets

 

Hey all,

 

Sorry, I’m late to this party. I’ve got a few tidbits to share that might be beneficial to you or others in the future. (And John that’s great info about the AWS SecretsManager)

 

One of the TIER requirements was to be able to read in password from an env parameter or from a file specified by env parameter. This doesn’t necessarily make sense for Grouper since we have to customize the property files anyways, but I’ve got some examples in the image source’s test directory:

Two examples of the EL code:

https://github.internet2.edu/docker/grouper/blob/master/test-compose/configs-and-secrets/grouper/grouper.hibernate.properties#L29

https://github.internet2.edu/docker/grouper/blob/master/test-compose/configs-and-secrets/grouper/subject.properties

And applying it using both methods: https://github.internet2.edu/docker/grouper/blob/master/test-compose/docker-compose.yml#L46-L47

 

If you were going with this method, I’d pick the one that is appropriate and just use that chunk, but it’s your env do as you want. 😊

 

Also, I’m working on some videos showing how to use the TIER Grouper image in Docker Swarm. I’m waiting for approval of the videos and then I’ll post about them on the list, but all of the step-by-step source material is here and shows how to use Docker Secrets: https://github.com/Unicon/tier-grouper-deployment. Using this method, the sensitive files and env specific config files are managed by Swarm. This has the upside of being able to run “docker service rollback XXX” and it switches configs/secrets to the previous config. (It’s actually pretty cool to call rollback several times and watch it switch your service back to the image/config several versions ago.)

 

Of course you can always use morphStrings to store your passwords, and mount them however you want to, and reference those locations in the properties file. I’d probably still manage the files with “docker config” since you’ll likely use databases and data sources that are different between test and prod. If the files are the same, then I’d just manage them as project source that gets backed into the image.

 

OK, my two cents.

 

John Gasper
IAM Consultant
Unicon, Inc.
PGP/GPG Key: 0xbafee3ef

From: <> on behalf of John Schrader <>
Date: Sunday, April 29, 2018 at 9:10 AM
To: Jack Stewart <>
Cc: "Hyzer, Chris" <>, Christopher Hubing <>, "" <>, csp study grouper <>
Subject: Re: [grouper-users] Containerized Grouper and Secrets

 

This is a great conversation..

 

For those of us running in AWS cloud, AWS has released "SecretsManager" [1]

 

SecretsManager opens up some interesting possibilities for both EC2 instances and containers(tasks) running in ECS.

 

As a POC, I've created an elConfig class (thanks to Chris for the capability) that retrieves values for hibernate properties:

hibernate.connection.url.sm.elConfig            = ${edu.internet2.middleware.grouperClient.config.SecretsManagerElClass.getSecret('/grouper/dev','url')}

hibernate.connection.url                        = $$hibernate.connection.url.sm$$

hibernate.connection.username.sm.elConfig       = ${edu.internet2.middleware.grouperClient.config.SecretsManagerElClass.getSecret('/grouper/dev','username')}

hibernate.connection.username                   = $$hibernate.connection.username.sm$$

hibernate.connection.password.sm.elConfig       = ${edu.internet2.middleware.grouperClient.config.SecretsManagerElClass.getSecret('/grouper/dev','password')}

hibernate.connection.password                   = $$hibernate.connection.password.sm$$

from SecretsManager.

 

Read/Decrypt access to the `/grouper/dev` secret is controlled by the role associated with an EC2 instance or ECS task.

Using a role helps with the initial credential bootstrapping and allows for immutability.

 

-John

 

[1] https://aws.amazon.com/secrets-manager/

 

 

 

 

 

 

 

 

On Thu, Apr 26, 2018 at 1:27 PM, Jack Stewart <> wrote:

Everyone,

Thank you all for your wonderful feedback!  I hope the discussion continues.

A few thoughts:

- Although Grouper can run with environment variables, it would take a lot of time to convert, and each major upgrade could possibly be made tricky as a result.  Or am I being too paranoid?

- I don’t quite see consensus on a clear way forward.  All the information presented here has been helpful, though.  I will check what others are doing locally.

Jack

 

-- 

Jack Stewart

Solutions Architect, Identity and Access Management

University of Michigan

4251 Plymouth Road

Ann Arbor, Michigan 48105-3640

(734) 764-0853

On Apr 26, 2018, at 12:39 PM, Hyzer, Chris <> wrote:

Fyi grouper config can happen in env vars, also the sources.xml can be migrated to subject.properties

https://spaces.internet2.edu/display/Grouper/Grouper+configuration+overlay#Grouperconfigurationoverlay-Environmentvariables

https://spaces.internet2.edu/display/Grouper/Grouper+sources.xml+conversion+to+subject.properties

Thanks
Chris

-----Original Message-----
From: [] On Behalf Of Christopher Hubing
Sent: Wednesday, April 25, 2018 1:00 PM
To: Jack Stewart <>
Cc: ; csp study grouper <>
Subject: Re: [grouper-users] Containerized Grouper and Secrets


For I2, we are storing secret things in an encrypted S3 bucket. The build
host has access to read from it, and then pushes the images to a private
Elastic Container Repo. The containers run in ECS.

Here's an example of our Dockerfile for the UI:
https://github.internet2.edu/gist/chubing/c4e663ab5a39fb73dccdcd748a92c5fe

Since the new Grouper container is pushed to Dockerhub (and have tags for
patches), it should make it pretty easy to manange (hopefully).

-c

On Wed, 25 Apr 2018, Jack Stewart wrote:

Everyone,

I would like to start out by saying that the new role-based Grouper containers are great!  It was very easy to build the images.

 

Now my question is, what are other schools doing with regard to their Grouper configurations?  Are you "burning them into" storing them in the containers themselves, or are you using

secrets?

 

Converting an application like Grouper to use secrets would be a LOT of work.  Effectively, you would need to convert all of the settings to environment variables.  How would you deal with

the sources.xml files which, by design, need to be customized?

 

Many thanks,

Jack

 

 

--

Jack Stewart

Solutions Architect, Identity and Access Management

University of Michigan

4251 Plymouth Road

Ann Arbor, Michigan 48105-3640

(734) 764-0853

 

 

 

--

John Schrader

Identity and Access Management

Office of Information Technologies

University of Notre Dame

 

EVERYTHING SHOULD BE MADE AS SIMPLE AS POSSIBLE, BUT NOT ANY SIMPLER

—ALBERT EINSTEIN

--

Jack Stewart

Solutions Architect, Identity and Access Management

University of Michigan

4251 Plymouth Road

Ann Arbor, Michigan 48105-3640

(734) 764-0853