Grouper Users - Open Discussion List

[Chris Hyzer <>]

Thanks Julio.  Yes, the XML export inport isn’t really for partial transfers.  What do you want to transfer?  Folders, groups, memberships, privileges?  Based on a parent folder?  Is it ok if create dates, last modify dates, member uuids, etc don’t match?   I could try to make something that generates some GSH for you…  if you need attributes, permissions, rules, etc, that is more complicated…

 

Thanks,

Chris

 

From: [mailto:] On Behalf Of Julio Polo
Sent: Tuesday, December 15, 2015 4:08 PM
To: Sean Mason
Cc:
Subject: Re: [grouper-users] Grouper Import-Export and managing objects

 

I suspect that you will spend a lot of time trying to get that import/export tool to do what you want.  I don't think it was designed for partial transfers.

I recently spent a lot of time trying to preserve one stem when migrating from one Grouper instance to another, and I ended up writing code to migrate *most* of what I needed, manually recreating certain things (fortunately my stem only had a handful of these things), and living with certain values being changed (original creator, timestamps, losing all audit logs).

In the end, the Grouper import/export tool was mostly used to verify that I didn't miss anything during the transfer.  I had to write a utility to de-normalize the data in the XML and change uuid values into group names and subject IDs. That made it easier for me to spot check that I didn't miss transferring anything important.

Some things I learned about the Grouper import/export tool as I used it:

gsh  -xmlexport  -excludeAudits  -stems some:small:stem  someOutputFile.xml

-excludeAudits was necessary; otherwise I would wait forever (days?) for the export to complete, even when exporting a stem with little data.  Not sure why you got audit data if you specified -excludeAudits

 

It doesn't look like folder privileges (stem, create) are exported (unless that's buried somewhere in <memberships>)

All subjects are exported, regardless of whether the subject is a member of the stem I wanted.  For us, that meant our entire population in LDAP was exported.  Not sure if that's the case when you specify -objectNames.

I tried modifying the exported XML to skip the subjects/memberships, and import only the folders, groups, attributes, but ran into some error during the gsh -xmlimport of this modified XML.  Maybe I broke the integrity of the exported XML when I did that.

Hopefully a future Grouper upgrade can better support partial exports/imports.   Best of luck.

 

-julio

Julio Polo
Enterprise Middleware, Identity and Access Management
Information Technology Services
University of Hawaii

 

 

 

On Mon, Dec 14, 2015 at 4:39 AM, Sean Mason <> wrote:

Hi There,

I am looking at introducing Grouper to provide some access management services for the campus, and am currently stuck trying to figure out how to move stem structures and group definition objects between instances to manage the development lifecycle of the product.  This will be a new implementation, using the latest released version of Grouper (2.2.2).  I had sent this question to the group early last week, but it seems to have been lost in the shuffle or perhaps I'm just too impatient.

I am attempting to build a process that would allow migration of new stem structures and preconfigured groups (like loader jobs) from development, to test, to production.  It would also be nice to be able to maintain an external repository of these types of objects so that new environments may be built in a more automated way.  It seemed to me the Import-Export functionality would be a good candidate for this, but I am either doing something wrong, or it does not operate as I current understand.

An example: gsh -xmlexport -excludeAudits -objectNames etc:LoaderJobs:SomeLoaderJob ~/SomeLoaderJob.xml The object in question is a group definition with attribute values to configure a (successful) SQL_GROUP_LIST loader job with no direct members.

My understanding of the documentation on https://spaces.internet2.edu/display/Grouper/Import-Export made me believe that the result would be an XML representation of the supplied group.  Instead the result is the export of a very large number (reportedly 600,000+, though it "may be less") of objects that includes thousands of memberships, audit events, and perhaps more since I haven't let a process complete through the extraction of audit events.

Have I misunderstood what the xmlexport utility is for and how it works, or perhaps I am doing something wrong?  The inclusion of audit events alone makes me believe I've missed something, since I thought I was explicitly avoiding the extraction of that type of object with the -excludeAudits option.

If this is not the utility to perform such tasks, does the community have suggestions on how this type of object management might be performed?

Thank you,
Sean