Skip to Content.
Sympa Menu

grouper-users - RE: [grouper-users] Penn's organizational hierarchy

Subject: Grouper Users - Open Discussion List

List archive

RE: [grouper-users] Penn's organizational hierarchy

Chronological Thread 
  • From: Chris Hyzer <>
  • To: "" <>
  • Cc: Grouper Users Mailing List <>, "" <>
  • Subject: RE: [grouper-users] Penn's organizational hierarchy
  • Date: Fri, 22 May 2009 08:08:41 -0400
  • Accept-language: en-US
  • Acceptlanguage: en-US

> It could well do, although I haven't yet digested it all. Perhaps you
> could add short summary at the beginning of the document in which you
> describe the general approach. For example I hadn't been aware that
> there was a loader pre-run hook.

I added the pre-hook for the loader work I did for generic org management
last month. I sent an email and wiki about it, but not a Jira, I just added

I added a summary of the Penn org implementation. Let me know if you need
more info:

General approach

* We have a view of orgs and parent id's, and a view of assignments of
org to person

* Since we want the orgs in a tree structure in Grouper, and the loader
expects the group_name matched with person id, we use the built-in grouper
hook on pre-run of loader job, to take the org table and calculate the
hierarchical group names. You could do this part a variety of ways,
including a db function (e.g. PLSQL for oracle)

* There are a few loader jobs:

o One that manages all the leaf node orgs groups (assigns people to
org groups)

o One that manages all the non-leaf rollup nodes (assigns orgs to
parent org groups)

o One that manages Penn "centers" which are our high level
organizations (assigns direct center children orgs to center groups)

o One that manages contractor groups. We dont have contractors
assigned in our org structure, so we manage them through attributes in our
person database. Then we can manually make other groups next to the org
rollup which includes the org rollup and the contractor group

* We will run all these jobs once daily after our payroll jobs run

* Grouper include/exclude automatically creates include and exclude lists
which are tacked on to the system of record group. This is useful in loader
jobs since the loader system of record list is resynced with the DB query
with each run, so manual changes will be undone. Penn actually only needs
"include" lists, not exclude at this point (use case is VP's are in a
different org than the org they manage, so we want to add them in). So we
will have include/exclude groups on the rollup groups, the centers groups,
and the contractor groups. We dont think we will need them on the leaf
nodes, though if we have a need later, we can add it in. The reason not to
do this is performance, it creates a bunch more groups and memberships.

Here are the steps to creating a loader SQL_GROUP_LIST job:

* First of all, you should prefer using views, and simply put simple
select from the view in the loader config. It is also easy to tell what the
loader is going to do without hunting through the loader configs, and easy to
make changes at runtime (though I believe the loader allows runtime query
changes as well)

* Note: if you are using include/exclude, then the group names should
have the system of record suffix which is configured in the

* Make a view of groups where each row represents a group.

o There is a col for the group name, display name (optional),
description (optional), and security (e.g. readers, viewers, etc: optional)

o This view will set these attributes of group and auto-create
groups which have no members (might be useful for orgs since apps can refer
to groups which have no members)

o Note: if you can give groups a unique suffix in the stem
structure, then the job can use the setting "grouperLoaderGroupsLike" which
will delete groups which are no longer in the group list

* Make the query which returns the subjectId (and sourceId if not the
default loader source), and group name

o For leaf nodes, this is generally a simple query that assigns
people to org groups

o For rollup nodes, this can be a little complex.

+ First of all, you might union the direct rollup children
with the direct rollup leaf nodes

+ The subjectId for groups is the group_id. for Grouper 1.4,
you can join to the grouper_attributes table. For 1.5 you can simply join to
the grouper_groups table. In both, you can join to grouper_groups_v if you
like. What I did for 1.4 is: grouper_attributes ga, grouper_fields gf where
gf.NAME = 'name' AND gf.ID = ga.field_id and ga.VALUE =
ocrv.MEMBER_GROUP_NAME. I would definitely keep these query in a view.

+ You should also specify the source id for groups: 'g:gsa'

* Configure the config group for each loader job. I put this next to the
top level loaded stem. I generally do this in GSH, though you could also do
this in the UI

* Kick off the loader job manually in GSH so you can verify the results
without waiting for the cron to run

* Restart your loader so it picks up the new job

> -----Original Message-----
> From: Loris Bennett
> [mailto:]
> Sent: Friday, May 22, 2009 4:31 AM
> To: Chris Hyzer
> Cc: Grouper Users Mailing List;
> Subject: Re: [grouper-users] Penn's organizational hierarchy
> On Thu, 2009-05-21 at 15:37 -0400, Chris Hyzer wrote:
> > Hey,
> >
> > I implemented the Grouper org hook at Penn. I have permission to
> share
> > my experience outside of Penn so I made a document here:
> >
> >
> onal+hierarchy
> >
> > This is the size of our org implementation:
> >
> > 27,000 people in orgs at Penn
> > 2,200 orgs
> > 3,000 org groups (more due to include/exclude lists)
> > 500,000 org memberships (there are a lot due to the rollups, and
> include/exclude lists)
> >
> > Note that the bulk of the work was in figuring out how our org data
> is
> > structured, and how I can expose it in views to something that the
> > loader can process. If you want to get this working at your
> > institution this document should be helpful.
> >
> > Btw, Loris, does this help with your org issues?
> It could well do, although I haven't yet digested it all. Perhaps you
> could add short summary at the beginning of the document in which you
> describe the general approach. For example I hadn't been aware that
> there was a loader pre-run hook.
> Cheers
> Loris
> > Thanks,
> > Chris
> --
> Dr. Loris Bennett (Mr.)
> Freie Universität Berlin
> ZEDAT - Zentraleinrichtung für Datenverarbeitung / Computer Center
> Compute & Media Service
> Fabeckstr. 32, Room 221
> D-14195 Berlin
> Tel ++49 30 838 51024
> Fax ++49 30 838 56721
> Email
> Web

Archive powered by MHonArc 2.6.16.

Top of Page