Grouper Users - Open Discussion List

[Loris Bennett <>]

Hi Chris,

Thanks for all your support. Here is the info you requested

- I am using Grouper 1.3.0 with PostgreSQL 8.2.4.
- The interval for the loader is 30s and I can see from the log that the
loader ran after the user was deleted from the database.
- I can't see *_count in the logs at all - what log4j properties do I
need?
- In log4j.properties I have: 
        ## GrouperLoader
        log4j.logger.edu.internet2.middleware.grouper.loader = DEBUG,
        grouper_debug
- The following error appears in the log when there is no subject for a
group name:
        2008-10-16 10:07:59,320 ERROR db.GrouperLoaderResultset: Problem
        with subjectId: null, subjectSourceId: null
        java.lang.IllegalArgumentException: null Subject Id
        
I have done some more testing and found the following behaviour.

- If I add a new group and a single user to the database, this is added
correctly in grouper.
- If this group and the user is deleted from the database, both the
group and the user remain in grouper
- If the group is added to the database with a different user, the
original user is removed and the new user added in grouper.

Thus it seems to be a problem with the case where there is no more
information about a group in the group-subjectId table. In this instance
the group and its members are not removed. I assume some sort of
comparison with the current state in grouper is needed in order to allow
the identification of groups which have been deleted completely in the
database.

But perhaps this has been rectified in 1.3.1.

Thanks again for all the help.

Regards

Loris



On Thu, 2008-10-16 at 01:52 -0400, Chris Hyzer wrote:
> It should absolutely work.  We will make it work if it doesn’t, or prove 
> that it does.  What version of grouper are you using, and what database 
> type?
>   I recommend at least 1.3.1 for using the loader.  What interval did you 
> have the loader running for that dynamic group? 
> Did you see entries in the group_loaderlog table which prove that it ran 
> (after you committed a delete from source table)
>  and did you see the columns for insert_count, update_count, and 
> delete_count (were they correct)?  What is the log4j level for the loader?  
> At least warn as default?  Any errors in the logs?  Send me your simple 
> test case (table, query, settings for the attributes, etc)
>  and I will try to reproduce.
> 
> I will put more documentation up on the wiki about this and let you know, 
> and I will make an official test case for loader membership removal...
> 
> Thanks for patience,
> Chris
> 
> 
> > -----Original Message-----
> > From: Loris Bennett 
> > [mailto:]
> > Sent: Wednesday, October 15, 2008 6:06 AM
> > To: Chris Hyzer
> > Cc: Grouper Users Mailing List
> > Subject: RE: [grouper-users] Grouper-WS: Strategies for moving
> > groups/stems?
> >
> > Hi Chris,
> >
> > I had overlooked SQL_GROUP_LIST. Perhaps we could extend the
> > documentation in the wiki regarding this point, as this is certainly in
> > principle a great feature.
> >
> > I have tried this out by generating a hierarchy of folders and then
> > setting up a dummy table containing the group-subject mapping (we don't
> > currently have an appropriate table or view). The groups seemed to be
> > filled OK across the hierarchy.
> >
> > However, when I delete a subject from the group-subject mapping, the
> > corresponding user does not get removed from the group. Should this
> > work?
> >
> > Thanks
> >
> > Loris
> >
> > On Tue, 2008-10-14 at 10:17 -0400, Chris Hyzer wrote:
> > > There is a grouperLoaderType: SQL_GROUP_LIST, where you have a query
> > that looks like this:
> > >
> > > Group_name     Subject_id
> > > a:orgs:org1    1234567
> > > a:orgs:org1    234567
> > > a:orgs:org2    543234
> > > a:orgs:org3    766463
> > > a:orgs:org2    1234567
> > >
> > > The groups and the stems will be auto-created if they do not exist.
> > So if the org structure changes,
> > > the new groups discovered in the query will be auto-created and the
> > new memberships will be created.
> > > If you want to try to get something like this working, let me know
> > and we can iron out the details.
> > > If that doesn’t work for you, out of curiosity, let me know what the
> > gaps are.
> > >
> > > Kind regards,
> > > Chris
> > >
> > > > -----Original Message-----
> > > > From: Loris Bennett 
> > > > [mailto:]
> > > > Sent: Tuesday, October 14, 2008 3:41 AM
> > > > To: Chris Hyzer
> > > > Cc: Grouper Users Mailing List
> > > > Subject: RE: [grouper-users] Grouper-WS: Strategies for moving
> > > > groups/stems?
> > > >
> > > > Hi Chris,
> > > >
> > > > Thanks for the information. My understanding is that grouper-loader
> > is
> > > > used to fill a group dynamically based on an SQL statement. So am I
> > > > right in thinking that you model the org structure in terms of an
> > > > essentially static hierarchy of groups which are then filled
> > > > automatically via grouper-loader?
> > > >
> > > > On the other hand, I must be missing something, because you say
> > that
> > > > the
> > > > loader will automatically reflect changes in the org structure.
> > > > Currently I can't see how this can be done.
> > > >
> > > > Regards
> > > >
> > > > Loris
> > > >
> > > > On Mon, 2008-10-13 at 12:45 -0400, Chris Hyzer wrote:
> > > > > We are starting to do this at Penn with our organizational
> > structure
> > > > (I say "starting" because we are in production,
> > > > > but only loading a subset of all the orgs we need).
> > > > >
> > > > > We have all our data in one warehouse database, and I made a view
> > > > which links users to orgs.  Then I use grouper-loader
> > > > > to sync up that view with our org groups.  If a user moved from
> > one
> > > > org to another, or another org was added, or an org
> > > > > was renamed, grouper-loader figures it all out... maybe it can
> > help
> > > > you (less people use the loader, but if you find issues,
> > > > > I will fix them... :)  ).  Note, in the loader if an org is
> > moved, or
> > > > renamed, then it will delete the old, create the new,
> > > > > which might give other problems... but maybe we can add some
> > features
> > > > there :)
> > > > >
> > > > > https://wiki.internet2.edu/confluence/display/GrouperWG/Grouper+-
> > > > +Loader
> > > > >
> > > > > Kind regards,
> > > > > Chris
> > > > >
> > > > > > -----Original Message-----
> > > > > > From: Loris Bennett 
> > > > > > [mailto:]
> > > > > > Sent: Monday, October 13, 2008 9:39 AM
> > > > > > To: Grouper Users Mailing List
> > > > > > Subject: [grouper-users] Grouper-WS: Strategies for moving
> > > > > > groups/stems?
> > > > > >
> > > > > > Hi,
> > > > > >
> > > > > > On the roadmap the "Namespace Transition Support" is listed. I
> > was
> > > > > > wondering what the status of this is and what strategies people
> > are
> > > > > > using as a workaround. Specifically I am interested in how to
> > deal
> > > > > > which
> > > > > > changes in an organizational structure.
> > > > > >
> > > > > > Currently I am doing the following:
> > > > > >
> > > > > >         1. read the organizational structure from a human
> > resources
> > > > > >         database
> > > > > >         2. construct a corresponding structure of folders in
> > > > Grouper
> > > > > >         using the web services interface
> > > > > >         3. add a standard group to each folders
> > > > > >         4. read the user data from the HR database
> > > > > >         5. add the users to the standard group in the
> > appropriate
> > > > > >         folders.
> > > > > >
> > > > > > If, say, HR decides to move a group from one unit within a
> > > > department
> > > > > > to
> > > > > > another, that group should remain a member of the department,
> > but
> > > > > > should
> > > > > > no longer be a member of any unit-specific groups.
> > > > > >
> > > > > > I would envisage doing something like:
> > > > > >
> > > > > >         1. create new group in the new unit
> > > > > >         2. add members of old group to new group
> > > > > >         3. determine memberships of old group
> > > > > >         4. recreate memberships which need to be preserved
> > > > > >         5. add any new memberships required
> > > > > >         6. delete all old memberships
> > > > > >         7. delete old group
> > > > > >
> > > > > > Steps 4 and 5 would require some human interaction regarding
> > what
> > > > > > exactly should be done.
> > > > > >
> > > > > > My questions are:
> > > > > >
> > > > > >         1. Is this the approach that other people are taking?
> > > > > >         2. Are there any alternatives?
> > > > > >         3. Will "Namespace Transition Support" provide this
> > > > > >         functionality?
> > > > > >         4. When might NTS be available?
> > > > > >
> > > > > > Thanks
> > > > > >
> > > > > > Loris
> > > > > >
> > > > > > --
> > > > > > Dr. Loris Bennett
> > > > > > Computer Centre
> > > > > > Freie Universität Berlin
> > > > > > Berlin, Germany
> > > > >
> > >
>