Skip to Content.
Sympa Menu

grouper-dev - loader for attribute definitions

Subject: Grouper Developers Forum

List archive

loader for attribute definitions


Chronological Thread 
  • From: Chris Hyzer <>
  • To: Grouper Dev <>
  • Subject: loader for attribute definitions
  • Date: Tue, 4 May 2010 02:25:50 -0400
  • Accept-language: en-US
  • Acceptlanguage: en-US

Hey,

 

This is done including an example of how Penn will manage permissions resources from our org chart.

 

https://spaces.internet2.edu/display/GrouperWG/Grouper+-+Loader+for+attribute+or+permission+definitions

 

https://bugs.internet2.edu/jira/browse/GRP-428

 

The Grouper Loader can be used to management attribute definitions (for attributes or permissions).  You can manage 4 parts of the attribute definition (each is optional, though you should pick one or wont need the loader):

  • AttributeDefNames: these are the attribute or permission names:  you can specify the name (mandatory), displayName (optional), and description (optional)
  • AttributeDefNameSets: relationships among attribute def names.  e.g. if one attribute def name implies another.  e.g. if org123 implies org1234
  • Actions: if you have actions that should be driven by database
  • ActionSets: if an action implies another, e.g. ADMIN implies READ and UPDATE

Specify the built in attributes in the grouper.properties:

#####################################

## attribute framework

#####################################

 

# root stem in grouper where built in attributes are put

grouper.attribute.rootStem = etc:attribute

 

# if the attribute loader attributes should be autoconfigured (created, etc)

grouper.attribute.loader.autoconfigure = true

 

The next time you start Grouper, it will auto-create the loader attributes that you can assign to an attributeDef.  The prefix is the attribute root stem in the grouper.properties (above), concatenated with "attrLoader", and the extension.  Note, the etc:attribute:attrLoader:attributeDefLoaderTypeDef is the definition for the loader "type".  You can control access to who can assign loader jobs with that loader.  By default only root or wheel can use this (probably how it should be or a restricted group of users for security reasons).

Attribute name (prefix etc:attribute:attrLoader configured in grouper.properties)

Meaning

etc:attribute:attrLoader:attributeLoader

Assign this to an attributeDef to designate it as a "loader" type.  Then the other a

etc:attribute:attrLoader:attributeLoaderType

Type of loader, e.g. ATTR_SQL_SIMPLE

etc:attribute:attrLoader:attributeLoaderDbName

DB name in grouper-loader.properties or default grouper db if blank

etc:attribute:attrLoader:attributeLoaderScheduleType

Type of schedule.  Defaults to CRON if a cron schedule is entered, or START_TO_START_INTERVAL if an interval is entered

etc:attribute:attrLoader:attributeLoaderQuartzCron

If a CRON schedule type, this is the cron setting string from the quartz product to run a job daily, hourly, weekly, etc.  e.g. daily at 7am: 0 0 7 * * ?

etc:attribute:attrLoader:attributeLoaderIntervalSeconds

If a START_TO_START_INTERVAL schedule type, this is the number of seconds between runs

etc:attribute:attrLoader:attributeLoaderPriority

Quartz has a fixed threadpool (max configured in the grouper-loader.properties), and when the max is reached, then jobs are prioritized by this integer.  The higher the better, and the default if not set is 5.

etc:attribute:attrLoader:attributeLoaderAttrsLike

If empty, then orphans (for attributeDefName and attributeDefNameSets) will be left alone.  If %, then all orphans deleted.  If a SQL like string, then only ones in that like string not in loader will be deleted

etc:attribute:attrLoader:attributeLoaderAttrQuery

SQL query with at least some of the following columns: attr_name, attr_display_name, attr_description

etc:attribute:attrLoader:attributeLoaderAttrSetQuery

SQL query with at least the following columns: if_has_attr_name, then_has_attr_name

etc:attribute:attrLoader:attributeLoaderActionQuery

SQL query with at least the following column: action_name

etc:attribute:attrLoader:attributeLoaderActionSetQuery

SQL query with at least the following columns: if_has_action_name, then_has_action_name

Here is an example of loading org units into an attribute definition for org unit permissions (e.g. READ on org123 or WRITE on org234) including hierarchies

Create two views, one for attribute definitions, one for the relationships among them.  Here is the attribute view that has 1100 rows

ORG_ATTRIBUTE_DEF_NAME

ORG_ATTRIBUTE_DEF_DISPLAY_NAME

penn:community:employee:org:TOPU:UNIV:UADM:91XX

penn:community:employee:org:TOPU:UNIV:UADM:91XX - Information Systems and Computing Parent

penn:community:employee:org:TOPU:UNIV:UADM:91XX:91YY

penn:community:employee:org:TOPU:UNIV:UADM:91XX:91YY - ISC Other Parent

penn:community:employee:org:TOPU:UNIV:UADM:91XX:91YY:9100

penn:community:employee:org:TOPU:UNIV:UADM:91XX:91YY:9100 - Information Systems and Computing

penn:community:employee:org:TOPU:UNIV:UADM:91XX:91YY:9101

penn:community:employee:org:TOPU:UNIV:UADM:91XX:91YY:9101 - ISC Finance and HR

penn:community:employee:org:TOPU:UNIV:UADM:91XX:AIS:9142

penn:community:employee:org:TOPU:UNIV:UADM:91XX:AIS:9142 - Administrative Information Technology and Data Admin

penn:community:employee:org:TOPU:UNIV:UADM:91XX:AIS:9147

penn:community:employee:org:TOPU:UNIV:UADM:91XX:AIS:9147 - Information Security Project Office and Technology

penn:community:employee:org:TOPU:UNIV:UADM:91XX:AIS:SEOG

penn:community:employee:org:TOPU:UNIV:UADM:91XX:AIS:SEOG - Systems Engineering & Operations Group Parent

penn:community:employee:org:TOPU:UNIV:UADM:91XX:AIS:SEOG:9143

penn:community:employee:org:TOPU:UNIV:UADM:91XX:AIS:SEOG:9143 - ISC-Systems Engineering

penn:community:employee:org:TOPU:UNIV:UADM:91XX:AIS:SEOG:9145

penn:community:employee:org:TOPU:UNIV:UADM:91XX:AIS:SEOG:9145 - Computer Operations

penn:community:employee:org:TOPU:UNIV:UADM:91XX:ITS:9153

penn:community:employee:org:TOPU:UNIV:UADM:91XX:ITS:9153 - ISC Support-On-Site

penn:community:employee:org:TOPU:UNIV:UADM:91XX:ITS:9156

penn:community:employee:org:TOPU:UNIV:UADM:91XX:ITS:9156 - ISC Communications Group

penn:community:employee:org:TOPU:UNIV:UADM:91XX:ITS:9157

penn:community:employee:org:TOPU:UNIV:UADM:91XX:ITS:9157 - Technology Support Services

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO - Network Operations Parent

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO:9131

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO:9131 - Network Engineering and Services

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO:9161

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO:9161 - Telecommunications Services

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO:9166

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO:9166 - Network Operations

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO:9181

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO:9181 - Metropolitan Area GigaPoP in Philadelphia for Internet2

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO:9182

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO:9182 - Next Generation PennNet Projects

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO:9183

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO:9183 - Penn Video Network Video Services

penn:community:employee:org:TOPU:UNIV:UADM:92XX

penn:community:employee:org:TOPU:UNIV:UADM:92XX - Human Resources Parent

Another view with the relationships among orgs that has 1100 immediate relationships (the size is a coincidence).  Note, only parent0child relationships need to be represented here, not grandparent or other relationships, those will be provided by Grouper automatically.

IF_HAS_ATTRIBUTE_DEF_NAME

THEN_HAS_ATTRIBUTE_DEF_NAME

penn:community:employee:org:TOPU:UNIV:UADM:90XX:DEVS

penn:community:employee:org:TOPU:UNIV:UADM:90XX:DEVS:9010

penn:community:employee:org:TOPU:UNIV:UADM:91XX

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO

penn:community:employee:org:TOPU:UNIV:UADM:91XX

penn:community:employee:org:TOPU:UNIV:UADM:91XX:ITS

penn:community:employee:org:TOPU:UNIV:UADM:91XX

penn:community:employee:org:TOPU:UNIV:UADM:91XX:91YY

penn:community:employee:org:TOPU:UNIV:UADM:91XX

penn:community:employee:org:TOPU:UNIV:UADM:91XX:AIS

penn:community:employee:org:TOPU:UNIV:UADM:91XX:91YY

penn:community:employee:org:TOPU:UNIV:UADM:91XX:91YY:9101

penn:community:employee:org:TOPU:UNIV:UADM:91XX:91YY

penn:community:employee:org:TOPU:UNIV:UADM:91XX:91YY:9100

penn:community:employee:org:TOPU:UNIV:UADM:91XX:AIS

penn:community:employee:org:TOPU:UNIV:UADM:91XX:AIS:SEOG

penn:community:employee:org:TOPU:UNIV:UADM:91XX:AIS

penn:community:employee:org:TOPU:UNIV:UADM:91XX:AIS:9142

penn:community:employee:org:TOPU:UNIV:UADM:91XX:AIS

penn:community:employee:org:TOPU:UNIV:UADM:91XX:AIS:9147

penn:community:employee:org:TOPU:UNIV:UADM:91XX:AIS:SEOG

penn:community:employee:org:TOPU:UNIV:UADM:91XX:AIS:SEOG:9143

penn:community:employee:org:TOPU:UNIV:UADM:91XX:AIS:SEOG

penn:community:employee:org:TOPU:UNIV:UADM:91XX:AIS:SEOG:9145

penn:community:employee:org:TOPU:UNIV:UADM:91XX:ITS

penn:community:employee:org:TOPU:UNIV:UADM:91XX:ITS:9157

penn:community:employee:org:TOPU:UNIV:UADM:91XX:ITS

penn:community:employee:org:TOPU:UNIV:UADM:91XX:ITS:9153

penn:community:employee:org:TOPU:UNIV:UADM:91XX:ITS

penn:community:employee:org:TOPU:UNIV:UADM:91XX:ITS:9156

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO:9183

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO:9181

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO:9161

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO:9182

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO:9166

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO

penn:community:employee:org:TOPU:UNIV:UADM:91XX:NETO:9131

penn:community:employee:org:TOPU:UNIV:UADM:92XX

penn:community:employee:org:TOPU:UNIV:UADM:92XX:HRS

Create an attribute definition:

grouperSession = GrouperSession.startRootSession();

orgAttributeDef = new AttributeDefSave(grouperSession).assignName("penn:community:employee:orgPermissions:orgs").assignAttributeDefType(AttributeDefType.perm).assignToEffMembership(true).assignToGroup(true).save();

orgAttributeDef.getAttributeDefActionDelegate().configureActionList(GrouperUtil.toSet(new Object[]{"read", "write"}));

 

Assign the loader type and attributes to that attribute definition

orgAttributeDef.getAttributeDelegate().assignAttributeByName(GrouperCheckConfig.attributeLoaderStemName() + ":attributeLoader");

orgAttributeDef.getAttributeValueDelegate().assignValue(GrouperCheckConfig.attributeLoaderStemName() + ":attributeLoaderType", "ATTR_SQL_SIMPLE");

orgAttributeDef.getAttributeValueDelegate().assignValue(GrouperCheckConfig.attributeLoaderStemName() + ":attributeLoaderQuartzCron", "0 0 7 * * ?");

orgAttributeDef.getAttributeValueDelegate().assignValue(GrouperCheckConfig.attributeLoaderStemName() + ":attributeLoaderAttrsLike", "%");

orgAttributeDef.getAttributeValueDelegate().assignValue(GrouperCheckConfig.attributeLoaderStemName() + ":attributeLoaderAttrQuery", "select oadf.ATTRIBUTE_NAME attr_name, oadf.ATTRIBUTE_DISPLAY_NAME attr_display_name from org_attribute_def_name oadf");

orgAttributeDef.getAttributeValueDelegate().assignValue(GrouperCheckConfig.attributeLoaderStemName() + ":attributeLoaderAttrSetQuery", "select oadns.IF_HAS_ATTRIBUTE_DEF_NAME if_has_attr_name, oadns.THEN_HAS_ATTRIBUTE_DEF_NAME then_has_attr_name  from org_attribute_def_name_set oadns");

 

Run the job once

loaderRunOneJobAttr(orgAttributeDef);

...

2010-05-04 02:19:47,723: [main] INFO  GrouperLoaderType.helperSyncAttributeDefNameSets(2108) - penn:community:employee:orgPermissions:orgs processed 1535 attributeDefNameSet records, finding new attributeDefNameSets to insert/remove, 500 of 1077 attributeDefNameSets

2010-05-04 02:20:54,663: [main] INFO  GrouperLoaderType.helperSyncAttributeDefNameSets(2108) - penn:community:employee:orgPermissions:orgs processed 2035 attributeDefNameSet records, finding new attributeDefNameSets to insert/remove, 1000 of 1077 attributeDefNameSets

2010-05-04 02:21:08,648: [main] INFO  GrouperLoaderType.syncOneAttributeDef(1743) - penn:community:employee:orgPermissions:orgs done syncing attributeDef, processed 2194 records.  Total members: 2111, inserts: 994, deletes: 0

loader ran successfully, inserted 994 memberships, deleted 0 records, total record count: 2194

 

sdaf



  • loader for attribute definitions, Chris Hyzer, 05/04/2010

Archive powered by MHonArc 2.6.16.

Top of Page