grouper-dev - Re: [grouper-dev] Re: [signet-dev] New ldappc snapshot fixes runner script problem
Subject: Grouper Developers Forum
List archive
Re: [grouper-dev] Re: [signet-dev] New ldappc snapshot fixes runner script problem
Chronological Thread
- From: Graham Seaman <>
- To: Kathryn Huxtable <>
- Cc: Grouper Dev <>,
- Subject: Re: [grouper-dev] Re: [signet-dev] New ldappc snapshot fixes runner script problem
- Date: Tue, 12 Aug 2008 11:47:05 +0100
Kathryn Huxtable wrote:
Yes, you *could* export all your externally provisioned groups and feed them into grouper, but that seems like overkill.My final requirement is to to be able to take some groups created in grouper and other groups already existing in the LDAP directory and assign them permissions using signet, feeding the permissions back into eduPermission attributes of individual person objects. Once set up these permissions will then be fed through to applications via Shibboleth on individual login.
So it seems likely that I will need to have grouper know about all my groups, including the externally created ones, and not only in order to stop existing isMemberOf values being deleted.
I had the impression before that my final requirement was quite achievable with current versions of signet/grouper/ldappc, but reading the conversation in the 'Proposal for ldappc provision scoping behavior' I'm no longer quite so sure. What do you think? Is this goal achievable with the current versions of grouper/signet/ldappc?
If you're currently provisioning your isMemberOf attributes from some other source, then I'd use a test LDAP directory until we have scoping working.Yes, it's well away from any live directory. However, my deadline is probably too tight to be able to wait on a final scoped ldappc version - if possible, I need to find temporary kludges. The real fix can be added in later.
The first thing I was trying to do was simply to provision the isMemberOf attribute of my person objects. I'm attaching the ldappc.xml part (the signet section of this is completely untested). My test group has the ID path in grouper of grouper:flame:test1. After running ldappc -memberships, members of the test1 group have all existing (non-grouper) isMemberOf values removed, and no new value added. The debug log says:
Not sure why it didn't provision your test group, though. Could you send me your (password-sanitized) ldappc.xml file please?
**** Starting Provisioning *****
2008-08-12 10:45:20,944 INFO logging.DebugLog: Started GrouperSession: 4f8d1be5-0757-4318-91bd-d80eee8db2bc,'GrouperSystem','application'
2008-08-12 10:45:54,476 INFO logging.DebugLog: Stopped GrouperSession: 4f8d1be5-0757-4318-91bd-d80eee8db2bc,'GrouperSystem','application'
2008-08-12 10:45:54,476 INFO logging.DebugLog: Subject ID Lookups: 0
2008-08-12 10:45:54,476 INFO logging.DebugLog: Subject Table Hits: 0
2008-08-12 10:45:54,477 INFO logging.DebugLog: Subject hit ratio: 0.0%
The ldap log show ldappc search through the 16k entries in the directory one by one, finding the members of the test group, and continuing; eg.
[12/Aug/2008:11:13:01 +0100] conn=115 op=12706 SRCH base="cn=XXXX,ou=Flame Users,dc=lse,dc=ac,dc=uk" scope=0 filter="(objectClass=*)" attrs="isMemberOf objectClass"^M
[12/Aug/2008:11:13:01 +0100] conn=115 op=12706 RESULT err=0 tag=101 nentries=1 etime=0
[12/Aug/2008:11:13:01 +0100] conn=115 op=12707 SRCH base="cn=XXXY,ou=Flame Users,dc=lse,dc=ac,dc=uk" scope=0 filter="(objectClass=*)" attrs="isMemberOf objectClass"
where 'XXXX' is a member of test1. I don't understand the filter value, given the filter defined in ldappc.xml. Although this is similar to the setup I had issues with a few month ago on grouper-users, some of the ldap details have since changed.
If I try to provision the test1 group itself, ldappc breaks. I assume this is due to the '/' character; I haven't found a list of allowed characters for group names yet, so I can sanitize my names..
[10:35 g_seaman@vaheva:/usr/local/src/ldappc-1.2.0-SNAPSHOT] ./bin/ldappc.sh -subject GrouperSystem -groups
2008-08-12 10:35:27,566 FATAL ErrorLog: [edu.internet2.middleware.ldappc.LdappcGrouperProvisioner] Grouper Provision Failed: Invalid name: "cn=lse:department:Short Stay Student 2007/08"
Graham
-K
On Aug 11, 2008, at 11:52 AM, Graham Seaman wrote:
Kathryn Huxtable wrote:
Graham Seaman pointed out that there was a problem in my runner scripts. I've fixed them.That now runs (though CLASSPATH needs explicitly exporting, at least on my box. Otherwise I get the dreaded:
I've deployed a new snapshot, ldappc-1.2.0 snapshot 20080811.152437-2, to the I2 site. It's linked on the wiki download page.
[16:58 g_seaman@vaheva:/usr/local/src/ldappc-1.2.0-SNAPSHOT] ./bin/ldappc.sh -subject GrouperSystem -memberships -subject GrouperSystem
Exception in thread "main" java.lang.NoClassDefFoundError: edu/internet2/middleware/ldappc/Ldappc)
Unfortunately my first attempt at a run with
./bin/ldappc.sh -subject GrouperSystem -memberships
simply removed all isMemberOf attributes from all my existing ldap data and didn't provision my little grouper test group at all. The data removal
presumably relates to the thread 'Proposal for ldappc provision scoping behavior', but until that is settled, what is the best fix? I assume I need to make sure all my externally provisioned groups are fed into grouper; do I need to do that through exporting xml from ldap and importing it to grouper, or is there a better way (the externally provisioned groups are updated on a daily basis)?
Graham
Graham
Sorry about that. I'd swear those scripts worked a few weeks ago...
-K
<?xml version="1.0" encoding="utf-8"?>
<!--
* Edited version of ldappcTemplate.xml GS
*
* It is important to note that the XML schema used to validate this file
* requires the elements to be in the order presented here.
-->
<ldappc>
<!--
* <grouper> is an optional element that contains the elements
* specific to provisioning data from Grouper. Note, if <grouper>
* is NOT defined, then <signet> must be. <grouper> contains the
* following elements.
*
* <group-queries> - This required element defines which groups are
* selected for the provisioning.
*
* <groups> - This optional element defines the configuration
* elements necessary for provisioning groups.
*
* <memberships> - This optional element defines the configuration
* elements necessary for provisioning memberships.
*
* Although both <groups> and <memberships> are optional, either one or
* both must be defined within <grouper>.
-->
<grouper><!-- 0 or 1 allowed -->
<!--
* <group-queries> is a required element identifying the
* queries used to select the groups used during provisioning.
* The set of groups created by the union of the results of each query
* determine the groups used for provisioning. It contains the following
* elements.
*
* <subordinate-stem-queries> - This optional element selects all
* groups that are subordinate to anyone of the naming
* stems identified in its <stem-list> element.
*
* <attribute-matching-queries> - This optional element selects all
* groups having an attribute containing the same value
* as anyone of the attribute name/value pairs identified in its
* <attribute-list> element.
*
* Although both <subordinate-stem-queries> and <attribute-matching-queries>
* are optional, either one or both must be defined within <grouper>.
-->
<group-queries><!-- exactly 1 required -->
<!--
* <subordinate-stem-queries> is a optional element defining a
* list of stems such that any group that is part of a subordinate
* stem is selected for provisioning. It contains the following
* element.
*
* <stem-list> - This required element defines the list of stems
*
* <stem-list> contains one or more of the following element.
*
* <stem> - This required element defines the stem.
*
* <stem>'s content is the the stem (e.g., uc:faculty:art).
-->
<subordinate-stem-queries><!-- 0 or 1 allowed-->
<stem-list><!-- exactly 1 required -->
<stem>grouper</stem><!-- 1 or more -->
</stem-list>
</subordinate-stem-queries>
</group-queries>
<!--
* <groups> is an optional element defining how the entries and DNs
* for provisioned groups are created. This element is REQUIRED if
* groups are being provisioned. The following attributes of <groups>
* define how the entries and DNs are created.
*
* structure - Defines the DN naming structure for groups. Its
* value must be one of the values "flat" or "bushy".
*
* root-dn - Defines the DN of the entry to be used as the root
* for storing the provisioned groups (e.g., "ou=groups,dc=example,dc=edu").
*
* initial-caches-size - Optional attribute specifying the initial
* size of the group cache. Setting this larger than the likely
* number of groups to be provisioned should improve performance.
*
* ldap-object-class - Defines the object class used to create
* each provisioned group's LDAP entry. Please note that if this
* object class has required attributes not populated by the
* provisioning process, then an error will occur.
*
* ldap-rdn-attribute - Defines the attribute in the ldap-object-class
* used as the RDN. This value may not be "ou" in order to prevent
* among other things naming collisions between stems and groups
* when structure is "bushy".
*
* grouper-attribute - Optional attribute defining whether the
* id or name attribute value of the group is to be used for the
* value of the ldap-rdn-attribute in the directory when structure
* is "flat". This is required when structure is "flat", but can be
* removed if structure is "bushy". The allowed values are "id" or "name".
*
* The <groups> element contains the following elements.
*
* <group-members-dn-list> - This optional element defines in which
* attribute the list of member DNs is maintained. If this
* element is NOT present, the list is NOT maintained.
*
* <group-members-name-list> - This optional element defines in which
* attribute the list of member names is maintained. If this
* element is NOT present, the list is NOT maintained.
*
* <group-attribute-mapping> - This optional element defines how
* group attributes are mapped to the group's directory entry.
* If this element is NOT present, then attributes are NOT maintained.
*
* Although <grouper-members-dn-list> and <group-members-name-list> are
* optional, either one or both must be defined within <groups>.
-->
<groups structure="flat"
root-dn="ou=Flame Groups,dc=lse,dc=ac,dc=uk"
ldap-object-class="groupOfNames"
ldap-rdn-attribute="cn"
grouper-attribute="id">
<!--
* The <group-members-dn-list> is an optional element defining
* whether or not provisioned group LDAP entries have an attribute
* listing all of the DNs of each Member's LDAP entry. If the
* element is defined, then the DNs of the Member's LDAP entries
* are stored in the LDAP attribute identified by the value of
* list-attribute.
*
* list-object-class - Optional attribute defining the
* object class the group entry must have to support the
* list-attribute. Please note that if this object class
* has required attributes not populated by the provisioning
* process, then an error may occur.
*
* list-attribute - Defines the LDAP entry attribute in which
* to store DN list (e.g., groupList).
*
* list-empty-value - Optional attribute defining the value to
* be placed in the list-attribute if no member DNs are
* stored there. If list-attribute is optional (i.e., a MAY
* attribute), this value is most likely not needed. If
* list-attribute is required (i.e., a MUST attribute), then
* this value should be defined.
*
-->
<group-members-dn-list list-object-class="groupOfNames"
list-attribute="member" list-empty-value="" /><!-- 0 or 1 allowed -->
<!--
* <group-members-name-list> is an optional element defining
* whether or not provisioned group LDAP entries have an attribute
* listing all of the members by name which belong to it. If this
* element is defined, then the list of members for a group will be
* stored in the group LDAP entry according to the following
* attributes.
*
* list-object-class - Optional attribute defining the object
* class the group entry must have to support the list-attribute.
* Please note that if this object class has required
* attributes not populated by the provisioning process,
* then an error may occur.
*
* list-attribute - Defines the LDAP entry attribute in which
* to store group list (e.g., groupList).
*
* list-empty-value - Optional attribute defining the value to
* be placed in the list-attribute if no member names are
* stored there. If list-attribute is optional (i.e., a MAY
* attribute), this value is most likely not needed. If
* list-attribute is required (i.e., a MUST attribute), then
* this value should be defined.
*
* <group-members-name-list> contains the following element.
*
* <source-subject-name-mapping> - This required element defines
* for each Source the Subject attribute to be used as the name
* of the Subject.
-->
<group-members-name-list
list-object-class="eduMember"
list-attribute="hasMember"
list-empty-value="">
<!--
* <source-subject-name-mapping> is a required element
* identifying for each Source which Subject attribute holds
* the value to be used as the name of the Subject. It contains
* one or more of the following element.
*
* <source-subject-name-map> - This required element defines
* single Source the Subject attribute holding the value
* to be used as the Subject's name.
-->
<source-subject-name-mapping><!-- Exactly 1 required -->
<!--
* <source-subject-name-map> is a required element identifying
* for a source the subject attribute containing the name of
* the subject.
*
* source - Defines the ID of the Source (e.g., g:gsa).
*
* subject-attribute - Defines the name of the Subject
* attribute containing the name of the subject
* (e.g., logonid).
-->
<!-- source id defined in sources.xml GS -->
<source-subject-name-map source="flame-person"
subject-attribute="employeeNumber" /><!-- 1 or more required -->
</source-subject-name-mapping>
</group-members-name-list>
<!--
* <group-attribute-mapping> is an optional element defining
* how attributes from a group are mapped to attributes
* of the group's LDAP entry.
*
* ldap-object-class - Optional attribute defining the
* object class the group entry must have to support
* the attribute mapping. Please note that if this object
* class has required attributes not populated by the
* provisioning process, then an error may occur.
*
* <group-attribute-mapping> contains one or more of the following
* element.
*
* <group-attribute-map> - This required element defines for
* a given group attribute which LDAP attribute will hold
* its value.
-->
</groups>
<!--
* <memberships> is an optional element defining how memberships
* are provisioned. This element is REQUIRED if Memberships are
* being provisioned. <memberships> contains the following element.
*
* <member-groups-list> - This required element defines how the
* list of groups to which a member belongs is maintained on
* the members LDAP entry.
*
-->
<memberships><!-- 0 or 1 allowed -->
<!--
* <member-groups-list> is a required element defining the
* attribute of provisioned group members LDAP entries that will
* list all the groups to which they belong.
*
* list-object-class - Optional attribute defining the
* object class the Member's entry must have to support
* the group list. Please note that if this object class
* has required attributes not populated by the provisioning
* process, then an error may occur.
*
* list-attribute - Defines the LDAP entry attribute in which
* to store group list (e.g., groupList).
*
* list-empty-value - Optional attribute defining the value to
* be placed in the list-attribute if no group names are
* stored there. If list-attribute is optional (i.e., a MAY
* attribute), this value is most likely not needed. If
* list-attribute is required (i.e., a MUST attribute), then
* this value should be defined.
*
* naming-attribute - Defines the group naming attribute to use
* when creating the list of groups for a member (e.g., "name" ).
*
* temporary-directory - Optional attribute defining a directory in
* which the membership update files will be written. Defaults to
* the current directory.
-->
<member-groups-list
list-object-class="eduMember"
list-attribute="isMemberOf"
naming-attribute="id"
list-empty-value=""
temporary-directory="" /><!-- exactly 1 required -->
</memberships>
</grouper>
<!--
* <signet> is an optional element that contains the elements specific to
* provisioning information from Signet. Note, if <signet> is NOT defined,
* then <grouper> must be. <signet> contains the following elements.
*
* <permissions-listing> - This required element defines how permissions
* are stored when provisioned.
*
* <permissions-queries> - This optional element defines additional
* criteria for determining which permissions are provisioned.
*
-->
<signet><!-- 0 or 1 allowed -->
<!--
* <permissions-listing> is a required element that defines how
* provisioned permissions are stored in a Subject's LDAP entry.
* The method storing provisioned permissions is wholly defined by
* the following attributes.
*
* stored-as - Defines how the permissions are stored. Its value
* must be one of "string" or "eduPermission".
*
* string-object-class - Optional attribute defining the object
* class to add to the Member's subject LDAP entry in order to
* support the string-attribute. Please note that if this object
* class has required attributes not populated by the
* provisioning process, then an error may occur.
*
* string-attribute - Optional attribute defining the LDAP entry
* attribute in which to store permissions if stored-as is set
* to "string" (e.g., permissionStrings). As implied this
* attribute is required if stored-as is "string".
*
* string-prefix - Optional attribute defining the prefix to use
* when storing permissions as strings.
*
* string-list-empty-value - Optional attribute defining the value to
* be placed in the list-attribute if no permission strings are
* stored there. If list-attribute is optional (i.e., a MAY
* attribute), this value is most likely not needed. If
* list-attribute is required (i.e., a MUST attribute), then
* this value should be defined. This attribute is only used when
* stored-as is "string".
*
-->
<permissions-listing
stored-as="string"
string-object-class="eduPerson"
string-attribute="eduPersonEntitlement"
string-prefix="signet:flame:"
string-list-empty-value="" /><!-- Exactly 1 required -->
</signet>
<!--
*
* <source-subject-identifiers> is a required element identifying by
* Subject Source how a Subject's LDAP entry is located. It contains
* one or more of the following element.
*
* <source-subject-identifier> - This required element identifies
* for a Subject Source how to locate a Subject's LDAP entry.
* One of these elements is to be defined for each Source.
-->
<source-subject-identifiers><!-- exactly 1 required -->
<!--
*
* <source-subject-identifier> is a required element that identifies
* for a given Subject Source the Subject naming attribute to be used for
* identifying Subjects in the directory, and an LDAP search filter
* that will uniquely identify Subjects for the Source using the
* given naming attribute value. It requires two attributes that define
* the Subject Source and the Subject naming attribute.
*
* initial-cache-size - Optional attribute specifying the initial cache
* size to use for caching subject DNs by subject ID. Specifying a
* larger number than the number of subjects should give better
* performance.
*
* source - The name of the Subject Source (e.g., "g:gsa")
*
* subject-attribute - The name of the Subject naming attribute to
* be used (e.g., "account"). If a value other than "id" (the subject
* ID) is specified, the performance will suffer as an extra lookup
* will need to be performed. It is recommended that the subject ID
* be in the subject's directory object and that it be indexed.
*
* <source-subject-identifier> contains the following element.
*
* <ldap-search> - This required element identifies the LDAP search
* criteria for finding a Subject's entry.
-->
<source-subject-identifier source="flame-person" subject-attribute="employeeNumber" initial-cache-size="">
<!-- 1 or more required -->
<!--
*
* <ldap-search> is a required element containing a parameterized
* LDAP search filter that uniquely identifies a Subject from the
* Subject Source. It has three required attributes defined below.
*
* base - Identifies the DN of the context or object to search
* (e.g., cn=tom,dc=example,dc=edu).
*
* scope - Identifies the scope of the search. The valid values
* are, ignoring case, the names of the JNDI scope constants
* defined in the class javax.naming.directory.SearchContext.
* These are object_scope, onelevel_scope, and subtree_scope.
*
* filter - Parameterized search filter (e.g., (&(uid={0})(objectclass=ucperson)))
* The string "{0}" will be replaced by the value of Subject's
* attribute identified by subject-attribute in
* source-subject-identifier element.
-->
<ldap-search
base="ou=Flame Users,dc=lse,dc=ac,dc=uk"
scope="subtree_scope"
filter="(employeeNumber={0})" />
<!-- exactly 1 required -->
</source-subject-identifier>
</source-subject-identifiers>
<!--
* <ldap> is a required element containing the information necessary to
* connect with the directory being populated. It contains the following
* element.
*
* <context> - This require element defines the parameters necessary
* to establish a JNDI LDAP context.
*
-->
<ldap><!-- Exactly 1 required -->
<!--
* <context> is a required element holding the parameters necessary
* to establish a JNDI LDAP context. It contains the following element.
*
* <parameters-list> - This required element contains the list of
* parameters needed to establish a JNDI LDAP context.
*
* <paramters-list> contains one or more of the follow element.
*
* <parameter> - This required element holds a parameter name/value
* pair.
*
* The <parameter>'s necessary to establish a JNDI LDAP context are
* listed below along with a description of the expected value. Note
* the parameter names must be the same, ignoring case, as the
* equivalent JNDI constant defined in the class javax.naming.LdapContext
* (e.g., a parameter name of provider_url for javax.naming.LdapContext.PROVIDER_URL).
*
* initial_context_factory - Fully qualified class name of a context
* factory (e.g., com.sun.jndi.ldap.LdapCtxFactory).
*
* provider_url - URL for the LDAP provider
* (e.g., ldap://localhost:389/o=JNDITutorial).
*
* security_authentication - Value of "none", "simple", "strong", or a
* provider-specific string (e.g., simple).
*
* security_principal - Specifies the name of the user/program doing
* the authentication (e.g., "cn=S. User,ou=NewHires,o=JNDITutorial").
*
* security_credentials - Specifies the credentials of the user/program
* doing the authentication (e.g., mysecret).
-->
<context><!-- Exactly 1 required -->
<parameter-list><!-- 0 or 1 allowed -->
<!-- 1 or more <parameter> elements are required within <parameter-list> -->
<parameter name="initial_context_factory" value="com.sun.jndi.ldap.LdapCtxFactory" />
<parameter name="provider_url" value="ldap://localhost:389"; />
<parameter name="security_authentication" value="simple" />
<parameter name="security_principal" value="XXXX" />
<parameter name="security_credentials" value="XXXX" />
</parameter-list>
</context>
</ldap>
</ldappc>
- New ldappc snapshot fixes runner script problem, Kathryn Huxtable, 08/11/2008
- Re: [signet-dev] New ldappc snapshot fixes runner script problem, Graham Seaman, 08/11/2008
- Re: [grouper-dev] Re: [signet-dev] New ldappc snapshot fixes runner script problem, Kathryn Huxtable, 08/11/2008
- Re: [grouper-dev] Re: [signet-dev] New ldappc snapshot fixes runner script problem, Graham Seaman, 08/12/2008
- Re: [grouper-dev] Re: [signet-dev] New ldappc snapshot fixes runner script problem, Tom Barton, 08/12/2008
- Re: [grouper-dev] Re: [signet-dev] New ldappc snapshot fixes runner script problem, Graham Seaman, 08/12/2008
- Re: [grouper-dev] Re: [signet-dev] New ldappc snapshot fixes runner script problem, Tom Barton, 08/12/2008
- Re: [grouper-dev] Re: [signet-dev] New ldappc snapshot fixes runner script problem, Graham Seaman, 08/12/2008
- Re: [grouper-dev] Re: [signet-dev] New ldappc snapshot fixes runner script problem, Kathryn Huxtable, 08/11/2008
- Re: [signet-dev] New ldappc snapshot fixes runner script problem, Graham Seaman, 08/11/2008
Archive powered by MHonArc 2.6.16.