Shibboleth Developers

[Ian Young <>]

RL 'Bob' Morgan wrote:

> On Thu, 20 Oct 2005, Ian Young wrote:
>
> > Just as a reminder, I have some code for the IdP that handles this 
> > particular case (IdP and some set of SPs behind a NAT) by rewriting 
> > the client's IP before sending it to an SP that is outside the NAT.
> >
> > There was a bit of a deafening silence around this last time round, 
> > but if people are interested, I could try reintegrating that with CVS 
> > HEAD again.
>
> Sounds like it could be useful.  I'd vote for having it be a feature, 
> modulo any extra complexity it might cause for other parts of the IdP or 
> its configuration.  I think we might use it here rather than try to make 
> all of our external SPs turn off checking.

Sorry to take so long to get back to this, but I've been kind of submerged.

I originally did a patch against the 1.2 IdP, but configuring it was a 
matter of editing the Java and recompiling.  No good for anyone other 
than me, in other words.

To make the code more generally useful, I split the problem into two 
parts, the first of which was a general mechanism for configuring 
plugins that fall outside the standard set, by introducing an Extensions 
element into the idp.xml schema closely modelled on the one provided by 
the C++ SP configuration scheme.  There was a bit of back-and-forth on 
this, but I think the timing was bad (1.3 crunch was looming) and in the 
end discussion petered out without the code being accepted.

For reference and more background, I'm attaching a copy of my last 
submission on this subject which includes the patches (although they 
probably won't work against CVS HEAD now given that its from July). 
Eyeballing them should give you a rough answer to your question about 
the complexity issues in the rest of the IdP.

I'd be happy to take another run at this if there was any prospect of it 
being of general value.

The second part (the NAT mapping thing based on that configuration 
scheme) is pretty straightforward once the configuration stuff is in 
place.  I didn't finish building that patch against 1.3 since the 
configuration changes weren't accepted, but essentially it just 
intercedes where the IdP is passing the client IP to the SP by passing 
it through the configured IP address mapper if there is one.  The basic 
mapper I built knows about the standard NATed ranges, and you tell it 
which entities are inside the NAT range and which are outside.  So, it 
can correctly handle the cases where you have some SPs inside the NAT 
(which you want to pass original client IPs to) and some outside (which 
need to be mapped to your global address).  Obviously you could extend 
this to work with IPv6 but I should stress that I didn't get that far, 
the problem I was solving was strictly IPv4.

        -- Ian

--- Begin Message ---

• From: Ian Young <>
• To:
• Cc: Scott Cantor <>, Chad La Joie <>
• Subject: proposed generic plugin mechanism for IdP
• Date: Mon, 11 Jul 2005 16:30:44 +0100
• List-archive: <https://mail.internet2.edu/wws/arc/shibboleth-dev>
• List-id: <shibboleth-dev.internet2.edu>

Here is a second cut at my generic plugin scheme for the IdP. I've moved it out into a separate thread to try and avoid cluttering Chad's original custom extensions thread.

This version takes Scott's comment into account by putting the <Plugin> element inside an <Extensions> element at the front of the configuration, just like the scheme used in the C++ SP and in SAML 2.0. I have also changed one of the attribute constraints in the schema from ##any to ##other to match the C++ SP's schema; this makes validation better but makes the example a little more complicated.

The other change I made was to move the plugin loading as far forward as I could, so that the plugins are available earlier. This is unlikely to make a big difference in practice, but I don't see it doing any harm.

Again, this is presented as three patches against the current CVS head.

patch1.txt alters the IdP configuration file schema to allow an <Extensions> element at the start, which can contain any number of <Plugin> elements followed by anything at all (for future expansion). Each <Plugin> must have name (random string you can use to refer to the plugin in code) and implementation (class name) attributes, but can have any other attributes *as long as they come from a different namespace* and any content model; it's up to the plugin to decode those.

patch2.txt is applied to the idp directory. Most of the workings are in PluginCollection.java, which is mainly a knock-off of ProtocolHandlerFactory.java. There is some bridging code in IdPProtocolSupport.java and some code to pick up the Plugin elements in IdPResponder.java.

A plugin class has to have a null constructor. If it also implements the (pre-existing) PluggableConfigurationComponent interface, the instance is passed the <Plugin> element to configure itself with.

I've included an example plugin as TestPlugin.java, and there is a second hunk of changes in IdPResponder which demonstrates accessing a plugin from code. Neither of these are really part of what would want to be checked in; they are just here as targets for discussion.

Finally, patch3.txt is applied to the idp.xml and shows what the configuration code looks like. In this version, note the required namespace declaration for the "foo" attribute; this is the effect of the ##other constraint.

Again, any comments are welcome. Absent that, this is probably as far as I can take this; it's good enough for me to use for my own purposes but ideally the next step would be for someone on the core team to adopt it (minus the example code, of course; I can provide a clean patch minus the example code if anyone wants it). After all, this kind of framework stuff really makes most sense if everyone already has it.

If it doesn't look like that will happen, I'll try and keep a clean and working set of patches available from the SDSS project site.

-- Ian
Index: dist.idp.xml
===================================================================
RCS file: /home/cvs/shibboleth/shibboleth/java/src/conf/dist.idp.xml,v
retrieving revision 1.9
diff -u -r1.9 dist.idp.xml
--- dist.idp.xml 8 Jul 2005 21:03:17 -0000 1.9
+++ dist.idp.xml 11 Jul 2005 12:56:45 -0000
@@ -13,6 +13,16 @@
defaultRelyingParty="urn:mace:shibboleth:examples"
providerId="https://idp.example.org/shibboleth";>

+ <!-- custom configuration -->
+ <Extensions>
+ <!-- example generic plugins -->
+ <Plugin name="my.plugin"
+
implementation="edu.internet2.middleware.shibboleth.TestPlugin"
+ xmlns:foo="http://iay.org.uk/foo#";
foo:foo="bar">
+ <SomeConfiguration>text</SomeConfiguration>
+ </Plugin>
+ <Plugin name="boring.plugin" implementation="java.util.Date"/>
+ </Extensions>

<!-- This section contains configuration options that apply only to a
site or group of sites
This would normally be adjusted when a new federation or
bilateral trust relationship is established -->
Index: IdPProtocolSupport.java
===================================================================
RCS file:
/home/cvs/shibboleth/shibboleth/java/src/edu/internet2/middleware/shibboleth/idp/IdPProtocolSupport.java,v
retrieving revision 1.16
diff -u -r1.16 IdPProtocolSupport.java
--- IdPProtocolSupport.java 20 Jun 2005 20:22:48 -0000 1.16
+++ IdPProtocolSupport.java 11 Jul 2005 12:56:27 -0000
@@ -71,9 +71,11 @@
private ArtifactMapper artifactMapper;
private Semaphore throttle;
private Trust trust = new ShibbolethTrust();
+ private final PluginCollection plugins;

IdPProtocolSupport(IdPConfig config, Logger transactionLog,
NameMapper nameMapper, ServiceProviderMapper spMapper,
- ArpEngine arpEngine, AttributeResolver resolver,
ArtifactMapper artifactMapper)
+ ArpEngine arpEngine, AttributeResolver resolver,
ArtifactMapper artifactMapper,
+ PluginCollection plugins)
throws ShibbolethConfigurationException {

this.transactionLog = transactionLog;
@@ -84,6 +86,7 @@
this.arpEngine = arpEngine;
this.resolver = resolver;
this.artifactMapper = artifactMapper;
+ this.plugins = plugins;

// Load a semaphore that throttles how many requests the IdP
will handle at once
throttle = new Semaphore(config.getMaxThreads());
@@ -179,6 +182,10 @@
}
}

+ public Object getPlugin(String name) {
+ return plugins.get(name);
+ }
+
public int providerCount() {

return metadata.size();
Index: IdPResponder.java
===================================================================
RCS file:
/home/cvs/shibboleth/shibboleth/java/src/edu/internet2/middleware/shibboleth/idp/IdPResponder.java,v
retrieving revision 1.38
diff -u -r1.38 IdPResponder.java
--- IdPResponder.java 8 Jul 2005 17:41:58 -0000 1.38
+++ IdPResponder.java 11 Jul 2005 12:56:29 -0000
@@ -76,6 +76,7 @@

private IdPConfig configuration;
private HashMap protocolHandlers = new HashMap();
+ private PluginCollection plugins = new PluginCollection();
private IdPProtocolSupport protocolSupport;

/*
@@ -113,6 +114,23 @@
// Load global configuration properties
configuration = new
IdPConfig(idPConfig.getDocumentElement());

+ // Load extensions
+ NodeList extElements =
idPConfig.getDocumentElement().getElementsByTagNameNS(IdPConfig.configNameSpace,
+ "Extensions");
+ if (extElements.getLength() > 1) {
+ log.warn("Encountered multiple <Extensions/>
configuration elements. Using first...");
+ }
+ if (extElements.getLength() > 0) {
+ Element extensions = (Element)
extElements.item(0);
+
+ // Load generic plugins
+ itemElements =
extensions.getElementsByTagNameNS(IdPConfig.configNameSpace,
+ "Plugin");
+ for (int i = 0; i < itemElements.getLength();
i++) {
+ plugins.add((Element)
itemElements.item(i));
+ }
+ }
+
// Load name mappings
NameMapper nameMapper = new NameMapper();
itemElements =
idPConfig.getDocumentElement().getElementsByTagNameNS(
@@ -191,7 +209,7 @@

// Load protocol handlers and support library
protocolSupport = new
IdPProtocolSupport(configuration, transactionLog, nameMapper, spMapper,
arpEngine,
- resolver, artifactMapper);
+ resolver, artifactMapper, plugins);
itemElements =
idPConfig.getDocumentElement().getElementsByTagNameNS(IdPConfig.configNameSpace,
"ProtocolHandler");

@@ -252,6 +270,23 @@
MDC.put("remoteAddr", request.getRemoteAddr());
log.debug("Received a request via GET for location (" +
request.getRequestURL() + ").");

+ /*
+ * Example code using plugins... not part of the
implementation.
+ */
+ Object plugged = protocolSupport.getPlugin("my.plugin");
+ if (plugged instanceof TestPlugin) {
+ ((TestPlugin)plugged).sayHowdy();
+ }
+ plugged = protocolSupport.getPlugin("boring.plugin");
+ if (plugged != null)
+ log.info("boring plugin says: " + plugged);
+ plugged = protocolSupport.getPlugin("missing.plugin");
+ if (plugged != null) {
+ log.info("missing plugin says: " + plugged);
+ } else {
+ log.info("missing plugin is missing");
+ }
+
try {
IdPProtocolHandler activeHandler =
lookupProtocolHandler(request);
// Pass request to the appropriate handler
Index: PluginCollection.java
===================================================================
RCS file: PluginCollection.java
diff -N PluginCollection.java
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ PluginCollection.java 1 Jan 1970 00:00:00 -0000
@@ -0,0 +1,105 @@
+/*
+ * Copyright [2005] [University Corporation for Advanced Internet
Development, Inc.]
+ *
+ * Licensed under the Apache License, Version 2.0 (the "License");
+ * you may not use this file except in compliance with the License.
+ * You may obtain a copy of the License at
+ *
+ * http://www.apache.org/licenses/LICENSE-2.0
+ *
+ * Unless required by applicable law or agreed to in writing, software
+ * distributed under the License is distributed on an "AS IS" BASIS,
+ * WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.
+ * See the License for the specific language governing permissions and
+ * limitations under the License.
+ */
+
+package edu.internet2.middleware.shibboleth.idp;
+
+import java.lang.reflect.Constructor;
+import java.lang.reflect.InvocationTargetException;
+import java.util.HashMap;
+import java.util.Map;
+
+import org.apache.log4j.Logger;
+import org.w3c.dom.Element;
+
+import
edu.internet2.middleware.shibboleth.common.PluggableConfigurationComponent;
+import
edu.internet2.middleware.shibboleth.common.ShibbolethConfigurationException;
+
+/**
+ * Collection class for loading generic plugin implementations based on XML
configuration.
+ *
+ * @author Ian Young, after Walter Hoehn
+ */
+public class PluginCollection {
+
+ private static Logger log =
Logger.getLogger(PluginCollection.class.getName());
+ private Map/*<String, Object>*/ plugins = new HashMap();
+
+ public Object get(String name) {
+ return plugins.get(name);
+ }
+
+ public void add(Element config) throws
ShibbolethConfigurationException {
+ String name = config.getAttribute("name");
+ if (name == null || name.equals("")) {
+ log.error("No plugin name specified. Attribute
(name) is "
+ + "required with element <Plugin/>.");
+ throw new ShibbolethConfigurationException("Invalid
configuration data supplied.");
+ }
+
+ String implementation = config.getAttribute("implementation");
+ if (implementation == null || implementation.equals("")) {
+ log.error("No plugin implementation specified.
Attribute (implementation) is "
+ + "required with element <Plugin/>.");
+ throw new ShibbolethConfigurationException("Invalid
configuration data supplied.");
+
+ } else {
+
+ log.info("Loading plugin \"" + name + "\"
implementation=" + implementation);
+ try {
+ Class implClass =
Class.forName(implementation);
+ Constructor constructor =
implClass.getConstructor(new Class[]{});
+ Object rawImpl = constructor.newInstance(new
Object[]{});
+
+ /*
+ * If the object is initializable, pass it
the element.
+ * If not, don't bother.
+ */
+ if (rawImpl instanceof
PluggableConfigurationComponent) {
+
((PluggableConfigurationComponent)rawImpl).initialize(config);
+ }
+
+ /*
+ * Register this plugin.
+ */
+ plugins.put(name, rawImpl);
+
+ } catch (ClassNotFoundException e) {
+ log.error("Invalid configuration, supplied
implementation class for the plugin "
+ + "could not be found: " +
e.getMessage());
+ throw new
ShibbolethConfigurationException("Invalid configuration data supplied.");
+
+ } catch (NoSuchMethodException e) {
+ log.error("Invalid configuration, supplied
implementation class for the plugin is "
+ + "not valid. A DOM Element
constructor is required: " + e.getMessage());
+ throw new
ShibbolethConfigurationException("Invalid configuration data supplied.");
+
+ } catch (InvocationTargetException e) {
+ Throwable cause = e.getCause();
+ if (cause != null) {
+ log.error(cause.getMessage());
+ }
+ log.error("Invalid configuration, supplied
implementation class for the plugin"
+ + " could not be loaded: " +
e.getMessage());
+ throw new
ShibbolethConfigurationException("Invalid configuration data supplied.");
+ } catch (Exception e) {
+ log.error("Invalid configuration, supplied
implementation class for the plugin"
+ + " could not be loaded: " +
e.getMessage());
+ throw new
ShibbolethConfigurationException("Invalid configuration data supplied.");
+ }
+ }
+ }
+
+}
Index: TestPlugin.java
===================================================================
RCS file: TestPlugin.java
diff -N TestPlugin.java
--- /dev/null 1 Jan 1970 00:00:00 -0000
+++ TestPlugin.java 1 Jan 1970 00:00:00 -0000
@@ -0,0 +1,22 @@
+package edu.internet2.middleware.shibboleth.idp;
+
+import org.apache.log4j.Logger;
+import org.w3c.dom.Element;
+
+import
edu.internet2.middleware.shibboleth.common.PluggableConfigurationComponent;
+
+public class TestPlugin implements PluggableConfigurationComponent {
+
+ private static Logger log =
Logger.getLogger(TestPlugin.class.getName());
+ private String foo = "(unset)";
+
+ public void initialize(Element config) {
+ foo = config.getAttributeNS("http://iay.org.uk/foo#";, "foo");
+ if (foo == null || foo.equals("")) foo = "(missing)";
+ log.info("TestPlugin initialized, foo='" + foo + "'");
+ }
+
+ public void sayHowdy() {
+ log.info("plugin says howdy: '" + foo + "'");
+ }
+}
Index: shibboleth-idpconfig-1.0.xsd
===================================================================
RCS file:
/home/cvs/shibboleth/shibboleth/java/src/schemas/shibboleth-idpconfig-1.0.xsd,v
retrieving revision 1.18
diff -u -r1.18 shibboleth-idpconfig-1.0.xsd
--- shibboleth-idpconfig-1.0.xsd 1 Jul 2005 21:12:39 -0000 1.18
+++ shibboleth-idpconfig-1.0.xsd 11 Jul 2005 12:56:13 -0000
@@ -38,6 +38,26 @@
<xs:complexType>
<xs:sequence>
<xs:sequence>
+ <xs:element name="Extensions"
minOccurs="0" maxOccurs="1">
+ <xs:annotation>
+
<xs:documentation>Container for extension libraries and custom
configuration</xs:documentation>
+ </xs:annotation>
+ <xs:complexType>
+ <xs:sequence>
+
<xs:element name="Plugin" minOccurs="0" maxOccurs="unbounded">
+
<xs:complexType>
+
<xs:sequence>
+
<xs:any namespace="##any" processContents="lax"
minOccurs="0" maxOccurs="unbounded"/>
+
</xs:sequence>
+
<xs:attribute name="implementation" type="xs:string" use="required"
/>
+
<xs:attribute name="name" type="xs:string" use="required" />
+
<xs:anyAttribute namespace="##other" processContents="lax"/>
+
</xs:complexType>
+
</xs:element>
+
<xs:any namespace="##other" processContents="lax" minOccurs="0"
maxOccurs="unbounded"/>
+ </xs:sequence>
+ </xs:complexType>
+ </xs:element>
<xs:element
name="RelyingParty" maxOccurs="unbounded">
<xs:complexType>
<xs:sequence
minOccurs="0" maxOccurs="unbounded">

--- End Message ---