Shibboleth Developers

["Scott Cantor" <>]

> Well, that's why I was asking about the work Howard and Will are
> doing.  What is it exactly that's being tested anyway?

They're unit testing components by relying (I would assume) on bypassing the
usual kinds of external pieces like authentication. They're wiring the raw
pieces together around mocked up HTTP wrappers that feed in the necessary
information, such as REMOTE_USER.

You can't do that outside of your sandbox and accomplish functional testing
of a deployed IdP. You could test the raw classes in the same way, but we
know the classes work, it's the actual deployed config you're trying to
test.

> So let's take SSO out of the picture.  We have a simple tool that
> makes an attribute query, but of course the AA can't authenticate the
> client so it falls back on defaultRelyingParty.  We can work around
> this in development but it is of little use in general, I'm afraid.

Of course. And you can't possibly know what my AA would or wouldn't be able
ot authenticate short of putting in a back-door, as I said. I as a deployer
would know, but if you expect me to be able to configure your test with a
key and certificate that would work, I suspect we're long past the set of
users that need your validation process anyway.

This is, as I said a long time ago, the difference between the result of the
initial default installation of Shibboleth (which is useless except as a
demo) and a real deployment (which takes much effort, local knowledge, etc.)
and can't be predicted or automated.

> Someone else in the group is working on a similar tool that does
> client authn, and so users will be able to test GridShib with this
> tool since a GridShib install injects a sample name mapping file into
> the config, which we can test against.  We do assume the principal
> name in the map file is a real principal name, so this is somewhat of
> a limitation, but in general we're in pretty good shape for
> post-install tests.

You would need to know a user that is in my domain. I could tell you that at
install time, I guess. Is that what you have in mind? That should work in
terms of the XML, as you say, since you're injecting a NameMapper that the
AA can use to decode a known subject. But you still can't run it, without
knowing how to authenticate as a trusted requester against my IdP. And there
is no way to know that.

> I don't know how to do pre-install testing either.  Seems like
> something is needed though, otherwise there's gonna be a lot of
> stepping on each other's toes.

We can insert instrumentation for tools to use to establish that an IdP is
functioning properly, which we need to do, but they wouldn't prove anything
specific to GridShib's requirements, I suspect.

Supporting this stuff is hard. I think we're trying to automate things
instead of educating people, and that's doomed to fail. Maybe both are, but
I don't think we can skip the education step.

-- Scott