Shibboleth Developers

[Will Norris <>]

(doh, hate it when you accidentally fat-finger the "send" keyboard  
shortcut)


On Nov 16, 2005, at 7:11 PM, Chad La Joie wrote:
> So, what I'd like to see, is a new thread started, here on the dev  
> list, that proposes a testing plan in a clear and concise manner.   
> Once we have a plan then we can figure out what needs to be done.

So maybe I'm not the most qualified person to start this, but here  
goes anyway.  (Apologies if I end up stating stuff that is painfully  
obvious to most.)

I think most everyone agrees on what the unit tests cover so there's  
no sense in belaboring that too much; the confusion seems to be with  
the integration tests.  The way I see it, we're wanting to test  
groups of components that *function together* to make sure they play  
nice together.  We've actually already documented where these  
integration tests are needed, whether or not we really thought about  
it this way (http://shibboleth.internet2.edu/docs/internet2-mace- 
shibboleth-arch-protocols-latest.pdf -- line 250).  Each transaction  
is a *potential* place for an integration test, where a "transaction"  
refers to some data being passed into a Shibboleth component and some  
other data coming out.  Referencing the steps in that diagram:

1 -> 2
Test SP ability to construct proper redirect URL for the initial request

2 -> 3
Test WAYF ability to redirect to proper IdP

3 -> (4) -> 5
Test IdP ability to receive SSO request and generate proper SAML  
response

5 -> 6
Test SP ability to receive SAML response and generate proper  
Attribute Query

6 -> 7
Test IdP ability to receive Attribute Query and generate proper SAML  
assertion

7 -> 8
Test SP ability to make proper authZ decision based on SAML assertion(s)


The two dealing directly with the IdP (3->5 && 6->7) are actually  
exactly what I checked in last month.  I don't see a need to write  
tests which cover multiple of these steps since it can be reasonably  
assumed that if I plug some data into test "5->6", and then take the  
result of that and plug it into test "6->7", the end result would be  
the same if I plugged the initial data into a test "5->6->7".  It's  
your basic mathematical replacement property -- "if (a == b && b ==  
c) then (a == c)".  Not only that, but it because more difficult to  
debug if there is an error because you've thrown so many additional  
variables into the mix.  If a test needs some additional component  
that is outside the scope of that particular test, then create a mock  
object to simulate the behavior of that additional component.

--
Will Norris
Information Technology
The University of Memphis

Attachment: smime.p7s
Description: S/MIME cryptographic signature