Shibboleth Developers

[Derek Atkins <>]

I've been researching a database backend for the SHAR and currently
I'm debating between using BerkeleyDB (BDB) and Embedded-MySQL
(MySQL).  Obviously any database is going to require storing the XML
data in string format, which would require parsing the XML to load it
from the database.  My hope is to implement a memory cache on top of
the database to try to limit the amount of XML parsing that needs to
happen.

The benefits of BDB are that it's small and it's mostly everywhere.
The downside is that all lookups are key:value "pairs".  While you can
create the concept of "rows" by having multiple values at a "key", you
cannot perform relational lookups based on these multiple columns.

BDB also allows the ability to have multiple "databases" in a single
file (in essence each "database" is a table).  This sort of gives you
a three-tuple of <database,key,value> in order to lookup data.

MySQL, on the other hand, is a real SQL-based database, and as of
version 4 provides an embedded-server library.  This means you can
link your applications against -lmysqld and you no longer need to
contact a database server.  This gives you all the benefits of a SQL
database including relational lookups and queries, as well as an
on-disk storage mechanism that hides the database from the user.

In addition to typed data (BDB has no internal data-typing), another
nice feature is that MySQL provides the ability to return only certain
columns in a query response.  BDB, on the other hand, has to return
the whole row.

On the call today I also claimed that a downside of BDB is that all
data is stored as byte-strings.  My argument was that writing
converters was a pain and took time.  Well, as I researched MySQL I
determined that while the data is typed inside the database, the
lookups all return strings, too, just like BDB.  This means that I
need the same string<->C-data converters in either case.

Currently I have a hierarchical (two-level) data model.  The primary
key is the session-id (the cookie).  That gives me the session handle
(the SAMLAuthenticationStatement) and other ancillary data (creation
time, last-used time, etc).  The second level is a keyed off of both
the session-id and resource, and maintains the SAMLResponse for the
given handle/resource tuple.

The three main goals are:
        1) look up a session based on the session-id
        2) look up an assertion-list based on session-id and resource
        3) easily delete a session based on session-id

My worry about using BDB is that it would require either one
"database" per session or one per resource in order to store the
second-level SAMLResponse data.  Unfortunately both approaches have
problems.  On one hand, dbopen events are considered heavy-weight so
one database per session isn't a good idea.  On the other hand, in
order to handle '3' it would require some way to iterate through each
of the resource databases, locking each one in turn in order to delete
a session.

So, I'm not sure which route to take, BDB or MySQL.  I think that
MySQL is probably a better route in terms of simplicity of coding,
simplicity of lookups, and longer-term issues (the ability to move to
a real database down the road).  The downside is that MySQL-4 is still
in "gamma" release and most certainly is not in common usage, let
alone in common distribution....

Comments?  Suggestions?

Thanks,

-derek
-- 
       Derek Atkins, SB '93 MIT EE, SM '95 MIT Media Laboratory
       Member, MIT Student Information Processing Board  (SIPB)
       URL: http://web.mit.edu/warlord/    PP-ASEL-IA     N1NWH
       

                        PGP key available

------------------------------------------------------mace-shib-design-+
For list utilities, archives, subscribe, unsubscribe, etc. please visit the
ListProc web interface at 

    http://archives.internet2.edu/

------------------------------------------------------mace-shib-design--