Shibboleth Developers

["Scott Cantor" <>]

> The way the datastore works is like a dictionary. You get values by
> supplying a key and set values with a key. Of course, you can set the
> same key to a new value, which alters it.

Yeah, but it's all the race conditions that are a concern. His FAQ suggests
that he doesn't understand the issue at all. Calling ACID databases
"overhead" also sends up red flags with me. Data integrity isn't just a nice
idea to be ignored when you need performance. You have to have both.
 
To have a writeable cache, you MUST prevent dirty writes (that is, nodes
writing back changes that conflict with each other). This isn't optional.
That's why I had to add versioning to my API. It doesn't take much work, but
the writer has to hold the version and pass it with the update so that the
data store can detect when the write is out of sync.

Now, I don't really need this feature yet because if all that's updated is
the last-access stamp, a race doesn't matter. But it matters a lot once you
start doing n-tier work and have to refresh SAML tokens and cache new ones.
It also is needed to do post-session-setup attribute resolution.

> Expiration times

Thanks, I didn't read this closely when I glanced at it. It supports the
semantic I use, which is for setting the actual expiration time.

> Given this new info, can you already tell if it can be done?

I think the race conditions would be a problem. Without any locking, you
can't be certain that you're not overwriting new data. The normal approach
would be to lock, select version, compare, then update. With no lock, you
can't protect the select/compare step.

I would say you've got two options:

- implement a plugin that doesn't support write integrity and always assumes
the data is in sync, which is probably good enough for most deployers

- implement versioning by embedding the version inside the key value

You could do things like "add" the new version and then if you conflicted,
you'd detect it, fetch the updated version, bump the version and try again,
etc.

> I too don't have much time but I would like to do this, if it is viable.
> Also, if there some example plugin already coded somewhere?

There's an in-memory plugin in the cpp-xmltooling code that does single-node
caching. The ODBC plugin in the cpp-sp repo is started but hasn't been
updated and completed yet.

(This is all 2.0 I'm referring to, I wouldn't even consider touching 1.3,
the API is horrible.)

-- Scott