perfsonar development work

[Maciej Glowiak <>]

Hi Jason,

Jason Zurawski wrote:
> > I am still curious what BDB improves? Well, I am quite sure it may be 
> > faster than eXist, but
> >   - were there made any performance tests and comparisons between eXist
> >     and BDB? I am sure you did something like this, but I didn't see any
> >     results and conclusions
> We are working on simple tests (not completed yet) that compare eXist, 
> BDB, and oracle XML (this is a part of another students work, but once 
> we get results we will be more than happy to share).  We will be doing 
> performance tests once the two instances are set up, and accepting 
> registrations.

That's ok. Before we chose eXist I had installed and tested a bit BDB. I 
liked it, but I remember we had problems with FreeBSD (Roman tested it).
Then XUpdate seemed to be the standard for modification very soon. Now 
it seems not to be modified for a very long time.

But I didn't tested performance of BDB at all, so I am just curious.
So, after you test it, please send me the results.

> >   - how far are both DB compatible? I thought BDB didn't support
> >     XUpdate, so single LS functionality wouldn't work with BDB unless we
> >     changed all database modification instructions. But for what? XQuery
> >     Update extension?
>
> BDB does not use XUpdate (one reason why I was interested in removing 
> the XUpdate functionality several weeks ago) and instead uses it's own 
> update API.  The XQuery/XPath statements are more or less the same, with 
> some notable changes regarding the organization of BDB's collections.  
> Yi did develop a seperate service engine to try and insert some 
> transparency into the calls to the various databases.

I was thinking about some transparency as well. But the point was that 
if we want to use BDB for LS we should implement these changes for 
single-LS as well.

> >   - but does BDB support XQuery Update extension?
> To my knowledge no, it uses a separate update API (there were always 
> promises or supporting XQuery update once it was finalized, this goes 
> back years however).

So that could be a problem.

> >   - is XQuery processed the same by eXist and BDB?
> Yes, they both should follow the standard.

But you also said that queries need to be a bit different because of 
differences in storage. It may be a problem for client which may not 
know what DB is used by particular LS.

> > I think LS (sLS+mLS) should be regarding as one LS, but I'd like to 
> > have code separation between multiLS and core LS. In fact multiLS 
> > should be independent protocol from LS. And in fact it is.
>
> If they are to be regarded as 'one' (i.e. would we still even want to 
> deploy single LS instances?) then I don't see why the need in separating 
> the code.

I think there is strong need for such a separation. One aim is to 
maintain the code. After merging sLS with mLS changes in one (mostly 
mLS) shouldn't interfere with single LS. Single LS should be operational 
even if mLS fails or is not stable.

Single LS is being functional tested now. Although we have been having 
"stable" LS for quite a long time, we found a few bugs. When will mLS be 
tested? It may be more difficult to be tested because you need to set up 
several instances of it. It means we will have "more stable" single LS 
and "unknown" mLS in the release.

> > > > Other my concern was on sending request and waiting for the response 
> > > > in request processing action (as far as I remember it was sending a 
> > > > request by LSRegisterRequest action and waiting for response).
> > >
> > > I was using timeouts to speed up the process of waiting for 
> > > responses.  We never really get a 'hang' situation that way, but 
> > > depending on latency and other network issues this could be a problem.
> >
> > You probably tested it in lab. In real 'bigger' network we may 
> > experience strange behaviour... :)
>
> I am sure there will be, but in my opinion all we can really do is go 
> for a best effort attempt, and handle the failures as best we can.

Yes, I agree. But we should also evaluate possible failure points and 
implement them in different way. After you finish your work on mLS I'll 
see if we can improve something with threading there

Best regards

Maciej

--

 --------------------------------------------------------------------
| Maciej Glowiak        Network Research and Development            ||
| 

     Poznan Supercomputing and Networking Center ||
| (+48 61) 858 2024     http://monstera.man.poznan.pl/              ||
 ====================================================================