perfsonar development work

[Jason Zurawski <>]

Luchesar;

One last one before I sign off for the night...

> And, finally, Jason (hmm, am I trying to excel you in replying in 
> single swipe? ;D ), you made a very good point, indeed, about leaving 
> space for further extension, and keeping the solution flexible, as I 
> indicated in my previous mail, and accordingly updated my proposal.


I have not seen your updated proposal, is this on the wiki?  Do you have 
a link?


> Now, about the "COMPONENT: Description" thing, I'd rather leave that 
> for after having some sleep too, but I'll still make one important 
> note before that. Don't forget that in the core of the perfSONAR 
> service monitoring will be the syslog messaging. In the end, there 
> should be no difference if an error has been encountered in processing 
> of regular user request, or during a self-test. Whatever goes wrong, 
> it must be clearly indicated and well described in a message sent to 
> syslog. So, getting to the point of it, there wouldn't be anything 
> wrong in a "COMPONENT: Description" content of the datum's if that's 
> what syslog collects too -- quite on the contrary, that is the goal.


I understand your reasoning: the designed clients will simply grab the 
data and stick it into logs.  Because these are the thoughts on how the 
data will be consumed I get your approach completely.  Unfortunately 
features like this have a tendency to stick around and evolve beyond 
original intent which is why I would personally favor structure (in the 
event that some other client down the road isn't simply slicing the data 
and trying to find deeper meaning).  Even with additional XML I 
proposed, wouldn't it still be rather easy to craft some type of 
python/perl script to consume the response and insert it into syslog as 
designed?


> > > Good point. However let's start with something easier, so how about 
> > > a generic test as a beginning?
> > >
> > > http://schemas.perfsonar.net/tools/admin/selftest/generic/full/1.0
> >
> > See my comment to VV for an in depth reasoning.  I am not opposed to 
> > 'all in one' as long as it is structured appropriately.
>
> Indeed, you made very good point, and that's why I changed the 
> proposal accordingly -- now you can easily add new self-tests in lieu 
> of generic.
>
> > This may be true, but it is far from clean or ideal.  We use the 
> > structured messages for a reason: there is a natural mechanism to 
> > find what we are really interested in by walking the XML.  Doing a 
> > regex across several datums to verify the results of the test offers 
> > a quick and correct solution to this problem, but we do have the 
> > tools at our disposal to avoid shoving all possible results into a 
> > single text bloated element.
>
> I totally agree in general. But I'm more a UNIX administrator than a 
> web service developer (the latest not at all, actually :D), and that's 
> perhaps why I'm more in peace with those kind of solutions.
>
> But what's really important here: I wouldn't even think to argue with 
> you, if we were talking about the "real" service operation. But we are 
> talking about monitoring, and that's _completely_ different thing. The 
> monitoring must be robust, and robust most always means simple: and 
> the simpler -- the better.
>
> I'd guess that you're not feeling nice with the currently proposed 
> approach, because it really does sound like a bad thing for a web 
> service to do, and I totally agree. But really, this is only a 
> supporting task, and it need not adhere to the "web service good 
> practices", let's call it this way. I'd rather totally pull it out of 
> the web-services themselves, but that's the way we've taken already.


Are we talking about just monitoring though?  This is a request and a 
response written using perfSONAR primitives making it just as much a 
part of perfSONAR as a measurement request.  I would agree this was 
simply monitoring if it was out of band, but if it interfaces into the 
service it should follow the same rules as everything else (e.g. this is 
required to follow "best practices" as you put it because its an exposed 
interface). 

Simple is also a fine goal, I don't think anything I submitted 
complicates matters to an insane degree and on the contrary makes it 
very easy for someone else (preferably not me) to define as many self 
tests as they desire.


> Finally, to just give you an example, you know that in critical 
> systems like aeroplanes, spaceships, etc., the fine-precision, 
> high-tech, complex systems are often backed up by simple, crude, and 
> straightforward ugly mechanical ones. Like that plain old magnetic 
> compass in the cockpit of an ultra-modern supersonic fighter plane. 
> It'll most likely work long after all complex MFDs and gyroscopes have 
> surrendered, and that's what I'd like to see in a monitoring system too.


A great story, but you will agree that the old magnetic compass must 
have been constructed properly using the appropriate materials to last 
as long as it has.  That is all I am asking for in this design :)

-jason