perfsonar development work

[Jochen Reinwand <>]

Hi Michalis,

Sorry for the long delay... Doing mail from at home was a little bit tricky 
to 
set up. I'm still at home, not really working, "enforced vacation"...

Find my comments inline.

On Wednesday 27 December 2006 07:27, Michael Michalis wrote:
> You have done a very thorough investigating work, and the information you
> have provided us is very useful. I haven't really tested soapUI but the
> ability to create requests based on a given WSWDL description is great and
> could save a lot of time. Also the schema validating ability is great,
> although I thing it can also be done by using a DTD specification in JDOM(I
> didn't have the time to check it out yet :) ).

Wouldn't it be a good idea to directly test the Relax-NG schemas we already 
have? It should be possible to use the classes of trang and/or jing, or just 
use jing itself. Just an idea haven't looked at it yet...

> I have some remarks though.

That's exactly why I sent my mail. To get feedback ;-)

> Firstly when you are functional testing a service you may need to provide
> requests that are not well formatted or not NMWG schema compliant or just
> missing some attributes or elements that are not important and the service
> can live without them. I'm not sure how these tools can provide the former
> kinds of requests.

Both tools allow free formed requests. You can send any defective and useless 
message you like. soapUI is just providing templates based on the WSDL 
schema. Always remember: It's virtually impossible to create a useful XML 
document from a schema automatically. What you normally can create is some 
sort of "fillout template". That's exactly what soapUI has really good 
support for.

> Secondly and most importantly creating, sending and 
> validating the requests to the service is not considered to be the hardest
> job. The most important and tricky job of functional testing is to check
> the data and the result code of the responses, against the actual data
> stored inside the service database. From what you have written these tools
> don't provide such functionality and thus scripts or classes doing this
> kind of work still need to be created.

Well, that's really the highest level of functional testing theoretically 
possible. But often you won't be able to do these kind of tests.
For most MPs for example you cannot predict what the result of the actual 
measurement will be. You just can say something like "it has to be a positive 
number not greater than xyz". These kind of tests are indeed possible using 
soapUI and also anteater.

> Last but not least functional 
> testing may need to be automated, this means that functional testing should
> be carried out without the presence of a human being, maybe running at late
> hours with the results being stored in a text file. I'm pretty sure that
> anteater can be automated but you mentioned that soapUI uses a GUI, meaning
> perhaps that it can not run without a human surveillance.

It can! You use the GUI to set up the tests you want to run. Afterwards you 
can run the tests automated via CLI. I haven't tested this feature, but it 
should work as expected.

> Concluding my 
> long remarks;), I have to say that these tools can really simplify the
> creation of requests and the schema validation of the responses but IMHO
> they can not do all that is needed for functional testing. Now because we
> seem to be on a tight schedule and functional testing is an extra work load
> for you, maybe using these tools in conjunction with some scripts or
> classes or even only these tools if time is not enough, may be the best
> solution.

This is exactly what I think. Time is running out and soapUI is very helpful 
in speeding up things since it is possible to use a GUI to "click" the tests 
together. Using XPath for testing is quite effective and soapUI really makes 
creating these tests very easy. You just send one request and afterwards 
create the tests based on the response you got.

Hopefully I will have enough time to finish my testing of soapUI and start 
working on creating tests. These meetings like the one in Cambridge next week 
really hinder you from doing the "real" work...

> Merry Christmas and a happy new year to you too Jochen and everyone else
> reading this mail.

Very best wishes for the new year to you and everyone else reading this!

greetings,
Jochen

-- 
Jochen Reinwand                                 Tel: +49 9131 852-8689
DFN Labor

Regionales Rechenzentrum Erlangen
Martensstrasse 1
91058 Erlangen
 
email: