perfsonar development work

[Loukik Kudarimoti <>]

This is a very good analysis of the SOAP UI tool. Very good job Michalis 
and Jochen!

Keeping in mind the meager manpower that we have to do such testing, it 
is important for us to use tools which reduce the amount of effort 
required for testing and also to use the same tool across all testing 
teams so that they can be re-used later on.

I have a couple of questions:
* If for every service we had a well specified wsdl and a more extensive 
rnc which can be converted to xsd which soap ui is happy with (we need 
to investigate what is lacking currently), does the SOAP UI tool then 
considerably reduce the effort required to test a service (compared to 
our current practice?)
* I am not sure of what level of automation can be achieved. I 
understand that requests and responses need to be either matched using 
xquery or compared against data values using scripts. If this is the 
case, there is preparatory work required to be done for each request. On 
an average how much time does it take to write up a request with SOAP UI?

Here is what I can think of as the next steps

* We need to investigate how to write up wsdls for our service. I 
believe writing up a wsdl is easy for a rpc/encoded style of web 
services but can be a bit tricky for document/literal.
* We need to investigate why the rncs that we have written are 
insufficient and what can be improved
* We need to investigate why the xsds generated from the rncs are not 
accepted by the soap UI
* Investigate if listing out all the error codes returned by the service 
will be helpful in testing

Lastly, we do not intend to discard the current scripts for services 
tested in 2.0 instead we can reuse them for newer versions in future.

thanks,
Loukik


Michael Michalis wrote:
> Hi all,
>
> It took me longer then I thought but finally I got a good picture 
> about soapUI.
>
> First off all I’ve have to say that soapUI is really hard to get into 
> J . The user guide was inadequate for the use of soapUI that I 
> indented and it wasn’t very clear about some things. But thanks to 
> Jochen and some help from the soapUI mailing list(the people there 
> were very helpful) I’ve managed.
>
> I didn’t produce a full range of tests for a particular type of 
> request but because it took a lot of time to figure out soapUI, I just 
> tried to find how soapUI could manage in the kind of functional 
> testing I had in mind.
>
> My requirements were:
>
> 1)Construct a test request
>
> 2) Feed that request into the service
>
> 3)Validate the response
>
> 4) Compare some data extracted from the response with data that are 
> considered known(nodes,links,etc)
>
> 5)Compare data extracted from the response against data taken from the 
> service database
>
> My findings per requirement were:
>
> 1) Construct a test request
>
> Firstly soapUI requires a wsdl description of the service being 
> tested. Unfortunately the wsdl description that is appearing on the 
> tomcat is very generic and inadequate for our needs.
>
> So one must write a wsdl description for any particular type request 
> type and its response. I followed Jochens example to do this(the only 
> experience with wsdl I had was a small wsdl file I wrote for a simple 
> service when I was still undergraduate). You also need the xsd files 
> describing the schema. Unfortunately our services rnc files cannot be 
> converted to xsd and loaded into soapUI. They need modifications which 
> in my case were made by Jochen(I hadn’t worked with rnc files before).
>
> Then the wsdl description are loaded to soapUI and then it can produce 
> an example request. Unfortunately it seems that our new rnc files were 
> not tottaly right and an example could not have been produced.
>
> But one can copy-paste a request into the request editor with ease. So 
> creating test request is no problem and each request can be easily 
> copied and modified
>
> 2) Feed that request into the service
>
> You just type the url of the service and with another click the 
> request is fed into the service and the response appears on the window.
>
> 3)Validate the response
>
> Schema validation is possible in soapUI, given that your wsdl and xsd 
> files are accepted. As I said before xsd files produced from our 
> services rnc files are not accepted and nedd modification. You also 
> have to write the wsdl of the service.
>
> 4) Compare some data extracted from the response with data that are 
> considered known(nodes,links,etc)
>
> This kind of tests can be easily done using the XPath assertions 
> feature of soapUI. One can type an appropriate XPath expression, test 
> it against the response and save the result if he wishes to be used 
> when the test is rerun, or simply type the result he expects and 
> compare it with the result from the XPath expression. So for this type 
> of testing soapUI is really creat and it can ease the testers work load
>
> 5)Compare data extracted from the response against data taken from the 
> service database
>
> For this requirement the use of the intergraded groovy scripting 
> language is compulsory. I tried to transfer data from the XPath 
> assertions to the groovy scripts or vies versa in order to use groovy 
> as little as possible and also exploit the ease of the XPath 
> assertions, but I wasn’t successful. So I had to rely only on groovy 
> to do the job done.
>
> Groovy is a scripting language very similar to java, so the transition 
> from java to groovy is easy. In fact groovy is more compact and faster 
> to develop than java once you have mastered it. I found groovy to be 
> quite fun and a powerful language, plus one can use all the available 
> libraries of java without any problem. The workspace for developing 
> groovy scripts in soapUI is quite ugly and tiresome I have to say. So 
> its best to create a template of your script in some other environment 
> (like eclipse) and then modified it in soapUI.
>
> Anyway I’ve manage through groovy to process the response, connect to 
> mysql and eXist and compare results.
>
> * Conclusions *
>
> * Pros *
>
> -One can easily create test requests
>
> -If the right wsdl and xsd files are present validation is an easy 
> process
>
> -With XPath assertions data can be easily retrieved and compared with 
> known values
>
> * Cons *
>
> -soapUI requires that the user is familiar with wsdl descriptions. In 
> our case the user must also be familiar with rnc in order to 
> appropriate change the services rnc files
>
> Another issue that arises here is if we SHOULD be changing rnc files 
> since the rnc files are essentially the perfsonar protocol and 
> tempering with them could mean changing the protocol its self. As 
> testers its not our job to do that.
>
> -Groovy is a different language and users have to learn how to use it
>
> - soapUI its self is quite a handful to learn
>
> So if I have to sum up all the above I would have to say that soapUI 
> can fulfil our functional testing needs. Can it do it better than we 
> have so far? The answer is no. Can it do it faster? It depends. An 
> experienced user can be very efficient with soapUI and produce a lot 
> of work in a short amount of time. A newbie, especially if he is not 
> familiar with wsdl or rnc or java, will take a lot time to do things. 
> Finally my suggestion is to keep the tests that we have done so far 
> for now and use soapUI for the testing of new services.
>
> P.S1 Sorry for the long email. It would be probably be best to have 
> this is a doc
>
> P.S2 Feel free to add any comments
>
> Best Regards,
>
> Michalis Michael