perfsonar development work

[Jochen Reinwand <>]

Hello everybody,

Because Loukik thought it might be a good idea, I took a closer look at 
anteater and found also another very interesting tool called soapUI. Since we 
haven't seen any code from Ilias that could help us in our functional testing 
ambitions and Michalis code is in Java, that we are not so familiar with, we 
really need some other framework for our functional testing. To start with 
the conclusion: Both anteater and soapUI seem to be sufficient for functional 
testing!

The concepts of both tools are a bit different, but the overall abilities in 
functional testing are very similar. Further on I will describe for both 
tools the typical work that needs to be done to get started with functional 
testing.



Anteater

Let's start with Anteater (http://aft.sourceforge.net/). Anteater is more or 
less an enhancement of Ant, that, as you all know, is already used in 
perfSONAR as deployment tool. But Ant can very easily be expanded to do a lot 
of other things. Anteater uses this features to enhance Ant with the 
capabilities for testing web services.

Here is the typical work cycle: You should have an example request for your 
service and an rnc schema for the corresponding response. Both things should 
be available for perfSONAR services, because they are expected to be found in 
the handover documentation ;-)
Now you just have to write an Ant XML file that uses the special targets of 
Anteater. You tell it to send the example XML as an request and to validate 
the result against the schema. The more accurate the schema, the better the 
testing. But you can do a lot more things:
- Analyse the header of the HTTP request.
- Use XPath syntax to extract parts of the document.
- Use regexps to validate the extracted values or the whole response.

There are some minor problems, when using Anteater. For example it does not 
generate nor parse the SOAP envelope and simply does an HTTP Get. So you have 
to take care about some SOAP specific things, which is not really bad and 
also very helpful in some cases, e.g. you can do functional testing on the 
SOAP part of the service itself.

I attached a tarball containing a simple testing framework for our BWCTL MP. 
It only sends a simple SetupDataRequest, validates the SetupDataResponse and 
extracts a value using XPath.

You will find the following files:

anteater-test/bwctl-mp.xml
This is the Anteater enhanced Ant file. After installing Anteater, which is 
very simple, you just have to enter the anteater-test directory and use 
"anteater -f bwctl-mp.xml" to start the test.

anteater-test/resources/requests/bwctl-req.xml
anteater-test/resources/schema/bwctl-res.rnc
These are the example request and the rnc schema for the response directly 
copied from our release documentation.

anteater-test/resources/soap-footer.xml
anteater-test/resources/soap-header.xml
Our requests need a SOAP envelope, that is not generated by Anteater 
automatically. By 'cat'ing together soap-header.xml, bwctl-req.xml and 
soap-footer.xml we get our full request

anteater-test/resources/schema/soap-bwctl-res.rnc
anteater-test/resources/schema/soap.rnc
Since the response contains the SOAP envelope, these rnc schemas help us 
validating the response against bwctl-req.xml by taking care of the SOAP 
part.

Another very interesting feature of Anteater is the built in web server. You 
can set up a simple SOAP server and test clients against it. You can even 
start a server and run test requests using the same Anteater process. Very 
funny.

But now the great drawback of Anteater: Seems like development has completely 
stopped in the year 2003! Since the Anteater package contains nearly all 
needed third party software, you won't realise it at once. But the bundled 
software is rather old and you sooner or later find e.g. missing Ant 
features, that can only be found in newer versions of Ant.

If we decide to use Anteater it might be useful/necessary to take over 
development. It seems like Ant is really powerful and working on updating and 
improving Anteater should be very simple. But I'm not an Ant or Java expert, 
so I don't want to say to much about this topic (yet). Perhaps someone else 
wants to take a look at it...



soapUI

Let me simply quote the web page (http://www.soapui.org/) as starting point 
for my writing about soapUI:
"soapUI is a desktop application for inspecting, invoking, developing and 
functional/load/compliance testing of web services over HTTP. It is mainly 
aimed at developers/testers providing and/or consuming web services 
(java, .net, etc). Functional and Load-Testing can be done both interactively 
in soapUI or within an automated build/integration process using the soapUI 
command-line tools. IDE-plugins are available for eclipse, IntelliJ IDEA, 
NetBeans and a specialized eclipse-plugin for JBossWS. soapUI requires java 
1.5 and is licensed under the LGPL license."

This said I should add that this tool seems to be the standard OpenSource 
tool 
for functional testing of web services. Perhaps Anteater was abandoned 
because of this tool...
Because soapUI is a graphical user interface it's quite easy to "click" 
together functional tests. The typical workcycle: You import your service 
specification in WSDL format. Based on this soapUI knows how to use your 
service. You can now generate template XML requests, fill them with data and 
send them to the service. By cloning you can easily create a lot of similar 
messages. You can even import data from text files and automatically fill the 
request with them. Based on these requests you can build test cases. soapUI 
will send the request and then do some tests on the response. Typical tests 
are quite similar to Anteater:
- Validate against schema (WSDL)
- Extract and test values via XPath (more features than Anteater!)
All these tests are easy to create and you can test them at once against the 
former received response. That's really comfortable!

There are some problems with soapUI that are more or less perfSONAR/NMWG 
specific. NMWG's request/response cannot be described using WSDL, like it 
should normally be done when using web services. Please note: My knowledge 
about SOAP and WSDL is far from being perfect! If you know how to describe a 
perfSONAR/NMWG service using WSDL or if you already have a WSDL file, please 
inform me about it! Having one will make functional testing for the described 
service possible in a few hours!

On the other hand it's no great problem to work around the WSDL problem. WSDL 
cannot (yet) use RNC/RNG for schema description, but both can be converted to 
XML Schema (XSD) using e.g. trang. Since XML Schema is not as powerful as 
Relax-NG I have some problems at the moment. I cannot present to you a ready 
and nice looking functional testing example, like I did for Anteater. Would 
be easy to hack something useful, but I first want to work on fixing some 
more problems. I will send a good example asap.

BTW: One drawback compared to Anteater could be the missing web server 
functionality for testing clients. soapUI is only a client...



Congratulations! You've managed to read through this "small" email!
As some sort of reward, let me wish you a Merry Christmas and a Happy New 
Year!
Jochen


On Tuesday 12 December 2006 14:26, Loukik Kudarimoti wrote:
> Jochen,
>
> I am not aware of any of us having tested this tool. From the
> description that you have given me, it looks very powerful and probably
> what we want. If you are going to do some testing with it, we would be
> very interested in hearing your experiences and outcomes from this testing.
>
> Thanks,
> Loukik.
>
> Jochen Reinwand wrote:
> > Hi everybody,
> >
> > I just came across the Open Source tool Anteater, which describes itself
> > as "Ant-based functional testing framework": http://aft.sourceforge.net/
> >
> > It seems to be very powerful in doing functional testing of web services
> > (including SOAP). For testing you have to write ant scripts extended by
> > special tags for the testing targets. I'm not very familiar with ant, but
> > the anteater scripts look very simple yet powerful. You select an xml
> > file to be sent to the service and can give conditions that the reply
> > should fullfill. You can e.g give a Relax-ng schema the reply must
> > validate against and you can do further tests on the content, like
> > pattern matching on parts of the reply denoted using XPath.
> > It's also possible to use anteater as "testing service" to validate
> > clients.
> >
> > Has anybody taken a closer look at this tool? I'm thinking about doing
> > some testing with it. If somebody knows, that anteater is not really
> > useful, please inform me to prevent me from doing stupid things ;-)
> >
> > regards,
> > Jochen

-- 
Jochen Reinwand                                 Tel: +49 9131 852-8689
DFN Labor

Regionales Rechenzentrum Erlangen
Martensstrasse 1
91058 Erlangen
 
email:

Attachment: anteater-test.tar.bz2
Description: application/tbz