perfsonar development work

[Jochen Reinwand <>]

Hello everybody!

Finally I sort of finished my soapUI evaluation!
To sum up I have to say I really like the tool!

I did my tests with the BWCTL MP, because I started with it and we had 
problems setting up the CLI MP (we have to test) quickly. As result I now 
have a package (see attachment) containing some sort of "working directory". 
I have included a README.txt file explaining (hopefully) most of the things 
about how to use the package. I have attached the README.txt also separately 
to make it easier accessable.

I have to admit that getting into soapUI is not that easy. Of course, 
starting 
prepared tests is not really difficult. Setting up tests is more complicated. 
But you can set up tests very fast once you got into using soapUI. It's very 
easy to use, if you know how ;-)

There is still one drawback: The Perl SOAP implementation is creating a SOAP 
envelope, that soapUI cannot validate. I don't know who's right on this 
point, but sooner or later I have to find a workaround in Perl or soapUI... 
See the README.txt for further information.

I came across some more advanced features soapUI provides, but haven't taken 
a 
closer look. Since they obviously should be useful for us, I listed them 
here:

- Data-Driven Testing/Template-Driven Testing: Show how to test using data 
from external sources.

- Surveillance Testing: Surveillance testing is here used to describe a setup 
where a number of both functional and load tests are run using some kind of 
scheduling tool to continuously validate the availability to some web 
service.

- Scripting seems to be possible as well!

That's hopefully all I have to say. Just look at the README.txt and test 
soapUI ;-)

Comments and questions are, of course, always welcome!

greetings,
Jochen

-- 
Jochen Reinwand                                 Tel: +49 9131 852-8689
DFN Labor

Regionales Rechenzentrum Erlangen
Martensstrasse 1
91058 Erlangen
 
email:

Attachment: soapui-bwctl-20070117.tar.gz
Description: application/tgz

Introduction
============

This package contains some sort of "working directory" for functional
testing the BWCTL MP using soapUI. It's more or less a first step. It shows
how things can work, but contains only a few tests.

The sections "Starting soapUI" and "Running tests" are a quick intro showing
how to use soapUI with the already created project for doing functional
testing. The following paragraphs explain how this directory was build up and
can be seen as an howto for creating further functional tests.

The only files from the package required for creating the soapUI project from
scratch are:
bwctl-mp-req.rnc
bwctl-mp-req.wsdl
bwctl-mp-res.rnc
bwctl-mp-res.wsdl
bwctl-req-example.xml

You need some tools to accomplish the following steps:
soapUI: This is, of course, always needed since functional testing won't work
        without it.
trang:  Is used for converting RNC schemas to XML Schema (XSD).
Java:   Both soapUI and trang need Java.

This package was created on SUSE 10.0, but should work on any Linux
Distribution providing the above mentioned tools. It should also be possible
to create a project under Windows based on the information in this document.
But I'm not sure whether the already created soapUI project can be loaded and
used. It most likely depends on the way soapUI and/or Java handle path names
and file content in Unix text file format.



Starting soapUI
===============

Installing and starting soapUI should be quite simple:
Download from http://www.soapui.org/ (use version 1.6) extract and
execute soapui-1.6/bin/soapui.sh .
There is also a soapui.bat ...

You should change to the working directory (the one created during extracting
the package and containing this README.txt and the project file
BWCTLMP-soapui-project.xml) before starting soapUI.
Something like the following should do it if you extracted soapUI in the
working directory:
> ./soapui-1.6/bin/soapui.sh
Or just put the "bin" directory in your path.



Running tests
==============

After starting soapUI, you can import (File -> Import Project) the project 
file
BWCTLMP-soapui-project.xml and start exploring soapUI. Or you (re)create the
project on your own by skipping this section and reading the following
sections.

To start the included test, expand the tree view (click on the "+") following
the path BWCTL MP -> Simple Tests -> SetupData .
On double clicking "SetupData" the "TestCase Editor" will be displayed.
Pressing the green "play" button in the upper bar the test will start.

Or, if you like it even more simple, use the CLI without starting the UI:

> ./soapui-1.6/bin/testrunner.sh BWCTLMP-soapui-project.xml

IMPORTANT: No matter which way you have chosen, you will get an error! The
test fails! This problem is related to some issue I have to take a closer look
at.

If you are interested, just double click the "SetupDataRequest 1" entry in the
TestCase Editor. A new window should open that shows you the send SOAP request
and the received response from the previous test. Below you will see the list
of tests applied. Only one should have failed (the one with the red circle,
the Schema Compliance test). Now skip to section "Further testing" for
further information.



Create WSDL
===========

As starting point we have:
bwctl-mp-req.rnc  -  The RNC schema for a SetupDataRequest
bwctl-mp-res.rnc  -  The RNC schema for a SetupDataResponse

soapUI can not handle RNC schemas, so we have to convert them to
XML Schema (XSD) using trang:
> mkdir bwctl-mp-req bwctl-mp-res
> trang bwctl-mp-req.rnc bwctl-mp-req/bwctl-mp-req.xsd
> trang bwctl-mp-res.rnc bwctl-mp-res/bwctl-mp-res.xsd

Now you have to create an appropriate WSDL file for the service including 
these
schemas...
Or you can use the ones included in this package:
bwctl-mp-req.wsdl and bwctl-mp-res.wsdl

Here is the problem: For every message (SetupDataRequest, SetupDataResponse,
MetadataKeyRequest etc.) a separate WSDL file is necessary. Normally there is
one WSDL file for every service containing all interfaces/messages. If 
somebody
would make it possible to load more than one schema with the same root element
name in WSDL, everything would be much easier with soapUI!!

But we now have our needed WSDL files an can start setting up the soapUI
project.



Setting up a soapUI project
===========================

First add a new project:
File -> New WSDL Project

I've chosen "BWCTL MP" as project name and saved the project file in working
directory where I put all the other files.

Now we add the service by adding the WSDL file bwctl-mp-req.wsdl:
<Context menu of BWCTL MP> -> Add WSDL from URL
In the box use file:./bwctl-mp-req.wsdl as URL!! This avoids absolute paths
and a lot of related problems! For the same reason please don't use "Add WSDL
from File"!

You are now asked, if you want to add an example request for the interface.
Just do it and also "Create optional elements in schema". Now you see what
soapUI can do for you!
Important: The RNC schemas are far to simple by now. The better and accurate
the schema is, the better the automatically created request is filled.

From within the popped up request window. You can send the request to the
service you see in the drop down menu in the upper bar by pressing the
green button left in the bar. The template request soapUI created is,
of course, not a correct request for the BWCTL MP!

Best is to use the example bwctl-req-example.xml that is supplied with the
package. Just copy and paste (use Ctrl-V in soapUI) its contents (without the
XML header <?xml...> !!) between <soapenv:Body> and </soapenv:Body>.

Now it should be possible to submit the request and the response will be
displayed in the right window.



Creating tests
==============

First create a test suite:
<Context menu BWCTL MP> -> New TestSuite

Than a new test case within the test suite:
<Context menu of test suite> -> New TestCase

You can (obviously) organise your tests in suites and cases. I don't know by
now what is the best way for organising the tests. Here I called the suite
"Simple Tests" and the first case "SetupData".

Now you can double click on "Test Steps" to see the editor for the steps.

But for adding a real test you have to go back to the formerly created 
request.
Click right on it and choose "Add to TestCase" from the context menu.
Now choose the test case (or create a new one). Give the test a name and also
add the Schema Assertion and the SOAP Fault Assertion.
The test (I called it "SetupDataRequest 1") will now be added as a step of the
test case and appear in the editor. A new window similar to the request window
earlier should also appear.

You can now push the green button and the request will be submitted and the
tests applied.

The SOAP assertion is ok, but what about the Schema Compliance? Remember it?
There is a special WSDL for the response. Double click the line with the red
circle and you can choose a different WSDL file. Choose "res" instead of 
"req".
Now the test will be done with the other WSDL file without requesting again!

Why is it still failing? Well, that's the greatest problem I have at the 
moment
regarding BWCTL MP.



Further testing
===============

As the line below the FAILED message implies, there is an
error in the SOAP header (in line 2), that we got from the BWCTL MP. The 
header
is added by the Perl SOAP modules (SOAP::Lite). "anyURI" is unknown. I don't
know who's right here: The Perl modules adding it or the soapUI not knowing 
it.

Can we continue without caring about that issue? - Yes!

You can edit the response in the right window (remove xsi:type="xsd:anyURI")
and via double clicking the line with the red circle again and pressing "OK"
the test will be rerun.

Now both circles should be green -> All tests passed!

Not very much testing so far?
Right click in the area with the green circles and choose "Add Assertion" from
the context menu. Choose "XPath Match" from the drop down menu and press OK.
Now you can enter an XPath expression in the upper text input field. But first
you should press "Declare" to automatically import Namespace definitions from
the response for your convenience. After the "declare" statements you can add
your own code. With "Select from current" the XPath expression will be
evaluated and the result displayed in the lower window. With "Test" a boolean
test is done on the result like the testing procedure will do it for the 
actual
test.

You want to know what XPath is? You can find a good introduction here:
http://en.wikipedia.org/wiki/XPath

For now a good test is to enter
/soap:Envelope/soap:Body/nmwg:message/nmwg:data/nmwg:datum[1]/@numBytes >= 0
here to test whether the first result of the BWCTL test is a positive number.

But what about the other results in the response?
The following expression counts all results not positive. Of course there
should be none to pass the test:
count(/soap:Envelope/soap:Body/nmwg:message/nmwg:data/nmwg:datum[@numBytes
 < 0]) = 0
Or, since we can even use XPath 2.0, which has a lot of additional features:
every $numBytes in 
/soap:Envelope/soap:Body/nmwg:message/nmwg:data/nmwg:datum/@numBytes 
satisfies ($numBytes >= 0)



Notes
=====

- http://www.soapui.org/userguide/
  Here you'll find a lot of information for using soapUI.
  Some interesting points might be:
  - Data-Driven Testing/Template-Driven Testing: Show how to test using
    data from external sources.
  - Surveillance Testing: Surveillance testing is here used to describe a
    setup where a number of both functional and load tests are run using some
    kind of scheduling tool to continuously validate the availability to some
    web service.
  - Scripting seems to be possible as well, but I haven't done any further
    investigation.
- Important when dealing with external files (mainly schema files):
  Always use relative paths, in order to make tests run everywhere not only
  your local installation!
  For import of WSDL use "Add WSDL from URL" and something like
  file:./schema.wsdl as URL.
- bwctl-mp-req.wsdl and bwctl-mp-res.wsdl differ in one line:
  
#############################################################################
  --- bwctl-mp-req.wsdl
  +++ bwctl-mp-res.wsdl
  -      <include schemaLocation="bwctl-mp-req/bwctl-mp-req.xsd"/>
  +      <include schemaLocation="bwctl-mp-res/bwctl-mp-res.xsd"/>
  
#############################################################################
  This is obviously very ugly!