perfsonar development work

[Jochen Reinwand <>]

On Friday 16 November 2007 07:16, Michael Michalis wrote:
> Prodromos Gerakios wrote:
> > Hi all,
> >
> > Namespaces:
> > I  have attempted to use trang so as to automatically generate the
> > xsd files. trang ran successfully *but* the files produced by trang
> > use the same namespace. For instance, in LSRegister there exist two
> > messages: a request and response message, which have some element
> > names in common but *just the names* not the *definitions*. I
> > noticed that the rnc files do indeed use the same namespaces. It is
> > therefore, impossible to write a single WSDL file for the entire web
> > service. In fact, it is necessary to provide a different WSDL file
> > for each operation. I noticed that other tests on soapUI (clmp and
> > rrdMA) use multiple WSDL files as a result of this issue.  I am not
> > sure how Relax-NG works but it should be possible to generate a
> > *single* and *complete* WSDL file.  As things stand, it is
> > impossible to validate request and response messages: current WSDL
> > files (of clmp and rrdma) import both request and response xsds but
> > really only use the response definitions for soapUI validation.
> > Additionally, if the LS (or some other)  service is contacted by an
> > external (in respect to perfSONAR) service/client then it must use a
> > single WSDL to contact it, right ?
>
> Prodromos,
> This is a long standing issue for perfSONAR and it has been raised a
> lot times in the past. But to be honest I'm not sure this will change
> soon.

This issue is fixed for services with correct NMWG schema. There is one 
schema 
file per service using common definitions by including common schema files. 
Having different schemas for request and response has been introduced 
accidentally in earlier releases. It never was planed for correct NWMG 
schemas. BWCTL MP schemas can already be converted to XSD and loaded by 
soapUI. Only minor changes are sometimes necessary because auf XSD<->RNG 
incompatibility issues.
I cannot recall how far this process is regarding Hades MA and common IPPM 
schemas, but this will also be addressed for the release.
There are also good news regarding RNC/RNG support in soapUI. The developers 
are willing to implement it, because it's also part of WSDL 2. But even until 
they implemented WSDL 2, there is a possibility to do RNG schema validation 
using RNG files. I already set up a soapUI project for BWCTL MP. We planed to 
use it for the release management process as well. I attached the 
documentation I created for the creation of a soapUI project. I wrote it some 
time ago and it's NOT complete if I remember it correctly...

Our view on functional testing doing supervised by a functional testing team:

Phase 1: Testing team is setting up a framework for functional testing. This 
includes testing of functionalities common for all services (e.g. 
EchoRequest), examples and howtos for typical test cases and ideas about how 
to create more complex test cases. Phase 1 should be completed shortly after 
developing of the service started to give developers powerful functional 
testing possibilities during service development.

Phase 2: Development and functional testing done in parallel to make an 
efficient development process possible. Functional tests are mainly created 
by developers, because during development and bug fixing they get a good idea 
of how their service should be tested. E.g. After fixing a bug it is very 
useful to create a test triggering the bug (if not already created during bug 
fixing!). If the bug or a similar one arises again later in development 
process it will be triggered by functional testing at once.

Phase 3: When the real management process starts (services nearly ready, 
feature freeze) the functional testing team starts verifying the test written 
by the developers. During this process the functional testing should be 
enhanced significantly, because it is very important that testing is tested 
independent from the developers!

regards,
Jochen

-- 
Jochen Reinwand, DFN-Labor
Friedrich-Alexander-Universität Erlangen-Nürnberg
Regionales RechenZentrum Erlangen (RRZE)
Martensstraße 1, 91058 Erlangen, Germany
Tel. +49 9131 85-28689, -28800, Fax +49 9131 302941

www.win-labor.dfn.de
Introduction
============

This file gives an introduction on using soapUI for functional testing of
web services. It was designed especially for NWMG/perfSONAR services using
version 2.0 of the schemas. soapUI 1.7.5 was used for testing the following
descriptions. Menu names and usage detail may differ between different soapUI
versions!

Note: See README.txt for a specific description of the soapUI project in this
directory. This file is a general introduction on how to test perfSONAR
services using soapUI.

The sections "Starting soapUI" and "Running tests" are a quick intro showing
how to use soapUI with an already created project for doing preconfigured
functional testing. The following paragraphs describe how to configure
functional tests with soapUI.

To accomplish the following steps you need soapUI itself and a Java runtime
environment since soapUI is written in Java.



Starting soapUI
===============

Installing and starting soapUI should be quite simple:
Download from http://www.soapui.org/ (e.g. version 1.7.5) extract and
execute soapui-1.7.5/bin/soapui.sh .
There is also a soapui.bat ...

Most likely you will find a workspace containing all related projects. You can
open it in soapUI or via command line parameter:

> <SOAPUI-PATH>/bin/soapui.sh perfSONAR-workspace.xml



Running tests
=============

After starting soapUI you can use the projects contained in the workspace or
you can import (File -> Import Project) project files to the current 
workspace.
If you have not loaded a workspace the default workspace will be used.
Or you (re)create a project on your own by skipping this section and reading
the following sections.

If you have loaded an existing project containing tests, you can directly 
start
them via the UI (user interface).

Instead of using the UI you can use the CLI (command line interface) for just
starting the tests:

> <SOAPUI-PATH>/bin/testrunner.sh <PROJECT-FILE>

For more information on using soapUI take a look at the user guide:
http://www.soapui.org/userguide/

For the CLI switches see:
http://www.soapui.org/userguide/commandline/functional.html

Also information about "JUnit Integration" and "JUnit Reporting" can be found
there.



Creating a soapUI project
=========================

The only files required for creating a soapUI project from scratch are: The
RELAX NG Compact Syntax (RNC) schema files, the Web Services Description
Language (WSDL) file(s) and some example XML requests.
 
There are two different ways for using the RNC schema files for schema
validation in soapUI. The normal approach is to load a XML Schema (XSD) file
via the WSDL file. At the moment only XML Schema can be used here! This may
change in the future, when soapUI is implementing WSDL2 that can use RELAX NG
for schema validation. On the other hand it is possible to do schema 
validation
directly using RELAX NG via a "Script Assertion" using Groovy. See below for
further information on this topic. 

All relevant WSDL and schema files are normally included in the soapUI project
file. But you cannot edit them from within soapUI. So if you plan to edit
these files, you should make sure that it is easy to "reload" these files from
within soapUI. Therefore it is important, that soapUI can at least find all
necessary files.

Best practice here is to put the WSDL files in the same directory as the
project file and load them in soapUI using the URL "file:./<WSDL-FILE>".
To make this work you should change your working directory to the directory
containing the project file(s) and the schema files before starting soapUI and
start soapUI from this directory. This way you can copy/extract this directory
everywhere you want and the WSDL files should be found.

The same is valid for the schema files (XSD) that are referenced from
the WSDL files. Additionally it is useful to use a subdirectory for the 
schema 
files since XSD schemas often consist of more than one file. A reference in
a WSDL file might look like this:
<include schemaLocation="xsd/service.xsd"/>

For notes on directly using and including RNG schema files see below.

Beside soapUI and a Java runtime environment, you will need the tool trang.
trang is used for converting RNC schemas to XML Schema (XSD) and RNG schemas.



Creating WSDL files
===================

A typical perfSONAR service (complying to schema version 2.0) has one RNC file
describing requests as well as responses. The file includes other (common) RNC
files for common NMWG/perfSONAR specifications.

Since soapUI cannot handle RNC schemas while loading WSDL files (for now), we
have to convert them to XML Schema (XSD) using trang. We assume the RNC files
are in subdirectory "rnc" and we want to write the resulting XSD files to the
subdirectory "xsd". The service we want to create XSD files for is described 
in
the RNC file service.rnc. The following commands will do the trick:

> mkdir xsd
> trang rnc/service.rnc xsd/service.xsd

trang will automatically convert all necessary RNC files to XSD files and 
place
them in the subdirectory "xsd".

Now we have to create an appropriate WSDL file for the service including these
schemas. This is not covered here at the moment!
Look for an appropriate example (in this directory) to get started...
In the following we assume that a file called service.wsdl was created
including xsd/service.xsd for schema validation.

IMPORTANT: If you don't want to use XSD schema files, because there is an 
error when soapUI tries to load it or you prefer to use direct RNG validation 
(see
below), you can put a simple schema definition directly into the WSDL file!



Setting up a soapUI project
===========================

It might be useful to create a new soapUI workspace for the services you are
going to test. The best way is to go to the directory you plan to put all 
tests
into and start soapUI from there creating a new workspace in this directory:

> <SOAPUI-PATH>/bin/soapui.sh perfSONAR-workspace.xml

You might get an error message, because no soapui-settings.xml containing the
soapUI settings exists. Don't worry, the file will be created. Just click
"No".

You can rename the workspace name (for the GUI) by clicking on the "Projects"
entry in the tree view.

Now add a new project:
File -> New WSDL Project

Choose an appropriate project name (e.g. the service name, like BWCTL-MP).
As initial WSDL you can use the service.wsdl created before. Please enter
file:./service.wsdl to denote the WSDL file!  This avoids absolute
paths and a lot of related problems! For the same reason please don't use
"Add WSDL from File" to add further WSDL files. Use "Add WSDL from URL" and
URLs starting with file:./ as described above.

You are also asked, if you want to add an example requests for all interfaces.
Just do it! Now you can see what soapUI can do for you!
Important: The RNC schemas are far too simple at the moment. The better and
accurate the schema is, the better the automatically created request is 
filled.
Keep in mind: The created requests can never be perfect!

Now save the project file in the working directory where the schema files are
located (see above).

Expand the project tree and double click on one of the requests "Request 1".
From within the popped up request window you can send the request to the
service, you see in the drop down menu in the upper tool bar, by pressing the
green button left in the bar. The template requests soapUI created are,
as said above, most likely not correct requests for the service!

Best is to find or create some example request for the service. Just copy and
paste (use Ctrl-V in soapUI) its content (without the XML header <?xml...> !!)
between <soapenv:Body> and </soapenv:Body> in the request window.

Now it should be possible to submit the request and the response will be
displayed in the right window.

IMPORTANT: Check the service URL you are contacting in the pull down menu in
the top tool bar.

IMPORTANT: The created project file (<PROJECTNAME>-soapui-project.xml) is
added to the workspace using an absolute path. To make everything relocatable
you should change the absolute directory to "./" using a normal text editor.
You will find the project as content of the element <con:project>.



Creating tests
==============

First create a test suite:
<Context menu service> -> New TestSuite

Than a new test case within the test suite:
<Context menu of test suite> -> New TestCase

You can (obviously) organise your tests in suites and cases.

Now you should add a "Test Step" to the TestCase:
<Context menu of test case> -> Insert Step -> Test Request

Give the test a name and select the operation to be used. Normally you do now
add the SOAP Response Assertion, the Schema Assertion, and the Not SOAP Fault
Assertion. Creating optional elements is normally not necessary, if you are
going to copy and paste the request. The test will now be added as a step of
the test case and appear in the editor on the left panel. A new window similar
to the request window earlier should also appear.

You can now push the green button and the request will be submitted and the
tests will be applied on the response.



Using RELAX NG (RNG/RNC) schemas
================================

The flexibility of soapUI allows to do RELAX NG schema validation although it
is not available via the typical WSDL mechanisms. The idea is to load the
classes available for RELAX NG validation in soapUI and than use a Groovy
"Script Assertion" to validate the response using these classes.

RELAX NG validation is normally not done using RNC files, but rather their
RNG version. You can use trang to convert RNC files to RNG:

> mkdir rng
> trang rnc/service.rnc rng/service.rng

In order to do RELAX NG schema validation in soapUI you first have to add the
necessary Java classes. Get the following ZIP archive:
http://weblogs.java.net/blog/kohsuke/archive/20060210/rng-validation.zip/rng-validation.zip

You just have to put the JAR files found in the ZIP file in the lib directory
into the ext directory. Normally soapUI searches for the ext directory in the
current working directory. During startup soapUI should inform about all
classes/modules being loaded.

Now just start soapUI and add a script assertion.

Here is the corresponding code:

-------------------------------------------------------------------------------
import javax.xml.XMLConstants
import javax.xml.transform.stream.StreamSource
import javax.xml.transform.dom.DOMSource
import javax.xml.validation.SchemaFactory

def groovyUtils = new com.eviware.soapui.support.GroovyUtils( context )
def holder = groovyUtils.getXmlHolder(messageExchange.getResponseContent())
def factory = SchemaFactory.newInstance(XMLConstants.RELAXNG_NS_URI)
def schema = factory.newSchema(new File("./rng/schema.rng"))
def validator = schema.newValidator()
def node = holder.getDomNode("//nmwg:message")
validator.validate(new DOMSource(node))
-------------------------------------------------------------------------------


IMPORTANT: Of course, the RNG schemas are not included in the project file
like the XSD schemas loaded by the WSDL file!



Further testing
===============

You can change the assertions and add new ones without sending the request
again. The tests will be applied to the old response until you explicitly
request a new one. You can even edit the response in the right window and 
check
the assertions on this modified response.

What kind of tests are available? Which one is useful?
Right click in the area with the green circles and choose "Add Assertion" from
the context menu. Choose "XPath Match" from the drop down menu and press OK.
Now you can enter an XPath expression in the upper text input field. But first
you should press "Declare" to automatically import namespace definitions from
the response for your convenience. After the "declare" statements you can add
your own code. With "Select from current" the XPath expression will be
evaluated and the result displayed in the lower window. With "Test" a boolean
test is done whether the expression in the upper window matches the text from
the lower window, like the testing procedure will do it for the actual test.

You want to know what XPath is? You can find a good introduction here:
http://en.wikipedia.org/wiki/XPath

A typical test for a perfSONAR service may look like:
/soap:Envelope/soap:Body/nmwg:message/nmwg:data/nmwg:datum[1]/@numBytes >= 0
This example tests whether the first result of a BWCTL run from the BWCTL MP 
is
a positive number.

But what about the other results in the response?
The following expression counts all results not positive. Of course there
should be none for a successful test:
count(
  
/soap:Envelope/soap:Body/nmwg:message/nmwg:data/nmwg:datum[@numBytes
 < 0]
) = 0
Or, since you can even use XPath 2.0, which has a lot of additional features:
every $numBytes in
  /soap:Envelope/soap:Body/nmwg:message/nmwg:data/nmwg:datum/@numBytes
  satisfies ($numBytes >= 0)



Notes
=====

- Here you'll find a lot of information for using soapUI:
  http://www.soapui.org/userguide/
- Data-Driven Testing/Template-Driven Testing: Testing using data from 
external
  sources.
- Surveillance Testing: Surveillance testing is here used to describe a
  setup where a number of both functional and load tests are run using some
  kind of scheduling tool to continuously validate the availability to some
  web service.
- Using Property Transfers it is possible to set up a test sending a
  MetadataKeyRequest, receiving a metadata key, and using this key for a 
  SetupDataRequest. More about Property Transfers:
  http://www.soapui.org/userguide/functional/propertytransfers.html
- You can use Groovy scripts to create soapUI tests:
  http://www.soapui.org/userguide/functional/groovystep.html
  What is Groovy?
  "An agile dynamic language for the Java Platform with many features that 
are 
  inspired by languages like Python, Ruby and Smalltalk, making them 
available 
  to Java developers using a Java-like syntax." http://groovy.codehaus.org/
  It is e.g. possible to verify the data from a NMWG response against the 
  content of a database using Groovy.
- There is a simple feature for flow control based on XPath tests:
  http://www.soapui.org/userguide/functional/gotostep.html
- Important when dealing with external files (mainly schema files):
  Always use relative paths, in order to make tests run everywhere not only
  your local installation!
  For import of WSDL use "Add WSDL from URL" and something like
  file:./schema.wsdl as URL.
- There is especially one problem when converting RNC to XSD:
  http://www.w3.org/TR/xmlschema-1/#cos-element-consistent
  trang is converting without warning, but soapUI will refuse to load the
  created schema.



Known issues
============

- If there is an error message about a SSL connection in the log and soapUI
  freezes you most likely used ./bla.wsdl as URL instead of file:./bla.wsdl .



Problems/TODO
=============

- Is it possible to use the same soapUI project files in Linux and Windows?