Discussion list for thrulay, a network testing tool

[stanislav shalunov <>]

-----BEGIN PGP SIGNED MESSAGE-----

                      thrulay-0.7 release notes

This release incorporates the results of the work of Bernhard Lutzmann
on his Google Summer of Code project.  Bernhard's project report is
appended to these release notes.  I would like to thank Bernhard for
his hard work on thrulay this summer and to congratulate him on the
excellent results.

We are in the process of incorporating of changes made by Huadong Liu
in his Google Summer of Code project into the main code base.  These
should be included in a future release.  Most importantly, they would
include (i) an API for programmatic access to thrulay tests (this is
useful for tools that wrap thrulay in other code for the purposes of
regular monitoring, coordination of tests or additional access
control) and (ii) improvements to on-the-fly quantile calculations.

In addition, Huadong will serve as a release engineer for the project.

The new thrulay release can be found at SourceForge or in the usual
direct location:

http://sourceforge.net/projects/thrulay/

http://www.internet2.edu/~shalunov/thrulay/thrulay-0.7.tar.gz

SHA1(thrulay-0.7.tar.gz) = 881709982ae9bde1477818dd14a20ca448400685

                                                        --Stanislav

-----BEGIN PGP SIGNATURE-----
Version: PGPfreeware 5.0i for non-commercial use

iQCVAwUBQ1QL15RUn1EgN49xAQF32gP6A+mj/Q5R1tFhf1SfGHhmluFw5Wxlsc3R
VNFlDDvRniMOjspMS7RPkv1da7aO1A0rJ+/Em+adkUWMMaVEYb7uqoNs+vCPfwNe
ESO9H6uiaE8OVGeaRpwephOHsSPHqnZJhdEaI0qjhxbt4jsPMAb48kHOGUg2BJum
HK5NGLicuLQ=
=lMxE
-----END PGP SIGNATURE-----
-- 
Stanislav Shalunov              http://www.internet2.edu/~shalunov/

Just my 0.086g of Ag.





                    thrulay-ng: Enhancements for thrulay
                                FINAL REPORT

                              Bernhard Lutzmann
                             
<>

                              September 1, 2005



1. INTRODUCTION

This is the final report for "thrulay-ng: Enhancements for thrulay",
a Google "Summer of Code" [1] project. It summarizes all the accomplishments
that have been made during the summer.

For additional information about thrulay-ng please visit the project homepage
at http://thrulay-ng.sf.net/



2. ACCOMPLISHMENTS

The features are listed in chronological order of implementation.

* GNU autotools
In the initial phase, the source code of thrulay 0.6 [2] has been adapted
and support for GNU autotools has been added. This allows users to easily
configure and build the programs from the source code. At this stage,
thrulay worked under the FreeBSD and Linux operating system. This source
code base has been integrated into the SourceForge.net CVS repository where
all future development took place.

* multiple TCP streams
Next, I decided to implement support to run multiple TCP streams. I started
out with using POSIX threads for this. Testing an initial version of the
multi-stream code on several operating systems revealed that implementing
this with threads was a bad idea. The results were everything else than
satisfactory. After discussing this with Stanislav and Jeff, Stanislav wrote
a very nice article [3] on the different possibilites on how to implement
parallel tasks. Based upon this article and digging in "Stevens, UNIX Network
Programming, Vol. 1" I descided to try an event-based approach. Changing from
threads to events was not that expensive, but fortunately results looked much
better now.

* API for programmatic execution of tests
While implementing multi-stream support, the client-code has also been
rearranged into several functions. This revealed in a basic client API that
could be used to run tests.

* Report MTU/MSS, set DSCP value for TOS byte
After this, code has been integrated to guess the Maximum Transmission Unit
(MTU) and Maximum Segment Size (MSS) if a TCP test is started. The client is
now also able to set a Differentiated Services Code Point (DSCP) value for
the Type of Service (TOS) byte in the IP header.

* SPARC/Solaris support
The source code has then been ported to SPARC/Solaris.

* IPv6 support
Some of the network code has also been changed to be less protocol specific
and that the client and server also work in an IPv6 environment.

* documentation: manual pages
To better document the usage of the client and server program, manual pages
have been written.

* UDP statistics: duplication, reordering
The next point was to improve the statistics reported in UDP mode. In a first
step, I fixed some little bugs in the current UDP code. Then I added support
so the server is able to calculate duplication and reordering [4] of packets.

* client authorization
After that, the server has been enhanced to be able to perform client
authorization based on netmasks.

* integration of TSC timekeeping projects
At this point of development, the TSC timekeeping projects fasttime [5] and
TSC-I2 [6] reached a usable state. So support for these have been added to
thrulay-ng. The configure script automatically detects if one of them is
installed and uses it as a replacement for the gettimeofday() system call.

* UDP statistics: quantiles
Then I added another statistic in UDP mode: quantiles. This has been
implemented as described in [7] using the "New Algorithm". thrulayd now
calculates the 0th (minimum), 50th (median) and 95th quantile.

* code review issues
In the middle of August, Jeff posted his valuable code review [8]. Fixing the
noted issues was the work of the following days. In an IRC session he also
mentioned that it would be fine to be able to easily build a release out of
the CVS version. Thus I have added a script and the file README.release,
which documents how to release a new version of thrulay.

* Mac OS X port
At the end of the project, thrulay has also been ported to Mac OS X and
currently supports Linux, BSD (FreeBSD, OpenBSD, NetBSD), Mac OS X and
Solaris.

Version 0.6.2 of thrulay-ng has been released at the end of August. This
package contains all the changes noted above. You are invited to give it
a trial. The source code can be downloaded from [9].



3. ACKNOWLEDGEMENT

First I would like to thank my mentors Jeff W. Boote and Stanislav Shalunov
of Internet2 for their great support and guidance.

Second I would like to thank all the students from the soc2005 mailing list.

Many thanks also to Google for sponsoring this project. The "Summer of Code"
was a nice happening.

Last but not least I would like to thank my friends Christian Schwind and
Bernhard Wachter for books, wine and discussion.



4. REFERENCES

[1] Google Summer of Code, http://code.google.com/summerofcode.html
[2] thrulay, http://www.internet2.edu/~shalunov/thrulay/
[3] "Threads, Tasks, Coroutines, Processes, and Events" by Stanislav Shalunov,
    http://www.internet2.edu/~shalunov/writing/threads-tasks.html
[4] "Definition of IP Packet Reordering Metric" by Stanislav Shalunov,
    
http://www.internet2.edu/~shalunov/ippm/draft-shalunov-reordering-definition-01.txt
[5] fasttime, http://fasttime.sf.net/
[6] TSC-I2, http://tsc-xluo.sf.net/
[7] "Approximate Medians and other Quantiles in One Pass and with Limited
    Memory" by Manku, Rajagopalan and Lindsay,
    http://www-db.stanford.edu/~manku/papers/98sigmod-quantiles.pdf
[8] https://mail.internet2.edu/wws/arc/soc2005/2005-08/msg00061.html
[9] http://sourceforge.net/project/showfiles.php?group_id=142602