NDT-DEV email list created

[Matt Mathis <>]

The underlying paradigm for both NPAD and NDT is to run a suite of
measurements and verify that they are consistent with the TCP model.
The model currently embedded in the code assumes standard TCP AIMD
(which Linux calls Reno).

At one time it was important because non-Reno did not include the
required instrumentation (2 lines of code!).
But now that that has been fixed, I actually don't know CUBIC well
enough to know how much the difference matters.

My sense is that pretending that CUBIC is supported may causes some
tests for marginal networks to false fail.   That is they really do
work well enough to be filled using CUBIC, but would not do well
enough using Reno.   I would expect these to be reported as failing,
when perhaps they should not.

This raises a general "calibration" question: both tools use several
heuristics that might be sensitive to details in either Web10g or
changes to TCP, such that their results are less accurate.    Rich and
I spent years tweaking some of these algorithms - I have no doubt that
they will appear to be correct, but they may be much less well
calibrated than we  would prefer.

Thanks,
--MM--
The best way to predict the future is to create it.  - Alan Kay

Privacy matters!  We know from recent events that people are using our
services to speak in defiance of unjust governments.   We treat
privacy and security as matters of life and death, because for some
users, they are.


On Tue, Jan 22, 2013 at 3:23 PM, Tiziana Refice 
<>
 wrote:
> Adding Matt, who could say more about what changes are needed to move NDT
> from reno to cubic.
>
>
> On Tue, Jan 22, 2013 at 3:13 PM, Richard Sanger 
> <>
> wrote:
>>
>> Hi Chris,
>>
>> Good to here you got this compiling, yeah the I2util is a bit of a pain.
>>
>> As for requiring reno this is not a KIS Web10G/Web100 requirement, rather
>> to do with the middlebox/ndt test itself.
>>
>> http://code.google.com/p/ndt/wiki/NDTTestMethodology - under
>> know-issues/limitations
>>
>> However cubic could be a perfectly fine substitute. I don't know too much
>> about the differences reno vs cubic myself nor why reno is a requirement of
>> ndt tests. If someone who knows a bit more about this could weigh in it
>> would be appreciated.
>>
>> As for porting process, I'll try do a write up when I get some time -
>> since its probably useful for everyone.
>>
>> Cheers,
>> Richard
>>
>>
>> On Wed, Jan 23, 2013 at 11:34 AM, rapier 
>> <>
>>  wrote:
>>>
>>> Hey, I got it to build. I was making a stupid mistake at first by copying
>>> over I2util before I ran autoreconf. It seems that following the
>>> instructions is useful. Anyway, I'm going to start running some tests soon
>>> by comparing results between web100 and web10g ndt servers.
>>>
>>> Hopefully the warning about requiring reno for accurate results isn't
>>> entirely true since we've extended the KIS to support BIC, CUBIC, and 
>>> H-TCP.
>>>
>>> I'll let you know more after I get some data together.
>>>
>>> Thanks again for doing this, you have no idea how much time you saved us
>>> on this.
>>>
>>> As for Aaron's request about splitting up the patch. That is, admittedly,
>>> a pain in the butt. If you can do that it would help with the adoption 
>>> quite
>>> a bit though. Our end goal is to have this be a central part of the
>>> perfSonar Web10G deployment (which will eventually obsolete the Web100
>>> deployment that is the base now). So if nothing else you'll know that the
>>> work you did is being used by the community.
>>>
>>> Also, when you have time I may want to talk to you a bit about the
>>> porting process. I'm the project leader (but not developer) and we're 
>>> really
>>> trying to get an idea of what it's like using the new API.
>>>
>>> Thanks again!
>>>
>>> Chris
>>>
>>>
>>>
>>> On 1/22/13 12:16 AM, Richard Sanger wrote:
>>>>
>>>> Or there is a .tar of the web10G version here
>>>> https://docs.google.com/folder/d/0B-PiO7DvhNkJUDNwcTZjVGtJNmc/edit with
>>>> the patch already applied.
>>>>
>>>>
>>>> On Tue, Jan 22, 2013 at 5:50 PM, Richard Sanger 
>>>> <
>>>> <mailto:>>
>>>>  wrote:
>>>>
>>>>
>>>>     The patch is for the current svn version being r796 retrieved from
>>>>     here https://code.google.com/p/ndt/ .
>>>>     Make sure you run autoreconf (automake/autoconf) to build new
>>>>     configuration files, these have been changed. Then remember to copy
>>>>     I2Util library (From the 3.5.6.1 release) into the ndt/ folder
>>>>     before running ./configure. If you have the web10g/web100 libraries
>>>>     installed it should build the corresponding versions of the server.
>>>>
>>>>     Cheers,
>>>>
>>>>
>>>>     On Tue, Jan 22, 2013 at 11:01 AM, rapier 
>>>> <
>>>>     
>>>> <mailto:>>
>>>>  wrote:
>>>>
>>>>         Quick question - which version of NDT did you build the patch
>>>>         against? Applying it against 3.5.6.1 (the latest I've been able
>>>>         to find) is producing a lot of failures.
>>>>
>>>>         Thanks!
>>>>
>>>>
>>>>
>>>>         On 1/17/13 1:16 AM, Richard Sanger wrote:
>>>>
>>>>             Hi,
>>>>
>>>>             As part of an internship at WAND, I've been working on a
>>>>             web10g version
>>>>             of ndt.
>>>>
>>>>             The patch is attached.
>>>>             I've also got some Web10g patched linux kernels for
>>>>             Debian(squeeze or
>>>>             wheezy should work) here.
>>>>
>>>> https://docs.google.com/__folder/d/0B-__PiO7DvhNkJUDNwcTZjVGtJNmc/edit
>>>>
>>>>
>>>> <https://docs.google.com/folder/d/0B-PiO7DvhNkJUDNwcTZjVGtJNmc/edit>
>>>>
>>>>             NOTE: The kernel needs a slight modification to the Web10g
>>>>             kernel patch.
>>>>             Which is included in the Google drive folder above.
>>>>
>>>>             I've incorporated Web10G into the code in such a way that
>>>>             both the
>>>>             web100srv and web10g can be built. Besides some bug fixes
>>>>             the web100srv
>>>>             is unchanged. The configuration process should detect web100
>>>>             and web10g
>>>>             libraries and build accordingly. Web10G variable logging is
>>>> not
>>>>             implemented - this shouldn't have any effects on the
>>>> results.
>>>>
>>>>             For the most part the client hasn't been changed, so an old
>>>>             client can
>>>>             connect without issues. The only things that could go wrong
>>>>             with an old
>>>>             client are TCP Timestamps/ECN may be incorrectly detected as
>>>> on.
>>>>
>>>>             This patch also includes a few fixes for both clients and
>>>>             servers,
>>>>             including those I logged as issues #69 #70 #71 #73.
>>>>
>>>>             Along with web10g this adds code in ndt/src/kernel_* files
>>>>             which will
>>>>             check some of the kernel configuration for a Web10G server
>>>>             and print
>>>>             warnings about incorrect configuration.(If the server is run
>>>>             with -k
>>>>             option these will be corrected).
>>>>
>>>>             Look at ndt/WEB10G README for more information.
>>>>
>>>>             I would be great to get this incorporated into the ndt trunk
>>>> (or
>>>>             branch). Although the Web10G code still needs more testing
>>>>             the web100
>>>>             parts of the code are unchanged.
>>>>
>>>>             Send me an email if you have any issues all feedback is
>>>> welcome.
>>>>
>>>>             Cheers,
>>>>             Richard
>>>>
>>>>
>>>>             _________________________________________________
>>>>             Web10g-user mailing list
>>>>             
>>>> 
>>>>  
>>>> <mailto:>
>>>>             https://lists.psc.edu/mailman/__listinfo/web10g-user
>>>>
>>>>             <https://lists.psc.edu/mailman/listinfo/web10g-user>
>>>>
>>>>             To UNSUBSCRIBE visit
>>>>             https://lists.psc.edu/mailman/__unsubscribe/web10g-user
>>>>             <https://lists.psc.edu/mailman/unsubscribe/web10g-user>
>>>>
>>>>
>>>>
>>
>