NDT-DEV email list created

[Aaron Brown <>]

Hey Jakub,

I put in some comments on various commits in the MultiplePorts branch, some 
minor and some major.

Cheers,
Aaron

> On May 4, 2015, at 4:01 AM, Jakub Slawinski 
> <>
>  wrote:
> 
> 
> Hi,
> 
> regarding the 10 seconds limit of the test, this has been already
> addressed in the MultiplePorts branch.
> 
> This branch will be merged into master by the next couple of days, so
> the next release of NDT should support configurable tests duration and
> the possibility to use multiple ports during the test.
> 
> 
> Regards,
>  Jakub.
> 
> On 05/01/2015 10:21 PM, Aaron Brown wrote:
>> Hi Jacques,
>> 
>> Comments inline
>> 
>> Cheers,
>> Aaron
>> 
>> On Apr 29, 2015, at 11:02 PM, Jacques Latour 
>> <<mailto:>>
>>  wrote:
>> 
>> Hi All!
>> 
>> As some of you may know, .ca (CIRA) is planning to publicly launch our 
>> Internet Performance Test site (http://cira.ca/performance and 
>> http://performance.cira.ca<http://performance.cira.ca/>) next week, it's 
>> based on the Flash NDT client, we're using the latest and greatest. We're 
>> really excited!!
>> 
>> Excellent, I’m glad you’re able to use it :)
>> 
>> Right now, we're in beta testing until May 5.
>> 
>> One thing that's bugging me is that we get different performance results 
>> based on different buffer size and browsers in a controlled environment.  
>> It seems that we would get better performance results if the flash client 
>> uses a bigger buffer, 32K seems to work everywhere, 48K likely candidate 
>> for us, and 64K seems to be correct but crashes in Chrome sometimes.  
>> (what's the science behind this number?)
>> 
>> The 8K buffer stems from the initial testing and diagnostics that were 
>> done way back when (over 10 years ago at this point…). The buffer can be 
>> modified as needed, and it shouldn’t pose a problem, though some of the 
>> misc. diagnostics (e.g. bottleneck link or what-have-you) may be less 
>> accurate. Given the advances in networks since then, some of those 
>> algorithms may produce inaccurate results anyway.
>> 
>> The second thing, the data in the buffer runs from 00 --> FF, so 
>> repetitive and compressible (if that's a word :), is that a problem?
>> 
>> It shouldn’t be a problem unless routers are being setup to compress 
>> random IP packets :) I’d imagine this may end up being more problematic 
>> with transparent proxies and WebSocket support, however.
>> 
>> So, by reading the C2S spec, it says the following; "must use 8K buffer" 
>> but other than with the java client, we get better results for larger 
>> buffer size.  See table below and attached document for results.
>> 
>> 
>> The NDT Client in the C2S Throughput test MUST use a 8192 Byte buffer to 
>> send the packets through the newly opened connection as fast as possible 
>> (i.e. without any delays) for 10 seconds. The content of the buffer SHOULD 
>> be initialized a single time and sent repeatedly. The contents of the 
>> buffer SHOULD avoid repeating content (to avoid any automatic compression 
>> mechanisms) and MUST include only US-ASCII printable characters.
>> 
>>        Chrome  Firefox IE
>> BUFFER  DOWNLOAD        UPLOAD  DOWNLOAD        UPLOAD  DOWNLOAD        
>> UPLOAD
>> 8K      27.9 Mbps       6.3 Mbps        26.8 Mbps       4.3 Mbps        
>> 26.2 Mbps       4.2 Mbps
>> 8K      26.2 Mbps       6.2 Mbps        26.6 Mbps       4.1 Mbps        
>> 27.5 Mbps       4.1 Mbps
>> 16K     29.6 Mbps       10.0 Mbps       28.9 Mbps       6.9 Mbps        
>> 23.2 Mbps       6.6 Mbps
>> 16K     27.3 Mbps       9.6 Mbps        22.3 Mbps       6.8 Mbps        
>> 23.7 Mbps       7.0 Mbps
>> 32K     28.5 Mbps       10.0 Mbps       24.8 Mbps       8.7 Mbps        
>> 25.5 Mbps       8.9 Mbps
>> 32K     29.7 Mbps       9.8 Mbps        26.7 Mbps       9.0 Mbps        
>> 28.5 Mbps       8.9 Mbps
>> 48K     28.0 Mbps       10.3 Mbps       28.8 Mbps       9.0 Mbps        
>> 25.8 Mbps       9.4 Mbps
>> 48K     29.7 Mbps       9.7 Mbps        23.1 Mbps       9.4 Mbps        
>> 26.8 Mbps       9.5 Mbps
>> 64K     25.7 Mbps       9.9 Mbps        23.7 Mbps       9.5 Mbps        
>> 21.3 Mbps       9.5 Mbps
>> 64K     28.1 Mbps       10.0 Mbps       28.7 Mbps       9.6 Mbps        
>> 26.3 Mbps       9.9 Mbps
>> Tests: Videotron FIBE 30 Download / 10 Upload
>> 
>> What about this known limitation?  Can we run 15 seconds?
>> 
>> (https://code.google.com/p/ndt/wiki/NDTTestMethodology)
>> Known Limitations (Client-To-Server Throughput Test)
>> A 10 second test may not be enough time for TCP to reach a steady-state on 
>> a high bandwidth, high latency link.
>> 
>> Let me know what you think, any insight, wisdom, assistance would be 
>> appreciated!
>> 
>> There are a number of hard coded references to the ’10 second’ aspect of 
>> the test, unfortunately. Also, since there isn’t a negotiation about test 
>> length, existing clients will only try to perform 10 second tests. 
>> However, if you go through the source and fix those hard coded references 
>> (patches please :)), and either control all the clients testing to you (so 
>> they all assume a new test length), or put some changes in to the protocol 
>> to allow a test length negotiation (again, patches please :)), it should 
>> work.
>> 
>> 
>> 
>> Thanks!
>> 
>> Jack
>> 
>> 
>> <NDT Testing Buffer Size.docx>
>> 
>>