|
Hi Jacques,
Comments inline
Cheers,
Aaron
On Apr 29, 2015, at 11:02 PM, Jacques Latour <> wrote:
Hi All!
As some of you may know, .ca (CIRA) is planning to publicly launch our Internet Performance Test site (http://cira.ca/performance and http://performance.cira.ca)
next week, it's based on the Flash NDT client, we're using the latest and greatest. We're really excited!!
Excellent, I’m glad you’re able to use it :)
Right now, we're in beta testing until May 5.
One thing that's bugging me is that we get different performance results based on different buffer size and browsers in a controlled environment. It seems that we would get better performance results if the flash client uses a bigger buffer, 32K seems to work
everywhere, 48K likely candidate for us, and 64K seems to be correct but crashes in Chrome sometimes. (what's the science behind this number?)
The 8K buffer stems from the initial testing and diagnostics that were done way back when (over 10 years ago at this point…). The buffer can be modified as needed, and it shouldn’t pose a problem, though some of the misc. diagnostics (e.g. bottleneck link or
what-have-you) may be less accurate. Given the advances in networks since then, some of those algorithms may produce inaccurate results anyway.
The second thing, the data in the buffer runs from 00 --> FF, so repetitive and compressible (if that's a word :), is that a problem?
It shouldn’t be a problem unless routers are being setup to compress random IP packets :) I’d imagine this may end up being more problematic with transparent proxies and WebSocket support, however.
So, by reading the C2S spec, it says the following; "must use 8K buffer" but other than with the java client, we get better results for larger buffer size. See table below
and attached document for results.
The NDT Client in the C2S Throughput test MUST use a 8192 Byte buffer to send the packets through the newly opened connection as fast as possible (i.e. without any delays) for 10 seconds. The content of the buffer SHOULD be initialized
a single time and sent repeatedly. The contents of the buffer SHOULD avoid repeating content (to avoid any automatic compression mechanisms) and MUST include only US-ASCII printable characters.
|
Chrome |
Firefox |
IE |
| BUFFER |
DOWNLOAD |
UPLOAD |
DOWNLOAD |
UPLOAD |
DOWNLOAD |
UPLOAD |
| 8K |
27.9 Mbps |
6.3 Mbps |
26.8 Mbps |
4.3 Mbps |
26.2 Mbps |
4.2 Mbps |
| 8K |
26.2 Mbps |
6.2 Mbps |
26.6 Mbps |
4.1 Mbps |
27.5 Mbps |
4.1 Mbps |
| 16K |
29.6 Mbps |
10.0 Mbps |
28.9 Mbps |
6.9 Mbps |
23.2 Mbps |
6.6 Mbps |
| 16K |
27.3 Mbps |
9.6 Mbps |
22.3 Mbps |
6.8 Mbps |
23.7 Mbps |
7.0 Mbps |
| 32K |
28.5 Mbps |
10.0 Mbps |
24.8 Mbps |
8.7 Mbps |
25.5 Mbps |
8.9 Mbps |
| 32K |
29.7 Mbps |
9.8 Mbps |
26.7 Mbps |
9.0 Mbps |
28.5 Mbps |
8.9 Mbps |
| 48K |
28.0 Mbps |
10.3 Mbps |
28.8 Mbps |
9.0 Mbps |
25.8 Mbps |
9.4 Mbps |
| 48K |
29.7 Mbps |
9.7 Mbps |
23.1 Mbps |
9.4 Mbps |
26.8 Mbps |
9.5 Mbps |
| 64K |
25.7 Mbps |
9.9 Mbps |
23.7 Mbps |
9.5 Mbps |
21.3 Mbps |
9.5 Mbps |
| 64K |
28.1 Mbps |
10.0 Mbps |
28.7 Mbps |
9.6 Mbps |
26.3 Mbps |
9.9 Mbps |
Tests: Videotron FIBE 30 Download / 10 Upload
What about this known limitation? Can we run 15 seconds?
Known Limitations (Client-To-Server Throughput Test)
A 10 second test may not be enough time for TCP to reach a steady-state on a high bandwidth, high latency link.
Let me know what you think, any insight, wisdom, assistance would be appreciated!
There are a number of hard coded references to the ’10 second’ aspect of the test, unfortunately. Also, since there isn’t a negotiation about test length, existing clients will only try to perform 10 second tests. However, if you go through the source and fix
those hard coded references (patches please :)), and either control all the clients testing to you (so they all assume a new test length), or put some changes in to the protocol to allow a test length negotiation (again, patches please :)), it should work.
<NDT Testing Buffer Size.docx>
|
