NDT-DEV email list created

[Will Hawkins <>]

Sebastian and Aaron (and everyone, really),

We at MLab are in the final throes of getting our platform ready for the
Flash client. In so doing, we've gotten deep into the weeds on the
performance/measurement implications of the Flash runtime.

I wanted to present you with our findings and get your input.

On all platforms (Windows, Mac, Linux), running the Flash client in
Google Chrome works a-ok. On Windows, running the Flash client in all
other browsers work great too. On Linux and Mac, however, that is not
the case.

On Mac and Linux where the user runs the NDT Flash client under Adobe's
runtime (i.e., browsers other than Chrome [even Chromium does not
work]), there is not enough network i/o fidelity to produce measurements
for connections with > ~ 50mbps throughput (on the s2c side).

In other words, Google's independent implementation of the Flash runtime
performs well enough on all platforms, but Adobe's implementation
performs well enough only on Windows.

Because MLab collects lots of data, we don't want our results to be
spoiled by tests whose results are artificially limited. We are
attempting to work out a way to "encourage" users to run the test in the
environment where we will get the most accurate results. To do so, we've
settled on the following plan:

1.  When the flash client detects that it is being run within a
non-optimal environment, display the following:  "Unfortunately, this
test cannot run within your current browser environment.  Please run
the test from within Google Chrome, or download M-Lab's Firefox
Extension."

2.  Disallow the flash client from running on any platform that
identifies itself as something other than Windows / Chrome.

What do people think about this plan? Reasonable?

As always, it's great working with everyone in NDT land. I look forward
to continuing the work :-)

Will

On 06/18/2014 03:06 AM, Sebastian Kostuch wrote:
> Hi Will
> indeed these changes results in higher speeds (I was able to get ~760 mbps
> which is quite similar to yours). Also I have checked flash client as
> background
> for new JS UI and both speeds are being updated correctly in real-time.
> This is
> really great work! Thanks for this. I have merged these changes into
> trunk so
> they would be also present in new release.
> Also as you probably have noticed I have fixed compilation error when using
> web100 so I closed appropriated issue related to this.
> 
> Regards
> Sebastian
> 
> On 17.06.2014 22:52, Will Hawkins wrote:
>> Hello Sebastian and Aaron (and MLab'ers CC'd),
>>
>> We found a way to juice the Flash client and achieve the performance
>> necessary to accurately measure high-speed network connections.
>>
>> The change is in r1082.
>>
>> The solution was counterintuitive (at least for me). Instead of
>> attempting to read data from the socket as quickly as possible, the
>> TestS2C class just gets out of the way for the duration of the test. It
>> registers only for the "socket close" event. The runtime has the
>> necessary speed to keep up with the network.
>>
>> The only problem with this method is keeping the user interface
>> up-to-date on progress throughout the test. Sebastian, this is where I
>> need you to double check. I believe that the code in r1082 properly
>> updates _s2cByteCount in the appropriate spot for those real-time
>> updates (specifically onSpeedUpdate). But, please double check!
>>
>> We were able to measure a connection at ~ 775 mbps using the Flash
>> client, but it would be great to have as many people test this as
>> possible. To do so, you can use a pre-compiled version at
>>
>> http://files.opentechinstitute.org/~hawkinsw/flash/test.html
>>
>> On a separate, but related note, please also review r1083 for inclusion
>> in the NDT release. This change allows the client to re-query JS for the
>> test's hostname just before launching a test. This gives the wrapper
>> page the opportunity to do a AJAX query to determine the test's
>> hostname. This is important for MLab because we do dynamic server
>> selection using a web-based API.
>>
>> I am eager to hear whether you have success with these changes or not. I
>> hope you do!
>>
>> Will
>>
>> On 06/17/2014 10:53 AM, Will Hawkins wrote:
>>> Just FYI: I believe that we have happened on to an actual solution for
>>> the "limit" to the performance of the S2C test. I am going to implement
>>> it today and do some testing.
>>>
>>> If possible, could we hold out on doing a release for a little while
>>> pending this investigation?
>>>
>>> Will
>>>
>>> On 06/16/2014 01:50 AM, Sebastian Kostuch wrote:
>>>> Hi Will,
>>>> what you mentioned seems reasonable. Would it be possible to merge
>>>> these
>>>> needed
>>>> changes into trunk in nearest time so we could make new release for
>>>> example tomorrow
>>>> or the day after tomorrow?
>>>>
>>>> Regards
>>>> Sebastian
>>>>
>>>> On 13.06.2014 17:36, Will Hawkins wrote:
>>>>> Sebastian,
>>>>>
>>>>> Thanks for sending this out! We (at MLab) are very interested in
>>>>> deploying a new version of the code so this 'release' looks like a
>>>>> good place to fit together! :-) In other words, we (again, MLab) are
>>>>> definitely going to do what you said in 3!
>>>>>
>>>>> I am out of the office, but I wanted to quickly respond to point 2.
>>>>> See below!
>>>>>
>>>>> Thanks again for driving this process forward!!
>>>>>
>>>>> Will
>>>>>
>>>>> On 6/13/14, 4:47 AM, Sebastian Kostuch wrote:
>>>>>> Hello,
>>>>>> NDT has been updated recently with some major changes (like
>>>>>> MSG_EXTENDED_LOGIN,
>>>>>> json support, new JS UI) so as I have mentioned in some earlier mail
>>>>>> that would be probably
>>>>>> good time to make new release. I have few questions related to this:
>>>>>> 1. New JS UI would fit well within this new release, however changes
>>>>>> related to this are still
>>>>>> on work branches and need review. Any feedback would be very
>>>>>> appreciated. These are
>>>>>> following issues: 129, 132, 133.
>>>>>> 2. Is Issue144 (URLLoader to flash) supposed to be contained also in
>>>>>> nearest release version?
>>>>>> If yes, then how does progress with work on this look like? If I
>>>>>> recall
>>>>>> correctly then you, Will,
>>>>>> mentioned that they are well tested and thus almost ready, right?
>>>>> Some of these are still up for debate. I am going to forward an email
>>>>> that I just sent to the MLab team with an update on the conundrum
>>>>> around the HTTP test to give you a sense about why they are up for
>>>>> debate.
>>>>>
>>>>> But, there are also changes in that branch that *need* to be
>>>>> incorporated into anything that gets released. For instance, r1070.
>>>>> There are parts of r1064 that will have to be integrated no matter
>>>>> what (any of the changes that have to do with splitting start test and
>>>>> finalize test stages).
>>>>>
>>>>> Does that make sense?
>>>>>
>>>>>> 3. Along with new release on googlecode what do you think about
>>>>>> pushing
>>>>>> these changes also
>>>>>> on M-Lab site? They bring many new features at all.
>>>>>>
>>>>>> Looking forward to your answers and I hope that next week could be
>>>>>> celebrated with making
>>>>>> new release of NDT :).
>>>>>>
>>>>>> Kind regards
>>>>>> Sebastian Kostuch
>>>>>>
>>>>
> 
>