NDT-DEV email list created

[Jakub Sławiński <>]

Hi,

is there any progress regarding new NDT release? What has left to be
done and how we can help to make it happen?


Regards,
  Jakub.

On 07/11/2014 03:15 AM, Richard Carlson wrote:
> Will;
> 
> The approach you describe sounds reasonable.  I would be happy to help
> test things out once you have them ready.
> 
> Rich
> 
> On 07/09/2014 08:53 PM, Will Hawkins wrote:
>> Mr. Carlson,
>>
>> Thank you for adding your insight on this. I certainly appreciate having
>> the chance to work with people as respected in the measurement world as
>> you and Matt.
>>
>> I agree that the goal should be not to tell the user that they are
>> incapable of choosing a good browser. In fact, I don't actually like
>> Chrome (shh, don't tell anyone!). I also agree that, in the case where
>> the user is running a "bad" browser/os combination, the measurements
>> made with this Flash client will be indicative of general performance on
>> their connection when running Flash applications (with the caveat that
>> HTTP-based network i/o is significantly faster than socket-based i/o
>> because it is implemented better by the runtime across platforms). As a
>> final note of agreement, I concur that NDT is more than just a "speed
>> test".
>>
>> The way that we are going to actually implement this is with a series of
>> configurable knobs. The person embedding the Flash client in their HTML
>> will have the option to treat an incompatible browser/os combination as
>> an error or a warning. They deployer will get to handle that condition
>> in a callback and display an appropriate message (so that they could say
>> something in "big bold letters", which I liked!).
>>
>> If that type of scenario is acceptable to you, the decision is more
>> about the appropriate default (error or warning). We are close to
>> finishing up an implementation. Would you be willing to review that when
>> it's ready and see if it meets your expectations?
>>
>> Thanks again for adding your thoughts! I was hoping that you'd weigh in.
>> We have Matt M on a separate thread adding his thoughts as well so I am
>> confident that we will come to a solid resolution.
>>
>> Will
>>
>> On 07/09/2014 08:40 PM, Richard Carlson wrote:
>>> Sebastian/Will;
>>>
>>> I'm not sure I agree with this idea.  While I agree that letting the
>>> user know that the test results are bad due to the Flash runtime
>>> problem, I'm not at all convinced that the proper response is to say
>>> 'use Chrome'.  It just seems a bit strong to me, like you are saying the
>>> user is too stupid to run a good browser.
>>>
>>> I'd prefer to let the test run and put in big bold letters across the
>>> results page or in some other manner indicate that the limiting factor
>>> is the client's flash runtime.  That way the user is being informed and
>>> the logged test results can also contain the same notation so they can
>>> be discarded if necessary.
>>>
>>> After all, they will get that performance when running other flash apps,
>>> so it would be good for them to know what the limits are.
>>>
>>> The whole purpose of the NDT system is to provide advanced diagnostic
>>> messages, not just run speed tests.  Showing the user that the test
>>> results are limited and explaining why would be much better than telling
>>> them to 'trust me and replace your browser's flash runtime'.
>>>
>>> So for me, the real question is can you make a simple and intuitive
>>> image or text that clearly shows that the problem is the flash runtime
>>> and not the NDT service.
>>>
>>> Rich
>>>
>>> On 07/08/2014 03:09 AM, Sebastian Kostuch wrote:
>>>> Hi Will,
>>>> IMHO it would be good to disable possibility of running test when
>>>> we detect that flash client is being run from within not supported
>>>> platform. With this there won't be faked results noted on server
>>>> side's statistics.
>>>> Also looking forward to any further information
>>>> regarding new release :).
>>>>
>>>> Regards
>>>> Sebastian
>>>>
>>>> On 03.07.2014 02:40, Will Hawkins wrote:
>>>>> Sebastian and Aaron (and everyone, really),
>>>>>
>>>>> We at MLab are in the final throes of getting our platform ready
>>>>> for the
>>>>> Flash client. In so doing, we've gotten deep into the weeds on the
>>>>> performance/measurement implications of the Flash runtime.
>>>>>
>>>>> I wanted to present you with our findings and get your input.
>>>>>
>>>>> On all platforms (Windows, Mac, Linux), running the Flash client in
>>>>> Google Chrome works a-ok. On Windows, running the Flash client in all
>>>>> other browsers work great too. On Linux and Mac, however, that is not
>>>>> the case.
>>>>>
>>>>> On Mac and Linux where the user runs the NDT Flash client under
>>>>> Adobe's
>>>>> runtime (i.e., browsers other than Chrome [even Chromium does not
>>>>> work]), there is not enough network i/o fidelity to produce
>>>>> measurements
>>>>> for connections with > ~ 50mbps throughput (on the s2c side).
>>>>>
>>>>> In other words, Google's independent implementation of the Flash
>>>>> runtime
>>>>> performs well enough on all platforms, but Adobe's implementation
>>>>> performs well enough only on Windows.
>>>>>
>>>>> Because MLab collects lots of data, we don't want our results to be
>>>>> spoiled by tests whose results are artificially limited. We are
>>>>> attempting to work out a way to "encourage" users to run the test
>>>>> in the
>>>>> environment where we will get the most accurate results. To do so,
>>>>> we've
>>>>> settled on the following plan:
>>>>>
>>>>> 1.  When the flash client detects that it is being run within a
>>>>> non-optimal environment, display the following:  "Unfortunately, this
>>>>> test cannot run within your current browser environment.  Please run
>>>>> the test from within Google Chrome, or download M-Lab's Firefox
>>>>> Extension."
>>>>>
>>>>> 2.  Disallow the flash client from running on any platform that
>>>>> identifies itself as something other than Windows / Chrome.
>>>>>
>>>>> What do people think about this plan? Reasonable?
>>>>>
>>>>> As always, it's great working with everyone in NDT land. I look
>>>>> forward
>>>>> to continuing the work :-)
>>>>>
>>>>> Will
>>>>>
>>>>> On 06/18/2014 03:06 AM, Sebastian Kostuch wrote:
>>>>>> Hi Will
>>>>>> indeed these changes results in higher speeds (I was able to get ~760
>>>>>> mbps
>>>>>> which is quite similar to yours). Also I have checked flash client as
>>>>>> background
>>>>>> for new JS UI and both speeds are being updated correctly in
>>>>>> real-time.
>>>>>> This is
>>>>>> really great work! Thanks for this. I have merged these changes into
>>>>>> trunk so
>>>>>> they would be also present in new release.
>>>>>> Also as you probably have noticed I have fixed compilation error when
>>>>>> using
>>>>>> web100 so I closed appropriated issue related to this.
>>>>>>
>>>>>> Regards
>>>>>> Sebastian
>>>>>>
>>>>>> On 17.06.2014 22:52, Will Hawkins wrote:
>>>>>>> Hello Sebastian and Aaron (and MLab'ers CC'd),
>>>>>>>
>>>>>>> We found a way to juice the Flash client and achieve the performance
>>>>>>> necessary to accurately measure high-speed network connections.
>>>>>>>
>>>>>>> The change is in r1082.
>>>>>>>
>>>>>>> The solution was counterintuitive (at least for me). Instead of
>>>>>>> attempting to read data from the socket as quickly as possible, the
>>>>>>> TestS2C class just gets out of the way for the duration of the
>>>>>>> test. It
>>>>>>> registers only for the "socket close" event. The runtime has the
>>>>>>> necessary speed to keep up with the network.
>>>>>>>
>>>>>>> The only problem with this method is keeping the user interface
>>>>>>> up-to-date on progress throughout the test. Sebastian, this is
>>>>>>> where I
>>>>>>> need you to double check. I believe that the code in r1082 properly
>>>>>>> updates _s2cByteCount in the appropriate spot for those real-time
>>>>>>> updates (specifically onSpeedUpdate). But, please double check!
>>>>>>>
>>>>>>> We were able to measure a connection at ~ 775 mbps using the Flash
>>>>>>> client, but it would be great to have as many people test this as
>>>>>>> possible. To do so, you can use a pre-compiled version at
>>>>>>>
>>>>>>> http://files.opentechinstitute.org/~hawkinsw/flash/test.html
>>>>>>>
>>>>>>> On a separate, but related note, please also review r1083 for
>>>>>>> inclusion
>>>>>>> in the NDT release. This change allows the client to re-query JS for
>>>>>>> the
>>>>>>> test's hostname just before launching a test. This gives the wrapper
>>>>>>> page the opportunity to do a AJAX query to determine the test's
>>>>>>> hostname. This is important for MLab because we do dynamic server
>>>>>>> selection using a web-based API.
>>>>>>>
>>>>>>> I am eager to hear whether you have success with these changes or
>>>>>>> not. I
>>>>>>> hope you do!
>>>>>>>
>>>>>>> Will
>>>>>>>
>>>>>>> On 06/17/2014 10:53 AM, Will Hawkins wrote:
>>>>>>>> Just FYI: I believe that we have happened on to an actual solution
>>>>>>>> for
>>>>>>>> the "limit" to the performance of the S2C test. I am going to
>>>>>>>> implement
>>>>>>>> it today and do some testing.
>>>>>>>>
>>>>>>>> If possible, could we hold out on doing a release for a little
>>>>>>>> while
>>>>>>>> pending this investigation?
>>>>>>>>
>>>>>>>> Will
>>>>>>>>
>>>>>>>> On 06/16/2014 01:50 AM, Sebastian Kostuch wrote:
>>>>>>>>> Hi Will,
>>>>>>>>> what you mentioned seems reasonable. Would it be possible to merge
>>>>>>>>> these
>>>>>>>>> needed
>>>>>>>>> changes into trunk in nearest time so we could make new release
>>>>>>>>> for
>>>>>>>>> example tomorrow
>>>>>>>>> or the day after tomorrow?
>>>>>>>>>
>>>>>>>>> Regards
>>>>>>>>> Sebastian
>>>>>>>>>
>>>>>>>>> On 13.06.2014 17:36, Will Hawkins wrote:
>>>>>>>>>> Sebastian,
>>>>>>>>>>
>>>>>>>>>> Thanks for sending this out! We (at MLab) are very interested in
>>>>>>>>>> deploying a new version of the code so this 'release' looks
>>>>>>>>>> like a
>>>>>>>>>> good place to fit together! :-) In other words, we (again, MLab)
>>>>>>>>>> are
>>>>>>>>>> definitely going to do what you said in 3!
>>>>>>>>>>
>>>>>>>>>> I am out of the office, but I wanted to quickly respond to
>>>>>>>>>> point 2.
>>>>>>>>>> See below!
>>>>>>>>>>
>>>>>>>>>> Thanks again for driving this process forward!!
>>>>>>>>>>
>>>>>>>>>> Will
>>>>>>>>>>
>>>>>>>>>> On 6/13/14, 4:47 AM, Sebastian Kostuch wrote:
>>>>>>>>>>> Hello,
>>>>>>>>>>> NDT has been updated recently with some major changes (like
>>>>>>>>>>> MSG_EXTENDED_LOGIN,
>>>>>>>>>>> json support, new JS UI) so as I have mentioned in some earlier
>>>>>>>>>>> mail
>>>>>>>>>>> that would be probably
>>>>>>>>>>> good time to make new release. I have few questions related to
>>>>>>>>>>> this:
>>>>>>>>>>> 1. New JS UI would fit well within this new release, however
>>>>>>>>>>> changes
>>>>>>>>>>> related to this are still
>>>>>>>>>>> on work branches and need review. Any feedback would be very
>>>>>>>>>>> appreciated. These are
>>>>>>>>>>> following issues: 129, 132, 133.
>>>>>>>>>>> 2. Is Issue144 (URLLoader to flash) supposed to be contained
>>>>>>>>>>> also in
>>>>>>>>>>> nearest release version?
>>>>>>>>>>> If yes, then how does progress with work on this look like? If I
>>>>>>>>>>> recall
>>>>>>>>>>> correctly then you, Will,
>>>>>>>>>>> mentioned that they are well tested and thus almost ready,
>>>>>>>>>>> right?
>>>>>>>>>> Some of these are still up for debate. I am going to forward an
>>>>>>>>>> email
>>>>>>>>>> that I just sent to the MLab team with an update on the conundrum
>>>>>>>>>> around the HTTP test to give you a sense about why they are up
>>>>>>>>>> for
>>>>>>>>>> debate.
>>>>>>>>>>
>>>>>>>>>> But, there are also changes in that branch that *need* to be
>>>>>>>>>> incorporated into anything that gets released. For instance,
>>>>>>>>>> r1070.
>>>>>>>>>> There are parts of r1064 that will have to be integrated no
>>>>>>>>>> matter
>>>>>>>>>> what (any of the changes that have to do with splitting start
>>>>>>>>>> test and
>>>>>>>>>> finalize test stages).
>>>>>>>>>>
>>>>>>>>>> Does that make sense?
>>>>>>>>>>
>>>>>>>>>>> 3. Along with new release on googlecode what do you think about
>>>>>>>>>>> pushing
>>>>>>>>>>> these changes also
>>>>>>>>>>> on M-Lab site? They bring many new features at all.
>>>>>>>>>>>
>>>>>>>>>>> Looking forward to your answers and I hope that next week
>>>>>>>>>>> could be
>>>>>>>>>>> celebrated with making
>>>>>>>>>>> new release of NDT :).
>>>>>>>>>>>
>>>>>>>>>>> Kind regards
>>>>>>>>>>> Sebastian Kostuch
>>>>>>>>>>>
>>>>>>
>>>>
>>>>
>>