NDT-DEV email list created

[Richard Carlson <>]

Will;

The approach you describe sounds reasonable.  I would be happy to help 
test things out once you have them ready.

Rich

On 07/09/2014 08:53 PM, Will Hawkins wrote:
> Mr. Carlson,
>
> Thank you for adding your insight on this. I certainly appreciate having
> the chance to work with people as respected in the measurement world as
> you and Matt.
>
> I agree that the goal should be not to tell the user that they are
> incapable of choosing a good browser. In fact, I don't actually like
> Chrome (shh, don't tell anyone!). I also agree that, in the case where
> the user is running a "bad" browser/os combination, the measurements
> made with this Flash client will be indicative of general performance on
> their connection when running Flash applications (with the caveat that
> HTTP-based network i/o is significantly faster than socket-based i/o
> because it is implemented better by the runtime across platforms). As a
> final note of agreement, I concur that NDT is more than just a "speed
> test".
>
> The way that we are going to actually implement this is with a series of
> configurable knobs. The person embedding the Flash client in their HTML
> will have the option to treat an incompatible browser/os combination as
> an error or a warning. They deployer will get to handle that condition
> in a callback and display an appropriate message (so that they could say
> something in "big bold letters", which I liked!).
>
> If that type of scenario is acceptable to you, the decision is more
> about the appropriate default (error or warning). We are close to
> finishing up an implementation. Would you be willing to review that when
> it's ready and see if it meets your expectations?
>
> Thanks again for adding your thoughts! I was hoping that you'd weigh in.
> We have Matt M on a separate thread adding his thoughts as well so I am
> confident that we will come to a solid resolution.
>
> Will
>
> On 07/09/2014 08:40 PM, Richard Carlson wrote:
> > Sebastian/Will;
> >
> > I'm not sure I agree with this idea.  While I agree that letting the
> > user know that the test results are bad due to the Flash runtime
> > problem, I'm not at all convinced that the proper response is to say
> > 'use Chrome'.  It just seems a bit strong to me, like you are saying the
> > user is too stupid to run a good browser.
> >
> > I'd prefer to let the test run and put in big bold letters across the
> > results page or in some other manner indicate that the limiting factor
> > is the client's flash runtime.  That way the user is being informed and
> > the logged test results can also contain the same notation so they can
> > be discarded if necessary.
> >
> > After all, they will get that performance when running other flash apps,
> > so it would be good for them to know what the limits are.
> >
> > The whole purpose of the NDT system is to provide advanced diagnostic
> > messages, not just run speed tests.  Showing the user that the test
> > results are limited and explaining why would be much better than telling
> > them to 'trust me and replace your browser's flash runtime'.
> >
> > So for me, the real question is can you make a simple and intuitive
> > image or text that clearly shows that the problem is the flash runtime
> > and not the NDT service.
> >
> > Rich
> >
> > On 07/08/2014 03:09 AM, Sebastian Kostuch wrote:
> > > Hi Will,
> > > IMHO it would be good to disable possibility of running test when
> > > we detect that flash client is being run from within not supported
> > > platform. With this there won't be faked results noted on server
> > > side's statistics.
> > > Also looking forward to any further information
> > > regarding new release :).
> > >
> > > Regards
> > > Sebastian
> > >
> > > On 03.07.2014 02:40, Will Hawkins wrote:
> > > > Sebastian and Aaron (and everyone, really),
> > > >
> > > > We at MLab are in the final throes of getting our platform ready for the
> > > > Flash client. In so doing, we've gotten deep into the weeds on the
> > > > performance/measurement implications of the Flash runtime.
> > > >
> > > > I wanted to present you with our findings and get your input.
> > > >
> > > > On all platforms (Windows, Mac, Linux), running the Flash client in
> > > > Google Chrome works a-ok. On Windows, running the Flash client in all
> > > > other browsers work great too. On Linux and Mac, however, that is not
> > > > the case.
> > > >
> > > > On Mac and Linux where the user runs the NDT Flash client under Adobe's
> > > > runtime (i.e., browsers other than Chrome [even Chromium does not
> > > > work]), there is not enough network i/o fidelity to produce measurements
> > > > for connections with > ~ 50mbps throughput (on the s2c side).
> > > >
> > > > In other words, Google's independent implementation of the Flash runtime
> > > > performs well enough on all platforms, but Adobe's implementation
> > > > performs well enough only on Windows.
> > > >
> > > > Because MLab collects lots of data, we don't want our results to be
> > > > spoiled by tests whose results are artificially limited. We are
> > > > attempting to work out a way to "encourage" users to run the test in the
> > > > environment where we will get the most accurate results. To do so, we've
> > > > settled on the following plan:
> > > >
> > > > 1.  When the flash client detects that it is being run within a
> > > > non-optimal environment, display the following:  "Unfortunately, this
> > > > test cannot run within your current browser environment.  Please run
> > > > the test from within Google Chrome, or download M-Lab's Firefox
> > > > Extension."
> > > >
> > > > 2.  Disallow the flash client from running on any platform that
> > > > identifies itself as something other than Windows / Chrome.
> > > >
> > > > What do people think about this plan? Reasonable?
> > > >
> > > > As always, it's great working with everyone in NDT land. I look forward
> > > > to continuing the work :-)
> > > >
> > > > Will
> > > >
> > > > On 06/18/2014 03:06 AM, Sebastian Kostuch wrote:
> > > > > Hi Will
> > > > > indeed these changes results in higher speeds (I was able to get ~760
> > > > > mbps
> > > > > which is quite similar to yours). Also I have checked flash client as
> > > > > background
> > > > > for new JS UI and both speeds are being updated correctly in real-time.
> > > > > This is
> > > > > really great work! Thanks for this. I have merged these changes into
> > > > > trunk so
> > > > > they would be also present in new release.
> > > > > Also as you probably have noticed I have fixed compilation error when
> > > > > using
> > > > > web100 so I closed appropriated issue related to this.
> > > > >
> > > > > Regards
> > > > > Sebastian
> > > > >
> > > > > On 17.06.2014 22:52, Will Hawkins wrote:
> > > > > > Hello Sebastian and Aaron (and MLab'ers CC'd),
> > > > > >
> > > > > > We found a way to juice the Flash client and achieve the performance
> > > > > > necessary to accurately measure high-speed network connections.
> > > > > >
> > > > > > The change is in r1082.
> > > > > >
> > > > > > The solution was counterintuitive (at least for me). Instead of
> > > > > > attempting to read data from the socket as quickly as possible, the
> > > > > > TestS2C class just gets out of the way for the duration of the
> > > > > > test. It
> > > > > > registers only for the "socket close" event. The runtime has the
> > > > > > necessary speed to keep up with the network.
> > > > > >
> > > > > > The only problem with this method is keeping the user interface
> > > > > > up-to-date on progress throughout the test. Sebastian, this is where I
> > > > > > need you to double check. I believe that the code in r1082 properly
> > > > > > updates _s2cByteCount in the appropriate spot for those real-time
> > > > > > updates (specifically onSpeedUpdate). But, please double check!
> > > > > >
> > > > > > We were able to measure a connection at ~ 775 mbps using the Flash
> > > > > > client, but it would be great to have as many people test this as
> > > > > > possible. To do so, you can use a pre-compiled version at
> > > > > >
> > > > > > http://files.opentechinstitute.org/~hawkinsw/flash/test.html
> > > > > >
> > > > > > On a separate, but related note, please also review r1083 for
> > > > > > inclusion
> > > > > > in the NDT release. This change allows the client to re-query JS for
> > > > > > the
> > > > > > test's hostname just before launching a test. This gives the wrapper
> > > > > > page the opportunity to do a AJAX query to determine the test's
> > > > > > hostname. This is important for MLab because we do dynamic server
> > > > > > selection using a web-based API.
> > > > > >
> > > > > > I am eager to hear whether you have success with these changes or
> > > > > > not. I
> > > > > > hope you do!
> > > > > >
> > > > > > Will
> > > > > >
> > > > > > On 06/17/2014 10:53 AM, Will Hawkins wrote:
> > > > > > > Just FYI: I believe that we have happened on to an actual solution
> > > > > > > for
> > > > > > > the "limit" to the performance of the S2C test. I am going to
> > > > > > > implement
> > > > > > > it today and do some testing.
> > > > > > >
> > > > > > > If possible, could we hold out on doing a release for a little while
> > > > > > > pending this investigation?
> > > > > > >
> > > > > > > Will
> > > > > > >
> > > > > > > On 06/16/2014 01:50 AM, Sebastian Kostuch wrote:
> > > > > > > > Hi Will,
> > > > > > > > what you mentioned seems reasonable. Would it be possible to merge
> > > > > > > > these
> > > > > > > > needed
> > > > > > > > changes into trunk in nearest time so we could make new release for
> > > > > > > > example tomorrow
> > > > > > > > or the day after tomorrow?
> > > > > > > >
> > > > > > > > Regards
> > > > > > > > Sebastian
> > > > > > > >
> > > > > > > > On 13.06.2014 17:36, Will Hawkins wrote:
> > > > > > > > > Sebastian,
> > > > > > > > >
> > > > > > > > > Thanks for sending this out! We (at MLab) are very interested in
> > > > > > > > > deploying a new version of the code so this 'release' looks like a
> > > > > > > > > good place to fit together! :-) In other words, we (again, MLab)
> > > > > > > > > are
> > > > > > > > > definitely going to do what you said in 3!
> > > > > > > > >
> > > > > > > > > I am out of the office, but I wanted to quickly respond to point 2.
> > > > > > > > > See below!
> > > > > > > > >
> > > > > > > > > Thanks again for driving this process forward!!
> > > > > > > > >
> > > > > > > > > Will
> > > > > > > > >
> > > > > > > > > On 6/13/14, 4:47 AM, Sebastian Kostuch wrote:
> > > > > > > > > > Hello,
> > > > > > > > > > NDT has been updated recently with some major changes (like
> > > > > > > > > > MSG_EXTENDED_LOGIN,
> > > > > > > > > > json support, new JS UI) so as I have mentioned in some earlier
> > > > > > > > > > mail
> > > > > > > > > > that would be probably
> > > > > > > > > > good time to make new release. I have few questions related to
> > > > > > > > > > this:
> > > > > > > > > > 1. New JS UI would fit well within this new release, however
> > > > > > > > > > changes
> > > > > > > > > > related to this are still
> > > > > > > > > > on work branches and need review. Any feedback would be very
> > > > > > > > > > appreciated. These are
> > > > > > > > > > following issues: 129, 132, 133.
> > > > > > > > > > 2. Is Issue144 (URLLoader to flash) supposed to be contained
> > > > > > > > > > also in
> > > > > > > > > > nearest release version?
> > > > > > > > > > If yes, then how does progress with work on this look like? If I
> > > > > > > > > > recall
> > > > > > > > > > correctly then you, Will,
> > > > > > > > > > mentioned that they are well tested and thus almost ready, right?
> > > > > > > > > Some of these are still up for debate. I am going to forward an
> > > > > > > > > email
> > > > > > > > > that I just sent to the MLab team with an update on the conundrum
> > > > > > > > > around the HTTP test to give you a sense about why they are up for
> > > > > > > > > debate.
> > > > > > > > >
> > > > > > > > > But, there are also changes in that branch that *need* to be
> > > > > > > > > incorporated into anything that gets released. For instance, r1070.
> > > > > > > > > There are parts of r1064 that will have to be integrated no matter
> > > > > > > > > what (any of the changes that have to do with splitting start
> > > > > > > > > test and
> > > > > > > > > finalize test stages).
> > > > > > > > >
> > > > > > > > > Does that make sense?
> > > > > > > > >
> > > > > > > > > > 3. Along with new release on googlecode what do you think about
> > > > > > > > > > pushing
> > > > > > > > > > these changes also
> > > > > > > > > > on M-Lab site? They bring many new features at all.
> > > > > > > > > >
> > > > > > > > > > Looking forward to your answers and I hope that next week could be
> > > > > > > > > > celebrated with making
> > > > > > > > > > new release of NDT :).
> > > > > > > > > >
> > > > > > > > > > Kind regards
> > > > > > > > > > Sebastian Kostuch