NDT-DEV email list created

[Aaron Brown <>]

Hey Jakub,

On Jul 22, 2014, at 10:52 AM, Jakub Sławiński 
<>
 wrote:

> 
> Hi,
> 
> I think we will be able to help with the mentioned issues.
> 
> Sebastian, can we already swap between flash and java backend on the
> html level? Moreover, can we help with packaging up RPMs?

If you could create RPMs for the flash components, that’d be great. There’s a 
spec file in the trunk of the repository that you can add an additional flash 
client component to.

Cheers,
Aaron

> 
> 
> Regards,
>  Jakub.
> 
> On 07/21/2014 04:58 PM, Aaron Brown wrote:
>> Hey Jakub,
>> 
>> The big open part is getting the flash client packaged so that an admin 
>> can do a “yum install [package]”, and have it just work. Among other 
>> things, it’ll need to package up something to serve the default flash 
>> socket xml stuff, and include a way, in the HTML, to swap between the 
>> flash or java applet as the ‘backend’ of the javascript so folks can run 
>> either. If you guys can work on packaging up RPMs, that’d be great.
>> 
>> Cheers,
>> Aaron
>> 
>> On Jul 21, 2014, at 10:48 AM, Jakub Sławiński 
>> <>
>>  wrote:
>> 
>>> 
>>> Hi,
>>> 
>>> is there any progress regarding new NDT release? What has left to be
>>> done and how we can help to make it happen?
>>> 
>>> 
>>> Regards,
>>> Jakub.
>>> 
>>> On 07/11/2014 03:15 AM, Richard Carlson wrote:
>>>> Will;
>>>> 
>>>> The approach you describe sounds reasonable.  I would be happy to help
>>>> test things out once you have them ready.
>>>> 
>>>> Rich
>>>> 
>>>> On 07/09/2014 08:53 PM, Will Hawkins wrote:
>>>>> Mr. Carlson,
>>>>> 
>>>>> Thank you for adding your insight on this. I certainly appreciate having
>>>>> the chance to work with people as respected in the measurement world as
>>>>> you and Matt.
>>>>> 
>>>>> I agree that the goal should be not to tell the user that they are
>>>>> incapable of choosing a good browser. In fact, I don't actually like
>>>>> Chrome (shh, don't tell anyone!). I also agree that, in the case where
>>>>> the user is running a "bad" browser/os combination, the measurements
>>>>> made with this Flash client will be indicative of general performance on
>>>>> their connection when running Flash applications (with the caveat that
>>>>> HTTP-based network i/o is significantly faster than socket-based i/o
>>>>> because it is implemented better by the runtime across platforms). As a
>>>>> final note of agreement, I concur that NDT is more than just a "speed
>>>>> test".
>>>>> 
>>>>> The way that we are going to actually implement this is with a series of
>>>>> configurable knobs. The person embedding the Flash client in their HTML
>>>>> will have the option to treat an incompatible browser/os combination as
>>>>> an error or a warning. They deployer will get to handle that condition
>>>>> in a callback and display an appropriate message (so that they could say
>>>>> something in "big bold letters", which I liked!).
>>>>> 
>>>>> If that type of scenario is acceptable to you, the decision is more
>>>>> about the appropriate default (error or warning). We are close to
>>>>> finishing up an implementation. Would you be willing to review that when
>>>>> it's ready and see if it meets your expectations?
>>>>> 
>>>>> Thanks again for adding your thoughts! I was hoping that you'd weigh in.
>>>>> We have Matt M on a separate thread adding his thoughts as well so I am
>>>>> confident that we will come to a solid resolution.
>>>>> 
>>>>> Will
>>>>> 
>>>>> On 07/09/2014 08:40 PM, Richard Carlson wrote:
>>>>>> Sebastian/Will;
>>>>>> 
>>>>>> I'm not sure I agree with this idea.  While I agree that letting the
>>>>>> user know that the test results are bad due to the Flash runtime
>>>>>> problem, I'm not at all convinced that the proper response is to say
>>>>>> 'use Chrome'.  It just seems a bit strong to me, like you are saying 
>>>>>> the
>>>>>> user is too stupid to run a good browser.
>>>>>> 
>>>>>> I'd prefer to let the test run and put in big bold letters across the
>>>>>> results page or in some other manner indicate that the limiting factor
>>>>>> is the client's flash runtime.  That way the user is being informed and
>>>>>> the logged test results can also contain the same notation so they can
>>>>>> be discarded if necessary.
>>>>>> 
>>>>>> After all, they will get that performance when running other flash 
>>>>>> apps,
>>>>>> so it would be good for them to know what the limits are.
>>>>>> 
>>>>>> The whole purpose of the NDT system is to provide advanced diagnostic
>>>>>> messages, not just run speed tests.  Showing the user that the test
>>>>>> results are limited and explaining why would be much better than 
>>>>>> telling
>>>>>> them to 'trust me and replace your browser's flash runtime'.
>>>>>> 
>>>>>> So for me, the real question is can you make a simple and intuitive
>>>>>> image or text that clearly shows that the problem is the flash runtime
>>>>>> and not the NDT service.
>>>>>> 
>>>>>> Rich
>>>>>> 
>>>>>> On 07/08/2014 03:09 AM, Sebastian Kostuch wrote:
>>>>>>> Hi Will,
>>>>>>> IMHO it would be good to disable possibility of running test when
>>>>>>> we detect that flash client is being run from within not supported
>>>>>>> platform. With this there won't be faked results noted on server
>>>>>>> side's statistics.
>>>>>>> Also looking forward to any further information
>>>>>>> regarding new release :).
>>>>>>> 
>>>>>>> Regards
>>>>>>> Sebastian
>>>>>>> 
>>>>>>> On 03.07.2014 02:40, Will Hawkins wrote:
>>>>>>>> Sebastian and Aaron (and everyone, really),
>>>>>>>> 
>>>>>>>> We at MLab are in the final throes of getting our platform ready
>>>>>>>> for the
>>>>>>>> Flash client. In so doing, we've gotten deep into the weeds on the
>>>>>>>> performance/measurement implications of the Flash runtime.
>>>>>>>> 
>>>>>>>> I wanted to present you with our findings and get your input.
>>>>>>>> 
>>>>>>>> On all platforms (Windows, Mac, Linux), running the Flash client in
>>>>>>>> Google Chrome works a-ok. On Windows, running the Flash client in all
>>>>>>>> other browsers work great too. On Linux and Mac, however, that is not
>>>>>>>> the case.
>>>>>>>> 
>>>>>>>> On Mac and Linux where the user runs the NDT Flash client under
>>>>>>>> Adobe's
>>>>>>>> runtime (i.e., browsers other than Chrome [even Chromium does not
>>>>>>>> work]), there is not enough network i/o fidelity to produce
>>>>>>>> measurements
>>>>>>>> for connections with > ~ 50mbps throughput (on the s2c side).
>>>>>>>> 
>>>>>>>> In other words, Google's independent implementation of the Flash
>>>>>>>> runtime
>>>>>>>> performs well enough on all platforms, but Adobe's implementation
>>>>>>>> performs well enough only on Windows.
>>>>>>>> 
>>>>>>>> Because MLab collects lots of data, we don't want our results to be
>>>>>>>> spoiled by tests whose results are artificially limited. We are
>>>>>>>> attempting to work out a way to "encourage" users to run the test
>>>>>>>> in the
>>>>>>>> environment where we will get the most accurate results. To do so,
>>>>>>>> we've
>>>>>>>> settled on the following plan:
>>>>>>>> 
>>>>>>>> 1.  When the flash client detects that it is being run within a
>>>>>>>> non-optimal environment, display the following:  "Unfortunately, this
>>>>>>>> test cannot run within your current browser environment.  Please run
>>>>>>>> the test from within Google Chrome, or download M-Lab's Firefox
>>>>>>>> Extension."
>>>>>>>> 
>>>>>>>> 2.  Disallow the flash client from running on any platform that
>>>>>>>> identifies itself as something other than Windows / Chrome.
>>>>>>>> 
>>>>>>>> What do people think about this plan? Reasonable?
>>>>>>>> 
>>>>>>>> As always, it's great working with everyone in NDT land. I look
>>>>>>>> forward
>>>>>>>> to continuing the work :-)
>>>>>>>> 
>>>>>>>> Will
>>>>>>>> 
>>>>>>>> On 06/18/2014 03:06 AM, Sebastian Kostuch wrote:
>>>>>>>>> Hi Will
>>>>>>>>> indeed these changes results in higher speeds (I was able to get 
>>>>>>>>> ~760
>>>>>>>>> mbps
>>>>>>>>> which is quite similar to yours). Also I have checked flash client 
>>>>>>>>> as
>>>>>>>>> background
>>>>>>>>> for new JS UI and both speeds are being updated correctly in
>>>>>>>>> real-time.
>>>>>>>>> This is
>>>>>>>>> really great work! Thanks for this. I have merged these changes into
>>>>>>>>> trunk so
>>>>>>>>> they would be also present in new release.
>>>>>>>>> Also as you probably have noticed I have fixed compilation error 
>>>>>>>>> when
>>>>>>>>> using
>>>>>>>>> web100 so I closed appropriated issue related to this.
>>>>>>>>> 
>>>>>>>>> Regards
>>>>>>>>> Sebastian
>>>>>>>>> 
>>>>>>>>> On 17.06.2014 22:52, Will Hawkins wrote:
>>>>>>>>>> Hello Sebastian and Aaron (and MLab'ers CC'd),
>>>>>>>>>> 
>>>>>>>>>> We found a way to juice the Flash client and achieve the 
>>>>>>>>>> performance
>>>>>>>>>> necessary to accurately measure high-speed network connections.
>>>>>>>>>> 
>>>>>>>>>> The change is in r1082.
>>>>>>>>>> 
>>>>>>>>>> The solution was counterintuitive (at least for me). Instead of
>>>>>>>>>> attempting to read data from the socket as quickly as possible, the
>>>>>>>>>> TestS2C class just gets out of the way for the duration of the
>>>>>>>>>> test. It
>>>>>>>>>> registers only for the "socket close" event. The runtime has the
>>>>>>>>>> necessary speed to keep up with the network.
>>>>>>>>>> 
>>>>>>>>>> The only problem with this method is keeping the user interface
>>>>>>>>>> up-to-date on progress throughout the test. Sebastian, this is
>>>>>>>>>> where I
>>>>>>>>>> need you to double check. I believe that the code in r1082 properly
>>>>>>>>>> updates _s2cByteCount in the appropriate spot for those real-time
>>>>>>>>>> updates (specifically onSpeedUpdate). But, please double check!
>>>>>>>>>> 
>>>>>>>>>> We were able to measure a connection at ~ 775 mbps using the Flash
>>>>>>>>>> client, but it would be great to have as many people test this as
>>>>>>>>>> possible. To do so, you can use a pre-compiled version at
>>>>>>>>>> 
>>>>>>>>>> http://files.opentechinstitute.org/~hawkinsw/flash/test.html
>>>>>>>>>> 
>>>>>>>>>> On a separate, but related note, please also review r1083 for
>>>>>>>>>> inclusion
>>>>>>>>>> in the NDT release. This change allows the client to re-query JS 
>>>>>>>>>> for
>>>>>>>>>> the
>>>>>>>>>> test's hostname just before launching a test. This gives the 
>>>>>>>>>> wrapper
>>>>>>>>>> page the opportunity to do a AJAX query to determine the test's
>>>>>>>>>> hostname. This is important for MLab because we do dynamic server
>>>>>>>>>> selection using a web-based API.
>>>>>>>>>> 
>>>>>>>>>> I am eager to hear whether you have success with these changes or
>>>>>>>>>> not. I
>>>>>>>>>> hope you do!
>>>>>>>>>> 
>>>>>>>>>> Will
>>>>>>>>>> 
>>>>>>>>>> On 06/17/2014 10:53 AM, Will Hawkins wrote:
>>>>>>>>>>> Just FYI: I believe that we have happened on to an actual solution
>>>>>>>>>>> for
>>>>>>>>>>> the "limit" to the performance of the S2C test. I am going to
>>>>>>>>>>> implement
>>>>>>>>>>> it today and do some testing.
>>>>>>>>>>> 
>>>>>>>>>>> If possible, could we hold out on doing a release for a little
>>>>>>>>>>> while
>>>>>>>>>>> pending this investigation?
>>>>>>>>>>> 
>>>>>>>>>>> Will
>>>>>>>>>>> 
>>>>>>>>>>> On 06/16/2014 01:50 AM, Sebastian Kostuch wrote:
>>>>>>>>>>>> Hi Will,
>>>>>>>>>>>> what you mentioned seems reasonable. Would it be possible to 
>>>>>>>>>>>> merge
>>>>>>>>>>>> these
>>>>>>>>>>>> needed
>>>>>>>>>>>> changes into trunk in nearest time so we could make new release
>>>>>>>>>>>> for
>>>>>>>>>>>> example tomorrow
>>>>>>>>>>>> or the day after tomorrow?
>>>>>>>>>>>> 
>>>>>>>>>>>> Regards
>>>>>>>>>>>> Sebastian
>>>>>>>>>>>> 
>>>>>>>>>>>> On 13.06.2014 17:36, Will Hawkins wrote:
>>>>>>>>>>>>> Sebastian,
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks for sending this out! We (at MLab) are very interested in
>>>>>>>>>>>>> deploying a new version of the code so this 'release' looks
>>>>>>>>>>>>> like a
>>>>>>>>>>>>> good place to fit together! :-) In other words, we (again, MLab)
>>>>>>>>>>>>> are
>>>>>>>>>>>>> definitely going to do what you said in 3!
>>>>>>>>>>>>> 
>>>>>>>>>>>>> I am out of the office, but I wanted to quickly respond to
>>>>>>>>>>>>> point 2.
>>>>>>>>>>>>> See below!
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Thanks again for driving this process forward!!
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Will
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On 6/13/14, 4:47 AM, Sebastian Kostuch wrote:
>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>> NDT has been updated recently with some major changes (like
>>>>>>>>>>>>>> MSG_EXTENDED_LOGIN,
>>>>>>>>>>>>>> json support, new JS UI) so as I have mentioned in some earlier
>>>>>>>>>>>>>> mail
>>>>>>>>>>>>>> that would be probably
>>>>>>>>>>>>>> good time to make new release. I have few questions related to
>>>>>>>>>>>>>> this:
>>>>>>>>>>>>>> 1. New JS UI would fit well within this new release, however
>>>>>>>>>>>>>> changes
>>>>>>>>>>>>>> related to this are still
>>>>>>>>>>>>>> on work branches and need review. Any feedback would be very
>>>>>>>>>>>>>> appreciated. These are
>>>>>>>>>>>>>> following issues: 129, 132, 133.
>>>>>>>>>>>>>> 2. Is Issue144 (URLLoader to flash) supposed to be contained
>>>>>>>>>>>>>> also in
>>>>>>>>>>>>>> nearest release version?
>>>>>>>>>>>>>> If yes, then how does progress with work on this look like? If 
>>>>>>>>>>>>>> I
>>>>>>>>>>>>>> recall
>>>>>>>>>>>>>> correctly then you, Will,
>>>>>>>>>>>>>> mentioned that they are well tested and thus almost ready,
>>>>>>>>>>>>>> right?
>>>>>>>>>>>>> Some of these are still up for debate. I am going to forward an
>>>>>>>>>>>>> email
>>>>>>>>>>>>> that I just sent to the MLab team with an update on the 
>>>>>>>>>>>>> conundrum
>>>>>>>>>>>>> around the HTTP test to give you a sense about why they are up
>>>>>>>>>>>>> for
>>>>>>>>>>>>> debate.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> But, there are also changes in that branch that *need* to be
>>>>>>>>>>>>> incorporated into anything that gets released. For instance,
>>>>>>>>>>>>> r1070.
>>>>>>>>>>>>> There are parts of r1064 that will have to be integrated no
>>>>>>>>>>>>> matter
>>>>>>>>>>>>> what (any of the changes that have to do with splitting start
>>>>>>>>>>>>> test and
>>>>>>>>>>>>> finalize test stages).
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Does that make sense?
>>>>>>>>>>>>> 
>>>>>>>>>>>>>> 3. Along with new release on googlecode what do you think about
>>>>>>>>>>>>>> pushing
>>>>>>>>>>>>>> these changes also
>>>>>>>>>>>>>> on M-Lab site? They bring many new features at all.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Looking forward to your answers and I hope that next week
>>>>>>>>>>>>>> could be
>>>>>>>>>>>>>> celebrated with making
>>>>>>>>>>>>>> new release of NDT :).
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Kind regards
>>>>>>>>>>>>>> Sebastian Kostuch
>>>>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>> 
>>>>>>> 
>>>>> 
>>> 
>> 
>