NDT-DEV email list created

[Aaron Brown <>]

Hey Sebastian,

That’s a tricky question. There are pros and cons with each:

Java: pro: it performs well on all platforms. con: people are disabling it 
(though with the signing cert, some of these problems may go away)
Flash: pro: pretty much everyone has it. con: it doesn’t perform well in 
certain configurations.

I guess my preference would be Java due to the performance issues with Flash. 
What do others think?

Cheers,
Aaron

On Jul 31, 2014, at 3:38 AM, Sebastian Kostuch 
<>
 wrote:

> Hi Aaron,
> thanks for your response. I will then make changes such as you proposed. 
> Would it be better to use flash or java as default if users have both 
> plugins installed on their system?
> 
> Regards
> Sebastian
> 
> On 29.07.2014 16:42, Aaron Brown wrote:
>> Hi Sebastian,
>> 
>> The best case scenario would be to have the index page choose a sensible 
>> default depending on whether flash or java are installed, and make it easy 
>> for them to select it on the page if they want something different.
>> 
>> Cheers,
>> Aaron
>> 
>> On Jul 25, 2014, at 1:42 AM, Sebastian Kostuch 
>> <>
>>  wrote:
>> 
>>> Hi Aaron,
>>> we will do our best to try to help you with mentioned task. When it comes 
>>> to swapping "backend"
>>> client then ATM there is script available ("set-active-client.sh) which 
>>> allows to choose if user prefers
>>> flash or java backend and properly updates HTML code. Is such a solution 
>>> acceptable or should we
>>> rather make it different way?
>>> 
>>> Regards
>>> Sebastian
>>> 
>>> On 22.07.2014 19:22, Aaron Brown wrote:
>>>> Hey Jakub,
>>>> 
>>>> On Jul 22, 2014, at 10:52 AM, Jakub Sławiński 
>>>> <>
>>>>  wrote:
>>>> 
>>>>> Hi,
>>>>> 
>>>>> I think we will be able to help with the mentioned issues.
>>>>> 
>>>>> Sebastian, can we already swap between flash and java backend on the
>>>>> html level? Moreover, can we help with packaging up RPMs?
>>>> If you could create RPMs for the flash components, that’d be great. 
>>>> There’s a spec file in the trunk of the repository that you can add an 
>>>> additional flash client component to.
>>>> 
>>>> Cheers,
>>>> Aaron
>>>> 
>>>>> Regards,
>>>>>  Jakub.
>>>>> 
>>>>> On 07/21/2014 04:58 PM, Aaron Brown wrote:
>>>>>> Hey Jakub,
>>>>>> 
>>>>>> The big open part is getting the flash client packaged so that an 
>>>>>> admin can do a “yum install [package]”, and have it just work. Among 
>>>>>> other things, it’ll need to package up something to serve the default 
>>>>>> flash socket xml stuff, and include a way, in the HTML, to swap 
>>>>>> between the flash or java applet as the ‘backend’ of the javascript so 
>>>>>> folks can run either. If you guys can work on packaging up RPMs, 
>>>>>> that’d be great.
>>>>>> 
>>>>>> Cheers,
>>>>>> Aaron
>>>>>> 
>>>>>> On Jul 21, 2014, at 10:48 AM, Jakub Sławiński 
>>>>>> <>
>>>>>>  wrote:
>>>>>> 
>>>>>>> Hi,
>>>>>>> 
>>>>>>> is there any progress regarding new NDT release? What has left to be
>>>>>>> done and how we can help to make it happen?
>>>>>>> 
>>>>>>> 
>>>>>>> Regards,
>>>>>>> Jakub.
>>>>>>> 
>>>>>>> On 07/11/2014 03:15 AM, Richard Carlson wrote:
>>>>>>>> Will;
>>>>>>>> 
>>>>>>>> The approach you describe sounds reasonable.  I would be happy to 
>>>>>>>> help
>>>>>>>> test things out once you have them ready.
>>>>>>>> 
>>>>>>>> Rich
>>>>>>>> 
>>>>>>>> On 07/09/2014 08:53 PM, Will Hawkins wrote:
>>>>>>>>> Mr. Carlson,
>>>>>>>>> 
>>>>>>>>> Thank you for adding your insight on this. I certainly appreciate 
>>>>>>>>> having
>>>>>>>>> the chance to work with people as respected in the measurement 
>>>>>>>>> world as
>>>>>>>>> you and Matt.
>>>>>>>>> 
>>>>>>>>> I agree that the goal should be not to tell the user that they are
>>>>>>>>> incapable of choosing a good browser. In fact, I don't actually like
>>>>>>>>> Chrome (shh, don't tell anyone!). I also agree that, in the case 
>>>>>>>>> where
>>>>>>>>> the user is running a "bad" browser/os combination, the measurements
>>>>>>>>> made with this Flash client will be indicative of general 
>>>>>>>>> performance on
>>>>>>>>> their connection when running Flash applications (with the caveat 
>>>>>>>>> that
>>>>>>>>> HTTP-based network i/o is significantly faster than socket-based i/o
>>>>>>>>> because it is implemented better by the runtime across platforms). 
>>>>>>>>> As a
>>>>>>>>> final note of agreement, I concur that NDT is more than just a 
>>>>>>>>> "speed
>>>>>>>>> test".
>>>>>>>>> 
>>>>>>>>> The way that we are going to actually implement this is with a 
>>>>>>>>> series of
>>>>>>>>> configurable knobs. The person embedding the Flash client in their 
>>>>>>>>> HTML
>>>>>>>>> will have the option to treat an incompatible browser/os 
>>>>>>>>> combination as
>>>>>>>>> an error or a warning. They deployer will get to handle that 
>>>>>>>>> condition
>>>>>>>>> in a callback and display an appropriate message (so that they 
>>>>>>>>> could say
>>>>>>>>> something in "big bold letters", which I liked!).
>>>>>>>>> 
>>>>>>>>> If that type of scenario is acceptable to you, the decision is more
>>>>>>>>> about the appropriate default (error or warning). We are close to
>>>>>>>>> finishing up an implementation. Would you be willing to review that 
>>>>>>>>> when
>>>>>>>>> it's ready and see if it meets your expectations?
>>>>>>>>> 
>>>>>>>>> Thanks again for adding your thoughts! I was hoping that you'd 
>>>>>>>>> weigh in.
>>>>>>>>> We have Matt M on a separate thread adding his thoughts as well so 
>>>>>>>>> I am
>>>>>>>>> confident that we will come to a solid resolution.
>>>>>>>>> 
>>>>>>>>> Will
>>>>>>>>> 
>>>>>>>>> On 07/09/2014 08:40 PM, Richard Carlson wrote:
>>>>>>>>>> Sebastian/Will;
>>>>>>>>>> 
>>>>>>>>>> I'm not sure I agree with this idea.  While I agree that letting 
>>>>>>>>>> the
>>>>>>>>>> user know that the test results are bad due to the Flash runtime
>>>>>>>>>> problem, I'm not at all convinced that the proper response is to 
>>>>>>>>>> say
>>>>>>>>>> 'use Chrome'.  It just seems a bit strong to me, like you are 
>>>>>>>>>> saying the
>>>>>>>>>> user is too stupid to run a good browser.
>>>>>>>>>> 
>>>>>>>>>> I'd prefer to let the test run and put in big bold letters across 
>>>>>>>>>> the
>>>>>>>>>> results page or in some other manner indicate that the limiting 
>>>>>>>>>> factor
>>>>>>>>>> is the client's flash runtime.  That way the user is being 
>>>>>>>>>> informed and
>>>>>>>>>> the logged test results can also contain the same notation so they 
>>>>>>>>>> can
>>>>>>>>>> be discarded if necessary.
>>>>>>>>>> 
>>>>>>>>>> After all, they will get that performance when running other flash 
>>>>>>>>>> apps,
>>>>>>>>>> so it would be good for them to know what the limits are.
>>>>>>>>>> 
>>>>>>>>>> The whole purpose of the NDT system is to provide advanced 
>>>>>>>>>> diagnostic
>>>>>>>>>> messages, not just run speed tests.  Showing the user that the test
>>>>>>>>>> results are limited and explaining why would be much better than 
>>>>>>>>>> telling
>>>>>>>>>> them to 'trust me and replace your browser's flash runtime'.
>>>>>>>>>> 
>>>>>>>>>> So for me, the real question is can you make a simple and intuitive
>>>>>>>>>> image or text that clearly shows that the problem is the flash 
>>>>>>>>>> runtime
>>>>>>>>>> and not the NDT service.
>>>>>>>>>> 
>>>>>>>>>> Rich
>>>>>>>>>> 
>>>>>>>>>> On 07/08/2014 03:09 AM, Sebastian Kostuch wrote:
>>>>>>>>>>> Hi Will,
>>>>>>>>>>> IMHO it would be good to disable possibility of running test when
>>>>>>>>>>> we detect that flash client is being run from within not supported
>>>>>>>>>>> platform. With this there won't be faked results noted on server
>>>>>>>>>>> side's statistics.
>>>>>>>>>>> Also looking forward to any further information
>>>>>>>>>>> regarding new release :).
>>>>>>>>>>> 
>>>>>>>>>>> Regards
>>>>>>>>>>> Sebastian
>>>>>>>>>>> 
>>>>>>>>>>> On 03.07.2014 02:40, Will Hawkins wrote:
>>>>>>>>>>>> Sebastian and Aaron (and everyone, really),
>>>>>>>>>>>> 
>>>>>>>>>>>> We at MLab are in the final throes of getting our platform ready
>>>>>>>>>>>> for the
>>>>>>>>>>>> Flash client. In so doing, we've gotten deep into the weeds on 
>>>>>>>>>>>> the
>>>>>>>>>>>> performance/measurement implications of the Flash runtime.
>>>>>>>>>>>> 
>>>>>>>>>>>> I wanted to present you with our findings and get your input.
>>>>>>>>>>>> 
>>>>>>>>>>>> On all platforms (Windows, Mac, Linux), running the Flash client 
>>>>>>>>>>>> in
>>>>>>>>>>>> Google Chrome works a-ok. On Windows, running the Flash client 
>>>>>>>>>>>> in all
>>>>>>>>>>>> other browsers work great too. On Linux and Mac, however, that 
>>>>>>>>>>>> is not
>>>>>>>>>>>> the case.
>>>>>>>>>>>> 
>>>>>>>>>>>> On Mac and Linux where the user runs the NDT Flash client under
>>>>>>>>>>>> Adobe's
>>>>>>>>>>>> runtime (i.e., browsers other than Chrome [even Chromium does not
>>>>>>>>>>>> work]), there is not enough network i/o fidelity to produce
>>>>>>>>>>>> measurements
>>>>>>>>>>>> for connections with > ~ 50mbps throughput (on the s2c side).
>>>>>>>>>>>> 
>>>>>>>>>>>> In other words, Google's independent implementation of the Flash
>>>>>>>>>>>> runtime
>>>>>>>>>>>> performs well enough on all platforms, but Adobe's implementation
>>>>>>>>>>>> performs well enough only on Windows.
>>>>>>>>>>>> 
>>>>>>>>>>>> Because MLab collects lots of data, we don't want our results to 
>>>>>>>>>>>> be
>>>>>>>>>>>> spoiled by tests whose results are artificially limited. We are
>>>>>>>>>>>> attempting to work out a way to "encourage" users to run the test
>>>>>>>>>>>> in the
>>>>>>>>>>>> environment where we will get the most accurate results. To do 
>>>>>>>>>>>> so,
>>>>>>>>>>>> we've
>>>>>>>>>>>> settled on the following plan:
>>>>>>>>>>>> 
>>>>>>>>>>>> 1.  When the flash client detects that it is being run within a
>>>>>>>>>>>> non-optimal environment, display the following:  "Unfortunately, 
>>>>>>>>>>>> this
>>>>>>>>>>>> test cannot run within your current browser environment.  Please 
>>>>>>>>>>>> run
>>>>>>>>>>>> the test from within Google Chrome, or download M-Lab's Firefox
>>>>>>>>>>>> Extension."
>>>>>>>>>>>> 
>>>>>>>>>>>> 2.  Disallow the flash client from running on any platform that
>>>>>>>>>>>> identifies itself as something other than Windows / Chrome.
>>>>>>>>>>>> 
>>>>>>>>>>>> What do people think about this plan? Reasonable?
>>>>>>>>>>>> 
>>>>>>>>>>>> As always, it's great working with everyone in NDT land. I look
>>>>>>>>>>>> forward
>>>>>>>>>>>> to continuing the work :-)
>>>>>>>>>>>> 
>>>>>>>>>>>> Will
>>>>>>>>>>>> 
>>>>>>>>>>>> On 06/18/2014 03:06 AM, Sebastian Kostuch wrote:
>>>>>>>>>>>>> Hi Will
>>>>>>>>>>>>> indeed these changes results in higher speeds (I was able to 
>>>>>>>>>>>>> get ~760
>>>>>>>>>>>>> mbps
>>>>>>>>>>>>> which is quite similar to yours). Also I have checked flash 
>>>>>>>>>>>>> client as
>>>>>>>>>>>>> background
>>>>>>>>>>>>> for new JS UI and both speeds are being updated correctly in
>>>>>>>>>>>>> real-time.
>>>>>>>>>>>>> This is
>>>>>>>>>>>>> really great work! Thanks for this. I have merged these changes 
>>>>>>>>>>>>> into
>>>>>>>>>>>>> trunk so
>>>>>>>>>>>>> they would be also present in new release.
>>>>>>>>>>>>> Also as you probably have noticed I have fixed compilation 
>>>>>>>>>>>>> error when
>>>>>>>>>>>>> using
>>>>>>>>>>>>> web100 so I closed appropriated issue related to this.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Regards
>>>>>>>>>>>>> Sebastian
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On 17.06.2014 22:52, Will Hawkins wrote:
>>>>>>>>>>>>>> Hello Sebastian and Aaron (and MLab'ers CC'd),
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> We found a way to juice the Flash client and achieve the 
>>>>>>>>>>>>>> performance
>>>>>>>>>>>>>> necessary to accurately measure high-speed network connections.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> The change is in r1082.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> The solution was counterintuitive (at least for me). Instead of
>>>>>>>>>>>>>> attempting to read data from the socket as quickly as 
>>>>>>>>>>>>>> possible, the
>>>>>>>>>>>>>> TestS2C class just gets out of the way for the duration of the
>>>>>>>>>>>>>> test. It
>>>>>>>>>>>>>> registers only for the "socket close" event. The runtime has 
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> necessary speed to keep up with the network.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> The only problem with this method is keeping the user interface
>>>>>>>>>>>>>> up-to-date on progress throughout the test. Sebastian, this is
>>>>>>>>>>>>>> where I
>>>>>>>>>>>>>> need you to double check. I believe that the code in r1082 
>>>>>>>>>>>>>> properly
>>>>>>>>>>>>>> updates _s2cByteCount in the appropriate spot for those 
>>>>>>>>>>>>>> real-time
>>>>>>>>>>>>>> updates (specifically onSpeedUpdate). But, please double check!
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> We were able to measure a connection at ~ 775 mbps using the 
>>>>>>>>>>>>>> Flash
>>>>>>>>>>>>>> client, but it would be great to have as many people test this 
>>>>>>>>>>>>>> as
>>>>>>>>>>>>>> possible. To do so, you can use a pre-compiled version at
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> http://files.opentechinstitute.org/~hawkinsw/flash/test.html
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On a separate, but related note, please also review r1083 for
>>>>>>>>>>>>>> inclusion
>>>>>>>>>>>>>> in the NDT release. This change allows the client to re-query 
>>>>>>>>>>>>>> JS for
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> test's hostname just before launching a test. This gives the 
>>>>>>>>>>>>>> wrapper
>>>>>>>>>>>>>> page the opportunity to do a AJAX query to determine the test's
>>>>>>>>>>>>>> hostname. This is important for MLab because we do dynamic 
>>>>>>>>>>>>>> server
>>>>>>>>>>>>>> selection using a web-based API.
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> I am eager to hear whether you have success with these changes 
>>>>>>>>>>>>>> or
>>>>>>>>>>>>>> not. I
>>>>>>>>>>>>>> hope you do!
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Will
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On 06/17/2014 10:53 AM, Will Hawkins wrote:
>>>>>>>>>>>>>>> Just FYI: I believe that we have happened on to an actual 
>>>>>>>>>>>>>>> solution
>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>> the "limit" to the performance of the S2C test. I am going to
>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>> it today and do some testing.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> If possible, could we hold out on doing a release for a little
>>>>>>>>>>>>>>> while
>>>>>>>>>>>>>>> pending this investigation?
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Will
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On 06/16/2014 01:50 AM, Sebastian Kostuch wrote:
>>>>>>>>>>>>>>>> Hi Will,
>>>>>>>>>>>>>>>> what you mentioned seems reasonable. Would it be possible to 
>>>>>>>>>>>>>>>> merge
>>>>>>>>>>>>>>>> these
>>>>>>>>>>>>>>>> needed
>>>>>>>>>>>>>>>> changes into trunk in nearest time so we could make new 
>>>>>>>>>>>>>>>> release
>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>> example tomorrow
>>>>>>>>>>>>>>>> or the day after tomorrow?
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>>>> Sebastian
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> On 13.06.2014 17:36, Will Hawkins wrote:
>>>>>>>>>>>>>>>>> Sebastian,
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Thanks for sending this out! We (at MLab) are very 
>>>>>>>>>>>>>>>>> interested in
>>>>>>>>>>>>>>>>> deploying a new version of the code so this 'release' looks
>>>>>>>>>>>>>>>>> like a
>>>>>>>>>>>>>>>>> good place to fit together! :-) In other words, we (again, 
>>>>>>>>>>>>>>>>> MLab)
>>>>>>>>>>>>>>>>> are
>>>>>>>>>>>>>>>>> definitely going to do what you said in 3!
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> I am out of the office, but I wanted to quickly respond to
>>>>>>>>>>>>>>>>> point 2.
>>>>>>>>>>>>>>>>> See below!
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Thanks again for driving this process forward!!
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Will
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> On 6/13/14, 4:47 AM, Sebastian Kostuch wrote:
>>>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>>> NDT has been updated recently with some major changes (like
>>>>>>>>>>>>>>>>>> MSG_EXTENDED_LOGIN,
>>>>>>>>>>>>>>>>>> json support, new JS UI) so as I have mentioned in some 
>>>>>>>>>>>>>>>>>> earlier
>>>>>>>>>>>>>>>>>> mail
>>>>>>>>>>>>>>>>>> that would be probably
>>>>>>>>>>>>>>>>>> good time to make new release. I have few questions 
>>>>>>>>>>>>>>>>>> related to
>>>>>>>>>>>>>>>>>> this:
>>>>>>>>>>>>>>>>>> 1. New JS UI would fit well within this new release, 
>>>>>>>>>>>>>>>>>> however
>>>>>>>>>>>>>>>>>> changes
>>>>>>>>>>>>>>>>>> related to this are still
>>>>>>>>>>>>>>>>>> on work branches and need review. Any feedback would be 
>>>>>>>>>>>>>>>>>> very
>>>>>>>>>>>>>>>>>> appreciated. These are
>>>>>>>>>>>>>>>>>> following issues: 129, 132, 133.
>>>>>>>>>>>>>>>>>> 2. Is Issue144 (URLLoader to flash) supposed to be 
>>>>>>>>>>>>>>>>>> contained
>>>>>>>>>>>>>>>>>> also in
>>>>>>>>>>>>>>>>>> nearest release version?
>>>>>>>>>>>>>>>>>> If yes, then how does progress with work on this look 
>>>>>>>>>>>>>>>>>> like? If I
>>>>>>>>>>>>>>>>>> recall
>>>>>>>>>>>>>>>>>> correctly then you, Will,
>>>>>>>>>>>>>>>>>> mentioned that they are well tested and thus almost ready,
>>>>>>>>>>>>>>>>>> right?
>>>>>>>>>>>>>>>>> Some of these are still up for debate. I am going to 
>>>>>>>>>>>>>>>>> forward an
>>>>>>>>>>>>>>>>> email
>>>>>>>>>>>>>>>>> that I just sent to the MLab team with an update on the 
>>>>>>>>>>>>>>>>> conundrum
>>>>>>>>>>>>>>>>> around the HTTP test to give you a sense about why they are 
>>>>>>>>>>>>>>>>> up
>>>>>>>>>>>>>>>>> for
>>>>>>>>>>>>>>>>> debate.
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> But, there are also changes in that branch that *need* to be
>>>>>>>>>>>>>>>>> incorporated into anything that gets released. For instance,
>>>>>>>>>>>>>>>>> r1070.
>>>>>>>>>>>>>>>>> There are parts of r1064 that will have to be integrated no
>>>>>>>>>>>>>>>>> matter
>>>>>>>>>>>>>>>>> what (any of the changes that have to do with splitting 
>>>>>>>>>>>>>>>>> start
>>>>>>>>>>>>>>>>> test and
>>>>>>>>>>>>>>>>> finalize test stages).
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>> Does that make sense?
>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> 3. Along with new release on googlecode what do you think 
>>>>>>>>>>>>>>>>>> about
>>>>>>>>>>>>>>>>>> pushing
>>>>>>>>>>>>>>>>>> these changes also
>>>>>>>>>>>>>>>>>> on M-Lab site? They bring many new features at all.
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Looking forward to your answers and I hope that next week
>>>>>>>>>>>>>>>>>> could be
>>>>>>>>>>>>>>>>>> celebrated with making
>>>>>>>>>>>>>>>>>> new release of NDT :).
>>>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>>>> Kind regards
>>>>>>>>>>>>>>>>>> Sebastian Kostuch
>>>>>>>>>>>>>>>>>> 
>