NDT-DEV email list created

[Aaron Brown <>]

Hey Will,

On Jun 25, 2014, at 9:51 PM, Will Hawkins 
<>
 wrote:

> I am happy to create a branch for this, but I think that might be
> overkill. Here's what I added
> 
> Index: src/Main.as
> ===================================================================
> --- src/Main.as       (revision 1112)
> +++ src/Main.as       (working copy)
> @@ -45,6 +45,14 @@
>     private function init(e:Event = null):void {
>       removeEventListener(Event.ADDED_TO_STAGE, init);
> 
> +      var selfUrl:String = this.loaderInfo.url;
> +      if (selfUrl.indexOf("http://";, 0) == 0) {
> +        selfUrl = selfUrl.replace("http://";, "");
> +        selfUrl = selfUrl.replace(/:[0-9]+/, "");
> +        selfUrl = selfUrl.replace(/\/.+$/, "");
> +        server_hostname = selfUrl;
> +      }
> +
>       // Set the properties of the SWF from HTML tags.
>       NDTUtils.initializeFromHTML(this.root.loaderInfo.parameters);

Looking at the parsing there, it will definitely fail on IPv6 addresses. It 
looks like actionscript has a URL library 
(http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/mx/utils/URLUtil.html)
 that would probably work for returning the address to test against.

> This will make it default to test against an NDT server running on the
> same host from which the swf was loaded. This does not preclude the user
> from overriding that value with their JS callback.
> 
> The other, perhaps easier, option is to do something similar in the JS
> that hosts the Flash applet. JS would parse window.location and then
> return that value in the getNDTServer() callback.
> 
> Let me know which route you prefer!

My general preference would be to encode that default into the Flash client 
to make it as simple as possible to embed the client in webpages.

Cheers,
Aaron


> Will
> 
> On 06/25/2014 02:22 PM, Aaron Brown wrote:
>> Hey Will,
>> 
>> On Jun 25, 2014, at 1:59 PM, Will Hawkins 
>> <>
>>  wrote:
>> 
>>> Hey Aaron!
>>> 
>>> I am working on this -- I promise :-)
>> 
>> Excellent, no rush :)
>> 
>>> Just to get some context, are you assuming that we would include an HTML
>>> wrapper for the Flash client and invoke the wrapped version to actually
>>> do the self testing? I'm just trying to figure out which of the
>>> Actionscript facilities I'll have access to in order to resolve this
>>> dilemma.
>> 
>> That’s what I’m thinking. Similar to how the java applet works (e.g. 
>> http://desk146.internet2.edu:7123/), you’d go to a webpage and the flash 
>> client would be embedded in the page. You could then just hit “start”, and 
>> it’d do a test from your machine to the server that served it.
>> 
>> Cheers,
>> Aaron
>> 
>>> 
>>> Will
>>> 
>>> On 06/24/2014 10:13 AM, Aaron Brown wrote:
>>>> Hey Will,
>>>> 
>>>> I’ve got the RPM working with the existing fakewww/Applet.
>>>> 
>>>> I’d like to include the flash client in the RPM if possible (otherwise, 
>>>> there’s not much benefit to the average RPM install for the new 
>>>> release). Unfortunately, it doesn’t look like there’s a good way to 
>>>> zero-configuration install (i.e. have it by default test back to the 
>>>> server it was served from) without using something like PHP (which would 
>>>> require swapping over to using Apache to serve the clients). Would it be 
>>>> possible to include a setting for the flash client to say “the server to 
>>>> use is the address of the server you downloaded the SWF from”?
>>>> 
>>>> Cheers,
>>>> Aaron
>>>> 
>>>> On Jun 20, 2014, at 8:02 PM, Will Hawkins 
>>>> <>
>>>>  wrote:
>>>> 
>>>>> Please let me know if/how I can help preparing the repo for RPM
>>>>> creation. I'm happy to give it a try on my own, but if there is
>>>>> expertise elsewhere (Aaron or Sebastian), then I am happy to take a back
>>>>> seat.
>>>>> 
>>>>> As for testers, we (MLab) can definitely find a few places to test. I
>>>>> know that we've been running trunk versions of the server on several
>>>>> test nodes for a while and haven't had any problems (besides the ones
>>>>> that I've submitted patches for).
>>>>> 
>>>>> Will
>>>>> 
>>>>> On 06/20/2014 09:54 AM, Aaron Brown wrote:
>>>>>> Hey Sebastian,
>>>>>> 
>>>>>> I’m untagging the release since I can guarantee we’ll need to make 
>>>>>> changes to the codebase for the NDT RPMs, and those are a requirement 
>>>>>> for outside testing since a very small percentage of NDT servers out 
>>>>>> there are compiled from source. I’ll commit some of the RPM bits, and 
>>>>>> then if you could go through, and make sure that it gets updated as 
>>>>>> needed for the changes (new web UI, etc), that’d be good. The goal of 
>>>>>> the NDT RPMs is that the administrators can do “yum install 
>>>>>> ndt-server”, start NDT and it just works. They don’t have to do any 
>>>>>> hand configuration to get an NDT server up-and-running.
>>>>>> 
>>>>>> Cheers,
>>>>>> Aaron
>>>>>> 
>>>>>> On Jun 20, 2014, at 5:15 AM, Sebastian Kostuch 
>>>>>> <>
>>>>>>  wrote:
>>>>>> 
>>>>>>> Hi
>>>>>>> Will, I have merged your last fixes into trunk so they will be 
>>>>>>> contained within
>>>>>>> new release :).
>>>>>>> Aaron, keeping in mind what you have said I have changed version 
>>>>>>> numbers
>>>>>>> to be release candidate and tagged this. Such tagged version is now 
>>>>>>> ready
>>>>>>> for being tested by outside folks. I hope that test stage will 
>>>>>>> perform smoothly
>>>>>>> and we would release new NDT version ASAP!
>>>>>>> Also I have updated wiki pages to match new protocol changes.
>>>>>>> 
>>>>>>> Regards
>>>>>>> Sebastian Kostuch
>>>>>>> 
>>>>>>> On 18.06.2014 23:00, Will Hawkins wrote:
>>>>>>>> Aaron,
>>>>>>>> 
>>>>>>>> Thanks for the information! We (M-Lab) are especially interested in
>>>>>>>> 
>>>>>>>> a) having the code "checked" to make sure that the measurements are
>>>>>>>> still valid
>>>>>>>> 
>>>>>>>> and
>>>>>>>> 
>>>>>>>> b) having a release to deploy on the platform.
>>>>>>>> 
>>>>>>>> Having said that, what is the best way to prepare a release 
>>>>>>>> candidate to
>>>>>>>> start the process rolling? Are you able to help verify the 
>>>>>>>> measurements
>>>>>>>> made with the new version of the tool? Will you be able to help,
>>>>>>>> generally, with the release process (since, I mean, you're awesome!)?
>>>>>>>> 
>>>>>>>> Thanks again for your email!
>>>>>>>> Will
>>>>>>>> 
>>>>>>>> On 06/18/2014 04:46 PM, Aaron Brown wrote:
>>>>>>>>> Hi Sebastian,
>>>>>>>>> 
>>>>>>>>> To do a full release, NDT will need to go through a release 
>>>>>>>>> candidate process to ensure that outside folks have a chance to 
>>>>>>>>> test it before something gets labeled as “released”. Beyond that, 
>>>>>>>>> NDT will need verified to make sure that something didn’t break 
>>>>>>>>> with existing clients (important since this is a less common 
>>>>>>>>> use-case). The upgrade process needs to be verified as well to make 
>>>>>>>>> sure that it doesn’t break an existing installation when someone 
>>>>>>>>> goes to upgrade.
>>>>>>>>> 
>>>>>>>>> As part of the upgrade procedure above, the RPMs will need updated 
>>>>>>>>> with any changes in the codebase since that’s the primary method 
>>>>>>>>> folks use to install NDT (the vast majority of NDT installations in 
>>>>>>>>> the wild are installed as part of the perfSONAR Performance 
>>>>>>>>> Toolkit). This procedure will need tested to make sure that both 
>>>>>>>>> new installs, and existing installs work as expected.
>>>>>>>>> 
>>>>>>>>> Once there’s a final release, the NDT RPMs will get added to the 
>>>>>>>>> Internet2 RPM repository (where the Toolkit users get their RPMs 
>>>>>>>>> from), and the source tarball will be added to the Internet2 site 
>>>>>>>>> at software.internet2.edu). Presumably, this link could be used as 
>>>>>>>>> canonical for NDT since any linked location is arbitrary from the 
>>>>>>>>> end user perspective.
>>>>>>>>> 
>>>>>>>>> Cheers,
>>>>>>>>> Aaron
>>>>>>>>> 
>>>>>>>>> On Jun 18, 2014, at 6:04 AM, Sebastian Kostuch 
>>>>>>>>> <>
>>>>>>>>>  wrote:
>>>>>>>>> 
>>>>>>>>>> Hi again,
>>>>>>>>>> new NDT version is coming soon, however there are last changes to 
>>>>>>>>>> be performed
>>>>>>>>>> before this. I have merged all changes to release into trunk, 
>>>>>>>>>> tested and fixed individual
>>>>>>>>>> small bugs. Next step will be to update our wiki pages to match 
>>>>>>>>>> new version which I will
>>>>>>>>>> be working on in nearest time.
>>>>>>>>>> However I wanted to ask two last questions to you:
>>>>>>>>>> 1. So far new release package has been published in "downloads" 
>>>>>>>>>> section of google code
>>>>>>>>>> project but recently Google has disabled this option. Therefore 
>>>>>>>>>> should we place such archive
>>>>>>>>>> in some external source like Google drive where users could 
>>>>>>>>>> directly download this (and update
>>>>>>>>>> wiki with such link)? Or maybe just publish link to release tagged 
>>>>>>>>>> revision?
>>>>>>>>>> 
>>>>>>>>>> 2. Also what about updating NDT links/software on another sites? I 
>>>>>>>>>> know that it will be performed
>>>>>>>>>> on M-lab but what about Internet2 website?
>>>>>>>>>> 
>>>>>>>>>> Regards
>>>>>>>>>> Sebastian
>>>>>>>>>> 
>>>>>>>>>> On 18.06.2014 09:06, Sebastian Kostuch wrote:
>>>>>>>>>>> Hi Will
>>>>>>>>>>> indeed these changes results in higher speeds (I was able to get 
>>>>>>>>>>> ~760 mbps
>>>>>>>>>>> which is quite similar to yours). Also I have checked flash 
>>>>>>>>>>> client as background
>>>>>>>>>>> for new JS UI and both speeds are being updated correctly in 
>>>>>>>>>>> real-time. This is
>>>>>>>>>>> really great work! Thanks for this. I have merged these changes 
>>>>>>>>>>> into trunk so
>>>>>>>>>>> they would be also present in new release.
>>>>>>>>>>> Also as you probably have noticed I have fixed compilation error 
>>>>>>>>>>> when using
>>>>>>>>>>> web100 so I closed appropriated issue related to this.
>>>>>>>>>>> 
>>>>>>>>>>> Regards
>>>>>>>>>>> Sebastian
>>>>>>>>>>> 
>>>>>>>>>>> On 17.06.2014 22:52, Will Hawkins wrote:
>>>>>>>>>>>> Hello Sebastian and Aaron (and MLab'ers CC'd),
>>>>>>>>>>>> 
>>>>>>>>>>>> We found a way to juice the Flash client and achieve the 
>>>>>>>>>>>> performance
>>>>>>>>>>>> necessary to accurately measure high-speed network connections.
>>>>>>>>>>>> 
>>>>>>>>>>>> The change is in r1082.
>>>>>>>>>>>> 
>>>>>>>>>>>> The solution was counterintuitive (at least for me). Instead of
>>>>>>>>>>>> attempting to read data from the socket as quickly as possible, 
>>>>>>>>>>>> the
>>>>>>>>>>>> TestS2C class just gets out of the way for the duration of the 
>>>>>>>>>>>> test. It
>>>>>>>>>>>> registers only for the "socket close" event. The runtime has the
>>>>>>>>>>>> necessary speed to keep up with the network.
>>>>>>>>>>>> 
>>>>>>>>>>>> The only problem with this method is keeping the user interface
>>>>>>>>>>>> up-to-date on progress throughout the test. Sebastian, this is 
>>>>>>>>>>>> where I
>>>>>>>>>>>> need you to double check. I believe that the code in r1082 
>>>>>>>>>>>> properly
>>>>>>>>>>>> updates _s2cByteCount in the appropriate spot for those real-time
>>>>>>>>>>>> updates (specifically onSpeedUpdate). But, please double check!
>>>>>>>>>>>> 
>>>>>>>>>>>> We were able to measure a connection at ~ 775 mbps using the 
>>>>>>>>>>>> Flash
>>>>>>>>>>>> client, but it would be great to have as many people test this as
>>>>>>>>>>>> possible. To do so, you can use a pre-compiled version at
>>>>>>>>>>>> 
>>>>>>>>>>>> http://files.opentechinstitute.org/~hawkinsw/flash/test.html
>>>>>>>>>>>> 
>>>>>>>>>>>> On a separate, but related note, please also review r1083 for 
>>>>>>>>>>>> inclusion
>>>>>>>>>>>> in the NDT release. This change allows the client to re-query JS 
>>>>>>>>>>>> for the
>>>>>>>>>>>> test's hostname just before launching a test. This gives the 
>>>>>>>>>>>> wrapper
>>>>>>>>>>>> page the opportunity to do a AJAX query to determine the test's
>>>>>>>>>>>> hostname. This is important for MLab because we do dynamic server
>>>>>>>>>>>> selection using a web-based API.
>>>>>>>>>>>> 
>>>>>>>>>>>> I am eager to hear whether you have success with these changes 
>>>>>>>>>>>> or not. I
>>>>>>>>>>>> hope you do!
>>>>>>>>>>>> 
>>>>>>>>>>>> Will
>>>>>>>>>>>> 
>>>>>>>>>>>> On 06/17/2014 10:53 AM, Will Hawkins wrote:
>>>>>>>>>>>>> Just FYI: I believe that we have happened on to an actual 
>>>>>>>>>>>>> solution for
>>>>>>>>>>>>> the "limit" to the performance of the S2C test. I am going to 
>>>>>>>>>>>>> implement
>>>>>>>>>>>>> it today and do some testing.
>>>>>>>>>>>>> 
>>>>>>>>>>>>> If possible, could we hold out on doing a release for a little 
>>>>>>>>>>>>> while
>>>>>>>>>>>>> pending this investigation?
>>>>>>>>>>>>> 
>>>>>>>>>>>>> Will
>>>>>>>>>>>>> 
>>>>>>>>>>>>> On 06/16/2014 01:50 AM, Sebastian Kostuch wrote:
>>>>>>>>>>>>>> Hi Will,
>>>>>>>>>>>>>> what you mentioned seems reasonable. Would it be possible to 
>>>>>>>>>>>>>> merge these
>>>>>>>>>>>>>> needed
>>>>>>>>>>>>>> changes into trunk in nearest time so we could make new 
>>>>>>>>>>>>>> release for
>>>>>>>>>>>>>> example tomorrow
>>>>>>>>>>>>>> or the day after tomorrow?
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>> Sebastian
>>>>>>>>>>>>>> 
>>>>>>>>>>>>>> On 13.06.2014 17:36, Will Hawkins wrote:
>>>>>>>>>>>>>>> Sebastian,
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Thanks for sending this out! We (at MLab) are very interested 
>>>>>>>>>>>>>>> in
>>>>>>>>>>>>>>> deploying a new version of the code so this 'release' looks 
>>>>>>>>>>>>>>> like a
>>>>>>>>>>>>>>> good place to fit together! :-) In other words, we (again, 
>>>>>>>>>>>>>>> MLab) are
>>>>>>>>>>>>>>> definitely going to do what you said in 3!
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> I am out of the office, but I wanted to quickly respond to 
>>>>>>>>>>>>>>> point 2.
>>>>>>>>>>>>>>> See below!
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Thanks again for driving this process forward!!
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Will
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> On 6/13/14, 4:47 AM, Sebastian Kostuch wrote:
>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>> NDT has been updated recently with some major changes (like
>>>>>>>>>>>>>>>> MSG_EXTENDED_LOGIN,
>>>>>>>>>>>>>>>> json support, new JS UI) so as I have mentioned in some 
>>>>>>>>>>>>>>>> earlier mail
>>>>>>>>>>>>>>>> that would be probably
>>>>>>>>>>>>>>>> good time to make new release. I have few questions related 
>>>>>>>>>>>>>>>> to this:
>>>>>>>>>>>>>>>> 1. New JS UI would fit well within this new release, however 
>>>>>>>>>>>>>>>> changes
>>>>>>>>>>>>>>>> related to this are still
>>>>>>>>>>>>>>>> on work branches and need review. Any feedback would be very
>>>>>>>>>>>>>>>> appreciated. These are
>>>>>>>>>>>>>>>> following issues: 129, 132, 133.
>>>>>>>>>>>>>>>> 2. Is Issue144 (URLLoader to flash) supposed to be contained 
>>>>>>>>>>>>>>>> also in
>>>>>>>>>>>>>>>> nearest release version?
>>>>>>>>>>>>>>>> If yes, then how does progress with work on this look like? 
>>>>>>>>>>>>>>>> If I recall
>>>>>>>>>>>>>>>> correctly then you, Will,
>>>>>>>>>>>>>>>> mentioned that they are well tested and thus almost ready, 
>>>>>>>>>>>>>>>> right?
>>>>>>>>>>>>>>> Some of these are still up for debate. I am going to forward 
>>>>>>>>>>>>>>> an email
>>>>>>>>>>>>>>> that I just sent to the MLab team with an update on the 
>>>>>>>>>>>>>>> conundrum
>>>>>>>>>>>>>>> around the HTTP test to give you a sense about why they are 
>>>>>>>>>>>>>>> up for
>>>>>>>>>>>>>>> debate.
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> But, there are also changes in that branch that *need* to be
>>>>>>>>>>>>>>> incorporated into anything that gets released. For instance, 
>>>>>>>>>>>>>>> r1070.
>>>>>>>>>>>>>>> There are parts of r1064 that will have to be integrated no 
>>>>>>>>>>>>>>> matter
>>>>>>>>>>>>>>> what (any of the changes that have to do with splitting start 
>>>>>>>>>>>>>>> test and
>>>>>>>>>>>>>>> finalize test stages).
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>> Does that make sense?
>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> 3. Along with new release on googlecode what do you think 
>>>>>>>>>>>>>>>> about pushing
>>>>>>>>>>>>>>>> these changes also
>>>>>>>>>>>>>>>> on M-Lab site? They bring many new features at all.
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Looking forward to your answers and I hope that next week 
>>>>>>>>>>>>>>>> could be
>>>>>>>>>>>>>>>> celebrated with making
>>>>>>>>>>>>>>>> new release of NDT :).
>>>>>>>>>>>>>>>> 
>>>>>>>>>>>>>>>> Kind regards
>>>>>>>>>>>>>>>> Sebastian Kostuch
>>>>>>>>>>>>>>>> 
>>>>>>>>>>> 
>>>>>>>>> 
>>>>>>> 
>>>>>> 
>>>> 
>>