NDT-DEV email list created

[Will Hawkins <>]

How about this?

Index: src/Main.as
===================================================================
--- src/Main.as (revision 1112)
+++ src/Main.as (working copy)
@@ -16,6 +16,7 @@
   import flash.display.Sprite;
   import flash.events.Event;
   import mx.resources.ResourceBundle;
+  import mx.utils.URLUtil;
   /**
    * @author Anant Subramanian
    */
@@ -45,6 +46,11 @@
     private function init(e:Event = null):void {
       removeEventListener(Event.ADDED_TO_STAGE, init);

+      var selfUrl:String = null;
+      if (selfUrl = URLUtil.getServerName(this.loaderInfo.url)) {
+        server_hostname = selfUrl;
+      }
+
       // Set the properties of the SWF from HTML tags.
       NDTUtils.initializeFromHTML(this.root.loaderInfo.parameters);


Will

On 06/26/2014 01:50 PM, Will Hawkins wrote:
> 
> 
> On 06/26/2014 09:13 AM, Aaron Brown wrote:
>> Hey Will,
>>
>> On Jun 25, 2014, at 9:51 PM, Will Hawkins 
>> <>
>>  wrote:
>>
>>> I am happy to create a branch for this, but I think that might be
>>> overkill. Here's what I added
>>>
>>> Index: src/Main.as
>>> ===================================================================
>>> --- src/Main.as     (revision 1112)
>>> +++ src/Main.as     (working copy)
>>> @@ -45,6 +45,14 @@
>>>     private function init(e:Event = null):void {
>>>       removeEventListener(Event.ADDED_TO_STAGE, init);
>>>
>>> +      var selfUrl:String = this.loaderInfo.url;
>>> +      if (selfUrl.indexOf("http://";, 0) == 0) {
>>> +        selfUrl = selfUrl.replace("http://";, "");
>>> +        selfUrl = selfUrl.replace(/:[0-9]+/, "");
>>> +        selfUrl = selfUrl.replace(/\/.+$/, "");
>>> +        server_hostname = selfUrl;
>>> +      }
>>> +
>>>       // Set the properties of the SWF from HTML tags.
>>>       NDTUtils.initializeFromHTML(this.root.loaderInfo.parameters);
>>
>> Looking at the parsing there, it will definitely fail on IPv6 addresses. 
>> It looks like actionscript has a URL library 
>> (http://help.adobe.com/en_US/FlashPlatform/reference/actionscript/3/mx/utils/URLUtil.html)
>>  that would probably work for returning the address to test against.
>>
> 
> Oh, I get it. You don't think that two quickly-hacked-together regexes
> covers all possible cases? You'd like me to use a library?
> 
> NO. :-)
> 
> Just kidding, of course. I was actually hoping that loaderInfo.url
> returned some type of object that I could easy inquire about the
> hostname. But, it seems like the URL library is the way to go. Standby
> for another rev.
> 
> Will
> 
> 
>>> This will make it default to test against an NDT server running on the
>>> same host from which the swf was loaded. This does not preclude the user
>>> from overriding that value with their JS callback.
>>>
>>> The other, perhaps easier, option is to do something similar in the JS
>>> that hosts the Flash applet. JS would parse window.location and then
>>> return that value in the getNDTServer() callback.
>>>
>>> Let me know which route you prefer!
>>
>> My general preference would be to encode that default into the Flash 
>> client to make it as simple as possible to embed the client in webpages.
>>
>> Cheers,
>> Aaron
>>
>>
>>> Will
>>>
>>> On 06/25/2014 02:22 PM, Aaron Brown wrote:
>>>> Hey Will,
>>>>
>>>> On Jun 25, 2014, at 1:59 PM, Will Hawkins 
>>>> <>
>>>>  wrote:
>>>>
>>>>> Hey Aaron!
>>>>>
>>>>> I am working on this -- I promise :-)
>>>>
>>>> Excellent, no rush :)
>>>>
>>>>> Just to get some context, are you assuming that we would include an HTML
>>>>> wrapper for the Flash client and invoke the wrapped version to actually
>>>>> do the self testing? I'm just trying to figure out which of the
>>>>> Actionscript facilities I'll have access to in order to resolve this
>>>>> dilemma.
>>>>
>>>> That’s what I’m thinking. Similar to how the java applet works (e.g. 
>>>> http://desk146.internet2.edu:7123/), you’d go to a webpage and the flash 
>>>> client would be embedded in the page. You could then just hit “start”, 
>>>> and it’d do a test from your machine to the server that served it.
>>>>
>>>> Cheers,
>>>> Aaron
>>>>
>>>>>
>>>>> Will
>>>>>
>>>>> On 06/24/2014 10:13 AM, Aaron Brown wrote:
>>>>>> Hey Will,
>>>>>>
>>>>>> I’ve got the RPM working with the existing fakewww/Applet.
>>>>>>
>>>>>> I’d like to include the flash client in the RPM if possible 
>>>>>> (otherwise, there’s not much benefit to the average RPM install for 
>>>>>> the new release). Unfortunately, it doesn’t look like there’s a good 
>>>>>> way to zero-configuration install (i.e. have it by default test back 
>>>>>> to the server it was served from) without using something like PHP 
>>>>>> (which would require swapping over to using Apache to serve the 
>>>>>> clients). Would it be possible to include a setting for the flash 
>>>>>> client to say “the server to use is the address of the server you 
>>>>>> downloaded the SWF from”?
>>>>>>
>>>>>> Cheers,
>>>>>> Aaron
>>>>>>
>>>>>> On Jun 20, 2014, at 8:02 PM, Will Hawkins 
>>>>>> <>
>>>>>>  wrote:
>>>>>>
>>>>>>> Please let me know if/how I can help preparing the repo for RPM
>>>>>>> creation. I'm happy to give it a try on my own, but if there is
>>>>>>> expertise elsewhere (Aaron or Sebastian), then I am happy to take a 
>>>>>>> back
>>>>>>> seat.
>>>>>>>
>>>>>>> As for testers, we (MLab) can definitely find a few places to test. I
>>>>>>> know that we've been running trunk versions of the server on several
>>>>>>> test nodes for a while and haven't had any problems (besides the ones
>>>>>>> that I've submitted patches for).
>>>>>>>
>>>>>>> Will
>>>>>>>
>>>>>>> On 06/20/2014 09:54 AM, Aaron Brown wrote:
>>>>>>>> Hey Sebastian,
>>>>>>>>
>>>>>>>> I’m untagging the release since I can guarantee we’ll need to make 
>>>>>>>> changes to the codebase for the NDT RPMs, and those are a 
>>>>>>>> requirement for outside testing since a very small percentage of NDT 
>>>>>>>> servers out there are compiled from source. I’ll commit some of the 
>>>>>>>> RPM bits, and then if you could go through, and make sure that it 
>>>>>>>> gets updated as needed for the changes (new web UI, etc), that’d be 
>>>>>>>> good. The goal of the NDT RPMs is that the administrators can do 
>>>>>>>> “yum install ndt-server”, start NDT and it just works. They don’t 
>>>>>>>> have to do any hand configuration to get an NDT server 
>>>>>>>> up-and-running.
>>>>>>>>
>>>>>>>> Cheers,
>>>>>>>> Aaron
>>>>>>>>
>>>>>>>> On Jun 20, 2014, at 5:15 AM, Sebastian Kostuch 
>>>>>>>> <>
>>>>>>>>  wrote:
>>>>>>>>
>>>>>>>>> Hi
>>>>>>>>> Will, I have merged your last fixes into trunk so they will be 
>>>>>>>>> contained within
>>>>>>>>> new release :).
>>>>>>>>> Aaron, keeping in mind what you have said I have changed version 
>>>>>>>>> numbers
>>>>>>>>> to be release candidate and tagged this. Such tagged version is now 
>>>>>>>>> ready
>>>>>>>>> for being tested by outside folks. I hope that test stage will 
>>>>>>>>> perform smoothly
>>>>>>>>> and we would release new NDT version ASAP!
>>>>>>>>> Also I have updated wiki pages to match new protocol changes.
>>>>>>>>>
>>>>>>>>> Regards
>>>>>>>>> Sebastian Kostuch
>>>>>>>>>
>>>>>>>>> On 18.06.2014 23:00, Will Hawkins wrote:
>>>>>>>>>> Aaron,
>>>>>>>>>>
>>>>>>>>>> Thanks for the information! We (M-Lab) are especially interested in
>>>>>>>>>>
>>>>>>>>>> a) having the code "checked" to make sure that the measurements are
>>>>>>>>>> still valid
>>>>>>>>>>
>>>>>>>>>> and
>>>>>>>>>>
>>>>>>>>>> b) having a release to deploy on the platform.
>>>>>>>>>>
>>>>>>>>>> Having said that, what is the best way to prepare a release 
>>>>>>>>>> candidate to
>>>>>>>>>> start the process rolling? Are you able to help verify the 
>>>>>>>>>> measurements
>>>>>>>>>> made with the new version of the tool? Will you be able to help,
>>>>>>>>>> generally, with the release process (since, I mean, you're 
>>>>>>>>>> awesome!)?
>>>>>>>>>>
>>>>>>>>>> Thanks again for your email!
>>>>>>>>>> Will
>>>>>>>>>>
>>>>>>>>>> On 06/18/2014 04:46 PM, Aaron Brown wrote:
>>>>>>>>>>> Hi Sebastian,
>>>>>>>>>>>
>>>>>>>>>>> To do a full release, NDT will need to go through a release 
>>>>>>>>>>> candidate process to ensure that outside folks have a chance to 
>>>>>>>>>>> test it before something gets labeled as “released”. Beyond that, 
>>>>>>>>>>> NDT will need verified to make sure that something didn’t break 
>>>>>>>>>>> with existing clients (important since this is a less common 
>>>>>>>>>>> use-case). The upgrade process needs to be verified as well to 
>>>>>>>>>>> make sure that it doesn’t break an existing installation when 
>>>>>>>>>>> someone goes to upgrade.
>>>>>>>>>>>
>>>>>>>>>>> As part of the upgrade procedure above, the RPMs will need 
>>>>>>>>>>> updated with any changes in the codebase since that’s the primary 
>>>>>>>>>>> method folks use to install NDT (the vast majority of NDT 
>>>>>>>>>>> installations in the wild are installed as part of the perfSONAR 
>>>>>>>>>>> Performance Toolkit). This procedure will need tested to make 
>>>>>>>>>>> sure that both new installs, and existing installs work as 
>>>>>>>>>>> expected.
>>>>>>>>>>>
>>>>>>>>>>> Once there’s a final release, the NDT RPMs will get added to the 
>>>>>>>>>>> Internet2 RPM repository (where the Toolkit users get their RPMs 
>>>>>>>>>>> from), and the source tarball will be added to the Internet2 site 
>>>>>>>>>>> at software.internet2.edu). Presumably, this link could be used 
>>>>>>>>>>> as canonical for NDT since any linked location is arbitrary from 
>>>>>>>>>>> the end user perspective.
>>>>>>>>>>>
>>>>>>>>>>> Cheers,
>>>>>>>>>>> Aaron
>>>>>>>>>>>
>>>>>>>>>>> On Jun 18, 2014, at 6:04 AM, Sebastian Kostuch 
>>>>>>>>>>> <>
>>>>>>>>>>>  wrote:
>>>>>>>>>>>
>>>>>>>>>>>> Hi again,
>>>>>>>>>>>> new NDT version is coming soon, however there are last changes 
>>>>>>>>>>>> to be performed
>>>>>>>>>>>> before this. I have merged all changes to release into trunk, 
>>>>>>>>>>>> tested and fixed individual
>>>>>>>>>>>> small bugs. Next step will be to update our wiki pages to match 
>>>>>>>>>>>> new version which I will
>>>>>>>>>>>> be working on in nearest time.
>>>>>>>>>>>> However I wanted to ask two last questions to you:
>>>>>>>>>>>> 1. So far new release package has been published in "downloads" 
>>>>>>>>>>>> section of google code
>>>>>>>>>>>> project but recently Google has disabled this option. Therefore 
>>>>>>>>>>>> should we place such archive
>>>>>>>>>>>> in some external source like Google drive where users could 
>>>>>>>>>>>> directly download this (and update
>>>>>>>>>>>> wiki with such link)? Or maybe just publish link to release 
>>>>>>>>>>>> tagged revision?
>>>>>>>>>>>>
>>>>>>>>>>>> 2. Also what about updating NDT links/software on another sites? 
>>>>>>>>>>>> I know that it will be performed
>>>>>>>>>>>> on M-lab but what about Internet2 website?
>>>>>>>>>>>>
>>>>>>>>>>>> Regards
>>>>>>>>>>>> Sebastian
>>>>>>>>>>>>
>>>>>>>>>>>> On 18.06.2014 09:06, Sebastian Kostuch wrote:
>>>>>>>>>>>>> Hi Will
>>>>>>>>>>>>> indeed these changes results in higher speeds (I was able to 
>>>>>>>>>>>>> get ~760 mbps
>>>>>>>>>>>>> which is quite similar to yours). Also I have checked flash 
>>>>>>>>>>>>> client as background
>>>>>>>>>>>>> for new JS UI and both speeds are being updated correctly in 
>>>>>>>>>>>>> real-time. This is
>>>>>>>>>>>>> really great work! Thanks for this. I have merged these changes 
>>>>>>>>>>>>> into trunk so
>>>>>>>>>>>>> they would be also present in new release.
>>>>>>>>>>>>> Also as you probably have noticed I have fixed compilation 
>>>>>>>>>>>>> error when using
>>>>>>>>>>>>> web100 so I closed appropriated issue related to this.
>>>>>>>>>>>>>
>>>>>>>>>>>>> Regards
>>>>>>>>>>>>> Sebastian
>>>>>>>>>>>>>
>>>>>>>>>>>>> On 17.06.2014 22:52, Will Hawkins wrote:
>>>>>>>>>>>>>> Hello Sebastian and Aaron (and MLab'ers CC'd),
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We found a way to juice the Flash client and achieve the 
>>>>>>>>>>>>>> performance
>>>>>>>>>>>>>> necessary to accurately measure high-speed network connections.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The change is in r1082.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The solution was counterintuitive (at least for me). Instead of
>>>>>>>>>>>>>> attempting to read data from the socket as quickly as 
>>>>>>>>>>>>>> possible, the
>>>>>>>>>>>>>> TestS2C class just gets out of the way for the duration of the 
>>>>>>>>>>>>>> test. It
>>>>>>>>>>>>>> registers only for the "socket close" event. The runtime has 
>>>>>>>>>>>>>> the
>>>>>>>>>>>>>> necessary speed to keep up with the network.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> The only problem with this method is keeping the user interface
>>>>>>>>>>>>>> up-to-date on progress throughout the test. Sebastian, this is 
>>>>>>>>>>>>>> where I
>>>>>>>>>>>>>> need you to double check. I believe that the code in r1082 
>>>>>>>>>>>>>> properly
>>>>>>>>>>>>>> updates _s2cByteCount in the appropriate spot for those 
>>>>>>>>>>>>>> real-time
>>>>>>>>>>>>>> updates (specifically onSpeedUpdate). But, please double check!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> We were able to measure a connection at ~ 775 mbps using the 
>>>>>>>>>>>>>> Flash
>>>>>>>>>>>>>> client, but it would be great to have as many people test this 
>>>>>>>>>>>>>> as
>>>>>>>>>>>>>> possible. To do so, you can use a pre-compiled version at
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> http://files.opentechinstitute.org/~hawkinsw/flash/test.html
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On a separate, but related note, please also review r1083 for 
>>>>>>>>>>>>>> inclusion
>>>>>>>>>>>>>> in the NDT release. This change allows the client to re-query 
>>>>>>>>>>>>>> JS for the
>>>>>>>>>>>>>> test's hostname just before launching a test. This gives the 
>>>>>>>>>>>>>> wrapper
>>>>>>>>>>>>>> page the opportunity to do a AJAX query to determine the test's
>>>>>>>>>>>>>> hostname. This is important for MLab because we do dynamic 
>>>>>>>>>>>>>> server
>>>>>>>>>>>>>> selection using a web-based API.
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> I am eager to hear whether you have success with these changes 
>>>>>>>>>>>>>> or not. I
>>>>>>>>>>>>>> hope you do!
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> Will
>>>>>>>>>>>>>>
>>>>>>>>>>>>>> On 06/17/2014 10:53 AM, Will Hawkins wrote:
>>>>>>>>>>>>>>> Just FYI: I believe that we have happened on to an actual 
>>>>>>>>>>>>>>> solution for
>>>>>>>>>>>>>>> the "limit" to the performance of the S2C test. I am going to 
>>>>>>>>>>>>>>> implement
>>>>>>>>>>>>>>> it today and do some testing.
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> If possible, could we hold out on doing a release for a 
>>>>>>>>>>>>>>> little while
>>>>>>>>>>>>>>> pending this investigation?
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> Will
>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>> On 06/16/2014 01:50 AM, Sebastian Kostuch wrote:
>>>>>>>>>>>>>>>> Hi Will,
>>>>>>>>>>>>>>>> what you mentioned seems reasonable. Would it be possible to 
>>>>>>>>>>>>>>>> merge these
>>>>>>>>>>>>>>>> needed
>>>>>>>>>>>>>>>> changes into trunk in nearest time so we could make new 
>>>>>>>>>>>>>>>> release for
>>>>>>>>>>>>>>>> example tomorrow
>>>>>>>>>>>>>>>> or the day after tomorrow?
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> Regards
>>>>>>>>>>>>>>>> Sebastian
>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>> On 13.06.2014 17:36, Will Hawkins wrote:
>>>>>>>>>>>>>>>>> Sebastian,
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks for sending this out! We (at MLab) are very 
>>>>>>>>>>>>>>>>> interested in
>>>>>>>>>>>>>>>>> deploying a new version of the code so this 'release' looks 
>>>>>>>>>>>>>>>>> like a
>>>>>>>>>>>>>>>>> good place to fit together! :-) In other words, we (again, 
>>>>>>>>>>>>>>>>> MLab) are
>>>>>>>>>>>>>>>>> definitely going to do what you said in 3!
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> I am out of the office, but I wanted to quickly respond to 
>>>>>>>>>>>>>>>>> point 2.
>>>>>>>>>>>>>>>>> See below!
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Thanks again for driving this process forward!!
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Will
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> On 6/13/14, 4:47 AM, Sebastian Kostuch wrote:
>>>>>>>>>>>>>>>>>> Hello,
>>>>>>>>>>>>>>>>>> NDT has been updated recently with some major changes (like
>>>>>>>>>>>>>>>>>> MSG_EXTENDED_LOGIN,
>>>>>>>>>>>>>>>>>> json support, new JS UI) so as I have mentioned in some 
>>>>>>>>>>>>>>>>>> earlier mail
>>>>>>>>>>>>>>>>>> that would be probably
>>>>>>>>>>>>>>>>>> good time to make new release. I have few questions 
>>>>>>>>>>>>>>>>>> related to this:
>>>>>>>>>>>>>>>>>> 1. New JS UI would fit well within this new release, 
>>>>>>>>>>>>>>>>>> however changes
>>>>>>>>>>>>>>>>>> related to this are still
>>>>>>>>>>>>>>>>>> on work branches and need review. Any feedback would be 
>>>>>>>>>>>>>>>>>> very
>>>>>>>>>>>>>>>>>> appreciated. These are
>>>>>>>>>>>>>>>>>> following issues: 129, 132, 133.
>>>>>>>>>>>>>>>>>> 2. Is Issue144 (URLLoader to flash) supposed to be 
>>>>>>>>>>>>>>>>>> contained also in
>>>>>>>>>>>>>>>>>> nearest release version?
>>>>>>>>>>>>>>>>>> If yes, then how does progress with work on this look 
>>>>>>>>>>>>>>>>>> like? If I recall
>>>>>>>>>>>>>>>>>> correctly then you, Will,
>>>>>>>>>>>>>>>>>> mentioned that they are well tested and thus almost ready, 
>>>>>>>>>>>>>>>>>> right?
>>>>>>>>>>>>>>>>> Some of these are still up for debate. I am going to 
>>>>>>>>>>>>>>>>> forward an email
>>>>>>>>>>>>>>>>> that I just sent to the MLab team with an update on the 
>>>>>>>>>>>>>>>>> conundrum
>>>>>>>>>>>>>>>>> around the HTTP test to give you a sense about why they are 
>>>>>>>>>>>>>>>>> up for
>>>>>>>>>>>>>>>>> debate.
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> But, there are also changes in that branch that *need* to be
>>>>>>>>>>>>>>>>> incorporated into anything that gets released. For 
>>>>>>>>>>>>>>>>> instance, r1070.
>>>>>>>>>>>>>>>>> There are parts of r1064 that will have to be integrated no 
>>>>>>>>>>>>>>>>> matter
>>>>>>>>>>>>>>>>> what (any of the changes that have to do with splitting 
>>>>>>>>>>>>>>>>> start test and
>>>>>>>>>>>>>>>>> finalize test stages).
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>> Does that make sense?
>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> 3. Along with new release on googlecode what do you think 
>>>>>>>>>>>>>>>>>> about pushing
>>>>>>>>>>>>>>>>>> these changes also
>>>>>>>>>>>>>>>>>> on M-Lab site? They bring many new features at all.
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Looking forward to your answers and I hope that next week 
>>>>>>>>>>>>>>>>>> could be
>>>>>>>>>>>>>>>>>> celebrated with making
>>>>>>>>>>>>>>>>>> new release of NDT :).
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>>>>>> Kind regards
>>>>>>>>>>>>>>>>>> Sebastian Kostuch
>>>>>>>>>>>>>>>>>>
>>>>>>>>>>>>>
>>>>>>>>>>>
>>>>>>>>>
>>>>>>>>
>>>>>>
>>>>
>>