NDT-DEV email list created

[Sebastian Kostuch <>]

Hi Will, hi Aaron!
I just would like to announce that if there is anything I can help with to
bring forward new NDT release then I will eagerly participate :).
Also, Will, when it comes to your last changes and question then IMHO
getting URL directly from flash client is good idea and might stay this way.
Also these changes aren't so major and looks good so probably we could
merge them ASAP?

Regards
Sebastian

On 26.06.2014 03:51, Will Hawkins wrote:
> I am happy to create a branch for this, but I think that might be
> overkill. Here's what I added
>
> Index: src/Main.as
> ===================================================================
> --- src/Main.as (revision 1112)
> +++ src/Main.as (working copy)
> @@ -45,6 +45,14 @@
>       private function init(e:Event = null):void {
>         removeEventListener(Event.ADDED_TO_STAGE, init);
>
> +      var selfUrl:String = this.loaderInfo.url;
> +      if (selfUrl.indexOf("http://";, 0) == 0) {
> +        selfUrl = selfUrl.replace("http://";, "");
> +        selfUrl = selfUrl.replace(/:[0-9]+/, "");
> +        selfUrl = selfUrl.replace(/\/.+$/, "");
> +        server_hostname = selfUrl;
> +      }
> +
>         // Set the properties of the SWF from HTML tags.
>         NDTUtils.initializeFromHTML(this.root.loaderInfo.parameters);
>
> This will make it default to test against an NDT server running on the
> same host from which the swf was loaded. This does not preclude the user
> from overriding that value with their JS callback.
>
> The other, perhaps easier, option is to do something similar in the JS
> that hosts the Flash applet. JS would parse window.location and then
> return that value in the getNDTServer() callback.
>
> Let me know which route you prefer!
> Will
>
> On 06/25/2014 02:22 PM, Aaron Brown wrote:
> > Hey Will,
> >
> > On Jun 25, 2014, at 1:59 PM, Will Hawkins 
> > <>
> >  wrote:
> >
> > > Hey Aaron!
> > >
> > > I am working on this -- I promise :-)
> > Excellent, no rush :)
> >
> > > Just to get some context, are you assuming that we would include an HTML
> > > wrapper for the Flash client and invoke the wrapped version to actually
> > > do the self testing? I'm just trying to figure out which of the
> > > Actionscript facilities I'll have access to in order to resolve this
> > > dilemma.
> > That’s what I’m thinking. Similar to how the java applet works (e.g. 
> > http://desk146.internet2.edu:7123/), you’d go to a webpage and the flash 
> > client would be embedded in the page. You could then just hit “start”, and 
> > it’d do a test from your machine to the server that served it.
> >
> > Cheers,
> > Aaron
> >
> > > Will
> > >
> > > On 06/24/2014 10:13 AM, Aaron Brown wrote:
> > > > Hey Will,
> > > >
> > > > I’ve got the RPM working with the existing fakewww/Applet.
> > > >
> > > > I’d like to include the flash client in the RPM if possible (otherwise, 
> > > > there’s not much benefit to the average RPM install for the new release). 
> > > > Unfortunately, it doesn’t look like there’s a good way to zero-configuration 
> > > > install (i.e. have it by default test back to the server it was served from) 
> > > > without using something like PHP (which would require swapping over to using 
> > > > Apache to serve the clients). Would it be possible to include a setting for 
> > > > the flash client to say “the server to use is the address of the server you 
> > > > downloaded the SWF from”?
> > > >
> > > > Cheers,
> > > > Aaron
> > > >
> > > > On Jun 20, 2014, at 8:02 PM, Will Hawkins 
> > > > <>
> > > >  wrote:
> > > >
> > > > > Please let me know if/how I can help preparing the repo for RPM
> > > > > creation. I'm happy to give it a try on my own, but if there is
> > > > > expertise elsewhere (Aaron or Sebastian), then I am happy to take a back
> > > > > seat.
> > > > >
> > > > > As for testers, we (MLab) can definitely find a few places to test. I
> > > > > know that we've been running trunk versions of the server on several
> > > > > test nodes for a while and haven't had any problems (besides the ones
> > > > > that I've submitted patches for).
> > > > >
> > > > > Will
> > > > >
> > > > > On 06/20/2014 09:54 AM, Aaron Brown wrote:
> > > > > > Hey Sebastian,
> > > > > >
> > > > > > I’m untagging the release since I can guarantee we’ll need to make changes to 
> > > > > > the codebase for the NDT RPMs, and those are a requirement for outside 
> > > > > > testing since a very small percentage of NDT servers out there are compiled 
> > > > > > from source. I’ll commit some of the RPM bits, and then if you could go 
> > > > > > through, and make sure that it gets updated as needed for the changes (new 
> > > > > > web UI, etc), that’d be good. The goal of the NDT RPMs is that the 
> > > > > > administrators can do “yum install ndt-server”, start NDT and it just works. 
> > > > > > They don’t have to do any hand configuration to get an NDT server 
> > > > > > up-and-running.
> > > > > >
> > > > > > Cheers,
> > > > > > Aaron
> > > > > >
> > > > > > On Jun 20, 2014, at 5:15 AM, Sebastian Kostuch 
> > > > > > <>
> > > > > >  wrote:
> > > > > >
> > > > > > > Hi
> > > > > > > Will, I have merged your last fixes into trunk so they will be contained 
> > > > > > > within
> > > > > > > new release :).
> > > > > > > Aaron, keeping in mind what you have said I have changed version numbers
> > > > > > > to be release candidate and tagged this. Such tagged version is now ready
> > > > > > > for being tested by outside folks. I hope that test stage will perform 
> > > > > > > smoothly
> > > > > > > and we would release new NDT version ASAP!
> > > > > > > Also I have updated wiki pages to match new protocol changes.
> > > > > > >
> > > > > > > Regards
> > > > > > > Sebastian Kostuch
> > > > > > >
> > > > > > > On 18.06.2014 23:00, Will Hawkins wrote:
> > > > > > > > Aaron,
> > > > > > > >
> > > > > > > > Thanks for the information! We (M-Lab) are especially interested in
> > > > > > > >
> > > > > > > > a) having the code "checked" to make sure that the measurements are
> > > > > > > > still valid
> > > > > > > >
> > > > > > > > and
> > > > > > > >
> > > > > > > > b) having a release to deploy on the platform.
> > > > > > > >
> > > > > > > > Having said that, what is the best way to prepare a release candidate to
> > > > > > > > start the process rolling? Are you able to help verify the measurements
> > > > > > > > made with the new version of the tool? Will you be able to help,
> > > > > > > > generally, with the release process (since, I mean, you're awesome!)?
> > > > > > > >
> > > > > > > > Thanks again for your email!
> > > > > > > > Will
> > > > > > > >
> > > > > > > > On 06/18/2014 04:46 PM, Aaron Brown wrote:
> > > > > > > > > Hi Sebastian,
> > > > > > > > >
> > > > > > > > > To do a full release, NDT will need to go through a release candidate process 
> > > > > > > > > to ensure that outside folks have a chance to test it before something gets 
> > > > > > > > > labeled as “released”. Beyond that, NDT will need verified to make sure that 
> > > > > > > > > something didn’t break with existing clients (important since this is a less 
> > > > > > > > > common use-case). The upgrade process needs to be verified as well to make 
> > > > > > > > > sure that it doesn’t break an existing installation when someone goes to 
> > > > > > > > > upgrade.
> > > > > > > > >
> > > > > > > > > As part of the upgrade procedure above, the RPMs will need updated with any 
> > > > > > > > > changes in the codebase since that’s the primary method folks use to install 
> > > > > > > > > NDT (the vast majority of NDT installations in the wild are installed as part 
> > > > > > > > > of the perfSONAR Performance Toolkit). This procedure will need tested to 
> > > > > > > > > make sure that both new installs, and existing installs work as expected.
> > > > > > > > >
> > > > > > > > > Once there’s a final release, the NDT RPMs will get added to the Internet2 
> > > > > > > > > RPM repository (where the Toolkit users get their RPMs from), and the source 
> > > > > > > > > tarball will be added to the Internet2 site at software.internet2.edu). 
> > > > > > > > > Presumably, this link could be used as canonical for NDT since any linked 
> > > > > > > > > location is arbitrary from the end user perspective.
> > > > > > > > >
> > > > > > > > > Cheers,
> > > > > > > > > Aaron
> > > > > > > > >
> > > > > > > > > On Jun 18, 2014, at 6:04 AM, Sebastian Kostuch 
> > > > > > > > > <>
> > > > > > > > >  wrote:
> > > > > > > > >
> > > > > > > > > > Hi again,
> > > > > > > > > > new NDT version is coming soon, however there are last changes to be performed
> > > > > > > > > > before this. I have merged all changes to release into trunk, tested and 
> > > > > > > > > > fixed individual
> > > > > > > > > > small bugs. Next step will be to update our wiki pages to match new version 
> > > > > > > > > > which I will
> > > > > > > > > > be working on in nearest time.
> > > > > > > > > > However I wanted to ask two last questions to you:
> > > > > > > > > > 1. So far new release package has been published in "downloads" section of 
> > > > > > > > > > google code
> > > > > > > > > > project but recently Google has disabled this option. Therefore should we 
> > > > > > > > > > place such archive
> > > > > > > > > > in some external source like Google drive where users could directly download 
> > > > > > > > > > this (and update
> > > > > > > > > > wiki with such link)? Or maybe just publish link to release tagged revision?
> > > > > > > > > >
> > > > > > > > > > 2. Also what about updating NDT links/software on another sites? I know that 
> > > > > > > > > > it will be performed
> > > > > > > > > > on M-lab but what about Internet2 website?
> > > > > > > > > >
> > > > > > > > > > Regards
> > > > > > > > > > Sebastian
> > > > > > > > > >
> > > > > > > > > > On 18.06.2014 09:06, Sebastian Kostuch wrote:
> > > > > > > > > > > Hi Will
> > > > > > > > > > > indeed these changes results in higher speeds (I was able to get ~760 mbps
> > > > > > > > > > > which is quite similar to yours). Also I have checked flash client as 
> > > > > > > > > > > background
> > > > > > > > > > > for new JS UI and both speeds are being updated correctly in real-time. This 
> > > > > > > > > > > is
> > > > > > > > > > > really great work! Thanks for this. I have merged these changes into trunk so
> > > > > > > > > > > they would be also present in new release.
> > > > > > > > > > > Also as you probably have noticed I have fixed compilation error when using
> > > > > > > > > > > web100 so I closed appropriated issue related to this.
> > > > > > > > > > >
> > > > > > > > > > > Regards
> > > > > > > > > > > Sebastian
> > > > > > > > > > >
> > > > > > > > > > > On 17.06.2014 22:52, Will Hawkins wrote:
> > > > > > > > > > > > Hello Sebastian and Aaron (and MLab'ers CC'd),
> > > > > > > > > > > >
> > > > > > > > > > > > We found a way to juice the Flash client and achieve the performance
> > > > > > > > > > > > necessary to accurately measure high-speed network connections.
> > > > > > > > > > > >
> > > > > > > > > > > > The change is in r1082.
> > > > > > > > > > > >
> > > > > > > > > > > > The solution was counterintuitive (at least for me). Instead of
> > > > > > > > > > > > attempting to read data from the socket as quickly as possible, the
> > > > > > > > > > > > TestS2C class just gets out of the way for the duration of the test. It
> > > > > > > > > > > > registers only for the "socket close" event. The runtime has the
> > > > > > > > > > > > necessary speed to keep up with the network.
> > > > > > > > > > > >
> > > > > > > > > > > > The only problem with this method is keeping the user interface
> > > > > > > > > > > > up-to-date on progress throughout the test. Sebastian, this is where I
> > > > > > > > > > > > need you to double check. I believe that the code in r1082 properly
> > > > > > > > > > > > updates _s2cByteCount in the appropriate spot for those real-time
> > > > > > > > > > > > updates (specifically onSpeedUpdate). But, please double check!
> > > > > > > > > > > >
> > > > > > > > > > > > We were able to measure a connection at ~ 775 mbps using the Flash
> > > > > > > > > > > > client, but it would be great to have as many people test this as
> > > > > > > > > > > > possible. To do so, you can use a pre-compiled version at
> > > > > > > > > > > >
> > > > > > > > > > > > http://files.opentechinstitute.org/~hawkinsw/flash/test.html
> > > > > > > > > > > >
> > > > > > > > > > > > On a separate, but related note, please also review r1083 for inclusion
> > > > > > > > > > > > in the NDT release. This change allows the client to re-query JS for the
> > > > > > > > > > > > test's hostname just before launching a test. This gives the wrapper
> > > > > > > > > > > > page the opportunity to do a AJAX query to determine the test's
> > > > > > > > > > > > hostname. This is important for MLab because we do dynamic server
> > > > > > > > > > > > selection using a web-based API.
> > > > > > > > > > > >
> > > > > > > > > > > > I am eager to hear whether you have success with these changes or not. I
> > > > > > > > > > > > hope you do!
> > > > > > > > > > > >
> > > > > > > > > > > > Will
> > > > > > > > > > > >
> > > > > > > > > > > > On 06/17/2014 10:53 AM, Will Hawkins wrote:
> > > > > > > > > > > > > Just FYI: I believe that we have happened on to an actual solution for
> > > > > > > > > > > > > the "limit" to the performance of the S2C test. I am going to implement
> > > > > > > > > > > > > it today and do some testing.
> > > > > > > > > > > > >
> > > > > > > > > > > > > If possible, could we hold out on doing a release for a little while
> > > > > > > > > > > > > pending this investigation?
> > > > > > > > > > > > >
> > > > > > > > > > > > > Will
> > > > > > > > > > > > >
> > > > > > > > > > > > > On 06/16/2014 01:50 AM, Sebastian Kostuch wrote:
> > > > > > > > > > > > > > Hi Will,
> > > > > > > > > > > > > > what you mentioned seems reasonable. Would it be possible to merge these
> > > > > > > > > > > > > > needed
> > > > > > > > > > > > > > changes into trunk in nearest time so we could make new release for
> > > > > > > > > > > > > > example tomorrow
> > > > > > > > > > > > > > or the day after tomorrow?
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > Regards
> > > > > > > > > > > > > > Sebastian
> > > > > > > > > > > > > >
> > > > > > > > > > > > > > On 13.06.2014 17:36, Will Hawkins wrote:
> > > > > > > > > > > > > > > Sebastian,
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Thanks for sending this out! We (at MLab) are very interested in
> > > > > > > > > > > > > > > deploying a new version of the code so this 'release' looks like a
> > > > > > > > > > > > > > > good place to fit together! :-) In other words, we (again, MLab) are
> > > > > > > > > > > > > > > definitely going to do what you said in 3!
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > I am out of the office, but I wanted to quickly respond to point 2.
> > > > > > > > > > > > > > > See below!
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Thanks again for driving this process forward!!
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Will
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > On 6/13/14, 4:47 AM, Sebastian Kostuch wrote:
> > > > > > > > > > > > > > > > Hello,
> > > > > > > > > > > > > > > > NDT has been updated recently with some major changes (like
> > > > > > > > > > > > > > > > MSG_EXTENDED_LOGIN,
> > > > > > > > > > > > > > > > json support, new JS UI) so as I have mentioned in some earlier mail
> > > > > > > > > > > > > > > > that would be probably
> > > > > > > > > > > > > > > > good time to make new release. I have few questions related to this:
> > > > > > > > > > > > > > > > 1. New JS UI would fit well within this new release, however changes
> > > > > > > > > > > > > > > > related to this are still
> > > > > > > > > > > > > > > > on work branches and need review. Any feedback would be very
> > > > > > > > > > > > > > > > appreciated. These are
> > > > > > > > > > > > > > > > following issues: 129, 132, 133.
> > > > > > > > > > > > > > > > 2. Is Issue144 (URLLoader to flash) supposed to be contained also in
> > > > > > > > > > > > > > > > nearest release version?
> > > > > > > > > > > > > > > > If yes, then how does progress with work on this look like? If I recall
> > > > > > > > > > > > > > > > correctly then you, Will,
> > > > > > > > > > > > > > > > mentioned that they are well tested and thus almost ready, right?
> > > > > > > > > > > > > > > Some of these are still up for debate. I am going to forward an email
> > > > > > > > > > > > > > > that I just sent to the MLab team with an update on the conundrum
> > > > > > > > > > > > > > > around the HTTP test to give you a sense about why they are up for
> > > > > > > > > > > > > > > debate.
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > But, there are also changes in that branch that *need* to be
> > > > > > > > > > > > > > > incorporated into anything that gets released. For instance, r1070.
> > > > > > > > > > > > > > > There are parts of r1064 that will have to be integrated no matter
> > > > > > > > > > > > > > > what (any of the changes that have to do with splitting start test and
> > > > > > > > > > > > > > > finalize test stages).
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > Does that make sense?
> > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > 3. Along with new release on googlecode what do you think about pushing
> > > > > > > > > > > > > > > > these changes also
> > > > > > > > > > > > > > > > on M-Lab site? They bring many new features at all.
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Looking forward to your answers and I hope that next week could be
> > > > > > > > > > > > > > > > celebrated with making
> > > > > > > > > > > > > > > > new release of NDT :).
> > > > > > > > > > > > > > > >
> > > > > > > > > > > > > > > > Kind regards
> > > > > > > > > > > > > > > > Sebastian Kostuch