NDT-DEV email list created

[Jakub Slawinski <>]

OK, but this is the future problem.

The -rc1 release should do the trick for the moment - at least it will
unblock a couple of things.
We will work on the random failures issue as soon as the ticket with the
detailed steps-to-reproduce will be created.


Regards,
  Jakub.

On 12/22/2014 08:52 PM, Jordan McCarthy wrote:
> Hi everybody,
>     I should mention that one of the M-Lab team members at Google has
> reported that Java is soon going to institute yet another layer of
> security that is likely to make it nigh-on impossible for the NDT Java
> applet to work correctly without significant intervention on the part
> of the end-user.  I'll let him elaborate, but it may be that trying to
> get Java fully stable at this point may be something of a lost cause.
>
> Jordan
>
> Jordan McCarthy
> Open Technology Institute | New America Foundation
> Public Key: 0xC08D8042 | 4A61 3D39 4125 127D 65EA DDC2 BFBD A2E9 C08D 8042
>
> On 12/22/2014 09:46 AM, Jakub Slawinski wrote:
>
> > OK, making -rc1 release will be a step forward. Moreover, it will
> > unblock all the actions I mentioned in my other mail.
>
> > What do you think about the following plan:
>
> > 1. Make 3.7.0-rc1 release and tag the repository accordingly 2.
> > Merge MultiplePorts branch into trunk 3. Move from code.google to
> > github 4. Work on the new issues / wait for feedback regarding
> > 3.7.0-rc1 / work on the "randomly doesn't work" issue
>
> > BTW. Aaron, can you create the ticket regarding "randomly doesn't
> > work" issue with all the details regarding how to reproduce the
> > issue?
>
> > Thank you in advance!
>
>
> > Regards, Jakub.
>
> > On 12/22/2014 03:23 PM, Aaron Brown wrote:
> >> Hey All,
> >>
> >> I’d be fine with doing an -rc1 release, but given this major
> > showstopper of “randomly doesn’t work”, I’m rather hesitant to call
> > it “3.7.0” and claim we’re done.
> >>
> >> Cheers, Aaron
> >>
> >> On Dec 22, 2014, at 9:10 AM, Aaron Brown
> > <<mailto:>>
> >  wrote:
> >>
> >> Hi Sebastian,
> >>
> >> This problem occurred in java 7 update 71, so it’s not just an
> >> older
> > version problem. The clients have been running OS X with java 7
> > installed, and it crops up whether in firefox or safari.
> >>
> >> Cheers, Aaron
> >>
> >> On Dec 18, 2014, at 5:15 AM, Sebastian Kostuch
> > <<mailto:>>
> >  wrote:
> >>
> >> Hey Aaron, does the problem still occur for testers on mentioned
> >> java version? If
> > yes, could you provide details about environment so I would be able
> > to reproduce this bug? If not then are we ready to release new
> > version of NDT?
> >>
> >> Best regards Sebastian
> >>
> >> W dniu 15.12.2014 o 15:59, Sebastian Kostuch pisze: Hi Aaron the
> >> problem with applet for me was related with old java version
> > (1.6). After changing to newer one (1.7) I haven't noticed any non
> > working tries (tested many times and all worked ok). Is it possible
> > that people who tested it have also similar issue?
> >>
> >> Regards Sebastian
> >>
> >> On 09.12.2014 16:50, Aaron Brown wrote: Hey Sebastian,
> >>
> >> We can do an initial -rc release, but “regularly doesn’t work for
> >> no
> > apparent reason” is definitely a show stopper for the final
> > release.
> >>
> >> Cheers, Aaron
> >>
> >> On Dec 7, 2014, at 11:56 PM, Sebastian Kostuch
> > <<mailto:>>
> >  wrote:
> >>
> >> Hi Aaron I have tested java applet using these RPMs and it turned
> >> out that
> > sometimes it didn't work (using fakewww), however I wasn't able to
> > find reason for such behaviour. Besides, for me it worked almost
> > all the time, only few times I have problem with running tests (no
> > errors, it just stopped working).I guess it may be something with
> > fakewww as only using it I have noticed such sporadic problems.
> > But, again, it worked almost all the time after all. Is it that
> > what blocks us from releasing NDT now or are there any other tasks
> > which should be completed before it could happen?
> >>
> >> Regards Sebastian
> >>
> >> On 02.12.2014 18:43, Aaron Brown wrote: Hey Sebastian,
> >>
> >> You can grab the RPMs from
> >
> http://ndb1.internet2.edu/~aaron/ndt-server-3.7.0-1.x86_64/<http://ndb1.internet2.edu/%7Eaaron/ndt-server-3.7.0-1.x86_64/>
>
>
> . The setup I’ve been using is the ‘fakewww’ one.
> >>
> >> Cheers, Aaron
> >>
> >> On Dec 2, 2014, at 8:38 AM, Sebastian Kostuch
> > <<mailto:>>
> >  wrote:
> >>
> >> Hi Aaron would it be helpful if you will send me this RPM and I
> >> will try if it
> > works ok on my system?
> >>
> >> Regards Sebastian
> >>
> >> On 01.12.2014 20:15, Aaron Brown wrote: Hey Sebastian,
> >>
> >> I wanted to make sure it wasn’t somehow a problem with the RPM I
> >> built
> > as opposed to the two different client systems that have tried it
> > so far.
> >>
> >> Cheers, Aaron
> >>
> >> On Nov 28, 2014, at 7:02 AM, Sebastian Kostuch
> > <<mailto:>>
> >  wrote:
> >>
> >> Hi Aaron, currently not but if needed then I can try to make some
> >> host public so
> > you will have access. However I'm wondering what exactly will you
> > be able to verify this way? The problem with applet not working
> > sometimes is related to which OS, web browser and java version
> > client uses, so will hosting page on another system make any
> > differences? Please correct me if I have not understood your
> > question right way :)
> >>
> >> Regards Sebastian
> >>
> >> On 25.11.2014 16:37, Aaron Brown wrote: Hey Sebastian,
> >>
> >> Do you have a publicly accessible host that I can test again?
> >> I’m
> > wondering if it’s some kind of OS X problem.
> >>
> >> Cheers, Aaron
> >>
> >> On Nov 24, 2014, at 4:41 AM, Sebastian Kostuch
> > <<mailto:>>
> >  wrote:
> >>
> >> Hi again after further investigation I have noticed that the
> >> problem on linux
> > is related with bug in these versions of java (based on discussion
> >
> here<http://www.webmo.net/discus/support/messages/3/710.html?1415693192>).
>
>
> When I installed older version as suggested there (7u67) I was finally
> > able to successfully run applet. Again, running it multiple times
> > resulted in working ok all the time so I wasn't able to reproduce
> > bug.
> >> What is current status of releasing NDT? Are there any things
> >> left
> > that should be done before we can do it?
> >>
> >> Regards Sebastian
> >>
> >> On 21.11.2014 17:36, Sebastian Kostuch wrote: Hi Aaron I have
> >> tried to reproduce bug with java applet working randomly (on
> > widget.html site). What I have noticed is that on linux systems
> > (tried on Ubuntu 14.04 and Fedora 20) I wasn't able to successfully
> > run test using applet and got warning that plugin was not loaded
> > properly and when I removed catching JS exception from script.js
> > file then it was " Liveconnect call for Applet ID [id] is not
> > allowed in this JVM instance" which is java security issue (unless
> > I have added site hosting it to exceptions list). It's odd that
> > using windows and same security settings I didn't have such error
> > (java version 8u25, on linux I have tried with the same version and
> > 7u72 also). However it resulted in working all the time on windows
> > and not working all the time on linux, wasn't able to reproduce it
> > the way you have mentioned. It would be helpfull to know what
> > exactly versions of java they were using and what system/browser.
> >>
> >> Regards Sebastian
> >>
> >> On 17.11.2014 21:12, Aaron Brown wrote: Hey Pawel,
> >>
> >> On Nov 17, 2014, at 10:51 AM, Paweł Gesek
> > <<mailto:>>
> >  wrote:
> >>
> >> Ok, thanks for clearing that up Aaron. As for the init script, if
> >> we
> > would decide to split the rpms, I believe we would either have to
> > have the RPMs conflict, or create a separate base
> > rpm(ndt-server-base) which would contain scripts, html, fakewww,
> > the daemon, etc. Then have two ndt-server rpms for just the
> > binaries, depending on the base package. The daemon would detect
> > which version is installed and it could be manually configured if
> > both were present. That would probably be the simplest route(aside
> > from having them conflict), but as you say, there is no web10g rpm
> > so thats for the future.
> >>
> >> That’s more the division I was thinking of: ndt-server-common
> >> (init
> > scripts, fakewww, flashpolicyd, etc.), ndt-server-web100
> > (web100srv), ndt-server-web10g (web10gsrv). Allow both -web100 and
> > -web10g RPMs to be installed, and have the existing “ndt-server”
> > RPM become a meta-package that grabs ndt-server-web100 and
> > ndt-server-common. Another option would be to have “ndt-server”
> > have web100srv, and have it depend on ndt-server-common. Upgrades
> > would then work as expected.
> >>
> >> Cheers, Aaron
> >>
> >>
> >> Regards, Paweł
> >>
> >> On 11/17/2014 03:22 PM, Aaron Brown wrote: Hi Pawel,
> >>
> >> On Nov 17, 2014, at 8:19 AM, Paweł Gesek
> > <<mailto:>>
> >  wrote:
> >>
> >> Aaron,
> >>
> >> I've noticed that with the new web10g userlands the RPM will fail
> >> to
> > build, since genplot10g is not mentioned in the spec file. I am
> > assuming this means that the rpms being built for distribution will
> > not contain the web10g binaries, since it passed on the build
> > system.
> >>
> >> Correct, there is no dependency for the RPM on web10g-userland.
> >>
> >> I am wondering whether this is ok for now, or should we include
> >> the
> > web10g stuff in the RPM distribution. I think that both web100 and
> > web10g binaries can be packaged together and the version used by
> > the daemon can be controlled by a configuration variable. Another
> > option is to create a separate ndt-server-web10g RPM for web10g,
> > since the web10g binaries do require the latest userlands as a
> > dependency. What is your opinion on this?
> >>
> >> Since no one is distributing RPMs for the web10g userland
> >> (AFAIK), I’d
> > prefer to leave well enough alone for -rc1. If we did support it,
> > i’d prefer splitting RPMs similar to how you noted, though, so that
> > folks without web10g-userland RPMs could install just the
> > web100-dependent package, and vice versa. I’m not sure how the init
> > script situation would work though.
> >>
> >> Cheers, Aaron
> >>
> >>
> >> Regards, Paweł
> >>
> >> On 11/14/2014 08:53 PM, Aaron Brown wrote: Hey Sebastian,
> >>
> >> I think with my commit to the Issue162 branch, the RPM is now in
> > reasonably good shape. One bit of oddity, and it’d be good if
> > someone else could try building the RPM and testing. When we
> > installed it here, the java applet would seem to switch randomly
> > between “working” and “failing”, and it wasn’t clear why it would
> > do it. We’ve had two different folks see this same issue.
> >>
> >> Cheers, Aaron
> >>
> >> On Nov 14, 2014, at 10:20 AM, Sebastian Kostuch
> > <<mailto:>>
> >  wrote:
> >>
> >> Hi Aaron, I have made some changes to this page. Now buttons are
> >> moved on top of
> > the site, also I have added proper info message there and when
> > plugin is not loaded properly and user clicks start button then
> > error message will be shown and test would not be started. Also I
> > have made plugin show only as a small bar. Please let me know if
> > new version of this page looks ok for you :)
> >>
> >> Regards Sebastian
> >>
> >> On 14.11.2014 13:44, Aaron Brown wrote: Hey Sebastian,
> >>
> >> The problem comes in with flash blockers, which are pretty
> >> common,
> > especially among network engineers. I’ve attached a screenshot of
> > what I see when I go to the page, and click “Use Flash”. It’s
> > completely unclear to me that I need to hit anything other than
> > “Start Test”. However, if I click “Start Test”, it will bomb out in
> > a bizarre fashion, because I haven’t clicked on the flash applet
> > below the buttons to get around the flash blocker, and since I
> > can’t scroll the page at all, I can’t even see that there is
> > something for me to click. I just have to blindly click in that box
> > that’s below the blue page, and I’m given no guidance to do that.
> > Beyond that, it’s rather jarring to have both appear in that page.
> > If we could shrink the applet down so that it’s a very small bar,
> > and specifically note the “flash block” issue in the text somewhere
> > (especially if they click “Start Test”, and it bombs out), that’d
> > be vastly more intuitive.
> >>
> >> If you want to try to see the issue for yourself, install a
> >> flash
> > blocker in Chrome or FF, and then navigate to
> > http://desk179.internet2.edu:7123/ and select “Use Flash”.
> >>
> >> Cheers, Aaron
> >>
> >> <Mail Attachment.png>
> >>
> >> On Nov 14, 2014, at 6:23 AM, Sebastian Kostuch
> > <<mailto:>>
> >  wrote:
> >>
> >> Hi Aaron, I'm not sure I understand you correctly. The purpose of
> >> this site is
> > to provide the user JS UI and the ability to choose which client
> > (java or flash) should be used as backend right? But this client
> > itself is not supposed to be used directly here as the GUI written
> > in HTML + JS does the whole work. If we want to provide user these
> > clients direct then we have separated pages for it (tcpbw100 and
> > tcpbw100-java). So what about hiding these controls on the
> > widget.html site so that only JS UI will be visible? Or am I
> > missing something and it should work different way?
> >> Also for ensuring: by writing that it worked you mean JS UI or
> >> running
> > flash client located on the bottom of the site?
> >>
> >> Kind regards Sebastian
> >>
> >> On 13.11.2014 15:43, Aaron Brown wrote: Hey Sebastian,
> >>
> >> Ah ok. I’ve got Flash Block on the browsers so the applet did
> >> load,
> > but because of how the page loads, wasn’t visible so I didn’t
> > realize I needed to click on it first. Once I clicked on that, it
> > worked. Given the prevelance of flash blockers, and the odd nature
> > of this HTML5 + Flash combo, is there a better way that we can
> > better locate the actual Applet on the page itself so that, and put
> > some text in, letting folks know they may need to click on the
> > hidden flash applet first?
> >>
> >> Cheers, Aaron
> >>
> >> On Nov 13, 2014, at 9:36 AM, Sebastian Kostuch
> > <<mailto:>>
> >  wrote:
> >>
> >> Hi Aaron, as Pawel mentioned there is currently problem with
> >> calling applet
> > functions from JS as from
> >> java 7u45 update there are more security restrictions to applets.
> >> You
> > can read more about it
> >
> here<http://www.oracle.com/technetwork/java/javase/7u45-relnotes-2016950.html#newft>.
> >>
>
> Basically now it is necessary to add Caller-Allowable-Codebase line to
> > manifest file (which I have already commited) and jar file must be
> > signed by a trusted CA.
> >> When it comes to issues you have mentioned, is there also problem
> >> with
> > calling start directly from applet or flash embedded on the same
> > site where JS UI is? Are there any errors?
> >>
> >> Kind regards Sebastian On 13.11.2014 15:20, Aaron Brown wrote:
> >> Hey Pawel,
> >>
> >> On Nov 13, 2014, at 9:11 AM, Paweł Gesek
> > <<mailto:>>
> >  wrote:
> >>
> >> Aaron,
> >>
> >> I've taken a look and talked with Sebastian about this. The new
> >> page
> > uses the applet by default if your browser supports java, however
> > there is some sort of problem with the javascript-applet
> > communication, making the gauges do nothing if you are using the
> > applet. Have you tried pressing the flash button below in order to
> > switch the client to flash?
> >>
> >> It doesn’t matter what I use, both bomb out, and don’t (as far as
> >> i
> > can tell anyway) perform any tests.
> >>
> >> One question regarding your changes, any reason to change
> > nobase_ndt_DATA to ndt_DATA in HTML5-frontend/Makefile.am? Without
> > the nobase modifier HTML5-frontend/images and HTML5-frontend/fonts
> > get installed in /usr/ndt without the fonts or images subdirectory,
> > which leads to missing resources on the webpage.
> >>
> >> The install had failed as is, and as part of my monkeying, i’d
> >> removed
> > it, and then forgot about it when it didn’t fix the issue. It’s
> > been restored in my latest commit.
> >>
> >> Cheers, Aaron
> >>
> >>
> >> Regards, Paweł
> >>
> >> On 11/12/2014 05:04 PM, Aaron Brown wrote: Hey Pawel,
> >>
> >> I couldn’t get the RPM to build using mock, so I committed some
> > changes so I could build the RPM. However, after installing the
> > RPM, going to the web and hitting start, it doesn’t seem to do
> > anything for me. Navigating to the flash client directly, however,
> > does seem to work.
> >>
> >> Cheers, Aaron
> >>
> >> On Nov 12, 2014, at 8:53 AM, Paweł Gesek
> > <<mailto:>>
> >  wrote:
> >>
> >> I've added the flashpolicyd script with some small modifications
> >> to
> > the NDT repository. It is started by the ndt daemon and can be
> > disabled in the sysconfig file in the same way as the fakewww
> > daemon.
> >>
> >> I believe I am finished with the RPM work, the changes are on
> >> the
> > branch "Issues162". Please review and let me know if I missed
> > something or you feel something needs to be changed.
> >>
> >> Regards, Paweł
> >>
> >> On 11/07/2014 04:14 PM, Paweł Gesek wrote: Great, I'll look at
> >> packaging that script with NDT. I assume we should
> > bundle it with the ndt-server rpm. I'm also thinking that we can
> > handle this in a similar fashion we do with fakewww - the ndt
> > daemon will handle starting and stopping of that script. Is that
> > okay?
> >>
> >> It's possible I missed fakewww, I'll take a look and make sure
> >> its
> > consistent with httpd. As for widget.html it allows switching
> > between the applet and flash client.
> >>
> >> Regards, Paweł
> >>
> >>
> >> On 11/07/2014 03:42 PM, Aaron Brown wrote: That script looks
> >> reasonable. As long as it can be reasonably be
> > packaged with NDT, I’d be fine with it.
> >>
> >> Cheers, Aaron
> >>
> >> On Nov 7, 2014, at 8:15 AM, Jordan McCarthy
> > <<mailto:>>
>
>
> wrote:
> >>
> >> For what it's worth, the M-Lab platform uses the second
> >> approach, embodied in this file:
> >> https://github.com/m-lab-tools/ndt-support/blob/master/flashpolicyd.py
> >>
> >>
> >>
> If this code looks reasonable to you all, please feel free to use it!
> >>
> >> Jordan
> >>
> >> Jordan McCarthy Open Technology Institute | New America
> >> Foundation Public Key: 0xC08D8042 | 4A61 3D39 4125 127D 65EA DDC2
> >> BFBD A2E9 C08D 8042
> >>
> >> On 11/07/2014 04:46 AM, Paweł Gesek wrote: Hello everyone,
> >>
> >> I have been looking at updating the NDT RPM so that the flash
> >> client there works out of the box(Issue162
> >> <https://code.google.com/p/ndt/issues/detail?id=162>). I've
> >> committed changes to the branch
> >> Issue162(https://code.google.com/p/ndt/source/detail?r=1144),
> >> which make the HTML5-frontend install with NDT and changed the
> >> index page in the apache configuration to the new widget.html.
> >>
> >> As far as running the Flash client out of the box goes, the NDT
> >> server has to expose the crossdomain.xml file, which is required
> >> for the Flash client to allow opening sockets to the server.
> >> Unfortunately the socket policy does not go through HTTP, but a
> >> simple proprietary protocol that uses the TCP port 843. I am
> >> wondering for the best way to handle this in the NDT rpm. We can
> >> possibly add a requirement for a package like
> >> https://code.google.com/p/flashpolicyd/ and use that to serve
> >> the file. We could also consider bundling some sort of a simple
> >> server script(like this
> >>
> https://github.com/xantus/mojo-websocket-examples/blob/master/script/flash-policy-server)
> >>
> >>
> >>
> >>
> that would serve the policy file. Do you preferences on how to resolve
> >> this?
> >>
> >> Regards, Paweł
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
> >>
>
>