NDT-DEV email list created

[Aaron Brown <>]

Hey Folks,

The big problem we run into is that we’re in a catch 22. The flash client, from what I’m told, doesn’t perform nearly as well as the Java client in a number of common cases. The Java applet, now signed, should be more acceptable, but it’s long term status is kaput. I think long term, our best bet is to move toward web sockets, but the code for that isn't ready currently. I think 3.8.0 should probably be the “web sockets” version, but for now, we leave 3.7.x as the java/flash client version.

Cheers,

Aaron

On Jan 5, 2015, at 12:07 PM, Peter Boothe <> wrote:

Hi! I'm the mentioned person!

As you have pointed out, the issue I mentioned shouldn't be a blocker for this release, but maybe it should help people choose where to aim development resources and time in the future. A good summary of what I was referring to is here:

  http://www.howtogeek.com/198300/oracle-cant-secure-the-java-plug-in-so-why-is-it-still-enabled-by-default/

By giving up on the applet sandbox, Oracle is making it so that all applets that want to run must be (a) signed and (b) have a super-scary security warning pop-up.  Choice (a) is good, and choice (b) is less so.

Because this UX is terrible and basically forces users to make uninformed security decisions ("Approve this applet signed by Joe Blow Inc to do whatever it wants to your computer? Y/N") browser makers have been moving away from supporting Java applets in any useful way due to security concerns (e.g. turning on click-to-play by default, no longer shipping with a JDK by default, etc). This process has been circling the drain for a while and Oracle doesn't seem to care, so I suspect (although I have no inside information one way or the other) that applet support is going to become more and more of an afterthought, which means that in the future most people aren't going to be able to run applets without installing the Java plugin themselves.

So the signs (as I read them) point to Java applets becoming more and more of a niche player for web content than they already are :(

  -Peter

On Mon, Dec 22, 2014 at 2:52 PM, Jordan McCarthy <> wrote:

-----BEGIN PGP SIGNED MESSAGE-----
Hash: SHA512

Hi everybody,
        I should mention that one of the M-Lab team members at Google has
reported that Java is soon going to institute yet another layer of
security that is likely to make it nigh-on impossible for the NDT Java
applet to work correctly without significant intervention on the part
of the end-user.  I'll let him elaborate, but it may be that trying to
get Java fully stable at this point may be something of a lost cause.

Jordan

Jordan McCarthy
Open Technology Institute | New America Foundation
Public Key: 0xC08D8042 | 4A61 3D39 4125 127D 65EA DDC2 BFBD A2E9 C08D 8042

On 12/22/2014 09:46 AM, Jakub Slawinski wrote:
>
> OK, making -rc1 release will be a step forward. Moreover, it will
> unblock all the actions I mentioned in my other mail.
>
> What do you think about the following plan:
>
> 1. Make 3.7.0-rc1 release and tag the repository accordingly 2.
> Merge MultiplePorts branch into trunk 3. Move from code.google to
> github 4. Work on the new issues / wait for feedback regarding
> 3.7.0-rc1 / work on the "randomly doesn't work" issue
>
> BTW. Aaron, can you create the ticket regarding "randomly doesn't
> work" issue with all the details regarding how to reproduce the
> issue?
>
> Thank you in advance!
>
>
> Regards, Jakub.
>
> On 12/22/2014 03:23 PM, Aaron Brown wrote:
>> Hey All,
>>
>> I’d be fine with doing an -rc1 release, but given this major
> showstopper of “randomly doesn’t work”, I’m rather hesitant to call
> it “3.7.0” and claim we’re done.
>>
>> Cheers, Aaron
>>
>> On Dec 22, 2014, at 9:10 AM, Aaron Brown
> <<mailto:>> wrote:
>>
>> Hi Sebastian,
>>
>> This problem occurred in java 7 update 71, so it’s not just an
>> older
> version problem. The clients have been running OS X with java 7
> installed, and it crops up whether in firefox or safari.
>>
>> Cheers, Aaron
>>
>> On Dec 18, 2014, at 5:15 AM, Sebastian Kostuch
> <<mailto:>> wrote:
>>
>> Hey Aaron, does the problem still occur for testers on mentioned
>> java version? If
> yes, could you provide details about environment so I would be able
> to reproduce this bug? If not then are we ready to release new
> version of NDT?
>>
>> Best regards Sebastian
>>
>> W dniu 15.12.2014 o 15:59, Sebastian Kostuch pisze: Hi Aaron the
>> problem with applet for me was related with old java version
> (1.6). After changing to newer one (1.7) I haven't noticed any non
> working tries (tested many times and all worked ok). Is it possible
> that people who tested it have also similar issue?
>>
>> Regards Sebastian
>>
>> On 09.12.2014 16:50, Aaron Brown wrote: Hey Sebastian,
>>
>> We can do an initial -rc release, but “regularly doesn’t work for
>> no
> apparent reason” is definitely a show stopper for the final
> release.
>>
>> Cheers, Aaron
>>
>> On Dec 7, 2014, at 11:56 PM, Sebastian Kostuch
> <<mailto:>> wrote:
>>
>> Hi Aaron I have tested java applet using these RPMs and it turned
>> out that
> sometimes it didn't work (using fakewww), however I wasn't able to
> find reason for such behaviour. Besides, for me it worked almost
> all the time, only few times I have problem with running tests (no
> errors, it just stopped working).I guess it may be something with
> fakewww as only using it I have noticed such sporadic problems.
> But, again, it worked almost all the time after all. Is it that
> what blocks us from releasing NDT now or are there any other tasks
> which should be completed before it could happen?
>>
>> Regards Sebastian
>>
>> On 02.12.2014 18:43, Aaron Brown wrote: Hey Sebastian,
>>
>> You can grab the RPMs from
> http://ndb1.internet2.edu/~aaron/ndt-server-3.7.0-1.x86_64/<http://ndb1.internet2.edu/%7Eaaron/ndt-server-3.7.0-1.x86_64/>
>
>
. The setup I’ve been using is the ‘fakewww’ one.
>>
>> Cheers, Aaron
>>
>> On Dec 2, 2014, at 8:38 AM, Sebastian Kostuch
> <<mailto:>> wrote:
>>
>> Hi Aaron would it be helpful if you will send me this RPM and I
>> will try if it
> works ok on my system?
>>
>> Regards Sebastian
>>
>> On 01.12.2014 20:15, Aaron Brown wrote: Hey Sebastian,
>>
>> I wanted to make sure it wasn’t somehow a problem with the RPM I
>> built
> as opposed to the two different client systems that have tried it
> so far.
>>
>> Cheers, Aaron
>>
>> On Nov 28, 2014, at 7:02 AM, Sebastian Kostuch
> <<mailto:>> wrote:
>>
>> Hi Aaron, currently not but if needed then I can try to make some
>> host public so
> you will have access. However I'm wondering what exactly will you
> be able to verify this way? The problem with applet not working
> sometimes is related to which OS, web browser and java version
> client uses, so will hosting page on another system make any
> differences? Please correct me if I have not understood your
> question right way :)
>>
>> Regards Sebastian
>>
>> On 25.11.2014 16:37, Aaron Brown wrote: Hey Sebastian,
>>
>> Do you have a publicly accessible host that I can test again?
>> I’m
> wondering if it’s some kind of OS X problem.
>>
>> Cheers, Aaron
>>
>> On Nov 24, 2014, at 4:41 AM, Sebastian Kostuch
> <<mailto:>> wrote:
>>
>> Hi again after further investigation I have noticed that the
>> problem on linux
> is related with bug in these versions of java (based on discussion
> here<http://www.webmo.net/discus/support/messages/3/710.html?1415693192>).
>
>
When I installed older version as suggested there (7u67) I was finally
> able to successfully run applet. Again, running it multiple times
> resulted in working ok all the time so I wasn't able to reproduce
> bug.
>> What is current status of releasing NDT? Are there any things
>> left
> that should be done before we can do it?
>>
>> Regards Sebastian
>>
>> On 21.11.2014 17:36, Sebastian Kostuch wrote: Hi Aaron I have
>> tried to reproduce bug with java applet working randomly (on
> widget.html site). What I have noticed is that on linux systems
> (tried on Ubuntu 14.04 and Fedora 20) I wasn't able to successfully
> run test using applet and got warning that plugin was not loaded
> properly and when I removed catching JS exception from script.js
> file then it was " Liveconnect call for Applet ID [id] is not
> allowed in this JVM instance" which is java security issue (unless
> I have added site hosting it to exceptions list). It's odd that
> using windows and same security settings I didn't have such error
> (java version 8u25, on linux I have tried with the same version and
> 7u72 also). However it resulted in working all the time on windows
> and not working all the time on linux, wasn't able to reproduce it
> the way you have mentioned. It would be helpfull to know what
> exactly versions of java they were using and what system/browser.
>>
>> Regards Sebastian
>>
>> On 17.11.2014 21:12, Aaron Brown wrote: Hey Pawel,
>>
>> On Nov 17, 2014, at 10:51 AM, Paweł Gesek
> <<mailto:>> wrote:
>>
>> Ok, thanks for clearing that up Aaron. As for the init script, if
>> we
> would decide to split the rpms, I believe we would either have to
> have the RPMs conflict, or create a separate base
> rpm(ndt-server-base) which would contain scripts, html, fakewww,
> the daemon, etc. Then have two ndt-server rpms for just the
> binaries, depending on the base package. The daemon would detect
> which version is installed and it could be manually configured if
> both were present. That would probably be the simplest route(aside
> from having them conflict), but as you say, there is no web10g rpm
> so thats for the future.
>>
>> That’s more the division I was thinking of: ndt-server-common
>> (init
> scripts, fakewww, flashpolicyd, etc.), ndt-server-web100
> (web100srv), ndt-server-web10g (web10gsrv). Allow both -web100 and
> -web10g RPMs to be installed, and have the existing “ndt-server”
> RPM become a meta-package that grabs ndt-server-web100 and
> ndt-server-common. Another option would be to have “ndt-server”
> have web100srv, and have it depend on ndt-server-common. Upgrades
> would then work as expected.
>>
>> Cheers, Aaron
>>
>>
>> Regards, Paweł
>>
>> On 11/17/2014 03:22 PM, Aaron Brown wrote: Hi Pawel,
>>
>> On Nov 17, 2014, at 8:19 AM, Paweł Gesek
> <<mailto:>> wrote:
>>
>> Aaron,
>>
>> I've noticed that with the new web10g userlands the RPM will fail
>> to
> build, since genplot10g is not mentioned in the spec file. I am
> assuming this means that the rpms being built for distribution will
> not contain the web10g binaries, since it passed on the build
> system.
>>
>> Correct, there is no dependency for the RPM on web10g-userland.
>>
>> I am wondering whether this is ok for now, or should we include
>> the
> web10g stuff in the RPM distribution. I think that both web100 and
> web10g binaries can be packaged together and the version used by
> the daemon can be controlled by a configuration variable. Another
> option is to create a separate ndt-server-web10g RPM for web10g,
> since the web10g binaries do require the latest userlands as a
> dependency. What is your opinion on this?
>>
>> Since no one is distributing RPMs for the web10g userland
>> (AFAIK), I’d
> prefer to leave well enough alone for -rc1. If we did support it,
> i’d prefer splitting RPMs similar to how you noted, though, so that
> folks without web10g-userland RPMs could install just the
> web100-dependent package, and vice versa. I’m not sure how the init
> script situation would work though.
>>
>> Cheers, Aaron
>>
>>
>> Regards, Paweł
>>
>> On 11/14/2014 08:53 PM, Aaron Brown wrote: Hey Sebastian,
>>
>> I think with my commit to the Issue162 branch, the RPM is now in
> reasonably good shape. One bit of oddity, and it’d be good if
> someone else could try building the RPM and testing. When we
> installed it here, the java applet would seem to switch randomly
> between “working” and “failing”, and it wasn’t clear why it would
> do it. We’ve had two different folks see this same issue.
>>
>> Cheers, Aaron
>>
>> On Nov 14, 2014, at 10:20 AM, Sebastian Kostuch
> <<mailto:>> wrote:
>>
>> Hi Aaron, I have made some changes to this page. Now buttons are
>> moved on top of
> the site, also I have added proper info message there and when
> plugin is not loaded properly and user clicks start button then
> error message will be shown and test would not be started. Also I
> have made plugin show only as a small bar. Please let me know if
> new version of this page looks ok for you :)
>>
>> Regards Sebastian
>>
>> On 14.11.2014 13:44, Aaron Brown wrote: Hey Sebastian,
>>
>> The problem comes in with flash blockers, which are pretty
>> common,
> especially among network engineers. I’ve attached a screenshot of
> what I see when I go to the page, and click “Use Flash”. It’s
> completely unclear to me that I need to hit anything other than
> “Start Test”. However, if I click “Start Test”, it will bomb out in
> a bizarre fashion, because I haven’t clicked on the flash applet
> below the buttons to get around the flash blocker, and since I
> can’t scroll the page at all, I can’t even see that there is
> something for me to click. I just have to blindly click in that box
> that’s below the blue page, and I’m given no guidance to do that.
> Beyond that, it’s rather jarring to have both appear in that page.
> If we could shrink the applet down so that it’s a very small bar,
> and specifically note the “flash block” issue in the text somewhere
> (especially if they click “Start Test”, and it bombs out), that’d
> be vastly more intuitive.
>>
>> If you want to try to see the issue for yourself, install a
>> flash
> blocker in Chrome or FF, and then navigate to
> http://desk179.internet2.edu:7123/ and select “Use Flash”.
>>
>> Cheers, Aaron
>>
>> <Mail Attachment.png>
>>
>> On Nov 14, 2014, at 6:23 AM, Sebastian Kostuch
> <<mailto:>> wrote:
>>
>> Hi Aaron, I'm not sure I understand you correctly. The purpose of
>> this site is
> to provide the user JS UI and the ability to choose which client
> (java or flash) should be used as backend right? But this client
> itself is not supposed to be used directly here as the GUI written
> in HTML + JS does the whole work. If we want to provide user these
> clients direct then we have separated pages for it (tcpbw100 and
> tcpbw100-java). So what about hiding these controls on the
> widget.html site so that only JS UI will be visible? Or am I
> missing something and it should work different way?
>> Also for ensuring: by writing that it worked you mean JS UI or
>> running
> flash client located on the bottom of the site?
>>
>> Kind regards Sebastian
>>
>> On 13.11.2014 15:43, Aaron Brown wrote: Hey Sebastian,
>>
>> Ah ok. I’ve got Flash Block on the browsers so the applet did
>> load,
> but because of how the page loads, wasn’t visible so I didn’t
> realize I needed to click on it first. Once I clicked on that, it
> worked. Given the prevelance of flash blockers, and the odd nature
> of this HTML5 + Flash combo, is there a better way that we can
> better locate the actual Applet on the page itself so that, and put
> some text in, letting folks know they may need to click on the
> hidden flash applet first?
>>
>> Cheers, Aaron
>>
>> On Nov 13, 2014, at 9:36 AM, Sebastian Kostuch
> <<mailto:>> wrote:
>>
>> Hi Aaron, as Pawel mentioned there is currently problem with
>> calling applet
> functions from JS as from
>> java 7u45 update there are more security restrictions to applets.
>> You
> can read more about it
> here<http://www.oracle.com/technetwork/java/javase/7u45-relnotes-2016950.html#newft>.
>>
>
Basically now it is necessary to add Caller-Allowable-Codebase line to
> manifest file (which I have already commited) and jar file must be
> signed by a trusted CA.
>> When it comes to issues you have mentioned, is there also problem
>> with
> calling start directly from applet or flash embedded on the same
> site where JS UI is? Are there any errors?
>>
>> Kind regards Sebastian On 13.11.2014 15:20, Aaron Brown wrote:
>> Hey Pawel,
>>
>> On Nov 13, 2014, at 9:11 AM, Paweł Gesek
> <<mailto:>> wrote:
>>
>> Aaron,
>>
>> I've taken a look and talked with Sebastian about this. The new
>> page
> uses the applet by default if your browser supports java, however
> there is some sort of problem with the _javascript_-applet
> communication, making the gauges do nothing if you are using the
> applet. Have you tried pressing the flash button below in order to
> switch the client to flash?
>>
>> It doesn’t matter what I use, both bomb out, and don’t (as far as
>> i
> can tell anyway) perform any tests.
>>
>> One question regarding your changes, any reason to change
> nobase_ndt_DATA to ndt_DATA in HTML5-frontend/Makefile.am? Without
> the nobase modifier HTML5-frontend/images and HTML5-frontend/fonts
> get installed in /usr/ndt without the fonts or images subdirectory,
> which leads to missing resources on the webpage.
>>
>> The install had failed as is, and as part of my monkeying, i’d
>> removed
> it, and then forgot about it when it didn’t fix the issue. It’s
> been restored in my latest commit.
>>
>> Cheers, Aaron
>>
>>
>> Regards, Paweł
>>
>> On 11/12/2014 05:04 PM, Aaron Brown wrote: Hey Pawel,
>>
>> I couldn’t get the RPM to build using mock, so I committed some
> changes so I could build the RPM. However, after installing the
> RPM, going to the web and hitting start, it doesn’t seem to do
> anything for me. Navigating to the flash client directly, however,
> does seem to work.
>>
>> Cheers, Aaron
>>
>> On Nov 12, 2014, at 8:53 AM, Paweł Gesek
> <<mailto:>> wrote:
>>
>> I've added the flashpolicyd script with some small modifications
>> to
> the NDT repository. It is started by the ndt daemon and can be
> disabled in the sysconfig file in the same way as the fakewww
> daemon.
>>
>> I believe I am finished with the RPM work, the changes are on
>> the
> branch "Issues162". Please review and let me know if I missed
> something or you feel something needs to be changed.
>>
>> Regards, Paweł
>>
>> On 11/07/2014 04:14 PM, Paweł Gesek wrote: Great, I'll look at
>> packaging that script with NDT. I assume we should
> bundle it with the ndt-server rpm. I'm also thinking that we can
> handle this in a similar fashion we do with fakewww - the ndt
> daemon will handle starting and stopping of that script. Is that
> okay?
>>
>> It's possible I missed fakewww, I'll take a look and make sure
>> its
> consistent with httpd. As for widget.html it allows switching
> between the applet and flash client.
>>
>> Regards, Paweł
>>
>>
>> On 11/07/2014 03:42 PM, Aaron Brown wrote: That script looks
>> reasonable. As long as it can be reasonably be
> packaged with NDT, I’d be fine with it.
>>
>> Cheers, Aaron
>>
>> On Nov 7, 2014, at 8:15 AM, Jordan McCarthy
> <<mailto:>>
>
>
wrote:
>>
>> For what it's worth, the M-Lab platform uses the second
>> approach, embodied in this file:
>> https://github.com/m-lab-tools/ndt-support/blob/master/flashpolicyd.py
>>
>>
>>
If this code looks reasonable to you all, please feel free to use it!
>>
>> Jordan
>>
>> Jordan McCarthy Open Technology Institute | New America
>> Foundation Public Key: 0xC08D8042 | 4A61 3D39 4125 127D 65EA DDC2
>> BFBD A2E9 C08D 8042
>>
>> On 11/07/2014 04:46 AM, Paweł Gesek wrote: Hello everyone,
>>
>> I have been looking at updating the NDT RPM so that the flash
>> client there works out of the box(Issue162
>> <https://code.google.com/p/ndt/issues/detail?id=162>). I've
>> committed changes to the branch
>> Issue162(https://code.google.com/p/ndt/source/detail?r=1144),
>> which make the HTML5-frontend install with NDT and changed the
>> index page in the apache configuration to the new widget.html.
>>
>> As far as running the Flash client out of the box goes, the NDT
>> server has to expose the crossdomain.xml file, which is required
>> for the Flash client to allow opening sockets to the server.
>> Unfortunately the socket policy does not go through HTTP, but a
>> simple proprietary protocol that uses the TCP port 843. I am
>> wondering for the best way to handle this in the NDT rpm. We can
>> possibly add a requirement for a package like
>> https://code.google.com/p/flashpolicyd/ and use that to serve
>> the file. We could also consider bundling some sort of a simple
>> server script(like this
>> https://github.com/xantus/mojo-websocket-examples/blob/master/script/flash-policy-server)
>>
>>
>>
>>
that would serve the policy file. Do you preferences on how to resolve
>> this?
>>
>> Regards, Paweł
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>>
>
>

-----BEGIN PGP SIGNATURE-----
Version: GnuPG v1.4.11 (GNU/Linux)

iQEcBAEBCgAGBQJUmHZ8AAoJEL+9ounAjYBCct4IAJHs8P6Z0zY8V9bDRgumhdxN
OnpDxdwVd6JPFs6XwChiX4tz4X1iGtRGj5etAtKZiQXDTIGkKIKxNxCATMy4OTpa
EJr7BL7ryx9fR4A+dWoh6/a8lrjMHkUJy+HapGWK8MlBHVW4fNpik54T6u5OY7zc
95t5dm6rYCXgy6qLW5FILfxvnpjJcjOVHdkkEG23S0NsSj4ZdLekFf8RsP9rWImc
YDqe0obTSgCMqHWJ3DchIwwGm9Guum3DkAyCAAK7kOeZ1hx4EwfVsq8hDFaYJAV5
RQRCvINagGrKsTD5C3KAb7IvGWx6hyyaQ8bcyxsNR28eNAPkUMrVlxwgWhxKZiY=
=9Dip
-----END PGP SIGNATURE-----

--

Peter Boothe | Software Engineer |  | +1 650-889-0879 (corp cell)