NDT-DEV email list created

[Sebastian Kostuch <>]

Hi all,
thanks a lot for your feedback. I have considered all of them and here 
are my thoughts:
in fact these all changes adding one more message at the beggining of 
each test was to
make changing results and protocols easier. My problem was to send some 
additional variables
at the end of middlebox test so that I could use them in JS UI to 
display RTT and network
speed restriction. I couldn't simply add them at the end of existing 
string as some clients
like Java one won't work properly then (it expects results in strict order).

My first idea was to somehow exchange information about what client 
expects and what
server could provide but after your opinions I think that it truly isn't 
necessary.
There was discussion about changing messages' body to JSON. And I think 
this would be
good way to go. If we contain all results information into JSON format 
then it would be
simple to add some additional variables in future (what is exactly what 
I need). Client
will then simply parse such JSON and retrieve only results it expects. 
Also using extended
login message we could determine if such client supports JSON format or 
not.

The essential minimum to get my middlebox results changes to work 
properly is to change
only results message body to JSON but wouldn't it be better to make such 
changes in all
existing messages?

Looking forward to your thoughts about it and thank you again for such 
spirited discussion :).

Regards,
Sebastian

On 16.04.2014 05:28, Richard Carlson wrote:
> Sebastian;
>
> Just one thought from me.  The client should not know about, or care 
> about, the web10g/100 variables.  It should process a list and display 
> them to the user, but it should treat them as opaque objects.
>
> When I first started working on the NDT system, the client processed 
> the variables and calculated the results.  This was fine when there 
> was only 1 client.  When I started working on the CLI client I now had 
> 2 clients that needed to be kept in sync as far as test results and 
> reporting mechanisms went.  This quickly became unmanageable as 
> changes in the java client needed to be duplicated in the C code, or 
> the other way around.
>
> I decided that the server should do all the work and then send the 
> results and raw data back to the client for display.  That way I could 
> update the test results once and not have to duplicate or forget 
> things when handling multiple clients.
>
> As this team begins to add more clients, I think it is even more 
> important that this concept be adhered to.  Having different clients 
> report different results will only confuse the user, or worse yet the 
> sys admin running the server.
>
> So do not put logic or extra protocol processing in the client. Keep 
> them as simple as possible and let the server handle the hard tasks. 
> That way the client just needs to read/display whatever the servers says.
>
> Rich
>
> On 04/14/2014 01:46 AM, Sebastian Kostuch wrote:
> > Hi
> > my intention was to make additional message at the beginning of each
> > test (not extending current login message which contains client 
> > version).
> > Anyway it could be some kind of login message too and JSON seems to be
> > good way to pack it into. This message would simply contain information
> > for client which protocol version and types server does support and
> > client would behave accordingly to it. When it comes to expected
> > variables then
> > naming web10/100 vars was just example but in fact it isn't the most
> > flexible solution. Should we first distinct which type of variable
> > client expects
> > (i.e. tcp_info, web10/100, another ones) and then deliver array of their
> > names or would it be better for example to enlist available results 
> > which
> > server could provide in some documentation and each client would then
> > just simply put names from it into such prepared message? Or maybe
> > do you have some different thoughts about it? Looking forward to it :).
> >
> > Regards,
> > Sebastian
> >
> > On 11.04.2014 20:49, Aaron Brown wrote:
> > > My preference if we’re going to go down the path of adding more stuff
> > > into the login message would be to have the body of the message be a
> > > JSON hash. This would allow adding more variables without breaking
> > > backwards compatibility and/or defining another schema for
> > > representing structured data.
> > >
> > > As to expected results, I’m not sure the best way to structure that
> > > since logically the web100/web10g variables being requested shouldn’t
> > > depend on web100/web10g (i.e. if web100g ever exists, or if we can
> > > provide some of them via TCP_INFO, it shouldn’t break things). Beyond
> > > that, there are some results that aren’t just web100-variables, but
> > > are combinations of web100 variables and other data (e.g. bottleneck
> > > link) that presumably would be good to have in there as well.
> > >
> > > Cheers,
> > > Aaron
> > >
> > > On Apr 11, 2014, at 4:03 AM, Sebastian Kostuch
> > > <>
> > >  wrote:
> > >
> > > > Hi
> > > > indeed we can make client reconnect after server receiving not
> > > > supported message. It would mean to have some redundant
> > > > established connections which in fact will be shut down quickly
> > > > but I think that in future old servers will get less and less popular
> > > > so such solution seems to be good and actually simplest than the
> > > > one with additional test. So I will too stick with the current 
> > > > solution.
> > > >
> > > > However there is also one thing which I think should be considered.
> > > > Currently we have client and server exchange only their versions
> > > > but as I have mentioned in some previous messages it would probably
> > > > be cleaner to make them also send additional protocol information
> > > > at the start of each test (assuming that they both are with new
> > > > changes, if client or server is old then of course such operation
> > > > will not be performed and both sides will behave as the protocol
> > > > version was the old one).
> > > >
> > > > I suggest to make them exchange information such as:
> > > > 1. Protocol version number
> > > > 2. Supported protocols (such as HTTP in future) - this could be
> > > > contained within
> > > > protocol version string as well i.e. S2C-2.0-HTTP
> > > > 3. Results set expected by client (i.e. names of web100/web10 
> > > > variables)
> > > > which will be sent for example right after protocol version message.
> > > > This
> > > > way it would be simple in future to make changes when we decide to
> > > > remove some existing results variables or add some new (when server
> > > > would not support such result variable then it will just respond with
> > > > some
> > > > kind of special value). Such message should be formatted for instance
> > > > with
> > > > string containing variable names separated with semicolon.
> > > >
> > > > So simply said the reason behind making such additional message 
> > > > exchange
> > > > at the beginning of tests is to make doing changes in protocols in
> > > > future more
> > > > easily. What is your opinion about this? I think we should consider
> > > > it ASAP as
> > > > we will need these changes to make some small protocol changes planned
> > > > earlier (for me it is adding additional variables to Middlebox test
> > > > results).
> > > >
> > > > Regards
> > > > Sebastian
> > > >
> > > > On 07.04.2014 15:03, Aaron Brown wrote:
> > > > > Hey Sebastian,
> > > > >
> > > > > Comment inline
> > > > >
> > > > > Cheers,
> > > > > Aaron
> > > > >
> > > > > On Apr 7, 2014, at 2:15 AM, Sebastian Kostuch
> > > > > <>
> > > > >  wrote:
> > > > >
> > > > > > Hi
> > > > > >
> > > > > > On 04.04.2014 19:59, Will Hawkins wrote:
> > > > > > > Pre PS: There is a running version of the new server at
> > > > > > > ndt.iupui.mlab4.nuq0t.measurement-lab.org. Feel free to send
> > > > > > > connections
> > > > > > > to it to test the new MSG_EXTENDED_LOGIN semantics.
> > > > > > >
> > > > > > > On 04/04/2014 06:35 AM, Sebastian Kostuch wrote:
> > > > > > > > Hi again
> > > > > > > > in addition to clients versioning I have mentioned in previous
> > > > > > > > email I
> > > > > > > > think that maybe it would be better to not determine
> > > > > > > > server actions based only on this information but rather using
> > > > > > > > the one
> > > > > > > > more detailed such as introducing versioning
> > > > > > > > of individual test protocols. I mean that server and client
> > > > > > > > should have
> > > > > > > > possibility to exchange information at the
> > > > > > > > beginning of each test which will contain version of protocol
> > > > > > > > supported
> > > > > > > > by both sites and such test will not be
> > > > > > > > performed if these versions differ significantly (which will mean
> > > > > > > > that
> > > > > > > > there were major protocol changes meantime
> > > > > > > > and much older clients can't be supported). Also this version
> > > > > > > > information could be used by server to determine
> > > > > > > > what messages send to such client during specific test etc. What
> > > > > > > > do you
> > > > > > > > all think about such solution?
> > > > > > > I think that is a great idea! It is one that I thought we would
> > > > > > > pursue
> > > > > > > at the start. However, it does not seem like the current test
> > > > > > > protocols
> > > > > > > allow for such an easy extension. At best, the test protocol 
> > > > > > > seem to
> > > > > > > exchange port numbers. There is some extended handshaking with the
> > > > > > > MID
> > > > > > > test and the SWF test. But, that does not seem to provide enough
> > > > > > > "room"
> > > > > > > for us to work.
> > > > > > >
> > > > > > > Aaron, do you agree with that?
> > > > > > >
> > > > > > > > And when it comes to cooperation between new client and old
> > > > > > > > server with
> > > > > > > > Will's changes would it be possible to re-send
> > > > > > > > login message by new client when the extended one fails? Or is
> > > > > > > > server
> > > > > > > > closing socket when first message fails and it couldn't
> > > > > > > > be done?
> > > > > > > I *think* that I answered this in my previous email. The problem 
> > > > > > > with
> > > > > > > making a backward compatible client is exactly as you say: The 
> > > > > > > server
> > > > > > > will immediately close the control connection when it receives a 
> > > > > > > bad
> > > > > > > protocol message. So, the burden will fall almost entirely on the
> > > > > > > client
> > > > > > > to be compatible with old servers. :-(
> > > > > > I see, so it indeed is problem. What about making additional test
> > > > > > running
> > > > > > before all the others (I think ATM there is possibility to
> > > > > > determine tests'
> > > > > > order?) which goal will be to exchange client/server versions. If
> > > > > > client
> > > > > > wouldn't request such test then server will know that client does 
> > > > > > not
> > > > > > support new protocol changes. However if such test will be requested
> > > > > > and performed then we can make server and client to exchange
> > > > > > additional
> > > > > > information at the beginning of each next tests such as mentioned
> > > > > > protocol
> > > > > > versions.
> > > > > > In such solution there shouldn't be problem with having old server
> > > > > > and new
> > > > > > client as new client will just send request for performing this
> > > > > > additional test
> > > > > > and old server which does not support it will just send response
> > > > > > with test suite
> > > > > > not containing this new test (then client will abandon sending
> > > > > > additional info
> > > > > > containing protocol versions and will assume that server supports
> > > > > > only the
> > > > > > oldest one). What do you think about such method?
> > > > > > Also looking forward for Aaron's confirmation about thoughts
> > > > > > mentioned above.
> > > > > I’m not sure there’s a problem with the MSG_LOGIN_EXTENDED either.
> > > > > The server spits out an error message about an invalid message
> > > > > received, and then the client reconnects. The only benefit I can see
> > > > > with the more complex test-exchange solution is that there wouldn’t
> > > > > be an error message showing up in the server logs. Unless there’s
> > > > > some other benefit to it, I’m not sure the added complexity is worth
> > > > > it.
> > > > >
> > > > > > > Also I would like to thank you all for making this discussion so
> > > > > > > > spirited :). I'm sure that with all ideas mentioned here we will
> > > > > > > > achieve our goal in the best way we can.
> > > > > > > I love working on this! Thank you both so much for your efforts!
> > > > > > >
> > > > > > > Will
> > > > > > >
> > > > > > > > Regards
> > > > > > > > Sebastian Kostuch
> > > > > > > >
> > > > > > > > On 04.04.2014 09:09, Sebastian Kostuch wrote:
> > > > > > > > > Hi Will
> > > > > > > > > shouldn't also compatibility between new client and old server be
> > > > > > > > > considered? Currently such client does not work and I'm getting
> > > > > > > > > "Logging to server: Received wrong type of the message" error.
> > > > > > > > > Also it probably would be good if we could number versions of our
> > > > > > > > > clients uniformly or maybe send additional information containing
> > > > > > > > > type of client (ATM flash client is differently versioned when
> > > > > > > > > compared to c/java one)?
> > > > > > > > >
> > > > > > > > > Regards,
> > > > > > > > > Sebastian Kostuch
> > > > > > > > >
> > > > > > > > > On 02.04.2014 15:43, Aaron Brown wrote:
> > > > > > > > > > Fine by me
> > > > > > > > > >
> > > > > > > > > > Cheers,
> > > > > > > > > > Aaron
> > > > > > > > > >
> > > > > > > > > > On Apr 2, 2014, at 1:05 AM, Will Hawkins
> > > > > > > > > > <>
> > > > > > > > > >  wrote:
> > > > > > > > > >
> > > > > > > > > > > Thank you for bringing up this discussion. I will need similar
> > > > > > > > > > > functionality in the very near future to support S2C tests 
> > > > > > > > > > > in the
> > > > > > > > > > > Flash applet. I like the idea of an extended login message 
> > > > > > > > > > > like:
> > > > > > > > > > >
> > > > > > > > > > > Type Length Value
> > > > > > > > > > > 11   (>= 1) (first byte has MSG_LOGIN semantics) (subsequent
> > > > > > > > > > > bytes
> > > > > > > > > > > are client version number in US-ASCII, the way that the 
> > > > > > > > > > > server's
> > > > > > > > > > > MSG_LOGIN is structured)
> > > > > > > > > > >
> > > > > > > > > > > If that is something that people would be okay with, I can 
> > > > > > > > > > > take a
> > > > > > > > > > > whack at implementing it on the server side. The client version
> > > > > > > > > > > number could be incorporated into the TestOptions struct 
> > > > > > > > > > > since it
> > > > > > > > > > > already gets passed to each of the test functions. The test
> > > > > > > > > > > functions can use that to guard non-default functionality.
> > > > > > > > > > >
> > > > > > > > > > > As I said, this type of message will really help me in the
> > > > > > > > > > > next few
> > > > > > > > > > > days as I try to improve the performance of the Flash client
> > > > > > > > > > > so I'm
> > > > > > > > > > > happy to work on it!
> > > > > > > > > > >
> > > > > > > > > > > Thanks again for bringing this matter up, Sebastian.
> > > > > > > > > > >
> > > > > > > > > > > Will
> > > > > > > > > > >
> > > > > > > > > > >
> > > > > > > > > > > On 4/1/14, 9:11 AM, Aaron Brown wrote:
> > > > > > > > > > > > It might be cleaner in the long run to introduce a new
> > > > > > > > > > > > message type
> > > > > > > > > > > > (MSG_EXTENDED_LOGIN or something). The client could try
> > > > > > > > > > > > logging in
> > > > > > > > > > > > with
> > > > > > > > > > > > that extended login message, and if it fails, it could fall
> > > > > > > > > > > > back to
> > > > > > > > > > > > MSG_LOGIN if applicable. We could probably also move to a JSON
> > > > > > > > > > > > structure
> > > > > > > > > > > > as the message content in the process. If we think through
> > > > > > > > > > > > that route,
> > > > > > > > > > > > it might make it easier down the road to do something that
> > > > > > > > > > > > works over
> > > > > > > > > > > > HTTP as well.
> > > > > > > > > > > >
> > > > > > > > > > > > Cheers,
> > > > > > > > > > > > Aaron
> > > > > > > > > > > >
> > > > > > > > > > > > On Mar 31, 2014, at 8:56 AM, Sebastian Kostuch
> > > > > > > > > > > > <
> > > > > > > > > > > > <mailto:>>
> > > > > > > > > > > >  wrote:
> > > > > > > > > > > >
> > > > > > > > > > > > > Hi,
> > > > > > > > > > > > > while working on Issue136
> > > > > > > > > > > > > <https://code.google.com/p/ndt/issues/detail?id=136&colspec=ID%20Type%20Status%20Severity%20Milestone%20Priority%20Owner%20Summary> 
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > >
> > > > > > > > > > > > > I have encountered some difficulties with making little
> > > > > > > > > > > > > changes in
> > > > > > > > > > > > > middlebox protocol
> > > > > > > > > > > > > without making old client not working with new server. So
> > > > > > > > > > > > > after some
> > > > > > > > > > > > > investigation I would like to propose
> > > > > > > > > > > > > following changes in order to solve this problem:
> > > > > > > > > > > > >
> > > > > > > > > > > > > To make protocol changes possible in future versions without
> > > > > > > > > > > > > losing
> > > > > > > > > > > > > backward compatibility both server and client
> > > > > > > > > > > > > need to know versions of each other. At this moment server
> > > > > > > > > > > > > has no
> > > > > > > > > > > > > information about client one. To fix this we
> > > > > > > > > > > > > need our clients to sent such information before tests are
> > > > > > > > > > > > > being run
> > > > > > > > > > > > > but we can't either change currently sent messages
> > > > > > > > > > > > > (such as the login message which could contain client
> > > > > > > > > > > > > version) either
> > > > > > > > > > > > > pass some new type of message from client to
> > > > > > > > > > > > > server (old server would know nothing about it).
> > > > > > > > > > > > >
> > > > > > > > > > > > > So probably it would be good to create some new test running
> > > > > > > > > > > > > before
> > > > > > > > > > > > > any other for only this versions exchange.
> > > > > > > > > > > > > This test should be obligatory (when server will not receive
> > > > > > > > > > > > > id of
> > > > > > > > > > > > > this one in client's request then it will treat client as the
> > > > > > > > > > > > > one with older versions, before proposed changes). On the
> > > > > > > > > > > > > other  hand
> > > > > > > > > > > > > when new client will send such test request to
> > > > > > > > > > > > > old server, then it would be just ignored (marked as not
> > > > > > > > > > > > > supported)
> > > > > > > > > > > > > and all will work ok also. This way backward
> > > > > > > > > > > > > compatibility should be assured.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Having both server and client know about their versions we
> > > > > > > > > > > > > can then
> > > > > > > > > > > > > perform some changes in tests protocol. And
> > > > > > > > > > > > > here goes the solution for Issue136: instead of always 
> > > > > > > > > > > > > sending
> > > > > > > > > > > > > additional RTT and TCP buffer size values server will
> > > > > > > > > > > > > just check which client he is connected to. If it is older
> > > > > > > > > > > > > one which
> > > > > > > > > > > > > does not support newest middlebox test protocol then
> > > > > > > > > > > > > these values will not be attached to final results.
> > > > > > > > > > > > >
> > > > > > > > > > > > > Also instead of checking only client/server version we could
> > > > > > > > > > > > > introduce
> > > > > > > > > > > > > numbering of individual tests protocols and server
> > > > > > > > > > > > > would sent such information to client before tests (i.e. 
> > > > > > > > > > > > > string
> > > > > > > > > > > > > containing entries such as "[testID]-[protocolVersion]").
> > > > > > > > > > > > >
> > > > > > > > > > > > > What do you think about such solution? Is this good way to
> > > > > > > > > > > > > assure
> > > > > > > > > > > > > backward compatibility and making introducing
> > > > > > > > > > > > > some protocol changes in future easier? Also if you have any
> > > > > > > > > > > > > questions
> > > > > > > > > > > > > feel free to ask them :).
> > > > > > > > > > > > >
> > > > > > > > > > > > > Regards,
> > > > > > > > > > > > > Sebastian Kostuch