Skip to Content.
Sympa Menu

ndt-dev - Re: [ndt-dev] Allowing protocol changes without losing backward compatibility

Subject: NDT-DEV email list created

List archive

Re: [ndt-dev] Allowing protocol changes without losing backward compatibility

Chronological Thread 
  • From: Aaron Brown <>
  • To: Sebastian Kostuch <>
  • Cc: "" <>
  • Subject: Re: [ndt-dev] Allowing protocol changes without losing backward compatibility
  • Date: Tue, 15 Apr 2014 14:05:27 +0000
  • Accept-language: en-US

Hey Sebastian,

My general sense would be to minimize changes unless we have a specific
use-case. I think a back-and-forth for each test type might be a good idea,
but I’m hesitant to add it in without something that specifically needs it.


On Apr 14, 2014, at 1:46 AM, Sebastian Kostuch

> Hi
> my intention was to make additional message at the beginning of each test
> (not extending current login message which contains client version).
> Anyway it could be some kind of login message too and JSON seems to be good
> way to pack it into. This message would simply contain information
> for client which protocol version and types server does support and client
> would behave accordingly to it. When it comes to expected variables then
> naming web10/100 vars was just example but in fact it isn't the most
> flexible solution. Should we first distinct which type of variable client
> expects
> (i.e. tcp_info, web10/100, another ones) and then deliver array of their
> names or would it be better for example to enlist available results which
> server could provide in some documentation and each client would then just
> simply put names from it into such prepared message? Or maybe
> do you have some different thoughts about it? Looking forward to it :).
> Regards,
> Sebastian
> On 11.04.2014 20:49, Aaron Brown wrote:
>> My preference if we’re going to go down the path of adding more stuff into
>> the login message would be to have the body of the message be a JSON hash.
>> This would allow adding more variables without breaking backwards
>> compatibility and/or defining another schema for representing structured
>> data.
>> As to expected results, I’m not sure the best way to structure that since
>> logically the web100/web10g variables being requested shouldn’t depend on
>> web100/web10g (i.e. if web100g ever exists, or if we can provide some of
>> them via TCP_INFO, it shouldn’t break things). Beyond that, there are some
>> results that aren’t just web100-variables, but are combinations of web100
>> variables and other data (e.g. bottleneck link) that presumably would be
>> good to have in there as well.
>> Cheers,
>> Aaron
>> On Apr 11, 2014, at 4:03 AM, Sebastian Kostuch
>> <>
>> wrote:
>>> Hi
>>> indeed we can make client reconnect after server receiving not
>>> supported message. It would mean to have some redundant
>>> established connections which in fact will be shut down quickly
>>> but I think that in future old servers will get less and less popular
>>> so such solution seems to be good and actually simplest than the
>>> one with additional test. So I will too stick with the current solution.
>>> However there is also one thing which I think should be considered.
>>> Currently we have client and server exchange only their versions
>>> but as I have mentioned in some previous messages it would probably
>>> be cleaner to make them also send additional protocol information
>>> at the start of each test (assuming that they both are with new
>>> changes, if client or server is old then of course such operation
>>> will not be performed and both sides will behave as the protocol
>>> version was the old one).
>>> I suggest to make them exchange information such as:
>>> 1. Protocol version number
>>> 2. Supported protocols (such as HTTP in future) - this could be contained
>>> within
>>> protocol version string as well i.e. S2C-2.0-HTTP
>>> 3. Results set expected by client (i.e. names of web100/web10 variables)
>>> which will be sent for example right after protocol version message. This
>>> way it would be simple in future to make changes when we decide to
>>> remove some existing results variables or add some new (when server
>>> would not support such result variable then it will just respond with some
>>> kind of special value). Such message should be formatted for instance with
>>> string containing variable names separated with semicolon.
>>> So simply said the reason behind making such additional message exchange
>>> at the beginning of tests is to make doing changes in protocols in future
>>> more
>>> easily. What is your opinion about this? I think we should consider it
>>> ASAP as
>>> we will need these changes to make some small protocol changes planned
>>> earlier (for me it is adding additional variables to Middlebox test
>>> results).
>>> Regards
>>> Sebastian
>>> On 07.04.2014 15:03, Aaron Brown wrote:
>>>> Hey Sebastian,
>>>> Comment inline
>>>> Cheers,
>>>> Aaron
>>>> On Apr 7, 2014, at 2:15 AM, Sebastian Kostuch
>>>> <>
>>>> wrote:
>>>>> Hi
>>>>> On 04.04.2014 19:59, Will Hawkins wrote:
>>>>>> Pre PS: There is a running version of the new server at
>>>>>> Feel free to send
>>>>>> connections
>>>>>> to it to test the new MSG_EXTENDED_LOGIN semantics.
>>>>>> On 04/04/2014 06:35 AM, Sebastian Kostuch wrote:
>>>>>>> Hi again
>>>>>>> in addition to clients versioning I have mentioned in previous email I
>>>>>>> think that maybe it would be better to not determine
>>>>>>> server actions based only on this information but rather using the one
>>>>>>> more detailed such as introducing versioning
>>>>>>> of individual test protocols. I mean that server and client should
>>>>>>> have
>>>>>>> possibility to exchange information at the
>>>>>>> beginning of each test which will contain version of protocol
>>>>>>> supported
>>>>>>> by both sites and such test will not be
>>>>>>> performed if these versions differ significantly (which will mean that
>>>>>>> there were major protocol changes meantime
>>>>>>> and much older clients can't be supported). Also this version
>>>>>>> information could be used by server to determine
>>>>>>> what messages send to such client during specific test etc. What do
>>>>>>> you
>>>>>>> all think about such solution?
>>>>>> I think that is a great idea! It is one that I thought we would pursue
>>>>>> at the start. However, it does not seem like the current test protocols
>>>>>> allow for such an easy extension. At best, the test protocol seem to
>>>>>> exchange port numbers. There is some extended handshaking with the MID
>>>>>> test and the SWF test. But, that does not seem to provide enough "room"
>>>>>> for us to work.
>>>>>> Aaron, do you agree with that?
>>>>>>> And when it comes to cooperation between new client and old server
>>>>>>> with
>>>>>>> Will's changes would it be possible to re-send
>>>>>>> login message by new client when the extended one fails? Or is server
>>>>>>> closing socket when first message fails and it couldn't
>>>>>>> be done?
>>>>>> I *think* that I answered this in my previous email. The problem with
>>>>>> making a backward compatible client is exactly as you say: The server
>>>>>> will immediately close the control connection when it receives a bad
>>>>>> protocol message. So, the burden will fall almost entirely on the
>>>>>> client
>>>>>> to be compatible with old servers. :-(
>>>>> I see, so it indeed is problem. What about making additional test
>>>>> running
>>>>> before all the others (I think ATM there is possibility to determine
>>>>> tests'
>>>>> order?) which goal will be to exchange client/server versions. If client
>>>>> wouldn't request such test then server will know that client does not
>>>>> support new protocol changes. However if such test will be requested
>>>>> and performed then we can make server and client to exchange additional
>>>>> information at the beginning of each next tests such as mentioned
>>>>> protocol
>>>>> versions.
>>>>> In such solution there shouldn't be problem with having old server and
>>>>> new
>>>>> client as new client will just send request for performing this
>>>>> additional test
>>>>> and old server which does not support it will just send response with
>>>>> test suite
>>>>> not containing this new test (then client will abandon sending
>>>>> additional info
>>>>> containing protocol versions and will assume that server supports only
>>>>> the
>>>>> oldest one). What do you think about such method?
>>>>> Also looking forward for Aaron's confirmation about thoughts mentioned
>>>>> above.
>>>> I’m not sure there’s a problem with the MSG_LOGIN_EXTENDED either. The
>>>> server spits out an error message about an invalid message received, and
>>>> then the client reconnects. The only benefit I can see with the more
>>>> complex test-exchange solution is that there wouldn’t be an error
>>>> message showing up in the server logs. Unless there’s some other benefit
>>>> to it, I’m not sure the added complexity is worth it.
>>>>>> Also I would like to thank you all for making this discussion so
>>>>>>> spirited :). I'm sure that with all ideas mentioned here we will
>>>>>>> achieve our goal in the best way we can.
>>>>>> I love working on this! Thank you both so much for your efforts!
>>>>>> Will
>>>>>>> Regards
>>>>>>> Sebastian Kostuch
>>>>>>> On 04.04.2014 09:09, Sebastian Kostuch wrote:
>>>>>>>> Hi Will
>>>>>>>> shouldn't also compatibility between new client and old server be
>>>>>>>> considered? Currently such client does not work and I'm getting
>>>>>>>> "Logging to server: Received wrong type of the message" error.
>>>>>>>> Also it probably would be good if we could number versions of our
>>>>>>>> clients uniformly or maybe send additional information containing
>>>>>>>> type of client (ATM flash client is differently versioned when
>>>>>>>> compared to c/java one)?
>>>>>>>> Regards,
>>>>>>>> Sebastian Kostuch
>>>>>>>> On 02.04.2014 15:43, Aaron Brown wrote:
>>>>>>>>> Fine by me
>>>>>>>>> Cheers,
>>>>>>>>> Aaron
>>>>>>>>> On Apr 2, 2014, at 1:05 AM, Will Hawkins
>>>>>>>>> <>
>>>>>>>>> wrote:
>>>>>>>>>> Thank you for bringing up this discussion. I will need similar
>>>>>>>>>> functionality in the very near future to support S2C tests in the
>>>>>>>>>> Flash applet. I like the idea of an extended login message like:
>>>>>>>>>> Type Length Value
>>>>>>>>>> 11 (>= 1) (first byte has MSG_LOGIN semantics) (subsequent bytes
>>>>>>>>>> are client version number in US-ASCII, the way that the server's
>>>>>>>>>> MSG_LOGIN is structured)
>>>>>>>>>> If that is something that people would be okay with, I can take a
>>>>>>>>>> whack at implementing it on the server side. The client version
>>>>>>>>>> number could be incorporated into the TestOptions struct since it
>>>>>>>>>> already gets passed to each of the test functions. The test
>>>>>>>>>> functions can use that to guard non-default functionality.
>>>>>>>>>> As I said, this type of message will really help me in the next few
>>>>>>>>>> days as I try to improve the performance of the Flash client so I'm
>>>>>>>>>> happy to work on it!
>>>>>>>>>> Thanks again for bringing this matter up, Sebastian.
>>>>>>>>>> Will
>>>>>>>>>> On 4/1/14, 9:11 AM, Aaron Brown wrote:
>>>>>>>>>>> It might be cleaner in the long run to introduce a new message
>>>>>>>>>>> type
>>>>>>>>>>> (MSG_EXTENDED_LOGIN or something). The client could try logging in
>>>>>>>>>>> with
>>>>>>>>>>> that extended login message, and if it fails, it could fall back
>>>>>>>>>>> to
>>>>>>>>>>> MSG_LOGIN if applicable. We could probably also move to a JSON
>>>>>>>>>>> structure
>>>>>>>>>>> as the message content in the process. If we think through that
>>>>>>>>>>> route,
>>>>>>>>>>> it might make it easier down the road to do something that works
>>>>>>>>>>> over
>>>>>>>>>>> HTTP as well.
>>>>>>>>>>> Cheers,
>>>>>>>>>>> Aaron
>>>>>>>>>>> On Mar 31, 2014, at 8:56 AM, Sebastian Kostuch
>>>>>>>>>>> <
>>>>>>>>>>> <mailto:>>
>>>>>>>>>>> wrote:
>>>>>>>>>>>> Hi,
>>>>>>>>>>>> while working on Issue136
>>>>>>>>>>>> <>
>>>>>>>>>>>> I have encountered some difficulties with making little changes
>>>>>>>>>>>> in
>>>>>>>>>>>> middlebox protocol
>>>>>>>>>>>> without making old client not working with new server. So after
>>>>>>>>>>>> some
>>>>>>>>>>>> investigation I would like to propose
>>>>>>>>>>>> following changes in order to solve this problem:
>>>>>>>>>>>> To make protocol changes possible in future versions without
>>>>>>>>>>>> losing
>>>>>>>>>>>> backward compatibility both server and client
>>>>>>>>>>>> need to know versions of each other. At this moment server has no
>>>>>>>>>>>> information about client one. To fix this we
>>>>>>>>>>>> need our clients to sent such information before tests are being
>>>>>>>>>>>> run
>>>>>>>>>>>> but we can't either change currently sent messages
>>>>>>>>>>>> (such as the login message which could contain client version)
>>>>>>>>>>>> either
>>>>>>>>>>>> pass some new type of message from client to
>>>>>>>>>>>> server (old server would know nothing about it).
>>>>>>>>>>>> So probably it would be good to create some new test running
>>>>>>>>>>>> before
>>>>>>>>>>>> any other for only this versions exchange.
>>>>>>>>>>>> This test should be obligatory (when server will not receive id
>>>>>>>>>>>> of
>>>>>>>>>>>> this one in client's request then it will treat client as the
>>>>>>>>>>>> one with older versions, before proposed changes). On the other
>>>>>>>>>>>> hand
>>>>>>>>>>>> when new client will send such test request to
>>>>>>>>>>>> old server, then it would be just ignored (marked as not
>>>>>>>>>>>> supported)
>>>>>>>>>>>> and all will work ok also. This way backward
>>>>>>>>>>>> compatibility should be assured.
>>>>>>>>>>>> Having both server and client know about their versions we can
>>>>>>>>>>>> then
>>>>>>>>>>>> perform some changes in tests protocol. And
>>>>>>>>>>>> here goes the solution for Issue136: instead of always sending
>>>>>>>>>>>> additional RTT and TCP buffer size values server will
>>>>>>>>>>>> just check which client he is connected to. If it is older one
>>>>>>>>>>>> which
>>>>>>>>>>>> does not support newest middlebox test protocol then
>>>>>>>>>>>> these values will not be attached to final results.
>>>>>>>>>>>> Also instead of checking only client/server version we could
>>>>>>>>>>>> introduce
>>>>>>>>>>>> numbering of individual tests protocols and server
>>>>>>>>>>>> would sent such information to client before tests (i.e. string
>>>>>>>>>>>> containing entries such as "[testID]-[protocolVersion]").
>>>>>>>>>>>> What do you think about such solution? Is this good way to assure
>>>>>>>>>>>> backward compatibility and making introducing
>>>>>>>>>>>> some protocol changes in future easier? Also if you have any
>>>>>>>>>>>> questions
>>>>>>>>>>>> feel free to ask them :).
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Sebastian Kostuch

Archive powered by MHonArc 2.6.16.

Top of Page