NDT-DEV email list created

[Aaron Brown <>]

Hey Sebastian,

My general sense would be to minimize changes unless we have a specific 
use-case. I think a back-and-forth for each test type might be a good idea, 
but I’m hesitant to add it in without something that specifically needs it.

Cheers,
Aaron

On Apr 14, 2014, at 1:46 AM, Sebastian Kostuch 
<>
 wrote:

> Hi
> my intention was to make additional message at the beginning of each test 
> (not extending current login message which contains client version).
> Anyway it could be some kind of login message too and JSON seems to be good 
> way to pack it into. This message would simply contain information
> for client which protocol version and types server does support and client 
> would behave accordingly to it. When it comes to expected variables then
> naming web10/100 vars was just example but in fact it isn't the most 
> flexible solution. Should we first distinct which type of variable client 
> expects
> (i.e. tcp_info, web10/100, another ones) and then deliver array of their 
> names or would it be better for example to enlist available results which
> server could provide in some documentation and each client would then just 
> simply put names from it into such prepared message? Or maybe
> do you have some different thoughts about it? Looking forward to it :).
> 
> Regards,
> Sebastian
> 
> On 11.04.2014 20:49, Aaron Brown wrote:
>> My preference if we’re going to go down the path of adding more stuff into 
>> the login message would be to have the body of the message be a JSON hash. 
>> This would allow adding more variables without breaking backwards 
>> compatibility and/or defining another schema for representing structured 
>> data.
>> 
>> As to expected results, I’m not sure the best way to structure that since 
>> logically the web100/web10g variables being requested shouldn’t depend on 
>> web100/web10g (i.e. if web100g ever exists, or if we can provide some of 
>> them via TCP_INFO, it shouldn’t break things). Beyond that, there are some 
>> results that aren’t just web100-variables, but are combinations of web100 
>> variables and other data (e.g. bottleneck link) that presumably would be 
>> good to have in there as well.
>> 
>> Cheers,
>> Aaron
>> 
>> On Apr 11, 2014, at 4:03 AM, Sebastian Kostuch 
>> <>
>>  wrote:
>> 
>>> Hi
>>> indeed we can make client reconnect after server receiving not
>>> supported message. It would mean to have some redundant
>>> established connections which in fact will be shut down quickly
>>> but I think that in future old servers will get less and less popular
>>> so such solution seems to be good and actually simplest than the
>>> one with additional test. So I will too stick with the current solution.
>>> 
>>> However there is also one thing which I think should be considered.
>>> Currently we have client and server exchange only their versions
>>> but as I have mentioned in some previous messages it would probably
>>> be cleaner to make them also send additional protocol information
>>> at the start of each test (assuming that they both are with new
>>> changes, if client or server is old then of course such operation
>>> will not be performed and both sides will behave as the protocol
>>> version was the old one).
>>> 
>>> I suggest to make them exchange information such as:
>>> 1. Protocol version number
>>> 2. Supported protocols (such as HTTP in future) - this could be contained 
>>> within
>>> protocol version string as well i.e. S2C-2.0-HTTP
>>> 3. Results set expected by client (i.e. names of web100/web10 variables)
>>> which will be sent for example right after protocol version message. This
>>> way it would be simple in future to make changes when we decide to
>>> remove some existing results variables or add some new (when server
>>> would not support such result variable then it will just respond with some
>>> kind of special value). Such message should be formatted for instance with
>>> string containing variable names separated with semicolon.
>>> 
>>> So simply said the reason behind making such additional message exchange
>>> at the beginning of tests is to make doing changes in protocols in future 
>>> more
>>> easily. What is your opinion about this? I think we should consider it 
>>> ASAP as
>>> we will need these changes to make some small protocol changes planned
>>> earlier (for me it is adding additional variables to Middlebox test 
>>> results).
>>> 
>>> Regards
>>> Sebastian
>>> 
>>> On 07.04.2014 15:03, Aaron Brown wrote:
>>>> Hey Sebastian,
>>>> 
>>>> Comment inline
>>>> 
>>>> Cheers,
>>>> Aaron
>>>> 
>>>> On Apr 7, 2014, at 2:15 AM, Sebastian Kostuch 
>>>> <>
>>>>  wrote:
>>>> 
>>>>> Hi
>>>>> 
>>>>> On 04.04.2014 19:59, Will Hawkins wrote:
>>>>>> Pre PS: There is a running version of the new server at
>>>>>> ndt.iupui.mlab4.nuq0t.measurement-lab.org. Feel free to send 
>>>>>> connections
>>>>>> to it to test the new MSG_EXTENDED_LOGIN semantics.
>>>>>> 
>>>>>> On 04/04/2014 06:35 AM, Sebastian Kostuch wrote:
>>>>>>> Hi again
>>>>>>> in addition to clients versioning I have mentioned in previous email I
>>>>>>> think that maybe it would be better to not determine
>>>>>>> server actions based only on this information but rather using the one
>>>>>>> more detailed such as introducing versioning
>>>>>>> of individual test protocols. I mean that server and client should 
>>>>>>> have
>>>>>>> possibility to exchange information at the
>>>>>>> beginning of each test which will contain version of protocol 
>>>>>>> supported
>>>>>>> by both sites and such test will not be
>>>>>>> performed if these versions differ significantly (which will mean that
>>>>>>> there were major protocol changes meantime
>>>>>>> and much older clients can't be supported). Also this version
>>>>>>> information could be used by server to determine
>>>>>>> what messages send to such client during specific test etc. What do 
>>>>>>> you
>>>>>>> all think about such solution?
>>>>>> I think that is a great idea! It is one that I thought we would pursue
>>>>>> at the start. However, it does not seem like the current test protocols
>>>>>> allow for such an easy extension. At best, the test protocol seem to
>>>>>> exchange port numbers. There is some extended handshaking with the MID
>>>>>> test and the SWF test. But, that does not seem to provide enough "room"
>>>>>> for us to work.
>>>>>> 
>>>>>> Aaron, do you agree with that?
>>>>>> 
>>>>>>> And when it comes to cooperation between new client and old server 
>>>>>>> with
>>>>>>> Will's changes would it be possible to re-send
>>>>>>> login message by new client when the extended one fails? Or is server
>>>>>>> closing socket when first message fails and it couldn't
>>>>>>> be done?
>>>>>> I *think* that I answered this in my previous email. The problem with
>>>>>> making a backward compatible client is exactly as you say: The server
>>>>>> will immediately close the control connection when it receives a bad
>>>>>> protocol message. So, the burden will fall almost entirely on the 
>>>>>> client
>>>>>> to be compatible with old servers. :-(
>>>>> I see, so it indeed is problem. What about making additional test 
>>>>> running
>>>>> before all the others (I think ATM there is possibility to determine 
>>>>> tests'
>>>>> order?) which goal will be to exchange client/server versions. If client
>>>>> wouldn't request such test then server will know that client does not
>>>>> support new protocol changes. However if such test will be requested
>>>>> and performed then we can make server and client to exchange additional
>>>>> information at the beginning of each next tests such as mentioned 
>>>>> protocol
>>>>> versions.
>>>>> In such solution there shouldn't be problem with having old server and 
>>>>> new
>>>>> client as new client will just send request for performing this 
>>>>> additional test
>>>>> and old server which does not support it will just send response with 
>>>>> test suite
>>>>> not containing this new test (then client will abandon sending 
>>>>> additional info
>>>>> containing protocol versions and will assume that server supports only 
>>>>> the
>>>>> oldest one). What do you think about such method?
>>>>> Also looking forward for Aaron's confirmation about thoughts mentioned 
>>>>> above.
>>>> I’m not sure there’s a problem with the MSG_LOGIN_EXTENDED either. The 
>>>> server spits out an error message about an invalid message received, and 
>>>> then the client reconnects. The only benefit I can see with the more 
>>>> complex test-exchange solution is that there wouldn’t be an error 
>>>> message showing up in the server logs. Unless there’s some other benefit 
>>>> to it, I’m not sure the added complexity is worth it.
>>>> 
>>>>>> Also I would like to thank you all for making this discussion so
>>>>>>> spirited :). I'm sure that with all ideas mentioned here we will
>>>>>>> achieve our goal in the best way we can.
>>>>>> I love working on this! Thank you both so much for your efforts!
>>>>>> 
>>>>>> Will
>>>>>> 
>>>>>>> Regards
>>>>>>> Sebastian Kostuch
>>>>>>> 
>>>>>>> On 04.04.2014 09:09, Sebastian Kostuch wrote:
>>>>>>>> Hi Will
>>>>>>>> shouldn't also compatibility between new client and old server be
>>>>>>>> considered? Currently such client does not work and I'm getting
>>>>>>>> "Logging to server: Received wrong type of the message" error.
>>>>>>>> Also it probably would be good if we could number versions of our
>>>>>>>> clients uniformly or maybe send additional information containing
>>>>>>>> type of client (ATM flash client is differently versioned when
>>>>>>>> compared to c/java one)?
>>>>>>>> 
>>>>>>>> Regards,
>>>>>>>> Sebastian Kostuch
>>>>>>>> 
>>>>>>>> On 02.04.2014 15:43, Aaron Brown wrote:
>>>>>>>>> Fine by me
>>>>>>>>> 
>>>>>>>>> Cheers,
>>>>>>>>> Aaron
>>>>>>>>> 
>>>>>>>>> On Apr 2, 2014, at 1:05 AM, Will Hawkins
>>>>>>>>> <>
>>>>>>>>>  wrote:
>>>>>>>>> 
>>>>>>>>>> Thank you for bringing up this discussion. I will need similar
>>>>>>>>>> functionality in the very near future to support S2C tests in the
>>>>>>>>>> Flash applet. I like the idea of an extended login message like:
>>>>>>>>>> 
>>>>>>>>>> Type Length Value
>>>>>>>>>> 11   (>= 1) (first byte has MSG_LOGIN semantics) (subsequent bytes
>>>>>>>>>> are client version number in US-ASCII, the way that the server's
>>>>>>>>>> MSG_LOGIN is structured)
>>>>>>>>>> 
>>>>>>>>>> If that is something that people would be okay with, I can take a
>>>>>>>>>> whack at implementing it on the server side. The client version
>>>>>>>>>> number could be incorporated into the TestOptions struct since it
>>>>>>>>>> already gets passed to each of the test functions. The test
>>>>>>>>>> functions can use that to guard non-default functionality.
>>>>>>>>>> 
>>>>>>>>>> As I said, this type of message will really help me in the next few
>>>>>>>>>> days as I try to improve the performance of the Flash client so I'm
>>>>>>>>>> happy to work on it!
>>>>>>>>>> 
>>>>>>>>>> Thanks again for bringing this matter up, Sebastian.
>>>>>>>>>> 
>>>>>>>>>> Will
>>>>>>>>>> 
>>>>>>>>>> 
>>>>>>>>>> On 4/1/14, 9:11 AM, Aaron Brown wrote:
>>>>>>>>>>> It might be cleaner in the long run to introduce a new message 
>>>>>>>>>>> type
>>>>>>>>>>> (MSG_EXTENDED_LOGIN or something). The client could try logging in
>>>>>>>>>>> with
>>>>>>>>>>> that extended login message, and if it fails, it could fall back 
>>>>>>>>>>> to
>>>>>>>>>>> MSG_LOGIN if applicable. We could probably also move to a JSON
>>>>>>>>>>> structure
>>>>>>>>>>> as the message content in the process. If we think through that 
>>>>>>>>>>> route,
>>>>>>>>>>> it might make it easier down the road to do something that works 
>>>>>>>>>>> over
>>>>>>>>>>> HTTP as well.
>>>>>>>>>>> 
>>>>>>>>>>> Cheers,
>>>>>>>>>>> Aaron
>>>>>>>>>>> 
>>>>>>>>>>> On Mar 31, 2014, at 8:56 AM, Sebastian Kostuch 
>>>>>>>>>>> <
>>>>>>>>>>> <mailto:>>
>>>>>>>>>>>  wrote:
>>>>>>>>>>> 
>>>>>>>>>>>> Hi,
>>>>>>>>>>>> while working on Issue136
>>>>>>>>>>>> <https://code.google.com/p/ndt/issues/detail?id=136&colspec=ID%20Type%20Status%20Severity%20Milestone%20Priority%20Owner%20Summary>
>>>>>>>>>>>> 
>>>>>>>>>>>> I have encountered some difficulties with making little changes 
>>>>>>>>>>>> in
>>>>>>>>>>>> middlebox protocol
>>>>>>>>>>>> without making old client not working with new server. So after 
>>>>>>>>>>>> some
>>>>>>>>>>>> investigation I would like to propose
>>>>>>>>>>>> following changes in order to solve this problem:
>>>>>>>>>>>> 
>>>>>>>>>>>> To make protocol changes possible in future versions without 
>>>>>>>>>>>> losing
>>>>>>>>>>>> backward compatibility both server and client
>>>>>>>>>>>> need to know versions of each other. At this moment server has no
>>>>>>>>>>>> information about client one. To fix this we
>>>>>>>>>>>> need our clients to sent such information before tests are being 
>>>>>>>>>>>> run
>>>>>>>>>>>> but we can't either change currently sent messages
>>>>>>>>>>>> (such as the login message which could contain client version) 
>>>>>>>>>>>> either
>>>>>>>>>>>> pass some new type of message from client to
>>>>>>>>>>>> server (old server would know nothing about it).
>>>>>>>>>>>> 
>>>>>>>>>>>> So probably it would be good to create some new test running 
>>>>>>>>>>>> before
>>>>>>>>>>>> any other for only this versions exchange.
>>>>>>>>>>>> This test should be obligatory (when server will not receive id 
>>>>>>>>>>>> of
>>>>>>>>>>>> this one in client's request then it will treat client as the
>>>>>>>>>>>> one with older versions, before proposed changes). On the other  
>>>>>>>>>>>> hand
>>>>>>>>>>>> when new client will send such test request to
>>>>>>>>>>>> old server, then it would be just ignored (marked as not 
>>>>>>>>>>>> supported)
>>>>>>>>>>>> and all will work ok also. This way backward
>>>>>>>>>>>> compatibility should be assured.
>>>>>>>>>>>> 
>>>>>>>>>>>> Having both server and client know about their versions we can 
>>>>>>>>>>>> then
>>>>>>>>>>>> perform some changes in tests protocol. And
>>>>>>>>>>>> here goes the solution for Issue136: instead of always sending
>>>>>>>>>>>> additional RTT and TCP buffer size values server will
>>>>>>>>>>>> just check which client he is connected to. If it is older one 
>>>>>>>>>>>> which
>>>>>>>>>>>> does not support newest middlebox test protocol then
>>>>>>>>>>>> these values will not be attached to final results.
>>>>>>>>>>>> 
>>>>>>>>>>>> Also instead of checking only client/server version we could
>>>>>>>>>>>> introduce
>>>>>>>>>>>> numbering of individual tests protocols and server
>>>>>>>>>>>> would sent such information to client before tests (i.e. string
>>>>>>>>>>>> containing entries such as "[testID]-[protocolVersion]").
>>>>>>>>>>>> 
>>>>>>>>>>>> What do you think about such solution? Is this good way to assure
>>>>>>>>>>>> backward compatibility and making introducing
>>>>>>>>>>>> some protocol changes in future easier? Also if you have any
>>>>>>>>>>>> questions
>>>>>>>>>>>> feel free to ask them :).
>>>>>>>>>>>> 
>>>>>>>>>>>> Regards,
>>>>>>>>>>>> Sebastian Kostuch
>> 
>