Skip to Content.
Sympa Menu

ndt-users - Re: Packets arriving out-of-order - which direction?

Subject: ndt-users list created

List archive

Re: Packets arriving out-of-order - which direction?


Chronological Thread 
  • From: Richard Carlson <>
  • To: Clarke Morledge <>,
  • Subject: Re: Packets arriving out-of-order - which direction?
  • Date: Thu, 13 Dec 2007 21:23:17 -0500

Hi Clarke;

All the detailed data reported by the NDT tool comes from the Server-to-Client test. This is because the web100 kernel captures the most details while sending. Since the server has the web100 enhanced kernel, the most details come when the server is sending data to the client.

The message below indicates that the client is generating a large percentage of duplicate ACKs. Duplicate ACKs occur when the client receives a packet out of order. If the packet was really lost, the server would also record a large number of fast retransmits. If the packets were just reordered, then the number of fast retransmits should be low.

To diagnose this problem, it would really help to set the expectation level correctly. Based on the fact that the S2C test is over 100 Mbps, can I safely assume that this is a gigabit Ethernet network? The RTT value says that this is a LAN envrionment, but there is not a large queue beiing built up on the server. The client is using a 64 KB buffer and window scaling is probably turned off. Based on the RTT and TCP buffer size this connection should max out around 135 Mbps, which is what you are getting.

One think you can try is to start the web100srv process with the "-r" option. This will cause it to log the web100 variable for the C2S test. There might be something in there that can help you gain a little more understanding. Another thing to try is add the "-t" option to the web100srv process. This will cause the server to generate tcpdump files for both s2c and c2s tests. These files can be analyzed to determine what is going on at the packet level.

Finally, what kind of infrastructure is between the client and server? How many switches and/or routers? Have you tired moving the client to another switch port? How about trying another Ethernet drop cable?

Rich
At 05:33 PM 12/13/2007, Clarke Morledge wrote:
I am trying to interpret some of the output from NDT (v5.4.8) that indicates assymetrical performance variance. The key component for me is the line below:

"No packet loss - but packets arrived out-of-order 12.58% of the time"

But the problem I have is that it isn't clear to me where the packets are arriving out-of-order. In this case I am assuming that it is from the client-to-server direction, but could there also be some out-of-order in the server-to-client direction, too? The output isn't clear.

However, is the packets arriving out-of-order component really helping me to diagnose the assymetry problem, or is this a red herring? Is there some relevant web-100 variable(s) that I should be looking at instead?

Thanks.
-----------------------

WEB100 Enabled Statistics:
Checking for Middleboxes . . . . . . . . . . . . . . . . . . Done
checking for firewalls . . . . . . . . . . . . . . . . . . . Done
running 10s outbound test (client-to-server [C2S]) . . . . . 40.08Mb/s
running 10s inbound test (server-to-client [S2C]) . . . . . . 134.32Mb/s

------ Client System Details ------
OS data: Name = Windows XP, Architecture = x86, Version = 5.1
Java data: Vendor = Sun Microsystems Inc., Version = 1.6.0_02

------ Web100 Detailed Analysis ------
622 Mbps OC-12 link found.
Link set to Full Duplex mode
No network congestion discovered.
Good network cable(s) found
Normal duplex operation found.

Web100 reports the Round trip time = 3.27 msec; the Packet size = 1460 Bytes; and
No packet loss - but packets arrived out-of-order 12.58% of the time
C2S throughput test: Packet queuing detected: 0.00%
S2C throughput test: Excessive packet queuing detected: 10.87%
This connection is receiver limited 58.26% of the time.
Increasing the the client's receive buffer (63.0 KB) will improve performance
This connection is sender limited 41.27% of the time.

-------------------------------------------------------------

Clarke Morledge
College of William and Mary
Information Technology - Network Engineering
Jones Hall (Room 18)
Williamsburg VA 23187

------------------------------------



Richard A. Carlson e-mail:

Network Engineer phone: (734) 352-7043
Internet2 fax: (734) 913-4255
1000 Oakbrook Dr; Suite 300
Ann Arbor, MI 48104



Archive powered by MHonArc 2.6.16.

Top of Page