This is the mail archive of the cygwin-patches mailing list for the Cygwin project.
Index Nav: | [Date Index] [Subject Index] [Author Index] [Thread Index] | |
---|---|---|
Message Nav: | [Date Prev] [Date Next] | [Thread Prev] [Thread Next] |
Other format: | [Raw text] |
At Windows we love what you are doing with Cygwin. However, we have been getting reports from our hardware vendors that iperf is slow on Windows. Iperf is of course compiled against the cygwin1.dll and we believe we have traced the problem down to the function fdsock in net.cc. SO_RCVBUF and SO_SNDBUF are being manually set. The comments indicate that the idea was to increase the buffer size, but, this code must have been written long ago because Windows has used autotuning for a very long time now. Please do not manually set SO_RCVBUF or SO_SNDBUF as this will limit your internet speed. I am providing a patch, an STC and my cygcheck -svr output. Hope we can fix this. Please let me know if I can help further. Simple Test Case: I have a script that pings 4 times and then iperfs for 10 seconds to debit.k-net.fr With patch $ bash buffer_test.sh 178.250.209.22 usage: bash buffer_test.sh <iperf server name> Pinging 178.250.209.22 with 32 bytes of data: Reply from 178.250.209.22: bytes=32 time=167ms TTL=34 Reply from 178.250.209.22: bytes=32 time=173ms TTL=34 Reply from 178.250.209.22: bytes=32 time=173ms TTL=34 Reply from 178.250.209.22: bytes=32 time=169ms TTL=34 Ping statistics for 178.250.209.22: Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), Approximate round trip times in milli-seconds: Minimum = 167ms, Maximum = 173ms, Average = 170ms ------------------------------------------------------------ Client connecting to 178.250.209.22, TCP port 5001 TCP window size: 64.0 KByte (default) ------------------------------------------------------------ [ 3] local 10.137.196.108 port 58512 connected with 178.250.209.22 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0- 1.0 sec 768 KBytes 6.29 Mbits/sec [ 3] 1.0- 2.0 sec 9.25 MBytes 77.6 Mbits/sec [ 3] 2.0- 3.0 sec 18.0 MBytes 151 Mbits/sec [ 3] 3.0- 4.0 sec 18.0 MBytes 151 Mbits/sec [ 3] 4.0- 5.0 sec 18.0 MBytes 151 Mbits/sec [ 3] 5.0- 6.0 sec 18.0 MBytes 151 Mbits/sec [ 3] 6.0- 7.0 sec 18.0 MBytes 151 Mbits/sec [ 3] 7.0- 8.0 sec 18.0 MBytes 151 Mbits/sec [ 3] 8.0- 9.0 sec 18.0 MBytes 151 Mbits/sec [ 3] 9.0-10.0 sec 18.0 MBytes 151 Mbits/sec [ 3] 0.0-10.0 sec 154 MBytes 129 Mbits/sec Without patch: dahavey@DMH-DESKTOP ~ $ bash buffer_test.sh 178.250.209.22 Pinging 178.250.209.22 with 32 bytes of data: Reply from 178.250.209.22: bytes=32 time=168ms TTL=34 Reply from 178.250.209.22: bytes=32 time=167ms TTL=34 Reply from 178.250.209.22: bytes=32 time=170ms TTL=34 Reply from 178.250.209.22: bytes=32 time=169ms TTL=34 Ping statistics for 178.250.209.22: Packets: Sent = 4, Received = 4, Lost = 0 (0% loss), Approximate round trip times in milli-seconds: Minimum = 167ms, Maximum = 170ms, Average = 168ms ------------------------------------------------------------ Client connecting to 178.250.209.22, TCP port 5001 TCP window size: 208 KByte (default) ------------------------------------------------------------ [ 3] local 10.137.196.108 port 58443 connected with 178.250.209.22 port 5001 [ ID] Interval Transfer Bandwidth [ 3] 0.0- 1.0 sec 512 KBytes 4.19 Mbits/sec [ 3] 1.0- 2.0 sec 1.50 MBytes 12.6 Mbits/sec [ 3] 2.0- 3.0 sec 1.50 MBytes 12.6 Mbits/sec [ 3] 3.0- 4.0 sec 1.50 MBytes 12.6 Mbits/sec [ 3] 4.0- 5.0 sec 1.50 MBytes 12.6 Mbits/sec [ 3] 5.0- 6.0 sec 1.50 MBytes 12.6 Mbits/sec [ 3] 6.0- 7.0 sec 1.50 MBytes 12.6 Mbits/sec [ 3] 7.0- 8.0 sec 1.50 MBytes 12.6 Mbits/sec [ 3] 8.0- 9.0 sec 1.50 MBytes 12.6 Mbits/sec [ 3] 9.0-10.0 sec 1.50 MBytes 12.6 Mbits/sec [ 3] 0.0-10.1 sec 14.1 MBytes 11.7 Mbits/sec The output shows that the RTT from my machine to the iperf server is similar in both cases (about 170ms) however with the patch the throughput averages 129 Mbps while without the patch the throughput only averages 11.7 Mbps. If we calculate the maximum throughput using Bandwidth = Queue/RTT we get (212992 * 8)/0.170 = 10.0231 Mbps. This is just about what iperf is showing us without the patch since the buffer size is set to 212992 I believe that the buffer size is limiting the throughput. With the patch we have no buffer limitation (autotuning) and can develop the full potential bandwidth on the link. If you want to duplicate the STC you will have to find an iperf server (I found an extreme case) that has a large enough RTT distance from you and try a few times. I get varying results depending on Internet traffic but without the patch never exceed the limit caused by the buffering. Here is the patch: --- net.cc_orig 2017-01-09 09:37:54.301210600 -0800 +++ net.cc 2017-01-09 14:15:57.998895500 -0800 @@ -517,7 +517,7 @@ bool fdsock (cygheap_fdmanip& fd, const device *dev, SOCKET soc) { - int size; +// int size; fd = build_fh_dev (*dev); if (!fd.isopen ()) @@ -584,6 +584,7 @@ fd->set_flags (O_RDWR | O_BINARY); debug_printf ("fd %d, name '%s', soc %p", (int) fd, dev->name (), soc); + /* Raise default buffer sizes (instead of WinSock default 8K). 64K appear to have the best size/performance ratio for a default @@ -608,6 +609,8 @@ of 1k, but since 64k breaks WSADuplicateSocket we use 63Kb. (*) Maximum normal TCP window size. Coincidence? */ + + #ifdef __x86_64__ ((fhandler_socket *) fd)->rmem () = 212992; ((fhandler_socket *) fd)->wmem () = 212992; @@ -615,6 +618,14 @@ ((fhandler_socket *) fd)->rmem () = 64512; ((fhandler_socket *) fd)->wmem () = 64512; #endif + +/* Please don't do this. Windows doesn't have a default buffer of 8K it uses autotuning. + The thing about network buffers is they have to be chosen dynamically. Both Windows + and Linux do this. However, this code sets the SO_SNDBUF and SO_RCVBUF size statically. + This will limit Internet speeds and cause bufferbloat. Let the OS dynamically choose + the SO_SNDBUF and SO_RCVBUF sizes. + + if (::setsockopt (soc, SOL_SOCKET, SO_RCVBUF, (char *) &((fhandler_socket *) fd)->rmem (), sizeof (int))) { @@ -632,7 +643,9 @@ (char *) &((fhandler_socket *) fd)->wmem (), (size = sizeof (int), &size))) system_printf ("getsockopt(SO_SNDBUF) failed, %u", WSAGetLastError ()); - } + } */ + + /* A unique ID is necessary to recognize fhandler entries which are duplicated by dup(2) or fork(2). This is used in BSD flock calls
Attachment:
cygcheck.out
Description: Binary data
Attachment:
net.cc.patch
Description: Binary data
Index Nav: | [Date Index] [Subject Index] [Author Index] [Thread Index] | |
---|---|---|
Message Nav: | [Date Prev] [Date Next] | [Thread Prev] [Thread Next] |