This is the mail archive of the cygwin mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: rsync very slow, but not a network issue


Brian Dessent schrieb:
Reini Urban wrote:
When the file is alredy there, rsync reports a speedup of about 70.
(When the file is not already there, the speed up is 1, of course.) I am
running rsync over ssh with pre-generated keys installed in my .ssh
directories.

This is a binary MP3. rsync (as diff) is not good in checking binary diffs. Please try it with a typical text file, where the patch is smaller than the source.

That is not correct at all, and rsync would be much less useful if it
were true. rsync works with arbitrary binary data just as well as it
does with text. It has no conception of 'lines' or any other type of
delimited text. All it does is split up the file into fixed-length
chunks and calculates a rolling checksum of each of these. It does not
use anything resembling 'diff' or 'patch'.


http://rsync.samba.org/tech_report/node2.html

Thanks for clarification. Makes sense. So as punishment I made another bench.

What is the maximum overhead?
It cannot be the reported difference (factor 500).

http://rsync.samba.org/tech_report/node6.html has about a similar filesize (2.1MB binary).
In this example with a hit:miss ration of 64247:948 the overhead is 5312200:5629158+1632284 (if read and write is synchronous) compared to rcp.
So about max 2.5 overhead, and not 500.


So check if the net overhead in the cygwin version is broken?
Could be easily tested out with two local files:

head -c 2100000 < /dev/random >test1.mp3
head -c 2100000 < /dev/random >test2.mp3
cp test1.mp3 test1-same.mp3
cp test2.mp3 test2.bak
time rsync test1.mp3 test2.mp3

  real    0m0.518s
  user    0m0.201s
  sys     0m0.171s

time rsync test1.mp3 test1-same.mp3

  real    0m0.524s
  user    0m0.154s
  sys     0m0.263s

RSYNC_RSH=ssh
TEST=othermachine
cp test2.bak test2.mp3
time scp test2.mp3 $TEST:

  63.2KB/s
real    0m36.619s
user    0m0.170s
sys     0m0.233s

# 100% different
$ time rsync test1.mp3 $TEST:test2.mp3
Server is very old version of rsync, upgrade recommended.

real    0m37.162s
user    0m0.388s
sys     0m0.202s

# 100% same
$ time rsync test2.mp3 $TEST:test2.bak
Server is very old version of rsync, upgrade recommended.

real    0m7.298s
user    0m1.201s
sys     0m2.217s

6x faster than scp, with same data.
0.5s (1.4%) slower than scp, with complete random data.

$ rsync --version
rsync  version 2.6.2  protocol version 28
$ ssh $TEST rsync --version
rsync version 2.4.6  protocol version 24
$ ssh -V
OpenSSH_3.8.1p1, OpenSSL 0.9.7d 17 Mar 2004
$ ssh $TEST ssh -V
OpenSSH_3.7.1p2, SSH protocols 1.5/2.0, OpenSSL 0.9.6c 21 dec 2001
$ uname -r
1.5.10(0.116/4/2)
--
Reini Urban
http://xarch.tu-graz.ac.at/home/rurban/


-- Unsubscribe info: http://cygwin.com/ml/#unsubscribe-simple Problem reports: http://cygwin.com/problems.html Documentation: http://cygwin.com/docs.html FAQ: http://cygwin.com/faq/


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]