Re: slow handling of large sets of files?

["Dante R. Chialvo" <d-chialvo at northwestern dot edu>]

I can't reproduce
(on a Sony Vaio running Windows XP Profesional, 2.0 GHz)

dchialvo@felipe ~
$ time gunzip -c DATA.tar.gz | tar xBpf -

real    0m18.556s
user    0m0.360s
sys     0m1.271s

dchialvo@felipe ~
$ time rm -rf DATA

real    0m2.476s
user    0m0.210s
sys     0m1.081s

Cheers
Dante


Lester Ingber wrote:
> 
> I don't understand why handling large data sets is so slow on my Cygwin?
> I have had this problem as far back as I can remember with Cygwin.
> Running executables prepared under gcc (without large file I/O) seem fine.
> 
> Below I include just the top part of `cygcheck -s`.
> 
> To be specific, I have tarred-gzipped directory of about 5300 files,
> % ls -l DATA.tar.gz
> [...] 92303 Jan 10 05:27 DATA.tar.gz
> which can be retrieved using a browser or just `wget` from
> http://www.alumni.caltech.edu/~ingber/DATA.tar.gz
> 
> I tested the following commands
> (1) gzcat DATA.tar.gz | tar xBpf -
> (2) rm -rf DATA
> on 4 machines/OS's:
> (a) ThinkPad (1.8 MHz)/Cygwin (see below) on mount c:/cygwin//home/ingber
> (b) Pentium 3 (700 MHz)/FreeBSD 4.10 (lots of users)
> (c) SPARC (300 MHz)/Solaris 8 (lots of users)
> (d) SPARC (500 MHz)/Solaris 9
> 
> The differences between (a) and (b)-(d) are so large that I think my
> estimates get the point across:
> Under Cygwin, it takes about 380 secs to perform (1), and about 480 secs
> to perform (2).
> On the other machines, it takes about 20 secs to perform (1).
> To perform (2), it takes about 3 secs on (b) and (d), and about 10 secs on (d).
> 
> Thanks.
> 
> Lester
> 
>     ------------8<------------ top cut -> bottom ------------->8------------
> Cygwin Configuration Diagnostics
> Current System Time: Mon Jan 10 05:44:49 2005
> 
> Windows XP Professional Ver 5.1 Build 2600 Service Pack 2
> 
> Path:   C:\cygwin\usr\local\bin
>         C:\cygwin\bin
>         C:\cygwin\bin
>         C:\cygwin\lib\subversion\bin
>         C:\cygwin\usr\X11R6\bin
>         C:\cygwin\usr\local\bin
>         C:\cygwin\bin
>         C:\cygwin\bin
>         .\
>         c:\WINDOWS\system32
>         c:\WINDOWS
>         .
> 
> Output from C:\cygwin\bin\id.exe (nontsec)
> UID: 1005(ingber) GID: 513(None)
> 513(None)
> 
> Output from C:\cygwin\bin\id.exe (ntsec)
> UID: 1005(ingber)   GID: 513(None)
> 0(root)             513(None)           544(Administrators) 545(Users)
> 
> SysDir: C:\WINDOWS\system32
> WinDir: C:\WINDOWS
> 
> CYGWIN = `check_case:strict'
> HOME = `C:\cygwin\home\ingber'
> MAKE_MODE = `unix'
> PWD = `/home/ingber'
> USER = `ingber'
> 
> Use `-r' to scan registry
> 
> a:  fd             N/A    N/A
> c:  hd  NTFS     55902Mb  51% CP CS UN PA FC     LESTER_C
> d:  cd             N/A    N/A
> i:  net NTFS     55902Mb  51% CP CS UN PA FC     LESTER_C
> 
> C:\cygwin                          /                         system  binmode
> c:\cygwin\managed                  /managed                  system  binmode
> C:\cygwin/bin                      /usr/bin                  system  binmode
> C:\cygwin/lib                      /usr/lib                  system  binmode
> C:\cygwin\usr\X11R6\lib\X11\fonts  /usr/X11R6/lib/X11/fonts  system  binmode
> .                                  /cygdrive                 system  binmode,cygdrive
> 
> Found: C:\cygwin\bin\awk.exe
> Found: C:\cygwin\bin\bash.exe
> Found: C:\cygwin\bin\cat.exe
> Found: C:\cygwin\bin\cp.exe
> Found: C:\cygwin\bin\cpp.exe
> Found: C:\cygwin\bin\find.exe
> Found: C:\cygwin\bin\gcc.exe
> Found: C:\cygwin\bin\gdb.exe
> Found: C:\cygwin\bin\grep.exe
> Found: C:\cygwin\bin\ld.exe
> Found: C:\cygwin\bin\ls.exe
> Found: C:\cygwin\bin\make.exe
> Found: C:\cygwin\bin\mv.exe
> Found: C:\cygwin\bin\rm.exe
> Found: C:\cygwin\bin\sed.exe
> Found: C:\cygwin\bin\sh.exe
> Found: C:\cygwin\bin\tar.exe
>     ------------8<------------ bottom cut <- top ------------->8------------
> 
> --
> Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
> Problem reports:       http://cygwin.com/problems.html
> Documentation:         http://cygwin.com/docs.html
> FAQ:                   http://cygwin.com/faq/

-- 
========================== 
Prof. Dante R. Chialvo 
http://www.chialvo.net
-------------------------- 
Cell: 310-3824810(in USA)
+34 679949564 (in Europe)
==========================

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/