This is the mail archive of the cygwin-developers mailing list for the Cygwin project.
Index Nav: | [Date Index] [Subject Index] [Author Index] [Thread Index] | |
---|---|---|
Message Nav: | [Date Prev] [Date Next] | [Thread Prev] [Thread Next] |
Other format: | [Raw text] |
On 12/12/2012 12:09 PM, Ryan Johnson wrote: > On 12/12/2012 2:21 PM, Christopher Faylor wrote: >> On Wed, Dec 12, 2012 at 12:11:46PM -0500, Ryan Johnson wrote: >>> On 12/12/2012 12:03 PM, Christopher Faylor wrote: >>>> On Tue, Dec 11, 2012 at 07:13:04PM -0800, Daniel Colascione wrote: >>>>> On 12/11/2012 5:06 PM, Daniel Colascione wrote: >>>>>> On 12/10/2012 7:51 PM, Daniel Colascione wrote: >>>>>>> The key to generating a binary that repros the problem is to unexec >>>>>>> emacs, then >>>>>>> try to repro with that generated binary, not a copy of it. >>>>>> The real explanation is a lot simpler: the binary is sparse. When you >>>>>> create a >>>>>> file mapping object for a sparse file, Windows discards all cached pages for >>>>>> that file. It makes sense that compilers (and Emacs unexec) would create >>>>>> sparse >>>>>> files as they seek around inside their outputs. >>>>> Anyway, the binary is sparse because our linker produces sparse files. >>>>> >>>>> Would the Cygwin developers accept this patch? With it, applications would >>>>> need >>>>> to explicitly use ftruncate to make files sparse. Considering the horrible and >>>>> unexpected performance implications of sparse files, I don't think generating >>>>> them automatically from a sequence of seeks and writes is the right thing >>>>> to do. >>>> I don't know if this was already done (don't see it in a quick glance at >>>> the archives) but, if this is just a simple case of executable files >>>> being sparse, it seems like an obvious optimization would be to just to >>>> do a, e.g., >>>> >>>> cp --sparse=never -p foo.exe foo.exe.tmp >>>> mv foo.exe.tmp foo.exe >>>> >>>> Wouldn't that remove the sparseness and wouldn't you see astounding >>>> performance improvments as a result? >>> Nope. You'd have to rm foo.exe first. >> "mv" does that automatically, doesn't it? > Oops. Thinko. I had "cp" in my head. > >>> Doing so fixes the problem nicely, though, as you suggest. >>>> I don't think we should be considering ripping code out of Cygwin >>>> without some actual data to back up claims. Testing something like the >>>> above should make it easier to justify. >>>> >>>> I'm actually rather surprised that setup.exe's tar code would maintain an >>>> executable's sparseness. >>> Setup is fine. It's home-brew stuff that suffers, unless/until invoking >>> `make install' copies the sparse file to its final destination, losing >>> the sparse property along the way. >>> >>> Personally, I'm still in shock that the loader barfs so badly over >>> sparse files... normal reads via mmap and fread use the fs cache just fine. >> If we're talking about the loader then why are the examples I asked for >> using "cp"? That's not really apples-to-apples. > I think I'm miscommunicating something here... let me try again. > > Problem: > > Attempts to execute a file with the NTFS "sparse" attribute force the Windows > loader to bypass the fs cache and fetch the file from disk. Almost. What actually happens is that the sparse attribute forces the system to flush the cache _for the whole file_ when av::fixup mmaps the binary, forcing the subsequent execution of the image we mmapped to reload everything from disk. Cygexec will work around the problem by avoiding the av::fixup mmap, but we shouldn't expect users to mount everything, including various build directories, cygexec. (I also haven't checked whether anything in /bin actually ends up sparse. It'd be interesting to see.) We could also work around the problem by making av::fixup use regular buffered reads instead of mmap. Regular buffered reads don't seem to trigger the cache flush. Still, I think that creating sparse files only on ftruncate is the right thing to do, at least for existing OS versions. Basically, almost nobody _uses_ sparse files in Windows (except us), so the paths that deal with them are optimized for simplicity and correctness, not performance.
Attachment:
signature.asc
Description: OpenPGP digital signature
Index Nav: | [Date Index] [Subject Index] [Author Index] [Thread Index] | |
---|---|---|
Message Nav: | [Date Prev] [Date Next] | [Thread Prev] [Thread Next] |