This is the mail archive of the cygwin-developers mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Broken autoconf mmap test


On Thu, Mar 24, 2011 at 10:59:45AM +0100, Corinna Vinschen wrote:
>On Mar 23 19:15, Corinna Vinschen wrote:
>> On Mar 23 11:51, Eric Blake wrote:
>> > I'm still debating if cygwin is compliant and the autoconf test is
>> > exploiting undefined behavior... this is tricky stuff to get right.
>> 
>> Cygwin tries to be as compliant as possible, and it tries to implement
>> expectations which are typical for some Unix systems even if they are
>> not covered by the standards.
>> 
>> And yes, the autoconf test tests defined behaviour, see the SUSv4 mmap
>> man page:
>> [etc]
>
>For the records and, hopefully, even for discussion if somebody feels
>a bit of pity with me.
>
>I've been mulling over this problem on and off for a long time.  I just
>don't see a solution for 64 bit systems by just using the OS calls as
>they are defined.  AT_ROUND_TO_PAGE is not supported, AT_EXTENDABLE_FILE
>changes the file size.
>
>So I was thinking of an entirely different way to implement mmap's on
>real files: Instead of really mapping them, mmap just generates an
>anonymous map.
>
>In case of MAP_PRIVATE, anonymous maps are implemented using
>VirtualAlloc.  The size of the allocated space is always a multiple of
>64K.  After creating it, the file content is simply copied over.
>Full stop.  Easy.
>
>In case of MAP_SHARED, anonymous maps are created using an actual
>file mapping (NtCreateSection/NtMapViewOfSection).  For files, mmap
>would still simply copy over the file content.
>
>The problem starts if the map has been opened using PROT_WRITE.  In that
>case we have to write the changed pages back to the file.  Fortunately
>the only times you can rely on the changes being written back to the
>file are msync, munmap, and process exit.
>
>But that's the tricky part.  How do we know a page has been written
>to?
>
>Easy solution:  Always write all of the mapping to the file.  Would that
>be possible?  This sounds wrong to me.  IIUC, msync should only write
>back the changes, not the entire content of the mapping.  It would
>potentially overwrite changes made in the file by other means.
>
>Starting with Windows 2000 there's a method to keep track of pages which
>have been written to:  Call VirtualAlloc with the MEM_WRITE_WATCH flag
>set, and then use GetWriteWatch to find the changed pages.
>Unfortunately the MEM_WRITE_WATCH flag works only for VirtualAlloc'ated
>memory, not for section maps created with MapViewOfFile, so it's not
>usable in this scenario.  We can't use VirtualAlloc for the MAP_SHARED
>case, otherwise the changes in memory are not visible to other processes
>sharing the map.
>
>What's left? 
>
>The hard way, apparently.  Always set the page protection to
>PAGE_READONLY.  If the process writes to it, an exception occurs.  Catch
>the exception, set the page protection to PAGE_READWRITE and return from
>the exception so the caller can write again.  When the changes are to be
>written back to the file, only write back the pages with PAGE_READWRITE
>protection.  If this is called from msync, reset the page protection to
>PAGE_READWRITE.
>
>Apart from being complicated and error prone, there are a couple of
>problems I can see:
>
>- The protection set by mmap and mprotect doesn't match the actual
>  protection.  So we need to keep track of the desired protection
>  per page.  This takes one byte per page on the cygheap.  So far we
>  only needed 1 bit per page.
>
>- It's darn tricky to make this thread-safe.
>
>- In all three possible solutions above: What if the original file
>  handle used in the mmap call has been closed and the file permissions
>  have been changed in the meantime so that the process does not have
>  write permissions anymore?

- Isn't it going to be really slow?  I guess if it only happens once
per page it won't be that bad but still: ouch.

cgf


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]