Re: inetutils 1.5 / ftpd problem: 426 Data connection: No buffer space available.

[Corinna Vinschen <corinna-cygwin at cygwin dot com>]

On May  7 20:28, antony baxter wrote:
> Corinna,
> 
> > IIUC, the testcase should exhibit the problem OOTB.  HAVE_MMAP is
> > defined and LARGE_TRANSFER_BLOCKSIZE is set to 32K.  I did what you
> > wrote above, I built server and client, added the example port to
> > /etc/services, created the ReallyBigFile from /dev/urandom as above...
> >
> > However, I can't reproduce any ill effect.  This testcase mmap's the
> > file exactly once and then calls as many 32K write's as necessary to
> > write the whole file.  I don't see any waste of memory at all.
> >
> > When examining the memory usage with Task Manager or, better, with
> > sysinternal's Process Explorer, you'll see how the memory usage goes up
> > over time.  But that's no problem.  What you see is the mapping of the
> > file into the physical memory of the machine.  With each write, the
> > process accesses another 32K bytes of the file mapping, so the OS has to
> > map another 32K of the file into the process memory.  Actually this is
> > done in 64K chunks, but that doesn't matter here.  What you see is quite
> > normal behaviour and has nothing to do with Cygwin's mmap implementation,
> > AFAICS.
> 
> Just to be clear, when the test server is serving up a 250MB file,
> you'd expect it to ultimately be using 250MB of memory according to
> Task Manager? On my server machine, this meant that by the time the

Yes, I expect that.  The reason is that ftpd (actually Chuck's example
code in my case) creates a memory map of the size of the whole file.  If
you access every single bit of the file, as ftpd naturally does, the
file has ultimately be mapped in memory in full size.  If the file is
bigger than the memory left in the virtual memory space of the process,
the mmap will fail.  As long as Windows can fit it into RAM, it will do
so.  If it can't, it will use the paging file.

A better approach using mmap would probably mmap only chunks of, say, 16
Megs into memory, unmap them after writing and map the next 16 Megs of
the file.  This way you can map any arbitrary sized file and you wouldn't
have paging tradeoffs.


Corinna

-- 
Corinna Vinschen                  Please, send mails regarding Cygwin to
Cygwin Project Co-Leader          cygwin AT cygwin DOT com
Red Hat

--
Unsubscribe info:      http://cygwin.com/ml/#unsubscribe-simple
Problem reports:       http://cygwin.com/problems.html
Documentation:         http://cygwin.com/docs.html
FAQ:                   http://cygwin.com/faq/