This is the mail archive of the cygwin-developers mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: Implementing aio_* and lio_* (async i/o) on Cygwin


Hi Corinna,

On Tue, 12 Dec 2017, Corinna Vinschen wrote:
On Dec 12 01:11, Mark Geisert wrote:
I've got a proof-of-concept implementation working but haven't integrated it
into my local Cygwin source tree yet.  This implementation uses a pool of
pthreads to issue reads and writes asynchronously.  Using threads allows to
operate transparently with the usual Cygwin-supplied file
descriptors/handles that are set up for synchronous-only operation.

If I were to use Win32 overlapped i/o, as Corinna suggested as a
possibility, there would have to be some supporting changes to Cygwin
internals, I think.  Either file descriptors would have to always permit
overlapped usage, or open() and/or fcntl() would have to allow some new flag
to specify overlapped capability when creating a new fd/handle.

If you use threads, you will have to change your code to use cygthreads
instead of pthreads.  Pthreads are a userspace concept, cygthreads are
used for internal tasks.

Understood in general. I had originally implemented a libaio that was built on top of Cygwin, so there was an "app" viewpoint in my thinking. But I'm coming to understand what you're talking about: really doing aio_* and lio_* as Cygwin syscalls using the same capabilities within Cygwin that other syscalls use. So cygthreads it is, only if they're needed.

However, using async IO, the only change necessary would be to change
prw_open to open an async handle, and pread/pwrite to wait for the
result if the current I/O is NOT async.  It may be necessary to add a
parameter or two to the pread/pwrite fhandler methods, but all the rest
of the functionality could be in the callers.

Please stop thinking in Win32 overlapped I/O.  I have no idea why
Microsoft uses this term.  Under the ntdll hood, a handle is either
synchronous (Keeping track of file position, waiting for read/write to
complete) or asynchronous (Not keeping track of file position, returning
STATUS_PENDING rather than waiting for completion, based on using the
FILE_SYNCHRONOUS_IO_{NON}ALERT.

On async handles, you can wait for an event object to be signalled, or
you can ask the NtReadFile/NtWriteFile functions to call a completion
routine.

OK.  I understand.

There is an O_ASYNC flag in some non-POSIX Unixes but what little
documentation

No, if we had to create a new open flag for this functionality it would
be wrong.

need here.  It's used to signal (via SIGIO arrival) that space is available
for a write, or that data is available for a read.  What it would need to do
for aio_* semantics is to signal that a write or a read has completed.

Yes, but with an arbitrary signal.  Alternatively aio can start a
pthread on completion, basically all the stuff sigevent is supposed to
handle.

I'm also unsure one can add overlapped capability to an existing Win32
handle.

That's what prw_open is for.

Ah, the light of enlightenment has lit.

Wouldn't it be easier (less modification of existing code) to just use the
thread model?

Not sure.  You will have to change the threading stuff anyway and what's
working in user space is pretty different from how you do it in Cygwin.
Also, wouldn't it be nice to use the methods already provided by the OS
for just such an opportunity?  Changing the pread/pwrite/prw_open
methods to support aio_read/aio_write should be pretty straightforward.

Yes, I've got the idea now. I'll go work on this direction and report back when I have something, modulo holidays and stuff.

Right now I have a patch to implement sigtimedwait() since it's similar to the already implemented sigwait() and sigwaitinfo(). I also have a patch that extends support for getting and setting cygthread and pthread names via cygwin_external(). I'll submit these two patches shortly.

Thanks for the patient internals explanations, as always.

..mark


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]