This is the mail archive of the cygwin-apps mailing list for the Cygwin project.


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]
Other format: [Raw text]

Re: [ITP] util-linux


Eric Blake wrote:

According to Yaakov S (Cygwin Ports) on 3/2/2006 3:06 PM:
No.  With Corinna's patch to add CYGWIN=transparent_exe option, we are
asking for problems if we distribute a script alongside an exe.
OK, but I can think of one *major* "violation" of this: libtool-built
applications, when linking against a to-be-installed library in the same
package.  In such a case, the real .exe is placed in .libs, and both an
extension-less libtool script  and a launcher .exe are placed in the
builddir; this allows one to run the application w/o installing it.

Yes, I'm aware of that, and I would love to spend some free time trying to solve another way for libtool to work without relying on the (subtle) difference between 'foo' and 'foo.exe' when testing uninstalled libtoolized applications. This libtool (ab)use of filenames has already had difficulties with managed mounts and trailing dots. Then there was the time when I tried to patch rm in coreutils-5.3.0 to handle .exe transparently (and immediately had to retract that change because of libtool).

For those following along at home, the issue behind libtool's "abuse" is as follows. (Sidenote: for eons foo.exe != foo and libtool relied on that. Later, changes in cygwin forced libtool to rely instead on foo.exe != foo. (with trailing dot). More "improvements" to cygwin's exe handling [but at least they are optional, thank goodness] now break even THAT and it's LIBTOOL's fault for "abusing" longstanding behavior? I don't think so.)


On all platforms, libtool creates wrapper scripts for two purposes: first, to set environment variables (PATH, LD_PRELOAD, etc) before launching the "real" uninstalled executable, so that the uninstalled executable can find its (uninstalled) dependencies. For instance,

mypackage/lib/ contains code for mylib shared library
mypackage/app/ contains code for myapp executable, which
               depends on mylib

After compile,
the mylib.so shared library is in mypackage/lib/.lib/ and
the myapp executable binary is in mypackage/src/.lib/
the myapp wrapper script is in mypackage/src/
and it will set LD_LIBRARY_PATH (or whatever variable is appropriate) to mypackage/lib/.lib, and then lauch mypackage/src/.lib/myapp. All is well.


Except on cygwin, but we'll get to that later.

The OTHER reason for the wrapper script is it serves to hold persistent state for the target executable -- libtool itself *parses* the script to read some of those variable values in "the next" libtool invocation after compilation (e.g. libtool --mode=install).

So, what's the problem on cygwin? Most makefiles -- especially those already tailored for cygwin or generated by autoconf'ing a Makefile.in, or automake'ing a Makefile.am->Makefile.in->Makefile, have their executable target names set as

myapp$(EXE_EXT): dependencies...
	rule....

When EXE_EXT= <empty>, as in most platforms, there is no problem. The WRAPPER script is named "myapp" and the target rule is "myapp" so the wrapper satisfies the rule. On cygwin, it does not: so make never believes that the rule has been satisfied, and always tries to relink the app. Over and over.

There are two solutions: (1) name the wrapper script myapp.exe instead of myapp. (2) create a wrapper *binary* named myapp.exe and lives in myproject/app/ NOT myproject/app/.lib, and which either [a] sets all the important environment variables and then exec's the "real" executable, or [b] exec's the wrapper script, which sets the environment and launches the "real" executable.

(1) is simple, clean, beautiful, and wrong. While current cygwin (at least, under the current bash shell; I didn't try others) will in fact treat the following:

----- foo.exe ----
#!/bin/sh
echo This is 'foo.exe' script
------------------

as an actual script, and use the sh-bang line to exec the correct interpreter, I do not think older cygwin's/shells did so. I'm pretty sure msys/mingw will barf. And I know it'll confuse users -- and if having the following two files in a directory:
foo.exe
foo
is considered "abuse" then it's absolutely true that having
foo.exe
which is NOT a binary executable is "abuse". (Also, I don't know how cygwin will treat a script named "foo.exe" on a remote share, with/without nosmbntsec, or local/remote with/without cygexec/exec mount options, ... This is just ASKING for even more trouble than we've got now.)


(2)[a] is problematic, because 'libtool --mode=install' can't source it, plus it duplicates knowledge; the same settings are stored in two places: the wrapper script (or its renamed replacement 'target-data-file') and in the binary wrapper.

More on this later.

(2)[b] So this is what we did. And it worked. Until other people broke it by changing cygwin. So we fixed it. And now we've got a new cygwin option (which I had NEVER heard about until now!) that breaks it again -- and the maintainer of coreutils is pushing for more widespread use.

E.g. pushing to deliberately break libtool

Eric, please don't *push* CYGWIN=transparent_exe until *somebody* implements the following libtool fix. Sadly, it's too late for any such major change to get into libtool-2.0 because libtool is effectively in code-slush right now. But, if *somebody* implements this fix I'll put it in cygwin's dist of libtool-2.0 and push for it to go into libtool-2.0.x.

The fix:
---------

Suppose that the following changes were made:

(1) the binary wrapper takes a command line argument that causes it to emit the current contents of the wrapper script on stdout

(2) the binary wrapper sets the environment variables itself, and then exec's the real binary executable directly instead of the wrapper script. This should be relatively easy: just make a copy of **environ, modify the entry in that copy with startswith PATH= (adding both the .lib dir which contains the target exe, and the .lib dirs which contain the DLLs on which it depends), and then use
execve(name-of-target-exe, ...)
instead of
execv(name-of-wrapper, ...).
Watch out for memory allocation issues wrt to the old environ[x] entry for PATH and the new one, tho.


(3) libtool (on cygwin|mingw) would then no longer generate the wrapper script. However, when it needs to source the contents of that script, it instead calls
'wrapper.exe --emit-script > .lib/wrapper-env-settings
sources THAT, and then rm -f's it. Note that (a) the env-settings script is both named something other than the actual name of the makefile target-without-.exe, AND lives in the .lib subdir.


-------

This would make everybody happy, wouldn't "duplicate" the storage of information, would keep the current good behavior of NOT relinking apps over-and-over, and could EVEN be extended across all platforms once it's working on cygwin/mingw. But it's a lot of work; fortunately each of the three items can be implemented and tested separately.

But, until these changes happen (and I'm not likely to have time to do them myself, although PTC and tested), my answer to complaints is going to be a pointer to this post, and "don't use transparent_exe".

--
Chuck


Index Nav: [Date Index] [Subject Index] [Author Index] [Thread Index]
Message Nav: [Date Prev] [Date Next] [Thread Prev] [Thread Next]