Re: (new) pcre package for consideration

[Charles Wilson <cwilson at ece dot gatech dot edu>]

Ronald Landheer-Cieslak wrote:

Looks like a typo...

> > > ran cygcheck against cygpcreposix.dll and got:

^ no -0
v with -0

> > > $ cygcheck cygpcreposix-0.dll 
> > > Found: .\cygpcreposix-0.dll
> > > cygpcreposix-0.dll
> > >  D:\cygwin\bin\cygwin1.dll
> > >    C:\WINNT\System32\KERNEL32.dll
> > >      C:\WINNT\System32\ntdll.dll
> > >  D:\cygwin\bin\cygpcre.dll

^ no -0
ALSO a typo? Or accurate transcription of the command output?

Given your reply below, I guess the first mismatch is a typo, and the 
second instance is NOT a typo.  If so, then you do have a problem.

> What I want to know is that it will find the cygcpre-0.dll when looking 
> for cygpcre.dll, without fixing anything. If that will work, it's fine by 
> me. (but it doesn't work for me and I doubt it'll work for you)

See above.

> The way I read these dependencies, cygpcreposix-0.dll depends on 
> cygpcre.dll and cygwin1.dll (which ...). cygpcre.dll will not exist after 
> installation - cygpcre-0.dll will.

Well, if that's the case, then you have a packaging problem.  You need 
to repackage things as described below, so as to avoid breaking other 
people's binaries that currently depend on cygpcre.dll.  However, the 
procedure below, while necessary, is not sufficient to solve the "my 
pcreposix dll is linking to the wrong pcre dll" problem.  See WAY below, 
for that.

Create 'semi-new' pcre packages contain EXACTLY the same binaries as in 
the current, official pcre (or, rebuilt binaries built using the OLD 
non-libtool method, so that the semi-new binaries "match" the existing 
ones).  It's probaby safest just to take the existing pcre package and 
manually split up the binaries into multiple tarballs.  That way you're 
assured drop-in compatibility.

Put the dlls, cygpcre.dll + cygpcreposix.dll, into their own 'libpcre' 
package.  Put everything else into the existing 'pcre' package.  The 
pcre package should depend on 'libpcre'.  You could also split the 
.la/.a/.dll.a and .h files into libpcre-devel, but that's probably overkill.

Release those new packages.  They should be 'drop-in' replacements for 
the existing pcre stuff.

Let simmer for approx 2 weeks.

Then, take your NEW libtool-enabled version of pcre, and put ITS 
cygpcre-0.dll and cygpcreposix-0.dll into a NEW package, 'libpcre0' 
(matching the '-0' version number on the DLLs).  Again, everything else 
goes into the pcre (NOT pcre0) package.  If you chose, above, to split 
the development stuff into a separate (e.g. third) -devel package, then 
do so again here -- but it should be libpcre-devel NOT libpcre0-devel.

Then, release these new packages.

End result, your users will be able to have installed

libpcre (containing cygpcre.dll + cygpcreposix.dll)
libpcre0 (continaing cygpcre-0.dll + cygpcreposix-0.dll)
libpcre-devel (maybe. Contains devel stuff for libpcre0)
pcre (contains everything else)

Thus, the "unversioned" pcre dlls can be installed alongside the new 
stuff, and carried forward to support existing apps that use them. 
HOWEVER, since the pcre (+ libpcre-devel?) packages contain ONLY stuff 
for the new cygpcre-0 libraries, all NEW builds will properly link 
against the NEW libraries.  Good.

-------------------------------------

Now, about the "Oops I linked it again" problem...

> > So I don't see what "problem" you're trying to fix.
>
> I'm not trying to fix anything, I'm trying to avoid having to: I want to 
> be sure that the mere name of the cygpcre.dll file won't matter. It 
> usually does, and I think it might in this case as cygpcre-0.dll is in the 
> same dir as cygpcreposix-0.dll, and should thus be found when looking for 
> the DLL before the installed cygcpre.dll is (that is, AFAIK, the way 
> Windows resolves this).
>
> Also, if I remove or rename cygpcre.dll (the installed one) 
> cygpcreposix-0.dll has a broken dependency. I want to be sure that doesn't 
> happen if the libraries are installed.

The problem, as you stated 3 emails ago but I missed, is that 
cygpcreposix-0.dll is being linked against cygpcre.dll NOT 
cygpcre-0.dll.  I'm starting to lose track, but I *thought* my patch 
fixed that.  I'll try again...

> > libtool --finish simply fixes up some stuff in libpcre.la and possible
> > relinks; on windows, it appears that it is almost never necessary.
>
> That's great, but it seems that removing my old pcre installation from
> Cygwin and replacing it with the one I made breaks anything that depends
> on pcre - and it will stay broken when recompiled against pcreposix.

Again, because your new cygpcreposix-0.dll is fubared.

> For example: I've uninstalled pcre - on which grep depends - and put 
> pcre-4.2-1 in its place. grep is still broken (complains about not finding 
> a DLL), which would mean that by installing pcre-4.2-1 grep will be broken 
> if not relinked.

Yes, this is the problem addressed by the packaging reorg outlined 
above.  I recently had to do this for gdbm.

2002-Feb-27: gdbm-1.8.0-4 release.  All in one binary package.

In March 2003, I wanted to update to gdbm-1.8.3 -- but because (a) 
libtool based building was now possible in 1.8.3, and (b) the API 
changed between 1.8.0 and 1.8.3, I knew that the DLL name would change. 
 To allow side-by-side coexistence of the old gdbm dlls and the new 
gdbm dlls, I ALSO knew I had to split up the previously monolithic gdbm 
package.

So:

2003-Mar-22: gdbm-1.8.0-5 released, consisted of
  libgdbm-1.8.0-5       : contains cyggdbm.dll
  libgdbm-devel-1.8.0-5 : contained .dll.a and .a and .h files (*)
  gdbm-1.8.0-5          : everything else (*)

(*) requires: libgdbm

Notice that 1.8.0-5 was IDENTICAL to 1.8.0-4 (and was based on the same 
'upstream' 1.8.0 release of gdbm).  The only difference was splitting 
the monolithic package into three parts.

Then, I released my new 1.8.3-1 version as a test release

2003-Mar-28: gdbm-1.8.3-1 released, consisted of
  libgdbm3-1.8.3-1      : contains cyggdbm-3.dll, cyggdbm_compat-3.dll
  libgdbm-1.8.3-1       : contains .dll.a, .a, .la, .h files (*)
  gdbm-1.8.3-1          : everything else (*)

(*) requires: libgdbm3

This 1.8.3-1 version is still TEST; I haven't yet promoted it to 'curr' 
(I'm waiting for feedback on my cvs package, first).  But, you see that 
the 1.8.3 version of -devel will (eventually) replace the 1.8.0 version 
of -devel.  The 1.8.3 version of the 'gdbm' package will replace the 
1.8.0 version of the gdbm package.  But the libgdbm3 package can coexist 
with the libgdbm package.  Thus, everyone will be happy.

The trick, of course, is to leave enough time between the compatible 
split release (1.8.0-5) and the upgrade (1.8.3-1) so that EVERYBODY has 
had time to update to 1.8.0-5 before you release 1.8.3-1.   Because if 
someone were to update directly from OLD 1.8.0-4 to the NEW 1.8.3-1, 
they'd miss the libgdbm package completely and never get it.  And then 
"grep" would break.  Figuratively speaking.

> I don't know which packages use pcreposix, but compiling and linking a 
> very simple program against pcreposix will result in an executable that 
> can't run because cygcpre.dll isn't there.

ASSUMING you fix the "Oops my cygpcreposix-0.dll is linked wrong" 
problem, the procedure described above means that you don't HAVE to know 
who is using your DLLs.  Although it's sometimes nice to do the following:

When splitting your package (e.g. gdbm-1.8.0-4 monolithic to compatible 
1.8.0-5 packages), search the setup.hints in the release heirarchy on 
cygwin mirrors for 'requires" lines that include the monolithic package 
('gdbm' == 'pcre').   Send an email to the cygwin-apps mailing list, 
like this one:

"ATTN: Apache, Perl, Python, and Exim maintainers"
http://www.cygwin.com/ml/cygwin-apps/2003-03/msg00675.html

> If we don't want to make a copy of cygpcre-0.dll called cygcpre-0.dll 
> (which would defeat the purpose of having the version in the name and 
> add a file that should not be needed to the distribution) I don't see how 
> to fix this.

Dead horse: see above.

> So, what I'd like to know is if there are any more things I may have 
> overlooked in the use of libtool, now that Gerrit's and your patches have 
> been applied, or whether this is a "doesn't work for me but works for you" 
> type of situation.

Okay, now down to the brass tacks: why is cygpcreposix-0.dll getting 
linked to cygpcre.dll and not to cygpcre-0.dll.

Okay, after the build (but before 'pcre-4.2-1.sh install), I looked in 
<srcdir>/.build/.libs:

$ cygcheck.exe cygpcreposix-0.dll
Found: .\cygpcreposix-0.dll
cygpcreposix-0.dll
  .\cygpcre-0.dll
    D:\cygwin\bin\cygwin1.dll
      D:\WINNT\System32\KERNEL32.dll
        D:\WINNT\System32\ntdll.dll

So, cygpcreposix-0.dll IS properly linked to cygpcre-0.dll.

Next, I did the 'pcre-4.2-1.sh install' and looked in 
<srcdir>/.inst/usr/bin.  And whaddaya know:

$ cygcheck cygpcreposix-0.dll
Found: .\cygpcreposix-0.dll
cygpcreposix-0.dll
  D:\cygwin\bin\cygwin1.dll
    D:\WINNT\System32\KERNEL32.dll
      D:\WINNT\System32\ntdll.dll
  D:\cygwin\bin\cygpcre.dll

While the relink of the executables didn't break them (e.g. the relinked 
.exe's still want cygpcre-0.dll NOT cygpcre.dll [*]), the relink of the 
posix dll DID break it.  Why?

[*] except in the case of pcretest -- which ITSELF links to 
cygpcre-0.dll, but also to the newly broken cygpcreposix-0.dll which 
wants cygpcre.dll.  So, pcretest now wants -0.dll directly, but .dll 
indirectly == badness.

The problem seems to be that the relink cmd that is generated by libtool 
contains this:
   -L/usr/lib -L/tmp/pcre/pcre-4.2/.inst/usr/lib -lpcre
Which measn that /usr/lib is searched before $inst_prefix_dir/usr/lib.

This is generated internal to libtool; it looks like a libtool bug to 
me.  I'll try to put together a small testcase and submit it to the 
libtool mailing list.

Now, on cygwin, I *think* that relinking is really unnecessary.  So, the 
workaround -- until libtool gets fixed and 1.5.1 is released -- would 
probably be something like the following HACK.

Modify the pcre-4.2-1.sh script so that at the end of build() [ or at 
the beginning of install() ], you explicitly make a "backup copy" of 
.build/.libs/cygpcreposix-0.dll.   (Yes, this is a total hack).  Then, 
at the end of install(), you replace the newly installed (and newly 
broken) cygpcreposix-0.dll with the saved "good" copy.

[Ack ach, cough cough. choke]  I know, I know, it's ugly -- but I really 
think this is a libtool bug, and it might be a while before it gets 
fixed since we'll want to wait for the 1.5.1 official release.

Sorry that I didn't have better news.

--Chuck


P.S. in the buildscript in prep(), after applying the patch, you should
  'chmod +x ${srcdir}/reconf-cygwin.sh'

P.P.S. libtool no longer distributes automake files 'missing' 
'install-sh' or 'mkinstalldirs'.  in the the reconf-cygwin.sh script, 
you should copy them from /usr/autotool/devel/share/automake-1.7/