Re: About the dll search algorithm of dlopen

[Michael Haubenwallner <michael dot haubenwallner at ssi-schaefer dot com>]

(and now with patches attached)

On 08/19/2016 06:37 PM, Michael Haubenwallner wrote:
> On 06/01/2016 10:17 PM, Corinna Vinschen wrote:
>> On Jun  1 16:26, Michael Haubenwallner wrote:
>>> On 06/01/2016 01:09 PM, Corinna Vinschen wrote:
>>>> On Jun  1 08:40, Michael Haubenwallner wrote:
>>>>> Hi,
>>>>>
>>>>> two issues with dlopen here (I'm about to prepare patches):
>>>>>
>>>>> *) The algorithm to combine dll file name variants with the search path
>>>>>    entries needs to be reordered, as in:
>>>>>    - for each dll file name variant:
>>>>>    -   for each search path:
>>>>>    + for each search path entry:
>>>>>    +   for each dll file name variant:
>>>>>          check if useable
> 
>>>> However, if you can speed up the search process ignore the
>>>> question...
> 
> Not sure if there's a speedup actually, ...
> 
>>>
>>> This also depends on whether find_exec really is necessary here.
>>
>> Not as such necessary, it's just the function used to search in a
>> search path.  If you want to change that you have to rewrite the
>> same logic again, just reversed.
>>
>> One way around YA code duplication could be some kind of path iterator
>> class which could be used from find_exec as well as from
>> get_full_path_of_dll.
> 
> ... but:
> 0001.patch is a draft for some new cygwin::pathfinder class, with
> 0002.patch adding the executable's directory as searchpath, and
> 0003.patch to search the PATH environment as well.
> 
> Thoughts?
> Any idea about different template nesting to be useful somewhere else?
> 
>>>>> *) The directory of the current main executable should be searched
>>>>>    after LD_LIBRARY_PATH and before /usr/bin:/usr/lib.
>>>>>    And PATH should be searched before /usr/bin:/usr/lib as well.
>>>>
>>>> Checking the executable path and $PATH are Windows concepts.  dlopen
>>>> doesn't do that on POSIX systems and we're not doing that either.
>>>
>>> Agreed, but POSIX also does have the concept of embedded RUNPATH,
>>> which is completely missing in Cygwin as far as I can see.
>>
>> RPATH and RUNPATH are ELF dynamic loader features, not supported by
>> PE/COFF.
> 
> In any case, to me it does feel quite important to have the (almost) same
> dll search algorithm with dlopen() as with CreateProcess().
> 
>>> However, when dlopen is about to search some path list, I can imagine to
>>> search the list of already loaded dlls first as well, but I'd prefer to
>>> leave this up to LoadLibrary...
>>
>> This problem would only occur if dlopen is not called with a path.  If
>> the given pathname is a plain filename, we could simply add a call to
>> GetModuleHandle and if it succeeds, return the handle (after checking
>> for RTLD_NODELETE).
> 
> But indeed this might lower the need for same search algorithms...
> 
>>> Side note:
>>> We also use some cl.exe/link.exe wrapper that supports LD_PRELOAD,
>>> LD_LIBRARY_PATH, embedded RUNPATH, as well as lazy loading for both
>>> LoadLibrary and CreateProcess: https://github.com/haubi/parity
>>> Basically I'm wondering why Cygwin doesn't provide that (yet?)...
>>
>> We discussed implementing our own dynamic loader once, but gave up due
>> to workload.  Parity is LGPLed and thus can't be included into Cygwin
>> itself.
> 
> 've mentioned Parity more as proof-of-concept than for inclusion,
> and accepting workload as reasoning.
> 
> Let's see if I can live without the ELF dynloader features.
> 
> Thanks!
> /haubi/
> 

>From fd7ed0d5cf2f255792e368bba3bc91c834d44896 Mon Sep 17 00:00:00 2001
From: Michael Haubenwallner <michael.haubenwallner@ssi-schaefer.com>
Date: Tue, 31 May 2016 13:09:11 +0000
Subject: [PATCH 1/3] dlopen: search each name within one single search dir

Search x/bin:x/lib if searching in x/lib.

Introduces and uses new pathfinder template, which
introduces and uses new samlist<vstring> templates.
---
 winsup/cygwin/cygheap_malloc.h |  15 ++
 winsup/cygwin/dlfcn.cc         |  71 +++---
 winsup/cygwin/pathfinder.h     | 116 ++++++++++
 winsup/cygwin/samlist.h        | 253 +++++++++++++++++++++
 winsup/cygwin/vstring.h        | 503 +++++++++++++++++++++++++++++++++++++++++
 5 files changed, 918 insertions(+), 40 deletions(-)
 create mode 100644 winsup/cygwin/pathfinder.h
 create mode 100644 winsup/cygwin/samlist.h
 create mode 100644 winsup/cygwin/vstring.h

diff --git a/winsup/cygwin/cygheap_malloc.h b/winsup/cygwin/cygheap_malloc.h
index 74f0bb6..dfed929 100644
--- a/winsup/cygwin/cygheap_malloc.h
+++ b/winsup/cygwin/cygheap_malloc.h
@@ -52,4 +52,19 @@ char *__reg1 cstrdup1 (const char *);
 void __reg2 cfree_and_set (char *&, char * = NULL);
 }
 
+#ifdef __cplusplus
+
+namespace cygwin {
+
+  template<cygheap_types HeapType>
+    struct allocator
+    {
+      static void * alloc (size_t s) { return cmalloc_abort (HeapType, s); }
+      static void free (void *p) { cfree (p); }
+    };
+
+}
+
+#endif
+
 #endif /*_CYGHEAP_MALLOC_H*/
diff --git a/winsup/cygwin/dlfcn.cc b/winsup/cygwin/dlfcn.cc
index 255a6d5..e2093ab 100644
--- a/winsup/cygwin/dlfcn.cc
+++ b/winsup/cygwin/dlfcn.cc
@@ -20,6 +20,7 @@ details. */
 #include "cygtls.h"
 #include "tls_pbuf.h"
 #include "ntdll.h"
+#include "pathfinder.h"
 
 static void
 set_dl_error (const char *str)
@@ -28,29 +29,6 @@ set_dl_error (const char *str)
   _my_tls.locals.dl_error = 1;
 }
 
-/* Look for an executable file given the name and the environment
-   variable to use for searching (eg., PATH); returns the full
-   pathname (static buffer) if found or NULL if not. */
-inline const char *
-check_path_access (const char *mywinenv, const char *name, path_conv& buf)
-{
-  return find_exec (name, buf, mywinenv, FE_NNF | FE_DLL);
-}
-
-/* Search LD_LIBRARY_PATH for dll, if it exists.  Search /usr/bin and /usr/lib
-   by default.  Return valid full path in path_conv real_filename. */
-static inline bool
-gfpod_helper (const char *name, path_conv &real_filename)
-{
-  if (strchr (name, '/'))
-    real_filename.check (name, PC_SYM_FOLLOW | PC_NULLEMPTY);
-  else if (!check_path_access ("LD_LIBRARY_PATH", name, real_filename))
-    check_path_access ("/usr/bin:/usr/lib", name, real_filename);
-  if (!real_filename.exists ())
-    real_filename.error = ENOENT;
-  return !real_filename.error;
-}
-
 static bool
 get_full_path_of_dll (const char* str, path_conv &real_filename)
 {
@@ -63,38 +41,50 @@ get_full_path_of_dll (const char* str, path_conv &real_filename)
       return false;		/* Yes.  Let caller deal with it. */
     }
 
-  tmp_pathbuf tp;
-  char *name = tp.c_get ();
+  cygwin::samlist<cygwin::vstring> basenames;
 
-  strcpy (name, str);	/* Put it somewhere where we can manipulate it. */
+  const char *basename = strrchr (str, '/');
+  basename = basename ? basename + 1 : str;
 
-  char *basename = strrchr (name, '/');
-  basename = basename ? basename + 1 : name;
-  char *suffix = strrchr (name, '.');
-  if (suffix && suffix < basename)
-    suffix = NULL;
+  int baselen = str + len - basename;
+  const char *suffix = strrchr (basename, '.');
 
+  char const * ext = "";
+  int extlen = 0;
   /* Is suffix ".so"? */
   if (suffix && !strcmp (suffix, ".so"))
     {
       /* Does the file exist? */
-      if (gfpod_helper (name, real_filename))
-	return true;
+      basenames.append (cygwin::vstring::constructor_args (basename, baselen, NULL));
       /* No, replace ".so" with ".dll". */
-      strcpy (suffix, ".dll");
+      baselen -= 3;
+      ext = ".dll";
+      extlen = 4;
     }
   /* Does the filename start with "lib"? */
   if (!strncmp (basename, "lib", 3))
     {
       /* Yes, replace "lib" with "cyg". */
-      strncpy (basename, "cyg", 3);
-      /* Does the file exist? */
-      if (gfpod_helper (name, real_filename))
-	return true;
+      basenames.append (cygwin::vstring::constructor_args ("cyg", 3, basename+3, baselen-3, ext, extlen, NULL));
       /* No, revert back to "lib". */
-      strncpy (basename, "lib", 3);
     }
-  if (gfpod_helper (name, real_filename))
+  basenames.append (cygwin::vstring::constructor_args (basename, baselen, ext, extlen, NULL));
+
+  cygwin::pathfinder finder (basenames);
+
+  if (basename > str)
+    finder.add_searchdir (str, basename - 1 - str);
+  else
+    {
+      /* NOTE: The Windows loader (for linked dlls) does
+	 not use the LD_LIBRARY_PATH environment variable. */
+      finder.add_envsearchpath ("LD_LIBRARY_PATH");
+
+      /* Finally we better have some fallback. */
+      finder.add_searchdir ("/usr/lib", -1);
+    }
+
+  if (finder.check_path_access (real_filename))
     return true;
 
   /* If nothing worked, create a relative path from the original incoming
@@ -113,6 +103,7 @@ dlopen (const char *name, int flags)
 {
   void *ret = NULL;
 
+  debug_printf ("flags %d for %s", flags, name);
   if (name == NULL)
     {
       ret = (void *) GetModuleHandle (NULL); /* handle for the current module */
diff --git a/winsup/cygwin/pathfinder.h b/winsup/cygwin/pathfinder.h
new file mode 100644
index 0000000..6845903
--- /dev/null
+++ b/winsup/cygwin/pathfinder.h
@@ -0,0 +1,116 @@
+/* pathfinder.h: find one of multiple file names in path
+
+   Copyright 2016 Red Hat, Inc.
+
+This file is part of Cygwin.
+
+This software is a copyrighted work licensed under the terms of the
+Cygwin license.  Please consult the file "CYGWIN_LICENSE" for
+details. */
+
+#include "samlist.h"
+#include "vstring.h"
+
+#ifdef __cplusplus
+
+namespace cygwin {
+
+class pathfinder
+{
+public:
+  typedef cygwin::samlist<cygwin::vstring> basenamelist;
+
+private:
+  pathfinder ();
+  pathfinder (pathfinder const &);
+  pathfinder & operator = (pathfinder const &);
+
+  basenamelist basenames_;
+  size_t basenames_maxlen_;
+
+  typedef cygwin::samlist<cygwin::vbuffer>  searchbufferlist;
+  searchbufferlist searchbuffers_;
+
+public:
+  ~pathfinder () {}
+
+  /* We need the basenames to search for first, to allow for optimized
+     memory allocation of each searchpath + basename combination. */
+  pathfinder (cygwin::samlist<cygwin::vstring> & basenames)
+    : basenames_ ()
+    , basenames_maxlen_ ()
+    , searchbuffers_()
+  {
+    basenames.swap(basenames_);
+
+    for (basenamelist::iterator basename = basenames_.begin ();
+	 basename != basenames_.end ();
+	 ++ basename)
+      {
+        if (basenames_maxlen_ < basename->stringlength ())
+	  basenames_maxlen_ = basename->stringlength ();
+      }
+  }
+
+  void add_searchdir (const char *dir, int dirlen)
+  {
+      if (dirlen < 0)
+        dirlen = strlen (dir);
+
+      if (!dirlen)
+        return;
+
+      /* Search "x/bin:x/lib" for "x/lib" */
+      if (dirlen >=4 && !strncmp (dir + dirlen - 4, "/lib", 4))
+	searchbuffers_.append (cygwin::vbuffer::constructor_args (dir, dirlen - 4, "/bin", 4, "/", 1 + basenames_maxlen_, NULL));
+//	searchbuffers_.appendv (dir, dirlen - 4, "/bin", 4, "/", 1 + basenames_maxlen_, NULL);
+
+      /* prealloc buffer in searchdir for any basename we will search for */
+      searchbuffers_.append (cygwin::vbuffer::constructor_args (dir, dirlen, "/", 1 + basenames_maxlen_, NULL));
+  }
+
+  void add_searchpath (const char *path)
+  {
+    while (path && *path)
+      {
+        const char *next = strchr (path, ':');
+        add_searchdir (path, next ? next - path : -1);
+	path = next ? next + 1 : next;
+      }
+  }
+
+  void add_envsearchpath (const char *envpath)
+    {
+      add_searchpath (getenv (envpath));
+    }
+
+  /* Within each searchdir registered, try each registered basename to
+     find as executable.  Returns found dir/basename in real_filename.
+     Returns true when found. */
+  bool check_path_access (path_conv& real_filename)
+  {
+    for (searchbufferlist::iterator dir = searchbuffers_.begin ();
+	 dir != searchbuffers_.end ();
+	 ++dir)
+      for (cygwin::samlist<cygwin::vstring>::iterator name = basenames_.begin ();
+	   name != basenames_.end ();
+	   ++name)
+	{
+	  /* complete the filename path to search for */
+	  memcpy (dir->buffer () + dir->stringlength (), name->string (), name->stringlength () + 1);
+	  debug_printf ("trying %s", dir->buffer ());
+	  real_filename.check (dir->string (), PC_SYM_FOLLOW | PC_POSIX);
+	  if (real_filename.exists () && !real_filename.isdir ())
+	    {
+	      debug_printf (" found %s", dir->buffer ());
+	      return true;
+	    }
+	}
+    real_filename.error = ENOENT;
+    return !real_filename.error;
+  }
+};
+
+}
+
+#endif /* __cplusplus */
diff --git a/winsup/cygwin/samlist.h b/winsup/cygwin/samlist.h
new file mode 100644
index 0000000..aec30ae
--- /dev/null
+++ b/winsup/cygwin/samlist.h
@@ -0,0 +1,253 @@
+/* samlist.h: cygwin::samlist
+
+   Copyright 2016 Red Hat, Inc.
+
+This file is part of Cygwin.
+
+This software is a copyrighted work licensed under the terms of the
+Cygwin license.  Please consult the file "CYGWIN_LICENSE" for
+details. */
+
+#ifdef __cplusplus
+
+namespace cygwin
+{
+  /* basic member for SinglyAllocatedMembers list */
+  class samlist_member
+  {
+    samlist_member * prev_;
+    samlist_member * next_;
+
+    /* no copy */
+    samlist_member (samlist_member const &);
+    samlist_member & operator = (samlist_member const &);
+
+  public:
+    samlist_member ()
+      : prev_ (this)
+      , next_ (this)
+    {}
+
+    class constructor_args
+    {
+      friend class samlist_member;
+
+      template<typename MemberType>
+	friend class samlist;
+
+      samlist_member * before;
+
+    public:
+      constructor_args () /* list anchor */
+	: before (NULL)
+      {}
+
+      constructor_args (samlist_member * b) /* list anchor */
+	: before (b)
+      {}
+    };
+
+    samlist_member (constructor_args & ca)
+    {
+      if (!ca.before) /* default constructor */
+	{
+	  /* anchor */
+	  prev_ = this;
+	  next_ = this;
+	  return;
+	}
+      /* instantly insert into list */
+      prev_ = ca.before->prev_;
+      next_ = ca.before;
+      prev_->next_ = this;
+      next_->prev_ = this;
+    }
+
+    ~samlist_member ()
+    {
+      /* remove from list */
+      samlist_member * next = next_;
+      samlist_member * prev = prev_;
+      next->prev_ = prev;
+      prev->next_ = next;
+      prev_ = NULL;
+      next_ = NULL;
+    }
+
+    samlist_member * next ()
+    {
+      return next_;
+    }
+
+    samlist_member * prev ()
+    {
+      return next_;
+    }
+  };
+
+  /* Double-linked list with SinglyAllocatedMember storage, that is,
+     double-linkage pointers are allocated alongside the member data. */
+  template<typename MemberType>
+    class samlist
+    {
+    public:
+      struct member_type
+	: public samlist_member
+	, public MemberType
+      {
+	struct constructor_args
+	  : public samlist_member::constructor_args
+	  , public MemberType::constructor_args
+	{
+	  constructor_args (samlist_member::constructor_args lca,
+			    typename MemberType::constructor_args mca)
+	    : samlist_member::constructor_args (lca)
+	    , MemberType::constructor_args (mca)
+	  {}
+	};
+
+	member_type (constructor_args & ca)
+	  : samlist_member (ca)
+	  , MemberType (ca)
+	{}
+
+        void * operator new (size_t classSize, constructor_args & ca)
+	{
+	  return MemberType::operator new (classSize, ca);
+	}
+      };
+
+      class iterator
+      {
+	samlist_member * current_;
+
+	iterator ();
+
+	friend class samlist;
+	iterator (samlist_member * current)
+	  : current_ (current)
+	{}
+
+      public:
+	iterator (iterator const & rhs)
+	  : current_ (rhs.current_)
+	{}
+
+	iterator & operator = (iterator const & rhs)
+	{
+	  current_ = rhs.current_;
+	  return *this;
+	}
+
+	iterator & operator ++ ()
+	{
+	  current_ = current_->next ();
+	  return *this;
+	}
+
+	iterator operator ++ (int)
+	{
+	  iterator ret (*this);
+	  current_ = current_->next ();
+	  return ret;
+	}
+
+	iterator & operator -- ()
+	{
+	  current_ = current_->prev ();
+	  return *this;
+	}
+
+	iterator operator -- (int)
+	{
+	  iterator ret (*this);
+	  current_ = current_->prev ();
+	  return ret;
+	}
+
+	bool operator == (iterator const & rhs) const
+	{
+	  return current_ == rhs.current_;
+	}
+
+	bool operator != (iterator const & rhs) const
+	{
+	  return !(*this == rhs);
+	}
+
+	member_type const & operator *  () const { return *ptr (); }
+	member_type       & operator *  ()       { return *ptr (); }
+	member_type const * operator -> () const { return  ptr (); }
+	member_type       * operator -> ()       { return  ptr (); }
+
+	operator member_type const * () const { return ptr (); }
+	operator member_type       * ()       { return ptr (); }
+
+	member_type const * ptr () const { return static_cast<member_type *>(current_); }
+	member_type       * ptr ()       { return static_cast<member_type *>(current_); }
+
+	void remove ()
+	{
+	  member_type * old = ptr ();
+	  ++ *this;
+	  delete old;
+	}
+      };
+
+    private:
+      samlist_member * anchor_;
+
+      /* no copy */
+      samlist (samlist const &);
+      samlist & operator = (samlist const &);
+
+    public:
+      iterator  begin () { return iterator (anchor_->next ()); }
+      iterator  end   () { return iterator (anchor_         ); }
+      iterator rbegin () { return iterator (anchor_->prev ()); }
+      iterator rend   () { return iterator (anchor_         ); }
+
+      samlist ()
+	: anchor_ (new samlist_member ())
+      {}
+
+      ~samlist ()
+      {
+	for (iterator it = begin (); it != end (); it.remove ());
+	delete anchor_;
+      }
+
+      void swap (samlist & that)
+      {
+	samlist_member * old = anchor_;
+	anchor_ = that.anchor_;
+	that.anchor_ = old;
+      }
+
+      member_type * append (typename MemberType::constructor_args ca)
+      {
+	typename member_type::constructor_args mca (samlist_member::constructor_args (anchor_), ca);
+	return new (mca) member_type (mca);
+      }
+
+      template<typename FirstType>
+	member_type * appendv (FirstType & firstArg, va_list const & moreArgs)
+	{
+	  typename member_type::constructor_args ca (firstArg, moreArgs);
+	  return append (ca);
+	}
+
+      template<typename FirstType>
+	member_type * appendv(FirstType & firstArg, ...)
+	{
+	  va_list moreArgs;
+	  va_start (moreArgs, firstArg);
+	  member_type * ret = appendv (firstArg, moreArgs);
+	  va_end (moreArgs);
+	  return ret;
+	}
+    };
+
+}
+
+#endif /* __cplusplus */
diff --git a/winsup/cygwin/vstring.h b/winsup/cygwin/vstring.h
new file mode 100644
index 0000000..0658699
--- /dev/null
+++ b/winsup/cygwin/vstring.h
@@ -0,0 +1,503 @@
+/* vstring.h: cygwin::vstring, cygwin::vbuffe
+
+   Copyright 2016 Red Hat, Inc.
+
+This file is part of Cygwin.
+
+This software is a copyrighted work licensed under the terms of the
+Cygwin license.  Please consult the file "CYGWIN_LICENSE" for
+details. */
+
+#include <assert.h>
+#include "cygheap_malloc.h"
+
+#ifdef __cplusplus
+
+namespace cygwin {
+
+  struct constructible {
+    struct constructor_args {};
+    constructible (constructor_args &) {}
+  };
+
+  namespace string {
+
+    template<typename CharType>
+      struct char_traits
+      {
+	typedef CharType char_type;
+	typedef size_t size_type;
+
+	/* see strlen */
+	static size_type length (char_type const *);
+
+	/* see stpcpy */
+	static char_type * copy (char_type *, char_type const *);
+
+	/* see stpncpy */
+	static char_type * copy (char_type *, char_type const *, size_type);
+      };
+
+    template<>
+      struct char_traits<char>
+      {
+	typedef char char_type;
+	typedef size_t size_type;
+
+	static size_type length(char_type const * s)
+	{
+	  return strlen (s);
+	}
+	static char_type * copy(char_type * d, char_type const * s)
+	{
+	  return stpcpy (d, s);
+	}
+	static char_type * copy(char_type * d, char_type const * s, size_type n)
+	{
+	  return stpncpy (d, s, n);
+	}
+      };
+
+    template<>
+      struct char_traits<wchar_t>
+      {
+	typedef wchar_t char_type;
+	typedef size_t size_type;
+
+	static size_type length(char_type const *s)
+	{
+	  return wcslen (s);
+	}
+	static char_type * copy(char_type *d, char_type const *s)
+	{
+	  return wcpcpy (d, s);
+	}
+	static char_type * copy(char_type *d, char_type const *s, size_type n)
+	{
+	  return wcpncpy (d, s, n);
+	}
+      };
+
+    namespace storage {
+
+      template<typename CharTraits, typename AllocatorType>
+	class varying_length /* must be last data member wherever used */
+	{
+	protected:
+	  typedef CharTraits char_traits;
+	  typedef AllocatorType allocator_type;
+	  typedef typename CharTraits::char_type char_type;
+	  typedef typename CharTraits::size_type size_type;
+
+	private:
+	  union {
+	    struct {
+	      size_type bufferlength_; /* initialized by operator new */
+	      size_type stringlength_; /* abused to check for operator new */
+	      /* must be last data member even in derived class */
+	      char_type buffer_[1]; /* we always have space for trailing zero */
+	    };
+	    double alignment_;
+	  };
+
+	  /* variable length disallows placement new */
+	  void * operator new (size_t classSize, void * p);
+
+	protected:
+	  varying_length ()
+	    : stringlength_ ()
+	    , buffer_ ()
+	  {}
+
+	  struct constructor_args
+	  {
+	    size_type bufferlength;
+
+	    constructor_args ()
+	      : bufferlength (0)
+	    {}
+
+	    constructor_args (size_type l)
+	      : bufferlength (l)
+	    {}
+
+	    void set (size_type l)
+	    {
+	      bufferlength = l;
+	    }
+	  };
+
+	  varying_length (constructor_args & ca)
+	  {
+	    /* check if our operator new was used as expected */
+	    assert (bufferlength_ == ca.bufferlength);
+	    assert (stringlength_ == (size_type) this);
+	    stringlength_ = (size_type) 0;
+	    buffer_[0] = (char_type) 0;
+	  }
+
+	  size_type   bufferlength () const { return bufferlength_; }
+	  size_type   stringlength () const { return stringlength_; }
+	  char_type const * buffer () const { return buffer_; }
+	  char_type       * buffer ()       { return buffer_; }
+
+	  void stringlength (size_type stringLength)
+	  {
+	    assert (stringLength <= bufferlength_);
+	    stringlength_ = stringLength;
+	  }
+
+	  void * operator new (size_t classSize, constructor_args & ca)
+	  {
+	    void *ret = allocator_type::alloc (classSize + ca.bufferlength);
+	    /* we are last data member, aligned to the end of real struct:
+	       find self to initialize bufferlength_ */
+	    union { void *v; char *b; varying_length *p; } self;
+	    self.v = ret;
+	    self.b += classSize - sizeof (varying_length);
+	    /* tell constructor about our allocation */
+	    self.p->bufferlength_ = ca.bufferlength;
+	    self.p->stringlength_ = (size_type) self.p;
+	    return ret;
+	  }
+
+	  void operator delete (void * p)
+	  {
+	    allocator_type::free (p);
+	  }
+	};
+
+    }
+
+    namespace access {
+
+      template<typename AllocatorType>
+	class readonly
+	  : protected AllocatorType /* hide AllocatorType::buffer () */
+	{
+	public:
+	  typedef AllocatorType storage_type;
+	  using typename storage_type::char_type;
+	  using typename storage_type::size_type;
+
+	  using typename AllocatorType::constructor_args;
+
+	  readonly (constructor_args & c)
+	    : AllocatorType (c)
+	  {}
+
+	  char_type const * string () const { return AllocatorType::buffer (); }
+	  size_type   stringlength () const { return AllocatorType::stringlength (); }
+	};
+
+      template<typename AllocatorType>
+	class writeable
+	  : public readonly<AllocatorType>
+	{
+	public:
+	  using typename readonly<AllocatorType>::storage_type;
+	  using typename readonly<AllocatorType>::char_type;
+	  using typename readonly<AllocatorType>::size_type;
+
+	  using typename readonly<AllocatorType>::constructor_args;
+
+	  writeable (constructor_args & c)
+	    : readonly<AllocatorType> (c)
+	  {}
+
+	  char_type *     buffer () { return storage_type::buffer (); }
+	  size_type bufferlength () { return storage_type::bufferlength (); }
+	};
+
+    }
+
+    /* Extend some ConstructibleBaseType by one string with length unknown before
+       construction.
+       The ConstructibleBaseType is required to provide the contructor_args type
+       to support polymorphic constructor arguments;
+       Provides 
+       The new operator needs to get the (varying) constructor
+       arguments for determining required memory size, while the constructor
+       itself assumes enough memory being allocated to copy the varying
+       string arguments.  */
+    template<typename AccessType, typename ConstructibleBaseType>
+      class varying_extension
+	: public ConstructibleBaseType
+	, public AccessType /* must be last data member */
+      {
+      public:
+	using typename AccessType::storage_type;
+
+	using typename AccessType::char_traits;
+	using typename AccessType::char_type;
+	using typename AccessType::size_type;
+
+	struct constructor_args
+	  : public ConstructibleBaseType::constructor_args
+	  , public AccessType::constructor_args
+	{
+	  char_type const * part0;
+	  va_list           parts;
+
+	  constructor_args ()
+	    : ConstructibleBaseType::constructor_args ()
+	    , AccessType::constructor_args ()
+	    , part0 (NULL)
+	    , parts ()
+	  {}
+
+	  constructor_args (char_type const * p0, va_list const & pn)
+	    : ConstructibleBaseType::constructor_args ()
+	    , AccessType::constructor_args ()
+	    , part0 (p0)
+	    , parts ()
+	  {
+	    if (part0)
+	      va_copy (parts, pn);
+	  }
+
+	  constructor_args (char_type const * p0, ...)
+	    : ConstructibleBaseType::constructor_args ()
+	    , AccessType::constructor_args ()
+	    , part0 (p0)
+	    , parts ()
+	  {
+	    if (part0)
+	      va_start (parts, p0);
+	  }
+
+	  void set (char_type const * p0, va_list const & pn)
+	  {
+	    if (part0)
+	      va_end (parts);
+	    part0 = p0;
+	    if (part0)
+	      va_copy (parts, pn);
+	  }
+
+	  void set (char_type const * p0, ...)
+	  {
+	    if (part0)
+	      va_end (parts);
+	    part0 = p0;
+	    if (part0)
+	      va_start (parts, p0);
+	  }
+
+	  ~constructor_args ()
+	  {
+	    if (part0)
+	      va_end (parts);
+	  }
+	};
+
+
+	void operator delete (void * p)
+	{
+	  AccessType::operator delete (p);
+	}
+
+	void * operator new (size_t classSize)
+	{
+	  constructor_args ca;
+	  return AccessType::operator new (classSize, ca);
+	}
+
+	void * operator new (size_t classSize, constructor_args & ca)
+	{
+	  char_type const * part0 = ca.part0;
+	  va_list parts;
+	  va_copy (parts, ca.parts);
+	  ca.bufferlength = 0;
+	  while (part0)
+	    {
+	      int part0len = va_arg (parts, int);
+	      if (part0len < 0)
+		part0len = char_traits::length (part0);
+	      ca.bufferlength += part0len;
+	      part0 = va_arg (parts, const char_type *);
+	    }
+	  va_end (parts);
+
+	  /* allocate member- and string-buffer at once */
+	  return AccessType::operator new (classSize, ca);
+	}
+
+	varying_extension (constructor_args & ca)
+	  : ConstructibleBaseType (ca)
+	  , AccessType (ca)
+	{
+	  char_type * dest = storage_type::buffer ();
+	  size_type bufferLength = 0;
+	  char_type const * part0 = ca.part0;
+	  va_list parts;
+	  va_copy (parts, ca.parts);
+	  while (part0)
+	    {
+	      int part0len = va_arg (parts, int);
+	      if (part0len < 0)
+		{
+		  char_type * old = dest;
+		  dest = char_traits::copy (old, part0);
+		  part0len = dest - old;
+		}
+	      else
+		dest = char_traits::copy (dest, part0, part0len);
+	      bufferLength += part0len;
+	      part0 = va_arg (parts, const char_type *);
+	    }
+	  va_end (parts);
+	  *dest = (char_type)0;
+	  storage_type::stringlength (dest - storage_type::buffer ());
+	  assert (bufferLength == storage_type::bufferlength ());
+	}
+
+	template<typename FinalType>
+	  static FinalType * create (typename FinalType::constructor_args & ca)
+	  {
+	    return new (ca) FinalType (ca);
+	  }
+
+	template<typename FinalType>
+	  static FinalType * create (typename FinalType::constructor_args & c, const char_type * part0, va_list const & parts)
+	  {
+	    c.set (part0, parts);
+	    return create (c);
+	  }
+
+	template<typename FinalType>
+	  static FinalType * create (typename FinalType::constructor_args & c, const char_type * part0, ...)
+	  {
+	    va_list parts;
+	    va_start (parts, part0);
+	    c.set (part0, parts);
+	    va_end (parts);
+	    return create (c);
+	  }
+
+	template<typename FinalType>
+	  static FinalType * create (const char_type * part0, va_list const & parts)
+	  {
+	    typename FinalType::constructor_args c;
+	    c.set (part0, parts);
+	    return create (c);
+	  }
+
+	template<typename FinalType>
+	  static FinalType * create (const char_type * part0, ...)
+	  {
+	    typename FinalType::constructor_args c;
+	    va_list parts;
+	    va_start (parts, part0);
+	    c.set (part0, parts);
+	    va_end (parts);
+	    return create (c);
+	  }
+      };
+  }
+
+  template<typename CharType, typename ConstructibleBaseType, typename Allocator>
+    struct basic_vstring_extension
+      : public string::varying_extension
+	< string::access::readonly
+	  < string::storage::varying_length
+	    < string::char_traits<CharType>
+	    , Allocator
+	  > >
+	, ConstructibleBaseType
+	>
+    {
+      using typename string::varying_extension
+	< string::access::readonly
+	  < string::storage::varying_length
+	    < string::char_traits<CharType>
+	    , Allocator
+	  > >
+	, ConstructibleBaseType
+	>::constructor_args;
+      basic_vstring_extension (constructor_args & ca)
+	: string::varying_extension
+	  < string::access::readonly
+	    < string::storage::varying_length
+	      < string::char_traits<CharType>
+	      , Allocator
+	    > >
+	  , ConstructibleBaseType
+	  > (ca)
+      {}
+    };
+
+  template<typename CharType, typename ConstructibleBaseType, typename Allocator>
+    struct basic_vbuffer_extension
+      : public string::varying_extension
+	< string::access::writeable
+	  < string::storage::varying_length
+	    < string::char_traits<CharType>
+	    , Allocator
+	  > >
+	, ConstructibleBaseType
+	>
+    {
+      using typename string::varying_extension
+	< string::access::writeable
+	  < string::storage::varying_length
+	    < string::char_traits<CharType>
+	    , Allocator
+	  > >
+	, ConstructibleBaseType
+	>::constructor_args;
+
+      basic_vbuffer_extension (constructor_args & ca)
+	: string::varying_extension
+	  < string::access::writeable
+	    < string::storage::varying_length
+	      < string::char_traits<CharType>
+	      , Allocator
+	    > >
+	  , ConstructibleBaseType
+	  > (ca)
+      {}
+    };
+
+  template<typename ConstructibleBaseType, typename Allocator>
+    struct vstring_extension
+      : public basic_vstring_extension<char, ConstructibleBaseType, Allocator>
+    {
+      using typename basic_vstring_extension<char, ConstructibleBaseType, Allocator>::constructor_args;
+      vstring_extension (constructor_args & ca)
+	: basic_vstring_extension<char, ConstructibleBaseType, Allocator> (ca)
+      {}
+    };
+
+  template<typename ConstructibleBaseType, typename Allocator>
+    struct wvstring_extension
+      : public basic_vstring_extension<wchar_t, ConstructibleBaseType, Allocator>
+    {
+      using typename basic_vstring_extension<wchar_t, ConstructibleBaseType, Allocator>::constructor_args;
+    };
+
+  template<typename ConstructibleBaseType, typename Allocator>
+    struct vbuffer_extension
+      : public basic_vbuffer_extension<char, ConstructibleBaseType, Allocator>
+    {
+      using typename basic_vbuffer_extension<char, ConstructibleBaseType, Allocator>::constructor_args;
+      vbuffer_extension (constructor_args & ca)
+	: basic_vbuffer_extension<char, ConstructibleBaseType, Allocator> (ca)
+      {}
+    };
+
+  template<typename ConstructibleBaseType, typename Allocator>
+    struct wvbuffer_extension
+      : public basic_vbuffer_extension<wchar_t, ConstructibleBaseType, Allocator>
+    {
+      using typename basic_vbuffer_extension<wchar_t, ConstructibleBaseType, Allocator>::constructor_args;
+    };
+
+  typedef vstring_extension<constructible, allocator<HEAP_STR> > vstring;
+  typedef vbuffer_extension<constructible, allocator<HEAP_STR> > vbuffer;
+
+  typedef wvstring_extension<constructible, allocator<HEAP_STR> > wvstring;
+  typedef wvbuffer_extension<constructible, allocator<HEAP_STR> > wvbuffer;
+}
+
+#endif /* __cplusplus */
-- 
2.8.3

>From 6e2739deccb06a4d739d2a41fa32355f21c151a1 Mon Sep 17 00:00:00 2001
From: Michael Haubenwallner <michael.haubenwallner@ssi-schaefer.com>
Date: Tue, 19 Jul 2016 14:56:48 +0200
Subject: [PATCH 2/3] dlopen: search main executable's directory

---
 winsup/cygwin/dlfcn.cc | 13 +++++++++++++
 1 file changed, 13 insertions(+)

diff --git a/winsup/cygwin/dlfcn.cc b/winsup/cygwin/dlfcn.cc
index e2093ab..4ea9984 100644
--- a/winsup/cygwin/dlfcn.cc
+++ b/winsup/cygwin/dlfcn.cc
@@ -80,6 +80,19 @@ get_full_path_of_dll (const char* str, path_conv &real_filename)
 	 not use the LD_LIBRARY_PATH environment variable. */
       finder.add_envsearchpath ("LD_LIBRARY_PATH");
 
+      /* Unless we have a special cygwin loader, there is no such thing like
+	 DT_RUNPATH on Windows we can use to search for dlls, except for the
+	 directory of the main executable. */
+      tmp_pathbuf tp;
+      PWCHAR exewname = tp.w_get ();
+      GetModuleFileNameW (NULL, exewname, NT_MAX_PATH);
+      char * exedir = tp.c_get ();
+      *exedir = '\0';
+      cygwin_conv_path (CCP_WIN_W_TO_POSIX, exewname, exedir, NT_MAX_PATH);
+      char * lastsep = strrchr (exedir, '/');
+      if (lastsep)
+	finder.add_searchdir (exedir, lastsep - exedir);
+
       /* Finally we better have some fallback. */
       finder.add_searchdir ("/usr/lib", -1);
     }
-- 
2.8.3

>From efe31198bf5747253a9c0d588c354a6b86234525 Mon Sep 17 00:00:00 2001
From: Michael Haubenwallner <michael.haubenwallner@ssi-schaefer.com>
Date: Tue, 19 Jul 2016 14:57:10 +0200
Subject: [PATCH 3/3] dlopen: search PATH environment variable

---
 winsup/cygwin/dlfcn.cc | 3 +++
 1 file changed, 3 insertions(+)

diff --git a/winsup/cygwin/dlfcn.cc b/winsup/cygwin/dlfcn.cc
index 4ea9984..302869d 100644
--- a/winsup/cygwin/dlfcn.cc
+++ b/winsup/cygwin/dlfcn.cc
@@ -93,6 +93,9 @@ get_full_path_of_dll (const char* str, path_conv &real_filename)
       if (lastsep)
 	finder.add_searchdir (exedir, lastsep - exedir);
 
+      /* Windows uses the PATH environment variable to search for dlls. */
+      finder.add_envsearchpath ("PATH");
+
       /* Finally we better have some fallback. */
       finder.add_searchdir ("/usr/lib", -1);
     }
-- 
2.8.3