Re: setup for 1.7 fails with Japanese characters in download

[Andy Koppe <andy dot koppe at gmail dot com>]

2009/10/3 Corinna Vinschen
> This leads to another question. ÂWhen unpacking distro tar archives,
> all archives hopefully only contain filenames with ASCII chars in them.
> However, there's no reason to keep it this way in the future, if we
> make sure that everybody uses the same charset.
>
> Therefore I'd like to propose that distro tar archives are in future
> *always* generated in the "C" locale, so setup can be sure all tar
> archives are UTF-8 encoded.

Good idea.

Any UTF-8 locale should be ok though, unless it turns out that the
language influences package builds.

> For setup it should be sufficient for now to make sure the installation
> directory gets converted to UTF-8, and to change the mbstowcs calls in
> filemanip.cc, function mklongpath() to MultiByteToWideChar (65001, ...)
> calls.
> [...]
> Not quite, of course.  Every path fetched from the system via
> GetCurrentDirectory or GetWindowsDirectory or some such, should
> be fetched via the W function and converted to UTF-8 explicitely.

Shame Windows doesn't allow programs to change their "ANSI" codepage,
in which case setup could just call SetACP(65001) and be done with it.

Anyway, here's another thing that would be needed: a call to
putenv("LC_ALL=C.UTF-8") to ensure that any Cygwin processes invoked
by setup use UTF-8, even if something else is set in Windows
environment variables.


> In the long run it would be better if setup,.exe uses widechar functions
> as much as possible, even for reading strings from the GUI.  This removes
> the dreaded codepage problem and just leaves one conversion, which is
> the conversion from UTF-8 filenames in tar archives to UTF-16.

Yep, but of course that's quite a project.

Andy