gvim crash when using UTF-8 locale and paste text from Windows clipboard

classic Classic list List threaded Threaded
7 messages Options
Reply | Threaded
Open this post in threaded view
|

gvim crash when using UTF-8 locale and paste text from Windows clipboard

Rainux
---------- Forwarded message ----------
From: Rainux <[hidden email]>
Date: 2005-6-19 下午11:25
Subject: gvim crash when using UTF-8 locale and paste text from
Windows clipboard
To: [hidden email]


Hi, I'm using gvim win32 in Windows XP SP2 Simplified Chinese version,
I've set the environment variable "LANG" to "zh_CN", then set
"encoding", "fileencoding" and "termencoding" to "cp936" in .vimrc
file, gvim works fine.
And recently I'v changed the environment variable "LANG" to
"zh_CN.UTF-8", set "encoding","fileencoding" and "termencoding" to
"utf-8" in .vimrc file, gvim also works fine. But, when paste text
from Windows clipboard to gvim, gvim often crashes, why?

Goddess light my path!


--
Goddess light my path!
Reply | Threaded
Open this post in threaded view
|

Re: gvim crash when using UTF-8 locale and paste text from Windows clipboard

Camillo Särs
Hi,

Rainux wrote:
> Hi, I'm using gvim win32 in Windows XP SP2 Simplified Chinese version,
> I've set the environment variable "LANG" to "zh_CN", then set
> "encoding", "fileencoding" and "termencoding" to "cp936" in .vimrc
> file, gvim works fine.
> And recently I'v changed the environment variable "LANG" to
> "zh_CN.UTF-8", set "encoding","fileencoding" and "termencoding" to
> "utf-8" in .vimrc file, gvim also works fine. But, when paste text
> from Windows clipboard to gvim, gvim often crashes, why?

Although I don't know the details about the crash, I think you should
try setting termencoding to the code page you use, i.e. "cp936", instead
of utf-8.  Windows doesn't natively speak utf-8.

Camillo
--
Camillo Särs <[hidden email]>             Aim for the impossible and you
http://camillo.särs.net                 will achieve the improbable
Reply | Threaded
Open this post in threaded view
|

Re: gvim crash when using UTF-8 locale and paste text from Windows clipboard

Rainux
Thanks for your help.

I've read more help text, and done more test, I found:
1. "termencoding" is never used in Win32 GUI version.
2. "fileencoding" is the encoding for file, do not have relative to this case.
3. When set $LANG to a UTF-8 locale, e.g. "zh-CN.UTF-8", set the
"encoding" to anything or let empty,  then gvim will crash while paste
LITTLE LONG text form Windows clipboard.
4. When not set $LANG, $LANG actually be "zh_CN" (the default locale I
use), set the "encoding" to "utf-8" or "cp936" (the code page I use),
gvim works fine.  But, if "encoding" is "utf-8", the messages gvim
displayed is not correctly, because the message text gvim selected is
encoded in "cp936" (because the $LANG is "zh_CN", not "zh_CN.UTF-8".).

I need set "encoding" to "utf-8", because this will let vim detect the
file encoding and convert it correctly. When use "cp936", the file
encoded in "cp950"(big5) and "ucs-2le" will not be detected and
converted.
Reply | Threaded
Open this post in threaded view
|

Re: gvim crash when using UTF-8 locale and paste text from Windows clipboard

Bram Moolenaar

Rainux wrote (about a month ago):

> Thanks for your help.
>
> I've read more help text, and done more test, I found:
> 1. "termencoding" is never used in Win32 GUI version.
> 2. "fileencoding" is the encoding for file, do not have relative to this case.
> 3. When set $LANG to a UTF-8 locale, e.g. "zh-CN.UTF-8", set the
> "encoding" to anything or let empty,  then gvim will crash while paste
> LITTLE LONG text form Windows clipboard.
> 4. When not set $LANG, $LANG actually be "zh_CN" (the default locale I
> use), set the "encoding" to "utf-8" or "cp936" (the code page I use),
> gvim works fine.  But, if "encoding" is "utf-8", the messages gvim
> displayed is not correctly, because the message text gvim selected is
> encoded in "cp936" (because the $LANG is "zh_CN", not "zh_CN.UTF-8".).
>
> I need set "encoding" to "utf-8", because this will let vim detect the
> file encoding and convert it correctly. When use "cp936", the file
> encoded in "cp950"(big5) and "ucs-2le" will not be detected and
> converted.

On my system I can't use the zh_CN locale, thus I can't even try to
reproduce this problem.

Could you at least specify the output of ":version", so that we can see
what Vim version you are using?

You might want to try a recent Vim, like what Tony Mechelynck makes
available, see http://users.skynet.be/antoine.mechelynck/vim/.

--
A hamburger walks into a bar, and the bartender says: "I'm sorry,
but we don't serve food here."

 /// Bram Moolenaar -- [hidden email] -- http://www.Moolenaar.net   \\\
///        Sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\              Project leader for A-A-P -- http://www.A-A-P.org        ///
 \\\     Buy LOTR 3 and help AIDS victims -- http://ICCF.nl/lotr.html   ///
Reply | Threaded
Open this post in threaded view
|

Re: gvim crash when using UTF-8 locale and paste text from Windows clipboard

Bram Moolenaar
In reply to this post by Rainux

Rainux -

> Sorry, after more test, I found Tony's build of Vim not crash when
> paste LITTLE LONG TEXT because it do not have a zh_CN.UTF-8 messages
> translation module ($VIMRUNTIME/lang/zh_CN.UTF-8/LC_MESSAGES/vim.mo).
>
> I copied this file and "libintl.dll" from official version to Tony's
> build, then do the paste test, it crashes.
>
> I removed this file from the official version, then the official
> version don't crash in the paste test, like Tony's.
>
> Hope this little information can help you to find and fix this
> "bug"(or not Vim's bug?), Bram. I'm sorry I've observed you're the
> author of Vim just now. ;)

Thus when you remove that vim.mo file it doesn't crash?  That means
there is a problem with a UTF-8 message handled like your codepage (was
it cp936?).  Someone reported before that the sprintf() in the MS
library causes this problem.  I changed several calls to vim_snprintf()
and checked the messages for using a "%" in the second byte, what else
would need to be done?

If you can, please try to reproduce the crash in a debugger, so that we
can see where it happens.  Tony's site gives hints how to compile Vim
with free tools.

- Bram

--
To keep milk from turning sour: Keep it in the cow.

 /// Bram Moolenaar -- [hidden email] -- http://www.Moolenaar.net   \\\
///        Sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\              Project leader for A-A-P -- http://www.A-A-P.org        ///
 \\\     Buy LOTR 3 and help AIDS victims -- http://ICCF.nl/lotr.html   ///
Reply | Threaded
Open this post in threaded view
|

Re: gvim crash when using UTF-8 locale and paste text from Windows clipboard

Bram Moolenaar
In reply to this post by Rainux

Rainux -

> Hi, I've used Cygwin's gcc to compiled a gvimd.exe, and tried to
> reproduce the crash in gdb, here is the conversation with gdb.
>
> M:\vim\vim63>gdb gvimd.exe -q
> (gdb) run
> Starting program: /cygdrive/m/vim/vim63/gvimd.exe
>
> (Here I switched to gVim and paste LITTLE LONG TEXT from Windows
> clipboard, then gdb caught the exception.)
>
> Program received signal SIGSEGV, Segmentation fault.
> 0x77c12a16 in wscanf () from /cygdrive/c/WINDOWS/system32/msvcrt.dll
> (gdb) info stack
> #0  0x77c12a16 in wscanf () from /cygdrive/c/WINDOWS/system32/msvcrt.dll
> #1  0x610e8987 in ?? ()
> #2  0x0054f020 in p_report ()
> #3  0x0022fde0 in ?? ()

It seems the stack was messed up.

> It seems the crash it not Vim's bug? And, I've viewed gvimd.exe and
> libintl.dll 's import functions from msvcrt.dll, I havn't founded the
> function wscanf(), why?

It's possibly called somewhere inside a library function.

> Do you need more infomation and more action? I'm just a newbie to
> machine-level-debugging.

Try setting breakpoints to further pinpoint where the crash occurs.
Perhaps clip_mch_request_selection() is a good one in this case:

        break clip_mch_request_selection

- Bram

--
Bad programs can be written in any language.

 /// Bram Moolenaar -- [hidden email] -- http://www.Moolenaar.net   \\\
///        Sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\              Project leader for A-A-P -- http://www.A-A-P.org        ///
 \\\     Buy LOTR 3 and help AIDS victims -- http://ICCF.nl/lotr.html   ///
Reply | Threaded
Open this post in threaded view
|

Re: gvim crash when using UTF-8 locale and paste text from Windows clipboard

Bram Moolenaar
In reply to this post by Rainux

Rainux wrote:

> Thanks for your hint, use this breakpoint I did a source-level trace
> and found a strange thing.

I'm glad you were able pin it down this far.  That helps a lot.

> The sources I used is full Unix sources, with patch 1-85 applied,
> compiled by gcc in Cygwin (follow Tony's Compiling HowTo).
>
> The correct description of the "bug" is follow:
>   When yank or put lines to/from Vim's register(It's independent of
>   Windows clipboard.), if lines more than 2(or bigger?), Vim will
>   display a message such as "8 lines yanked" or "10 more lines". When
>   using zh_CN.UTF-8 locale, Vim will try to display the localized
>   string of the message, the localized yank message will be OK, but
>   try to display the localized put message will cause Vim crash.
>
> The yank message is print by ops.c:2871, and the put message is print by
> misc1.c:3028.
>
> ops.c:2871
> smsg((char_u *)_("%ld lines yanked"), yanklines);
>                 // ^ This line display the localized yank message.
>
> misc1.c:3028
> sprintf((char *)msg_buf, _("%ld more lines"), pn);
>                 // ^ This line try to format the localized put message but
>                 // crash in wscanf().
>    else
> sprintf((char *)msg_buf, _("%ld fewer lines"), pn);
>
> The conversation with gdb:

I first thought "msg_buf" could be too small, but you have the same
problem with smsg(), thus that's not the case.

There must be a problem in gettext(), which is what _() translates into.
I'm afraid I can't do anything for that.  This is the libintl.dll
library.  You could try getting another version of libintl.dll.  Are you
using the one in the Vim distribution?

You could split up the code to make sure the crash is inside
gettext().  Change:

                 sprintf((char *)msg_buf, _("%ld more lines"), pn);

To:
                 {
                         char *ts = _("%ld more lines");
                         sprintf((char *)msg_buf, ts, pn);
                 }

If it now crashes in sprintf() then it's a different problem.  But from
your description it should crash in _().

> Program received signal SIGSEGV, Segmentation fault.
> 0x77c12a16 in wscanf () from /cygdrive/c/WINDOWS/system32/msvcrt.dll

I guess it might have something to do with this msvcrt.dll being
incompatible with the libintl.dll.  Try using the libintl.dll that comes
with cygwin.

--
BLACK KNIGHT: None shall pass.
ARTHUR:       I have no quarrel with you, brave Sir knight, but I must cross
              this bridge.
BLACK KNIGHT: Then you shall die.
                 "Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD

 /// Bram Moolenaar -- [hidden email] -- http://www.Moolenaar.net   \\\
///        Sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\              Project leader for A-A-P -- http://www.A-A-P.org        ///
 \\\     Buy LOTR 3 and help AIDS victims -- http://ICCF.nl/lotr.html   ///