gVim stripping diacritics not in keyboard charset

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

gVim stripping diacritics not in keyboard charset

Tuetschek
Hi,
I am using gVim 7.2 under Windows 7 and I have customized my En-Us
keyboard  using the MS Keyboard Layout Creator, so that it contains
not only English characters, but also Czech ones and others (e.g.
Latin long
vocals), which have diacritical marks. It works perfectly well in all
programs but gVim (even Vim console version and Vim console Cygwin
version work correctly).
The problem is that some of the diacritical marks get just stripped
away, e.g. I type "ř" and get just "r". I noticed this happens only
with characters outside of the Latin-1 (or CP1252?) charset (e.g. I
can type "ä" all right), but it doesn't depend on the encoding, file
encoding, terminal encoding or language setting in gVim.
Could anyone help me how to solve or at least understand the problem?
Thanks,
Tuetschek

--
You received this message from the "vim_use" maillist.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: gVim stripping diacritics not in keyboard charset

Cyril Slobin-4
On Tue, Jan 19, 2010 at 10:40 PM, Tuetschek <[hidden email]> wrote:

> The problem is that some of the diacritical marks get just stripped
> away, e.g. I type "ř" and get just "r".

I second this. I have recently installed an Esperanto extension of the
standard US English layout, and found that letters like ĉ or ŝ aren't
entered properly in gVim. Sad.

--
http://slobin.pp.ru/ `When I use a word,' Humpty Dumpty said,
<[hidden email]> `it means just what I choose it to mean'

--
You received this message from the "vim_use" maillist.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: gVim stripping diacritics not in keyboard charset

bill lam
ĵaŭ, 21 Jan 2010, Cyril Slobin skribis:
> On Tue, Jan 19, 2010 at 10:40 PM, Tuetschek <[hidden email]> wrote:
>
> > The problem is that some of the diacritical marks get just stripped
> > away, e.g. I type "ř" and get just "r".
>
> I second this. I have recently installed an Esperanto extension of the
> standard US English layout, and found that letters like ĉ or ŝ aren't
> entered properly in gVim. Sad.
>

I just solved a similar problem in linux mentioned in another thread.
I guess the problem is not gvim itself.  Which OS does you use?  Did
it also failed in other gui editors?  I guess you already aware the
esperanto keymap as a workaround.

--
regards,
====================================================
GPG key 1024D/4434BAB3 2008-08-24
gpg --keyserver subkeys.pgp.net --recv-keys 4434BAB3

--
You received this message from the "vim_use" maillist.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: gVim stripping diacritics not in keyboard charset

Tuetschek
> I just solved a similar problem in linux mentioned in another thread.
> I guess the problem is not gvim itself.  Which OS does you use?  Did
> it also failed in other gui editors?  I guess you already aware the
> esperanto keymap as a workaround.
>

Mine is currently on Windows 7, but I've had the same problems with
Windows XP, so I think it's for all (current) versions of Windows.
It's clearly a gVim-related problem with keyboard input handling, I've
no similar problems with any other program (except for PuTTy, but I
think there's some similar problem there).
Changing the keymap is exactly what I'm trying to avoid. And I can use
any Windows keyboard layout for any language. But instead of switching
keymaps every few seconds, which feels kind of schizophrenic to me,
I've created a custom one that contains all the characters I need. And
sadly gVim is not compatible with it.

--
You received this message from the "vim_use" maillist.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: gVim stripping diacritics not in keyboard charset

Tony Mechelynck
In reply to this post by Tuetschek
On 19/01/10 20:40, Tuetschek wrote:

> Hi,
> I am using gVim 7.2 under Windows 7 and I have customized my En-Us
> keyboard  using the MS Keyboard Layout Creator, so that it contains
> not only English characters, but also Czech ones and others (e.g.
> Latin long
> vocals), which have diacritical marks. It works perfectly well in all
> programs but gVim (even Vim console version and Vim console Cygwin
> version work correctly).
> The problem is that some of the diacritical marks get just stripped
> away, e.g. I type "ř" and get just "r". I noticed this happens only
> with characters outside of the Latin-1 (or CP1252?) charset (e.g. I
> can type "ä" all right), but it doesn't depend on the encoding, file
> encoding, terminal encoding or language setting in gVim.
> Could anyone help me how to solve or at least understand the problem?
> Thanks,
> Tuetschek
>

It can be an OS configuration problem, a Vim configuration problem, or
(especially in console mode) a misunderstanding between the OS and Vim,
as follows:

OS configuration problems
-------------------------
The keyboard can only generate keys which are in your system locale, as
known in Unix parlance as $LC_CTYPE, and, at Vim startup and before
running any scripts (including not only your own vimrc but possibly also
a "system vimrc" of which not every user is aware), as 'encoding' and
possibly v:ctype

Vim configuration problems
--------------------------
* Vim can only accept into memory characters which exist in the charset
defined by the present value of 'encoding'

* Vim can only display glyphs which are in your present font. In console
mode this means the console font (which is out of Vim's control), in GUI
mode it is your present 'guifont', see
http://vim.wikia.com/wiki/Setting_the_font_in_the_GUI

* Vim can only write to a disk file characters which exist in the file's
'fileencoding'

* When reading an existing file, Vim tries to determine the file's
'fileencoding' (singular) with the help of the global option
'fileencodings' (plural). You can bypass this heuristic if you know the
"right" value, see :help ++enc

* When creating a new file, Vim sets its 'fileencoding' to the option's
global value (again, unless you override it, e.g. with ++enc). An empty
'fileencoding' (the default) means: use 'encoding'.

Misunderstandings between the OS and Vim
----------------------------------------
Vim believes that the characters sent by the keyboard (and, in console
mode, those accepted by the display) are in the charset defined by
'termencoding'. The default value of 'termencoding' is (except in GUI
mode in gvim with GTK2 GUI) the empty string, which means: use
'encoding' (IIUC in GUI mode on Win32 gvim always gets keyboard input in
Unicode regardless of 'termencoding'). This empty string is OK at
startup because the default value of 'encoding' is your locale's ctype
charset, but it is probably not OK if you have changed 'encoding', for
instance in order to edit Unicode. In that case I recommend to set
'termencoding' (if it was empty) to the "old" value of 'encoding' before
you set a "new" one. (This won't harm in the cases where Vim disregards
the value, and it is essential when it doesn't.)

To check the "actual" value of a character which may be displayed
differently (e.g. because the console cannot display the actual value,
or because Vim "thinks" that it can't) put the Normal-mode cursor on the
character and type ga (then look at bottom left of the Vim screen).

Inputting characters not in your OS locale
------------------------------------------
Vim can accept into memory any character present in 'encoding', even
those not accepted by your OS locale. There are several ways to do it:

- Keymaps (but you say you don't want them)
- Character value in hex, or, for single-byte characters, also in
decimal or octal, see :help i_CTRL-V_digit
- Digraphs: see :help digraph.txt and in particular :help
digraphs-default. IMHO these are best for sporadic Latin accented
characters not on your keyboard, and to a lesser degree even for
sporadic non-Latin characters for which a digraph exists. (By sporadic I
mean: just a few at a time, not long sentences of them.)



For details, see http://vim.wikia.com/wiki/Working_with_Unicode as well
as the help topics mentioned there.



Best regards,
Tony.
--
Information Center, n.:
        A room staffed by professional computer people whose job it is
to tell you why you cannot have the information you require.

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

To unsubscribe from this group, send email to vim_use+unsubscribegooglegroups.com or reply to this email with the words "REMOVE ME" as the subject.
Reply | Threaded
Open this post in threaded view
|

Re: gVim stripping diacritics not in keyboard charset

Tuetschek
Thanks, but I'm sorry, your suggestions didn't help ... it really
doesn't matter which terminal or file encoding I have set, my font is
capable of displaying such characters (I can paste a "ř"' through
clipboard or open any file that contains it all right) and that
character is present in my locale "Czech_Republic.1250".

I think it's got something to do with the way Vim handles Windows
input messages (since the console Windows version works OK).

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

To unsubscribe from this group, send email to vim_use+unsubscribegooglegroups.com or reply to this email with the words "REMOVE ME" as the subject.
Reply | Threaded
Open this post in threaded view
|

Re: gVim stripping diacritics not in keyboard charset

Tuetschek
On Tuesday, March 30, 2010 9:01:36 PM UTC+2, Tuetschek wrote:
> Thanks, but I'm sorry, your suggestions didn't help ... it really
> doesn't matter which terminal or file encoding I have set, my font is
> capable of displaying such characters (I can paste a "ř"' through
> clipboard or open any file that contains it all right) and that
> character is present in my locale "Czech_Republic.1250".
>
> I think it's got something to do with the way Vim handles Windows
> input messages (since the console Windows version works OK).

I have finally solved the problem ... and it's kinda strange. I just downloaded Vim sources and built them using Visual Studio C++ Express 2010 according to the instructions at http://vim.wikia.com/wiki/Build_Vim_in_Windows_with_Visual_Studio . I was planning to have closer look at the problem, put some debug mesagges in etc., but it dissapeared by itself after the first build.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.


Reply | Threaded
Open this post in threaded view
|

Re: gVim stripping diacritics not in keyboard charset

Tony Mechelynck
On 22/04/13 23:22, Tuetschek wrote:

> On Tuesday, March 30, 2010 9:01:36 PM UTC+2, Tuetschek wrote:
>> Thanks, but I'm sorry, your suggestions didn't help ... it really
>> doesn't matter which terminal or file encoding I have set, my font is
>> capable of displaying such characters (I can paste a "ř"' through
>> clipboard or open any file that contains it all right) and that
>> character is present in my locale "Czech_Republic.1250".
>>
>> I think it's got something to do with the way Vim handles Windows
>> input messages (since the console Windows version works OK).
>
> I have finally solved the problem ... and it's kinda strange. I just downloaded Vim sources and built them using Visual Studio C++ Express 2010 according to the instructions at http://vim.wikia.com/wiki/Build_Vim_in_Windows_with_Visual_Studio . I was planning to have closer look at the problem, put some debug mesagges in etc., but it dissapeared by itself after the first build.
>

Well, I'm happy to know you finally got rid of the problem. I suppose
we'll never know if the cure was due to a different procedure used to
build Vim, or to one of the many bugfixes which happened in the three
years (and almost a month) since then.

For people who want to see the begin of this thread (e.g. if, like me,
they read this by POP rather than on the Google Groups web interface),
it is at
https://groups.google.com/group/vim_use/browse_frm/thread/8ee40c183ebf4d27/


Happy Vimming!
Tony.
--
Due to circumstances beyond your control, you are master of your fate
and captain of your soul.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.