VIM 7.0 under Windows and Unicode fonts

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

VIM 7.0 under Windows and Unicode fonts

Alexander C. Gaber
Hi all,

I decided to use VIM 7.0 as a text editor under Windows, mainly because it allows me to use only my keyboard to edit files.

My problem is that I always work with UTF-8 or Unicode (codepage 1200) files and I have problems reading characters in other languages. I tried searching around and found the :set guifontwide command, but it seems I can't use any of the fonts from a base Windows install, or I haven't quite figured out how.

If you can suggest the most complete Unicode font (the used languages, I don't care about math symbols, but definitely care about *all* used languages today). I seriously switch through all languages in a week's worth and definitely three character sets a day, I'd hate having to change fonts based on what I need to edit right then.

Or, if I can use Windows fonts, please let me know how.

For information, I have all languages support installed on my Windows XP system, and I can see the needed characters if I open the files in other editors (i.e. Notepad). I tested VIM with the UTF8 HTML page saved as text (http://www.columbia.edu/kermit/utf8.html), and the default font "Fixedsys" doesn't display many of the characters correctly. I changed font around a bit and what I discovered is that the very few of the fonts I can choose have only a couple of languages supported (i.e. Western, Greek, Turkish, Central European and Cyrillic for Lucida Console). When I choose Lucida Console, the languages supported display fine, but not the others (i.e. Japanese, etc.)

Any help will be appreciated. Thanks in advance,

Alex

Reply | Threaded
Open this post in threaded view
|

Re: VIM 7.0 under Windows and Unicode fonts

A.J.Mechelynck
Alexander C. Gaber wrote:

> Hi all,
>
> I decided to use VIM 7.0 as a text editor under Windows, mainly because it allows me to use only my keyboard to edit files.
>
> My problem is that I always work with UTF-8 or Unicode (codepage 1200) files and I have problems reading characters in other languages. I tried searching around and found the :set guifontwide command, but it seems I can't use any of the fonts from a base Windows install, or I haven't quite figured out how.
>
> If you can suggest the most complete Unicode font (the used languages, I don't care about math symbols, but definitely care about *all* used languages today). I seriously switch through all languages in a week's worth and definitely three character sets a day, I'd hate having to change fonts based on what I need to edit right then.
>
> Or, if I can use Windows fonts, please let me know how.
>
> For information, I have all languages support installed on my Windows XP system, and I can see the needed characters if I open the files in other editors (i.e. Notepad). I tested VIM with the UTF8 HTML page saved as text (http://www.columbia.edu/kermit/utf8.html), and the default font "Fixedsys" doesn't display many of the characters correctly. I changed font around a bit and what I discovered is that the very few of the fonts I can choose have only a couple of languages supported (i.e. Western, Greek, Turkish, Central European and Cyrillic for Lucida Console). When I choose Lucida Console, the languages supported display fine, but not the others (i.e. Japanese, etc.)
>
> Any help will be appreciated. Thanks in advance,
>
> Alex
>
>

The following assumes that you have a version of gvim.exe with the  Unicode
capabilities compiled-in, i.e., ":echo has('multi_byte') " returns 1. (If it
returns zero you need a different executable.)

I have tried a third-party font called "Code 2000" in the past: it has glyphs
for the whole BMP (U+0000 to U+FFFF) IIUC, or at least for those codepoints
where characters are defined; but they are ugly; and I don't think it is
Vim-compatible anyway. If you still want to try it out, I guess Google is your
friend.

In gvim under Windows I use MingLiU for CJK (Chinese-Japanese-Korean) and
Courier_New for the rest (except that when editing only Latin-alphabet text I
use Lucida_Console).

It is a known limitation of gvim that, although it allows you to edit any
Unicode codepoint, all codepoints above U+FFFF will display a question mark.
You can still use ga to display the codepoint in decimal, hex and octal, g8 to
see the UTF-8 byte sequence (assuming, of course, that you have 'encoding' set
to UTF-8), and enter any Unicode codepoint using (for instance) the keystrokes
mentioned under ":help i_CTRL-V_digit".

Note that you can leave 'guifontwide' and 'guifontset' unset (even in gvim
versions where they are defined) and use only 'guifont'. You can even mention
several fonts in 'guifont' but they must have the same width, height, and
ratios of lowercase body height, stem height and tail height to total height
(i.e., the relative heights of x, p and d to the character cell height, as
well as the width of all glyphs, must be the same among all those fonts) which
isn't easy to guarantee.

You may want to use mappings, for example as follows:

        :map    <F6>      :set gfn=Courier_New:h9:cDEFAULT<CR>
        :map    <S-F6>    :set gfn=MingLiU:h16:cDEFAULT<CR>
        :imap   <F6>      <C-O>:set gfn=Courier_New:h9:cDEFAULT<CR>
        :imap   <S-F6>    <C-O>:set gfn=MingLiU:h16:cDEFAULT<CR>

Then hit Shift-F6 when editing sinograms, and F6 when editing other languages
(I tested Courier_New with French, Russian and Arabic; and I'm confident that
it will support a number of other languages too).

You can also choose fonts via a meny, using ":set guifont=*", or have Vim
display the current value (exactly as it must be written in your vimrc or
gvimrc) using ":set guifont=<Tab>". The latter also allows edit-in-place to
tweak the current settings.


Best regards,
Tony.
Reply | Threaded
Open this post in threaded view
|

Re: VIM 7.0 under Windows and Unicode fonts

Karl Guertin
In reply to this post by Alexander C. Gaber
On 10/28/06, Alexander C. Gaber <[hidden email]> wrote:
> If you can suggest the most complete Unicode font (the used languages, I don't care about math symbols, but definitely care about *all* used languages today). I seriously switch through all languages in a week's worth and definitely three character sets a day, I'd hate having to change fonts based on what I need to edit right then.
>
> Or, if I can use Windows fonts, please let me know how.

I hit that page on ubuntu linux using bitstream vera [1] and the
coverage is decent.

It's missing Old Irish, Gothic, Burmese, Mongolian, Tibetan and has a
few characters missing from Braille,  Turkish (Ottoman), Urdu, Pashto,
Vietnamese (nôm). The rest have glyphs but I have no clue whether
they're correct or not.

[1] http://packages.ubuntulinux.org/edgy/x11/ttf-bitstream-vera
Reply | Threaded
Open this post in threaded view
|

Re: VIM 7.0 under Windows and Unicode fonts

Alexander C. Gaber
In reply to this post by A.J.Mechelynck
A.J.Mechelynck wrote:
> The following assumes that you have a version of gvim.exe with the
> Unicode capabilities compiled-in, i.e., ":echo has('multi_byte') "
> returns 1. (If it returns zero you need a different executable.)
I am using the version available on the Vim web site, which has
multi-byte compiled in. I also tested with the command you mentioned and
it returns the value 1.
> I have tried a third-party font called "Code 2000" in the past: it has
> glyphs for the whole BMP (U+0000 to U+FFFF) IIUC, or at least for
> those codepoints where characters are defined; but they are ugly; and
> I don't think it is Vim-compatible anyway. If you still want to try it
> out, I guess Google is your friend.
Unfortunately you are right, it's not Vim-compatible :( It does not
appear in the list of fonts I can choose from. As a matter of fact, I
downloaded Vim 6.4 and for some reason I see more choices in Vim 6.4
than I see in Vim 7.0 on the same computer. Any idea why?

The problem with your proposed mappings is that I absolutely need to see
all languages at once. The most important part of my job for which I
wanted to use VIM was the ability to see differences between two files,
including differences between characters. For example, I have two files
like this:

English text;
   French translation,
   Greek translation,
   Russian translation,
   Chinese translation,
   Japanese translation,
   Arabic translation;

If the Arabic translation has even one character different, I would like
to spot it right away and the same goes for all languages; here, Vim's
diff functionality would greatly help, but if I can't see the graphical
difference, I may disregard it as a glitch in Vim's colouring. The other
problem is I sometimes need to verify the characters with the ones
displayed in another program (i.e. in a web page displayed by my browser
or an application) where I can change between translations in a second.
Mapping different characters, while probably a good solution, would
simply slow down my work.

But thank you for your answer and instructions on how to accomplish all
that, it's much appreciated.
Reply | Threaded
Open this post in threaded view
|

Re: VIM 7.0 under Windows and Unicode fonts

Alexander C. Gaber
In reply to this post by Karl Guertin
Karl Guertin wrote:
> I hit that page on ubuntu linux using bitstream vera [1] and the
> coverage is decent.
>
> It's missing Old Irish, Gothic, Burmese, Mongolian, Tibetan and has a
> few characters missing from Braille,  Turkish (Ottoman), Urdu, Pashto,
> Vietnamese (nôm). The rest have glyphs but I have no clue whether
> they're correct or not.
>
> [1] http://packages.ubuntulinux.org/edgy/x11/ttf-bitstream-vera
The biggest problem with that font is that I don't see it in the list of
Vim's fonts.
Reply | Threaded
Open this post in threaded view
|

Re: VIM 7.0 under Windows and Unicode fonts

A.J.Mechelynck
In reply to this post by Alexander C. Gaber
Alexander C. Gaber wrote:

> A.J.Mechelynck wrote:
>> The following assumes that you have a version of gvim.exe with the
>> Unicode capabilities compiled-in, i.e., ":echo has('multi_byte') "
>> returns 1. (If it returns zero you need a different executable.)
> I am using the version available on the Vim web site, which has
> multi-byte compiled in. I also tested with the command you mentioned and
> it returns the value 1.
>> I have tried a third-party font called "Code 2000" in the past: it has
>> glyphs for the whole BMP (U+0000 to U+FFFF) IIUC, or at least for
>> those codepoints where characters are defined; but they are ugly; and
>> I don't think it is Vim-compatible anyway. If you still want to try it
>> out, I guess Google is your friend.
> Unfortunately you are right, it's not Vim-compatible :( It does not
> appear in the list of fonts I can choose from. As a matter of fact, I
> downloaded Vim 6.4 and for some reason I see more choices in Vim 6.4
> than I see in Vim 7.0 on the same computer. Any idea why?
>
> The problem with your proposed mappings is that I absolutely need to see
> all languages at once. The most important part of my job for which I
> wanted to use VIM was the ability to see differences between two files,
> including differences between characters. For example, I have two files
> like this:
>
> English text;
>    French translation,
>    Greek translation,
>    Russian translation,
>    Chinese translation,
>    Japanese translation,
>    Arabic translation;
>
> If the Arabic translation has even one character different, I would like
> to spot it right away and the same goes for all languages; here, Vim's
> diff functionality would greatly help, but if I can't see the graphical
> difference, I may disregard it as a glitch in Vim's colouring. The other
> problem is I sometimes need to verify the characters with the ones
> displayed in another program (i.e. in a web page displayed by my browser
> or an application) where I can change between translations in a second.
> Mapping different characters, while probably a good solution, would
> simply slow down my work.
>
> But thank you for your answer and instructions on how to accomplish all
> that, it's much appreciated.
>

Well, vimdiff will always fold away identical parts and display different
parts, regardless of whether your current font actually has glyphs for the
different characters. As long as the different characters are in the Unicode
BMP, toggling the font with the mappings in my previous post should be fast
enough. It might alter the global size of the Vim window because the character
cell size differs; here's an improved version to always keep Vim maximized:

  :map  <F6>   :set gfn=Courier_New:h9:cDEFAULT lines=999 columns=9999<CR>
  :imap <F6>   <C-O>:set gfn=Courier_New:h9:cDEFAULT lines=999 columns=9999<CR>
  :map  <S-F6> :set gfn=MingLiU:h16:cDEFAULT lines=999 columns=9999<CR>
  :imap <S-F6> <C-O>:set gfn=MingLiU:h16:cDEFAULT lines=999 columns=9999<CR>

Similarly, gvim will display the whole (split) windows either left-to-right or
right-to-left; it doesn't offer true bidi functionality. You may want to map

  :map  <F8>   :setlocal invarabic<CR>
  :imap <F8>   <C-O>:setlocal invarabic<CR>

to toggle between Arabic (right-to-left) and Latin (left-to-right) view.
Another possibility is to display the same file in two split-windows, one LTR
and the other RTL.


Best regards,
Tony.
Reply | Threaded
Open this post in threaded view
|

Re: VIM 7.0 under Windows and Unicode fonts

Yongwei Wu
In reply to this post by Karl Guertin
On 10/29/06, Karl Guertin <[hidden email]> wrote:

> On 10/28/06, Alexander C. Gaber <[hidden email]> wrote:
> > If you can suggest the most complete Unicode font (the used languages, I don't care about math symbols, but definitely care about *all* used languages today). I seriously switch through all languages in a week's worth and definitely three character sets a day, I'd hate having to change fonts based on what I need to edit right then.
> >
> > Or, if I can use Windows fonts, please let me know how.
>
> I hit that page on ubuntu linux using bitstream vera [1] and the
> coverage is decent.
>
> It's missing Old Irish, Gothic, Burmese, Mongolian, Tibetan and has a
> few characters missing from Braille,  Turkish (Ottoman), Urdu, Pashto,
> Vietnamese (nôm). The rest have glyphs but I have no clue whether
> they're correct or not.
>
> [1] http://packages.ubuntulinux.org/edgy/x11/ttf-bitstream-vera

Karl, I think you are mistaken. Although Bitstream Vera Sans Mono is a
decent font, it does not have better effect than Courier New in this
case. It contains 269 glyphs, while Courier New contains 1318 glyphs.
When using Courier New, I can display ASCII, Latin1, Greek, Cyrillic,
Arabic, Hebrew, Yiddish, as well as Chinese and Japanese (NSimsun
seems automatically used), text at the same time.

Alex, if you cannot display Chinese while choosing Courier New, you
may try setting your default locale (Control Panel > Regional and
Language Options > Advanced > Language for non-Unicode programs) to
Chinese (PRC). It should work. And then you will be able to edit
English, French, Greek, Russian, Chinese, Japanese, and Arabic
simultaneously.

Best regards,

Yongwei
--
Wu Yongwei
URL: http://wyw.dcweb.cn/
Reply | Threaded
Open this post in threaded view
|

Re: VIM 7.0 under Windows and Unicode fonts

panshizhu
"Yongwei Wu" <[hidden email]> 写于 2006-10-30 10:41:21:

> Alex, if you cannot display Chinese while choosing Courier New, you
> may try setting your default locale (Control Panel > Regional and
> Language Options > Advanced > Language for non-Unicode programs) to
> Chinese (PRC). It should work. And then you will be able to edit
> English, French, Greek, Russian, Chinese, Japanese, and Arabic
> simultaneously.
>
> Best regards,
>
> Yongwei

A little off-topic, but AFAIK this works perfect in Windows but not
identical in Linux, since Windows do not connect the internal processing
locale to the locale of user interfaces. The UI will never change when you
change the locale. (That's a good design IMO)

When you change the locale in Linux, the messages, menu texts are all
changed to the targeting locale after reboot. So it is hard to have a
Chinese UI and retain English locale, and it might be more difficult to
have a Chinese locale while retain English UI.

Alex's case is an example, if what he want is to display chinese correctly,
set locale to chinese is an easy way, but he might be troubled if all the
messages in the UI also changed into chinese.

--
Sincerely, Pan, Shi Zhu. ext: 2606
Reply | Threaded
Open this post in threaded view
|

Re: VIM 7.0 under Windows and Unicode fonts

Alexander C. Gaber
In reply to this post by Yongwei Wu
Yongwei Wu wrote:
> When using Courier New, I can display ASCII, Latin1, Greek, Cyrillic,
> Arabic, Hebrew, Yiddish, as well as Chinese and Japanese (NSimsun
> seems automatically used), text at the same time.
Thank you so very much! I think I found a solution to my problems by
using the NSimsun font as my guifontwide:

:set guifont=Courier_New:h11
:set guifontwide=NSimsun:h11

I can't quite test now if I can see all characters, but I should
definitely give this a try. I also tried :set
guifontwide=Bitstream_Cyberbit:h11 and :set
guifontwide=TITUS_Cyberbit_Basic:h11 but, while this seems to work fine
in Vim 6.4, it is no longer recognized as a valid font (???) in Vim 7.0.

Now, the best thing would be a monospaced font having all the Unicode
characters ;) but the day that will exist, it would be my dream come
true (and probably, by then, I had moved on from this, and no longer
need it...).
Reply | Threaded
Open this post in threaded view
|

Re: VIM 7.0 under Windows and Unicode fonts

Yongwei Wu
On 10/30/06, Alexander C. Gaber <[hidden email]> wrote:
> Yongwei Wu wrote:
> > When using Courier New, I can display ASCII, Latin1, Greek, Cyrillic,
> > Arabic, Hebrew, Yiddish, as well as Chinese and Japanese (NSimsun
> > seems automatically used), text at the same time.
> Thank you so very much! I think I found a solution to my problems by
> using the NSimsun font as my guifontwide:
>
> :set guifont=Courier_New:h11
> :set guifontwide=NSimsun:h11

That works (though it does not look as good as the automatic font
mapping). Because of the different font widths, the following lines
may work better than yours:

:set guifont=Courier_New:h10
:set guifontwide=NSimsun:h12

> I can't quite test now if I can see all characters, but I should
> definitely give this a try. I also tried :set
> guifontwide=Bitstream_Cyberbit:h11 and :set
> guifontwide=TITUS_Cyberbit_Basic:h11 but, while this seems to work fine
> in Vim 6.4, it is no longer recognized as a valid font (???) in Vim 7.0.

Cyberbit is not a monospaced font, I suppose.

> Now, the best thing would be a monospaced font having all the Unicode
> characters ;) but the day that will exist, it would be my dream come
> true (and probably, by then, I had moved on from this, and no longer
> need it...).

I would prefer better a user-configurable font mapping (doable in
Linux; don't know how to achieve it in Windows), like the Java
font.properties. Because of the Unification of Han characters, the
shape of some Chinese characters can hardly be perfect for all users
of Simplified Chinese, Traditional Chinese, Japanese, and Korean (some
characters in Cyberbit look really ugly in my eyes). In the world of
Unicode, pure text without language tags is becoming even a little
more irrelevant :-(.

Best regards,

Yongwei

--
Wu Yongwei
URL: http://wyw.dcweb.cn/
Reply | Threaded
Open this post in threaded view
|

Re: VIM 7.0 under Windows and Unicode fonts

A.J.Mechelynck
In reply to this post by panshizhu
[hidden email] wrote:

> "Yongwei Wu" <[hidden email]> 写于 2006-10-30 10:41:21:
>> Alex, if you cannot display Chinese while choosing Courier New, you
>> may try setting your default locale (Control Panel > Regional and
>> Language Options > Advanced > Language for non-Unicode programs) to
>> Chinese (PRC). It should work. And then you will be able to edit
>> English, French, Greek, Russian, Chinese, Japanese, and Arabic
>> simultaneously.
>>
>> Best regards,
>>
>> Yongwei
>
> A little off-topic, but AFAIK this works perfect in Windows but not
> identical in Linux, since Windows do not connect the internal processing
> locale to the locale of user interfaces. The UI will never change when you
> change the locale. (That's a good design IMO)
>
> When you change the locale in Linux, the messages, menu texts are all
> changed to the targeting locale after reboot. So it is hard to have a
> Chinese UI and retain English locale, and it might be more difficult to
> have a Chinese locale while retain English UI.
>
> Alex's case is an example, if what he want is to display chinese correctly,
> set locale to chinese is an easy way, but he might be troubled if all the
> messages in the UI also changed into chinese.
>
> --
> Sincerely, Pan, Shi Zhu. ext: 2606
>
>

To keep English UI, add at the _top_ of your vimrc (before sourcing the
vimrc_example and before "filetype on" if you use either):

        if has("unix")
                language messages C
        else
                language messages en
        endif

If, in addition, you want to force Chinese character locale regardless of what
was set by the OS, you may want to add also

        if &tenc == ""
                let &tenc = &enc
        endif
        set enc=utf-8
        language ctype zh_CN.utf-8

(or something). The exact setting may or may not be OS-dependent, I didn't try
it (while I tested the "messages" setting on both WinXP and Linux -- not in a
Chinese locale but in a French one).

See ":help :language"


Best regards,
Tony.
Reply | Threaded
Open this post in threaded view
|

Re: VIM 7.0 under Windows and Unicode fonts

Yongwei Wu
In reply to this post by panshizhu
On 10/30/06, [hidden email] <[hidden email]> wrote:

> "Yongwei Wu" <[hidden email]> 写于 2006-10-30 10:41:21:
> > Alex, if you cannot display Chinese while choosing Courier New, you
> > may try setting your default locale (Control Panel > Regional and
> > Language Options > Advanced > Language for non-Unicode programs) to
> > Chinese (PRC). It should work. And then you will be able to edit
> > English, French, Greek, Russian, Chinese, Japanese, and Arabic
> > simultaneously.
> >
> > Best regards,
> >
> > Yongwei
>
> A little off-topic, but AFAIK this works perfect in Windows but not
> identical in Linux, since Windows do not connect the internal processing
> locale to the locale of user interfaces. The UI will never change when you
> change the locale. (That's a good design IMO)
>
> When you change the locale in Linux, the messages, menu texts are all
> changed to the targeting locale after reboot. So it is hard to have a
> Chinese UI and retain English locale, and it might be more difficult to
> have a Chinese locale while retain English UI.
>
> Alex's case is an example, if what he want is to display chinese correctly,
> set locale to chinese is an easy way, but he might be troubled if all the
> messages in the UI also changed into chinese.

The Windows UI will not change, but the Vim UI may change. Apart from
hard-coding in _vimrc (as already discussed by Tony), one may use
Control Panel and environment variables to change the Vim UI language,
which are my preferred ways. In Regional and Language Options, the
language under Standards and formats determines the default UI
language of Vim, unless it is overridden by the environment variable
LANG.

I generally set the default encoding for legacy applications to CP936
(Chinese (PRC)), but I set the language under Standards and formats to
English (United Kingdom) to get an English UI in Vim (as well as make
Firefox treat ambiguous-width characters in a UTF-8 web page as single
width by default). To test the Chinese UI of Vim, I type `set
LANG=zh_CN' in the command prompt.

Best regards,

Yongwei
--
Wu Yongwei
URL: http://wyw.dcweb.cn/