Quantcast

Help with print encoding

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Help with print encoding

Lifepillar
Hello everyone,

I am writing to this list because I cannot find a solution to the
following problem. I am trying to print some documents, and a few
characters turn out as upside-down question marks. Even after reading
the manual, I cannot figure out how to fix that.

I am using Vim 8.0.172 under macOS. Vim is compiled with +iconv. These
are the relevant settings in my .vimrc:

set encoding=utf-8
if has('mac')
   fun! LFPrintFile(fname)
     call system('pstopdf ' . a:fname)
     call system('open -a Preview ' . a:fname . '.pdf')
     call delete(a:fname)
     call delete(a:fname.'.pdf')
     return v:shell_error
   endf
   set printexpr=LFPrintFile(v:fname_in)
endif
set printoptions=syntax:n,number:y
set printfont=:h9

The buffers to be printed are UTF-8 encoded, and printencoding is empty.
As an example, if I print the following text:

Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1

all the quotes (U+2018, U+2019) become upside-down question marks. What
is wrong in my configuration?

Thanks,
Life.



--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Help with print encoding

Mike Williams
Hi,

Have you tried setting printencoding to macroman?

The reason you see the upside-down question marks is because VIM
converts the buffer content from utf-8 to latin1, but the latin1
character set does not include the left and right single quotes and VIM
will change them to upside-down quotes so they stand out.  On Macs there
is special code to convert utf-8 to Mac encoding where the left and
right quotes exist, so that should work for you.  Unfortunately I don't
have access to a Mac to confirm this.

Cheers.

On 13/02/2017 08:08, Lifepillar wrote:

> Hello everyone,
>
> I am writing to this list because I cannot find a solution to the
> following problem. I am trying to print some documents, and a few
> characters turn out as upside-down question marks. Even after reading
> the manual, I cannot figure out how to fix that.
>
> I am using Vim 8.0.172 under macOS. Vim is compiled with +iconv. These
> are the relevant settings in my .vimrc:
>
> set encoding=utf-8
> if has('mac')
>    fun! LFPrintFile(fname)
>      call system('pstopdf ' . a:fname)
>      call system('open -a Preview ' . a:fname . '.pdf')
>      call delete(a:fname)
>      call delete(a:fname.'.pdf')
>      return v:shell_error
>    endf
>    set printexpr=LFPrintFile(v:fname_in)
> endif
> set printoptions=syntax:n,number:y
> set printfont=:h9
>
> The buffers to be printed are UTF-8 encoded, and printencoding is empty.
> As an example, if I print the following text:
>
> Signif. codes:  0 ‘***’ 0.001 ‘**’ 0.01 ‘*’ 0.05 ‘.’ 0.1 ‘ ’ 1
>
> all the quotes (U+2018, U+2019) become upside-down question marks. What
> is wrong in my configuration?
>
> Thanks,
> Life.
>
>
>

Mike
--
Nothing matters, but everything counts.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Help with print encoding

Lifepillar
On 13/02/2017 18:08, Mike Williams wrote:
> Hi,
>
> Have you tried setting printencoding to macroman?

Yes, and the result is the same.

> The reason you see the upside-down question marks is because VIM
> converts the buffer content from utf-8 to latin1, but the latin1
> character set does not include the left and right single quotes and VIM
> will change them to upside-down quotes so they stand out.  On Macs there
> is special code to convert utf-8 to Mac encoding where the left and
> right quotes exist, so that should work for you.  Unfortunately I don't
> have access to a Mac to confirm this.

I do not know much about print encondings, but... wouldn't it be
possible to define new print encodings, namely a UTF-8 print encoding?

My current workaround is to use :TOhtml then print the resulting HTML
from a browser, but I'd like to have a direct way.

Thanks,
Life.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Help with print encoding

Mike Williams
On 14/02/2017 07:25, Lifepillar wrote:
> On 13/02/2017 18:08, Mike Williams wrote:
>> Hi,
>>
>> Have you tried setting printencoding to macroman?
>
> Yes, and the result is the same.

Well it was worth a shot.

>> The reason you see the upside-down question marks is because VIM
>> converts the buffer content from utf-8 to latin1, but the latin1
>> character set does not include the left and right single quotes and VIM
>> will change them to upside-down quotes so they stand out.  On Macs there
>> is special code to convert utf-8 to Mac encoding where the left and
>> right quotes exist, so that should work for you.  Unfortunately I don't
>> have access to a Mac to confirm this.
>
> I do not know much about print encondings, but... wouldn't it be
> possible to define new print encodings, namely a UTF-8 print encoding?

That would make life easier, but history is against you I'm afraid.

> My current workaround is to use :TOhtml then print the resulting HTML
> from a browser, but I'd like to have a direct way.

That would be nicer.  I'll raise it on the vim dev list as I have some
ideas.

TTFN

Mike
--
Every time I know the answers, they change the questions on me.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Help with print encoding

Lifepillar
On 14/02/2017 16:43, Mike Williams wrote:

> On 14/02/2017 07:25, Lifepillar wrote:
>> On 13/02/2017 18:08, Mike Williams wrote:
>>> Hi,
>>>
>>> Have you tried setting printencoding to macroman?
>>
>> Yes, and the result is the same.
>
> Well it was worth a shot.
>
>>> The reason you see the upside-down question marks is because VIM
>>> converts the buffer content from utf-8 to latin1, but the latin1
>>> character set does not include the left and right single quotes and VIM
>>> will change them to upside-down quotes so they stand out.  On Macs there
>>> is special code to convert utf-8 to Mac encoding where the left and
>>> right quotes exist, so that should work for you.  Unfortunately I don't
>>> have access to a Mac to confirm this.
>>
>> I do not know much about print encondings, but... wouldn't it be
>> possible to define new print encodings, namely a UTF-8 print encoding?
>
> That would make life easier, but history is against you I'm afraid.

Would you care to elaborate? Is PostScript the problem? If so, can Vim
bypass generating a PostScript file?

>> My current workaround is to use :TOhtml then print the resulting HTML
>> from a browser, but I'd like to have a direct way.
>
> That would be nicer.  I'll raise it on the vim dev list as I have some
> ideas.

Great :) That is a minor annoyance (I do not need to print often), but
it would be nice if the process could be streamlined.

Thanks,
Life.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Help with print encoding

Mike Williams
On 14/02/2017 20:32, Lifepillar wrote:

> On 14/02/2017 16:43, Mike Williams wrote:
>> On 14/02/2017 07:25, Lifepillar wrote:
>>> On 13/02/2017 18:08, Mike Williams wrote:
>>>> The reason you see the upside-down question marks is because VIM
>>>> converts the buffer content from utf-8 to latin1, but the latin1
>>>> character set does not include the left and right single quotes and VIM
>>>> will change them to upside-down quotes so they stand out.  On Macs there
>>>> is special code to convert utf-8 to Mac encoding where the left and
>>>> right quotes exist, so that should work for you.  Unfortunately I don't
>>>> have access to a Mac to confirm this.
>>>
>>> I do not know much about print encondings, but... wouldn't it be
>>> possible to define new print encodings, namely a UTF-8 print encoding?
>>
>> That would make life easier, but history is against you I'm afraid.
>
> Would you care to elaborate? Is PostScript the problem? If so, can Vim
> bypass generating a PostScript file?

Almost all printing technology pre-dates Unicode and earlier unified
character set technology.  Early products would be sales region specific
- for example Western printers would not print East Asian text and vice
versa (OK they did ASCII but not the wider European character set, but I
digress).

The early printers were very limited in processing power and available
memory.  The original methods of font definition and use were organised
around minimising font sizes and a lot of the print file text encoding
methods were designed to support this.  They provide some internal fonts
but because of font licensing and hardware constraints there were only a
few fonts covering the basic design styles - fixed width, serif,
sans-serif, and symbols.

This means if you wanted to use any other fonts on the printer they have
to be downloaded to the printer.  These fonts could be large, and if you
use multiple fonts your print file size balloons, and then your printer
runs out of memory and fails to print.  Techniques developed to reduce
the final file size to help printing these complex files, but they were
not trivial.

VIM printing support was added in the late 90s.  It uses four fonts
(each of the Courier normal, bold, italic and bold-italic variations are
distinct fonts) which are all provided by default in PostScript printing
systems.  Importantly this also means that everything is known about the
fonts (glyph sizes, complement, etc.).

Most printing technology in use today (PS, PDF, XPS) supports fonts with
a large number of glyphs, and multi-byte character encodings (including
various Unicode encodings) for the text to be printed.  It would be
possible to add support to VIM given time - it would need to include a
font reader, font rewriter (to avoid including a few megabytes of font
data not needed), four fonts to be used for the four styles (although it
could implement a poor man's version for bold and italic versions) which
are suitably licensed for inclusion with VIM.  Adding support for
arbitrary fonts provided by the user raises the bar further due to the
various font formats available and varying glyph complements.  And you
still have the issue of what to do if a font does not contain the glyph
for the character you want printed.

At the end of the day VIM is a text editor.  It supports basic printing
with limited styling - it is not a full Unicode scripting engine with
printing.  Generally text files are written in a single well defined
script, or character set, which can normally be handled by an 8-bit
encoding (there is support for CJK MBCS printing as well).  The
immediate problem is handling the transcoding of Unicode to a suitable
single byte character set.  There are two possible solutions, selection
of single byte character set based on glyphs to be printed, or use
Unicode character alternates when transcoding to a character set.  For
example converting your left and right quotes to apostrophes.  I will be
suggesting the latter one to start with to the dev team.

Basically yes it could be done, but the amount of work and additions to
VIM seems out of proportion to the benefit gained.  Some may say they
would love a full Unicode print support, possibly even with proportional
fonts but they their will be issues with composing characters and
possibly a layout-engine.  These are far from trivial to implement and
most VIM users would have other priorities.

I hope that helps understand some of the issues to do with printing
these days.

TTFN

Mike
--
Where there's smoke, there's toast.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Help with print encoding

Lifepillar
On 19/02/2017 10:11, Mike Williams wrote:

> On 14/02/2017 20:32, Lifepillar wrote:
>> On 14/02/2017 16:43, Mike Williams wrote:
>>> On 14/02/2017 07:25, Lifepillar wrote:
>>>> On 13/02/2017 18:08, Mike Williams wrote:
>>>>> The reason you see the upside-down question marks is because VIM
>>>>> converts the buffer content from utf-8 to latin1, but the latin1
>>>>> character set does not include the left and right single quotes and
>>>>> VIM
>>>>> will change them to upside-down quotes so they stand out.  On Macs
>>>>> there
>>>>> is special code to convert utf-8 to Mac encoding where the left and
>>>>> right quotes exist, so that should work for you.  Unfortunately I
>>>>> don't
>>>>> have access to a Mac to confirm this.
>>>>
>>>> I do not know much about print encondings, but... wouldn't it be
>>>> possible to define new print encodings, namely a UTF-8 print encoding?
>>>
>>> That would make life easier, but history is against you I'm afraid.
>>
>> Would you care to elaborate? Is PostScript the problem? If so, can Vim
>> bypass generating a PostScript file?
>
> Almost all printing technology pre-dates Unicode and earlier unified
> character set technology.  Early products would be sales region specific
> - for example Western printers would not print East Asian text and vice
> versa (OK they did ASCII but not the wider European character set, but I
> digress).
>
> The early printers were very limited in processing power and available
> memory.  The original methods of font definition and use were organised
> around minimising font sizes and a lot of the print file text encoding
> methods were designed to support this.  They provide some internal fonts
> but because of font licensing and hardware constraints there were only a
> few fonts covering the basic design styles - fixed width, serif,
> sans-serif, and symbols.
>
> This means if you wanted to use any other fonts on the printer they have
> to be downloaded to the printer.  These fonts could be large, and if you
> use multiple fonts your print file size balloons, and then your printer
> runs out of memory and fails to print.  Techniques developed to reduce
> the final file size to help printing these complex files, but they were
> not trivial.
>
> VIM printing support was added in the late 90s.  It uses four fonts
> (each of the Courier normal, bold, italic and bold-italic variations are
> distinct fonts) which are all provided by default in PostScript printing
> systems.  Importantly this also means that everything is known about the
> fonts (glyph sizes, complement, etc.).
>
> Most printing technology in use today (PS, PDF, XPS) supports fonts with
> a large number of glyphs, and multi-byte character encodings (including
> various Unicode encodings) for the text to be printed.  It would be
> possible to add support to VIM given time - it would need to include a
> font reader, font rewriter (to avoid including a few megabytes of font
> data not needed), four fonts to be used for the four styles (although it
> could implement a poor man's version for bold and italic versions) which
> are suitably licensed for inclusion with VIM.  Adding support for
> arbitrary fonts provided by the user raises the bar further due to the
> various font formats available and varying glyph complements.  And you
> still have the issue of what to do if a font does not contain the glyph
> for the character you want printed.
>
> At the end of the day VIM is a text editor.  It supports basic printing
> with limited styling - it is not a full Unicode scripting engine with
> printing.  Generally text files are written in a single well defined
> script, or character set, which can normally be handled by an 8-bit
> encoding (there is support for CJK MBCS printing as well).  The
> immediate problem is handling the transcoding of Unicode to a suitable
> single byte character set.  There are two possible solutions, selection
> of single byte character set based on glyphs to be printed, or use
> Unicode character alternates when transcoding to a character set.  For
> example converting your left and right quotes to apostrophes.  I will be
> suggesting the latter one to start with to the dev team.
>
> Basically yes it could be done, but the amount of work and additions to
> VIM seems out of proportion to the benefit gained.  Some may say they
> would love a full Unicode print support, possibly even with proportional
> fonts but they their will be issues with composing characters and
> possibly a layout-engine.  These are far from trivial to implement and
> most VIM users would have other priorities.
>
> I hope that helps understand some of the issues to do with printing
> these days.

Thanks for the thorough explanation!
Life.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Help with print encoding

arocker
In reply to this post by Mike Williams

> At the end of the day VIM is a text editor.  It supports basic printing
> with limited styling - it is not a full Unicode scripting engine with
> printing.

Very interesting discussion. At some point in the evolution from the basic
text editor, using something like LaTeX makes more sense.

There's no useful concept in software, no matter how simple, that won't be
seized immediately and abused beyond its authors wildest speculations.
(Back when the Osborne portable computer was sold with a spreadsheet and
word processor bundle, the world immediately divided into 2 groups.
Spreadsheet users who wanted to write an occasional letter would create a
66-character wide column and type in it, and writers who would do any
arithmetic required with Wordstar's primitive commands.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Loading...