noeol noendofline binary fileformat

classic Classic list List threaded Threaded
16 messages Options
Reply | Threaded
Open this post in threaded view
|

noeol noendofline binary fileformat

Michael Ludwig-3

I have text files in DOS format with no CRLF at the end of file. I know
text files are supposed to have a newline at eof, but some tools do not
deem this necessary. I'm editing these files using VIM and do not want
to put a newline at eof when it is missing. But VIM does. How can I tell
it not to?

The solution seems to be to set the "binary" option.

    http://vimdoc.sourceforge.net/vimfaq.html - see 5.4.
    http://vimdoc.sourceforge.net/htmldoc/usr_23.html - see *23.4*

I also found this tip:

    http://vim.sourceforge.net/tips/tip.php?tip_id=1369

But this changes the "fileformat" to "unix". This is surprising - if it
actually were a binary file this conversion would be wrong.

C:\temp :: vim --version
VIM - Vi IMproved 7.0 (2006 May 7, compiled May  7 2006 16:18:30)
MS-Windows 32 Bit Konsolen-Version
[...]

C:\temp :: od -c test.txt
0000000   e   i   n   s  \r  \n   z   w   e   i  \r  \n   d   r   e   i
0000020

C:\temp :: vim test.txt
# [noeol] already set (setting it explicitly does not change anything)
# :set binary
# :w

C:\temp :: od -c test.txt
0000000   e   i   n   s  \n   z   w   e   i  \n   d   r   e   i
0000016

No eol indeed, but no CRLF either.

Same story on Linux Ubuntu Feisty using:

VIM - Vi IMproved 7.0 (2006 May 7, compiled May 22 2007 21:27:09)
Inklusive der Korrekturen: 1-164, 234-235

First question: Is this the correct behaviour?

Second question: What can I do to preserve both the "fileformat" and
"noeol" properties?

Starting VIM with the -b (binary) switch is not satisfying either; this
does not convert line endings to UNIX format, but newly inserted lines
do not get DOS line endings, but UNIX line endings.

Michael

--~--~---------~--~----~------------~-------~--~----~
You received this message from the "vim_use" maillist.
For more information, visit http://www.vim.org/maillist.php
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: noeol noendofline binary fileformat

Ben Schmidt

> First question: Is this the correct behaviour?

I think so, yes.

Unix lineendings must be used in 'binary' mode, otherwise Vim also has
to remember whether each line had a CR or not, rather than just whether
to tack an LF at the end. As you have noticed, though, setting 'binary'
after loading a file, rather than before, will result in the line
endings changing.

> Second question: What can I do to preserve both the "fileformat" and
> "noeol" properties?

It's a hack, but the best thing I can think of is to modify the
autocommands from the tip you found to insert the CRs as well as set
binary, simulating the effect you get when reading the file with -b:



" preserve noeol (missing trailing eol) when saving file
" in order to preserve missing-trailing-eol, we need
" to temporarily 'set binary' for the duration of file writing.
" Thanks to Aaron and A.Mechelynck for motivation and hints
" leading to this solution.

augroup automatic_noeol
au!

au BufWritePre  * :call TempSetBinaryForNoeol()
au BufWritePost * :call TempRestoreBinaryForNoeol()

fun! TempSetBinaryForNoeol()
   let g:save_binary = &binary
   if ! &eol && ! &binary
     setlocal binary
     if &ff == "dos"
       silent 1,$-1s/$/\="\\".nr2char(13)
     endif
   endif
endfun

fun! TempRestoreBinaryForNoeol()
   if ! &eol && ! g:save_binary
     if &ff == "dos"
       silent 1,$-1s/\r$/
     endif
     setlocal nobinary
   endif
endfun

aug END



It won't work for Mac line endings. All the CRs will be changed to LFs.
But I think it will work for Unix and DOS.

Cheers,

Ben.



Send instant messages to your online friends http://au.messenger.yahoo.com 


--~--~---------~--~----~------------~-------~--~----~
You received this message from the "vim_use" maillist.
For more information, visit http://www.vim.org/maillist.php
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: noeol noendofline binary fileformat

Ben Schmidt

I have updated the tip on the Wiki to incorporate my improvement. Hope
that wasn't inappropriate. Someone can change it back if it was, and let
me know what to do in future. I am a newcomer to the community.

http://vim.wikia.com/wiki/Save_file_that_had_missing-trailing-end-of-line_automatically_without_trailing-end-of-line

Ben.



Ben Schmidt wrote:

>> First question: Is this the correct behaviour?
>
> I think so, yes.
>
> Unix lineendings must be used in 'binary' mode, otherwise Vim also has
> to remember whether each line had a CR or not, rather than just whether
> to tack an LF at the end. As you have noticed, though, setting 'binary'
> after loading a file, rather than before, will result in the line
> endings changing.
>
>> Second question: What can I do to preserve both the "fileformat" and
>> "noeol" properties?
>
> It's a hack, but the best thing I can think of is to modify the
> autocommands from the tip you found to insert the CRs as well as set
> binary, simulating the effect you get when reading the file with -b:
>
>
>
> " preserve noeol (missing trailing eol) when saving file
> " in order to preserve missing-trailing-eol, we need
> " to temporarily 'set binary' for the duration of file writing.
> " Thanks to Aaron and A.Mechelynck for motivation and hints
> " leading to this solution.
>
> augroup automatic_noeol
> au!
>
> au BufWritePre  * :call TempSetBinaryForNoeol()
> au BufWritePost * :call TempRestoreBinaryForNoeol()
>
> fun! TempSetBinaryForNoeol()
>    let g:save_binary = &binary
>    if ! &eol && ! &binary
>      setlocal binary
>      if &ff == "dos"
>        silent 1,$-1s/$/\="\\".nr2char(13)
>      endif
>    endif
> endfun
>
> fun! TempRestoreBinaryForNoeol()
>    if ! &eol && ! g:save_binary
>      if &ff == "dos"
>        silent 1,$-1s/\r$/
>      endif
>      setlocal nobinary
>    endif
> endfun
>
> aug END
>
>
>
> It won't work for Mac line endings. All the CRs will be changed to LFs.
> But I think it will work for Unix and DOS.
>
> Cheers,
>
> Ben.
>
>
>
> Send instant messages to your online friends http://au.messenger.yahoo.com 
>
>
> >
>
Send instant messages to your online friends http://au.messenger.yahoo.com 


--~--~---------~--~----~------------~-------~--~----~
You received this message from the "vim_use" maillist.
For more information, visit http://www.vim.org/maillist.php
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

RE: noeol noendofline binary fileformat

JohnBeckett

Ben Schmidt wrote:
> I have updated the tip on the Wiki...

http://vim.wikia.com/wiki/VimTip1369

Thanks Ben - very nice result.

If anyone could help by fixing a typo in any tip, or deleting a few junk
comments, or combining useful comments, that would be great. Or, you could
fix a whole tip like Ben did.

We are still feeling our way while fixing the tips, and there are no "don't
do that" rules. You might like to read the guidelines:

http://vim.wikia.com/wiki/Vim_Tips_Wiki:General_guidelines

Most tips have a "This page needs review" box which contains a link to the
guidelines.

Ideally, you would make yourself an account (use the login link at the top
of any page). Then the fact that you edited the page would be recorded in
the history. It's traditional on a wiki to NOT credit authors, and I would
be inclined to delete the two-line attribution comment in VimTip1369.

The code block should probably have the leading spaces removed (while
keeping the indents), and <pre> ... </pre> tags should be inserted around
the block. That gives a better result for someone copying code from the tip.

Since you fixed the tip, you could also delete the first line:
 {{Review}}

That line includes Template:Review which displays the "This page needs
review" box. At the bottom of the tip, you can also see that the tip is in
Category:Review (and Category:VimTip). When {{Review}} is removed, the
Category:Review will be removed, and there will be one less tip we have to
review!

The initial wording could be improved, so you might argue that {{Review}}
should stay. It doesn't really matter. I'm changing "vim" to "Vim" (I see
you fixed "windows" to "Windows").

Finally, we want to put every tip in a suitable category. For example, if a
reader were looking for tips on searching, they would browse the Searching
category, for example, from the category tree on the main page (needs
javascript):

http://vim.wikia.com/wiki/Main_Page

On the "General guidelines" page mentioned above, you will find a link to
the "Category guidelines". I'm seeking help on what to call the new category
we will need for this tip. See "Categories" at end of the mail archive:

http://lists.wikia.com/pipermail/vim-l/2007-August/thread.html

John


--~--~---------~--~----~------------~-------~--~----~
You received this message from the "vim_use" maillist.
For more information, visit http://www.vim.org/maillist.php
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: noeol noendofline binary fileformat

Sebastian Menge
In reply to this post by Ben Schmidt

Am Fri, 10 Aug 2007 10:54:33 +1000 schrieb Ben Schmidt:

> I have updated the tip on the Wiki to incorporate my improvement. Hope
> that wasn't inappropriate.

That's how it is meant to be. :-)

In an ideal world the experts for some topic (specifc language, plugin,
folding, syntax etc) on this list would be monitoring a couple of tips
and improve them when someone here asks, or new ideas evolve at this list.

At the moment we need a _lot_ of help, especially with categorization of
tips. Everyone is welcome :-)

Sebastian.


--~--~---------~--~----~------------~-------~--~----~
You received this message from the "vim_use" maillist.
For more information, visit http://www.vim.org/maillist.php
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: noeol noendofline binary fileformat

Ben Schmidt
In reply to this post by JohnBeckett

Further updated, and changed name, which didn't quite work the way I
thought it would, but there you have it...I presume if the result sucks
someone can put it back...

http://vim.wikia.com/wiki/Preserve_missing_end-of-line_at_end_of_text_files

Gave it the 'fileformat' category but without doing anything to that
category (so I presume it's become some kind of orphan or something). I
wonder whether that category is a bit too specific, though, to be all
that useful--there can't be all that many tips related to it, surely,
and the category namespace will just get huge if categories are so
specific. At the same time it is a really good idea to have
categories/keywords/something related to the Vim help tags. 'eol' or
'endofline' would also be an appropriate category with this in mind. And
perhaps 'writing'.

Ben.



John Beckett wrote:

> Ben Schmidt wrote:
>> I have updated the tip on the Wiki...
>
> http://vim.wikia.com/wiki/VimTip1369
>
> Thanks Ben - very nice result.
>
> If anyone could help by fixing a typo in any tip, or deleting a few junk
> comments, or combining useful comments, that would be great. Or, you could
> fix a whole tip like Ben did.
>
> We are still feeling our way while fixing the tips, and there are no "don't
> do that" rules. You might like to read the guidelines:
>
> http://vim.wikia.com/wiki/Vim_Tips_Wiki:General_guidelines
>
> Most tips have a "This page needs review" box which contains a link to the
> guidelines.
>
> Ideally, you would make yourself an account (use the login link at the top
> of any page). Then the fact that you edited the page would be recorded in
> the history. It's traditional on a wiki to NOT credit authors, and I would
> be inclined to delete the two-line attribution comment in VimTip1369.
>
> The code block should probably have the leading spaces removed (while
> keeping the indents), and <pre> ... </pre> tags should be inserted around
> the block. That gives a better result for someone copying code from the tip.
>
> Since you fixed the tip, you could also delete the first line:
>  {{Review}}
>
> That line includes Template:Review which displays the "This page needs
> review" box. At the bottom of the tip, you can also see that the tip is in
> Category:Review (and Category:VimTip). When {{Review}} is removed, the
> Category:Review will be removed, and there will be one less tip we have to
> review!
>
> The initial wording could be improved, so you might argue that {{Review}}
> should stay. It doesn't really matter. I'm changing "vim" to "Vim" (I see
> you fixed "windows" to "Windows").
>
> Finally, we want to put every tip in a suitable category. For example, if a
> reader were looking for tips on searching, they would browse the Searching
> category, for example, from the category tree on the main page (needs
> javascript):
>
> http://vim.wikia.com/wiki/Main_Page
>
> On the "General guidelines" page mentioned above, you will find a link to
> the "Category guidelines". I'm seeking help on what to call the new category
> we will need for this tip. See "Categories" at end of the mail archive:
>
> http://lists.wikia.com/pipermail/vim-l/2007-August/thread.html
>
> John
>
>
> >
>
Send instant messages to your online friends http://au.messenger.yahoo.com 


--~--~---------~--~----~------------~-------~--~----~
You received this message from the "vim_use" maillist.
For more information, visit http://www.vim.org/maillist.php
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

RE: noeol noendofline binary fileformat

Gene Kwiecinski
In reply to this post by Michael Ludwig-3

>I have text files in DOS format with no CRLF at the end of file. I know
>text files are supposed to have a newline at eof, but some tools do not
>deem this necessary. I'm editing these files using VIM and do not want
>to put a newline at eof when it is missing. But VIM does. How can I
tell
>it not to?

I don't have an answer to your question, but if I may ask a question in
return... why would it kill you to have a newline at the end of the
file?  Those "tools" might deem it to be unnecessary, but does the
trailing newline actually *break* them?

I don't know who started that idiocy of newlines only being line
*separators* instead of line *terminators* ('though I can take one good
guess:  M$), but that just breaks 4+ decades of standard practice as far
as non-binary text files.

I go as far as to explicitly add an extra blank line at the end of any
text file (source code, html, batch files, email, config files, etc.),
so my text doesn't end up "missing" its last line.

--~--~---------~--~----~------------~-------~--~----~
You received this message from the "vim_use" maillist.
For more information, visit http://www.vim.org/maillist.php
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: noeol noendofline binary fileformat

Michael Ludwig-3
In reply to this post by Ben Schmidt

Ben Schmidt schrieb am 10.08.2007 um 10:25:33 (+1000):

>
> > First question: Is this the correct behaviour?
>
> I think so, yes.
>
> Unix lineendings must be used in 'binary' mode, otherwise Vim also has
> to remember whether each line had a CR or not, rather than just whether
> to tack an LF at the end. As you have noticed, though, setting 'binary'
> after loading a file, rather than before, will result in the line
> endings changing.

Maybe I'm on the wrong track here, but I used to think that "binary"
implies there be no concept of a line ending - just bytes, which are to
be kept as they are. Of course, when I hit the enter key to add new
lines, this will have to be interpreted in a certain way, preferably in
any one of the established conventions for line endings on various
systems.

> > Second question: What can I do to preserve both the "fileformat" and
> > "noeol" properties?
>
> It's a hack, but the best thing I can think of is to modify the
> autocommands from the tip you found to insert the CRs as well as set
> binary, simulating the effect you get when reading the file with -b:

Your hack works - thanks a lot!

Cheers, Michael

--~--~---------~--~----~------------~-------~--~----~
You received this message from the "vim_use" maillist.
For more information, visit http://www.vim.org/maillist.php
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: noeol noendofline binary fileformat

Michael Ludwig-3
In reply to this post by Gene Kwiecinski

Gene Kwiecinski schrieb am 10.08.2007 um 11:37:22 (-0400):

>
> >I have text files in DOS format with no CRLF at the end of file. I know
> >text files are supposed to have a newline at eof, but some tools do not
> >deem this necessary. I'm editing these files using VIM and do not want
> >to put a newline at eof when it is missing. But VIM does. How can I
> tell
> >it not to?
>
> I don't have an answer to your question, but if I may ask a question in
> return... why would it kill you to have a newline at the end of the
> file?  Those "tools" might deem it to be unnecessary, but does the
> trailing newline actually *break* them?

I wanted to reduce textual difference when editing source code featuring
"noeol" at EOF. Said tools (used by co-workers) would re-drop the eol I
would have added. Less diff, less noise. But it wouldn't kill me, I'd
actually prefer those tools would behave correctly.

And then, I just wanted to know how to do the trick in VIM - and I
discovered this list, so I took my question as a pretext for signing up.

> I don't know who started that idiocy of newlines only being line
> *separators* instead of line *terminators* ('though I can take one good
> guess:  M$), but that just breaks 4+ decades of standard practice as far
> as non-binary text files.

The tool in question is an IDE for PHP called "Zend Studio", Windows
version.

Another tool disrespecting 4+ decades of standard practice is the Saxon
XSLT processor. And probably many more XML tools. XML, as we know,
manages to be human-readable without old-fashioned crap such as lines.

> I go as far as to explicitly add an extra blank line at the end of any
> text file (source code, html, batch files, email, config files, etc.),
> so my text doesn't end up "missing" its last line.

So "G" conveniently lands you on an empty line from where to start a new
paragraph.

--~--~---------~--~----~------------~-------~--~----~
You received this message from the "vim_use" maillist.
For more information, visit http://www.vim.org/maillist.php
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

RE: noeol noendofline binary fileformat

Gene Kwiecinski

>>>I have text files in DOS format with no CRLF at the end of file. I
know
>>>text files are supposed to have a newline at eof, but some tools do
not
>>>deem this necessary. I'm editing these files using VIM and do not
want
>>>to put a newline at eof when it is missing. But VIM does. How can I
>>>it not to?

>>I don't have an answer to your question, but if I may ask a question
in
>>return... why would it kill you to have a newline at the end of the
>>file?  Those "tools" might deem it to be unnecessary, but does the
>>trailing newline actually *break* them?

>I wanted to reduce textual difference when editing source code
featuring
>"noeol" at EOF. Said tools (used by co-workers) would re-drop the eol I

Even with an explicit blank line at the end?  Eerie...


>would have added. Less diff, less noise. But it wouldn't kill me, I'd
>actually prefer those tools would behave correctly.

Mmm.  I tend to "reformat" text to behave correctly whenever possible,
eg, the blank line at the end, kill off any trailing whitespace on each
line, etc.  Just my own personal preference to have things look nice and
*act* nice.


>And then, I just wanted to know how to do the trick in VIM - and I
>discovered this list, so I took my question as a pretext for signing
up.

Well, welcome aboard!  :D


>>I don't know who started that idiocy of newlines only being line
>>*separators* instead of line *terminators* ('though I can take one
good
>>guess:  M$), but that just breaks 4+ decades of standard practice as
far
>>as non-binary text files.

>The tool in question is an IDE for PHP called "Zend Studio", Windows
>version.

Hmmm.  Remind me to avoid it...  :D


>Another tool disrespecting 4+ decades of standard practice is the Saxon
>XSLT processor. And probably many more XML tools. XML, as we know,
>manages to be human-readable without old-fashioned crap such as lines.

Yeah, but organisationally, it's nice to have readable indentation,
etc., to be able to read it more clearly.  Eg, I can read sideways- and
even inverted text, albeit more slowly than when it's oriented normally,
but just because I *can* read it that way doesn't mean I *should*.
Whatever aids me in reading it, I prefer.  If that means I add
lines/whitespace to make it clearer, then I'll do that.  Worst case, can
"prettify" it when reading/editing, then rejoin to one single Uberline
when I'm done.


>>I go as far as to explicitly add an extra blank line at the end of any
>>text file (source code, html, batch files, email, config files, etc.),
>>so my text doesn't end up "missing" its last line.

>So "G" conveniently lands you on an empty line from where to start a
new
>paragraph.

There's that, too, but it's a minor issue.

I just like to be able to park on a blank line so I know it actually
ends there cleanly, vs getting to a line of text that abruptly ends,
with no EOL sequence, so I'm left guessing as to whether it was
unexpectedly truncated or actually *does* end there.  A friend was
sending me email that for some reason would end like that, and I'd
answer back what I saw, but he'd later ask why I didn't respond to any
points subsequent to that.  "Huh?  *Where*?"  So he got into the habit
of adding "---" as the last line of text so I'd know it actually ended
there.  My blank-line is my own equivalent of that.

--~--~---------~--~----~------------~-------~--~----~
You received this message from the "vim_use" maillist.
For more information, visit http://www.vim.org/maillist.php
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: noeol noendofline binary fileformat

Tony Mechelynck
In reply to this post by Michael Ludwig-3

Michael Ludwig wrote:

> Ben Schmidt schrieb am 10.08.2007 um 10:25:33 (+1000):
>>> First question: Is this the correct behaviour?
>> I think so, yes.
>>
>> Unix lineendings must be used in 'binary' mode, otherwise Vim also has
>> to remember whether each line had a CR or not, rather than just whether
>> to tack an LF at the end. As you have noticed, though, setting 'binary'
>> after loading a file, rather than before, will result in the line
>> endings changing.
>
> Maybe I'm on the wrong track here, but I used to think that "binary"
> implies there be no concept of a line ending - just bytes, which are to
> be kept as they are. Of course, when I hit the enter key to add new
> lines, this will have to be interpreted in a certain way, preferably in
> any one of the established conventions for line endings on various
> systems.
>
>>> Second question: What can I do to preserve both the "fileformat" and
>>> "noeol" properties?
>> It's a hack, but the best thing I can think of is to modify the
>> autocommands from the tip you found to insert the CRs as well as set
>> binary, simulating the effect you get when reading the file with -b:
>
> Your hack works - thanks a lot!
>
> Cheers, Michael

The problem is precisely that: keeping bytes as they are.

Vim interprets CRLF or LF (in "dos" fileformat mode), LF (in "unix" fileformat
mode) or CR (in "mac" fileformat mode) as a line ending. Notice than in dos
mode, reading LF-alone will write CR-LF, making dos mode unsuitable for binary
files. Happily, as said under ":help 'binary'", in binary mode the file is
always read like in "unix" fileformat mode.

In Unix mode, CRLF will be seen as a ^M at the end of a line. This is no
probem as long as you keep it unchanged.

Binary mode is meant for, for instance, changing a text string in a program,
to correct a typo or to translate to a different language. Binary programs
almost always have binary pointers (absolute or relative) to other places in
the program, so it is of primordial importance not to add or delete anything
which could make one part of the program move by respect to another. If you
did, you could end up, at best, with an "invalid opcode" exception when
running the program.

Using binary mode to force Vim to omit the last line's ending is a mistake: by
making sure that the last line has an end-of-line, Vim is doing "the right
thing", as can be seen by doing

(in a Windows shell such as cmd.exe) copy file1.txt+file2.txt result.txt
(or, in a Unix shell such as bash) cat file1.txt file2.txt > result.txt

If file1.txt had a missing end-of-line at the end, the last line of file1.txt
will be run into the first line of file2.txt, making them one long line
somewhere near the middle of result.txt.



Best regards,
Tony.
--
There is no right or wrong, there is only your personal opinion.
                  (Bram Moolenaar)

--~--~---------~--~----~------------~-------~--~----~
You received this message from the "vim_use" maillist.
For more information, visit http://www.vim.org/maillist.php
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: noeol noendofline binary fileformat

Michael Ludwig-3

Tony Mechelynck schrieb am 10.08.2007 um 21:09:51 (+0200):

>
> Michael Ludwig wrote:
> > Ben Schmidt schrieb am 10.08.2007 um 10:25:33 (+1000):
> >>> First question: Is this the correct behaviour?
> >> I think so, yes.
> >>
> >> Unix lineendings must be used in 'binary' mode, otherwise Vim also has
> >> to remember whether each line had a CR or not, rather than just whether
> >> to tack an LF at the end. As you have noticed, though, setting 'binary'
> >> after loading a file, rather than before, will result in the line
> >> endings changing.
> >
> > Maybe I'm on the wrong track here, but I used to think that "binary"
> > implies there be no concept of a line ending - just bytes, which are to
> > be kept as they are. Of course, when I hit the enter key to add new
> > lines, this will have to be interpreted in a certain way, preferably in
> > any one of the established conventions for line endings on various
> > systems.
> >
> >>> Second question: What can I do to preserve both the "fileformat" and
> >>> "noeol" properties?
> >> It's a hack, but the best thing I can think of is to modify the
> >> autocommands from the tip you found to insert the CRs as well as set
> >> binary, simulating the effect you get when reading the file with -b:
> >
> > Your hack works - thanks a lot!
> >
> > Cheers, Michael
>
> The problem is precisely that: keeping bytes as they are.
>
> Vim interprets CRLF or LF (in "dos" fileformat mode), LF (in "unix"
> fileformat mode) or CR (in "mac" fileformat mode) as a line ending.
> Notice than in dos mode, reading LF-alone will write CR-LF, making dos
> mode unsuitable for binary files. Happily, as said under ":help
> 'binary'", in binary mode the file is always read like in "unix"
> fileformat mode.
>
> In Unix mode, CRLF will be seen as a ^M at the end of a line. This is
> no probem as long as you keep it unchanged.
>
> Binary mode is meant for, for instance, changing a text string in a
> program, to correct a typo or to translate to a different language.
> Binary programs almost always have binary pointers (absolute or
> relative) to other places in the program, so it is of primordial
> importance not to add or delete anything which could make one part of
> the program move by respect to another. If you did, you could end up,
> at best, with an "invalid opcode" exception when running the program.

So let's edit a binary file that happens to contain a CR-LF sequence
using the -b switch to start VIM. Everything will be fine. But let's do
the same setting "binary" after opening the file (using ":e test.txt")
and on writing our CR-LF sequence is reduced to just LF, which might be
wrong.

What makes me wonder is that the "binary" option apparently does not
cause VIM to behave the same way as the -b switch.

> Using binary mode to force Vim to omit the last line's ending is a
> mistake: by making sure that the last line has an end-of-line, Vim is
> doing "the right thing", as can be seen by doing
>
> (in a Windows shell such as cmd.exe) copy file1.txt+file2.txt
> result.txt (or, in a Unix shell such as bash) cat file1.txt file2.txt
> > result.txt
>
> If file1.txt had a missing end-of-line at the end, the last line of
> file1.txt will be run into the first line of file2.txt, making them
> one long line somewhere near the middle of result.txt.

I agree all text files should have an eol at eof.

Regards, Michael

--~--~---------~--~----~------------~-------~--~----~
You received this message from the "vim_use" maillist.
For more information, visit http://www.vim.org/maillist.php
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: noeol noendofline binary fileformat

Michael Ludwig-3
In reply to this post by Gene Kwiecinski

Gene Kwiecinski schrieb am 10.08.2007 um 15:09:13 (-0400):

[...]

> >>I don't have an answer to your question, but if I may ask a question
> in
> >>return... why would it kill you to have a newline at the end of the
> >>file?  Those "tools" might deem it to be unnecessary, but does the
> >>trailing newline actually *break* them?
>
> >I wanted to reduce textual difference when editing source code
> featuring
> >"noeol" at EOF. Said tools (used by co-workers) would re-drop the eol I
>
> Even with an explicit blank line at the end?  Eerie...

To be honest, I don't know exactly. But I guess disposing of the eol at
eof is part of the PHP IDE's reformatting feature. Web browsers don't
need newlines, and the IDE doesn't either.

[...]

> >Another tool disrespecting 4+ decades of standard practice is the Saxon
> >XSLT processor. And probably many more XML tools. XML, as we know,
> >manages to be human-readable without old-fashioned crap such as lines.
>
> Yeah, but organisationally, it's nice to have readable indentation,
> etc., to be able to read it more clearly.  Eg, I can read sideways- and
> even inverted text, albeit more slowly than when it's oriented normally,
> but just because I *can* read it that way doesn't mean I *should*.
> Whatever aids me in reading it, I prefer.  If that means I add
> lines/whitespace to make it clearer, then I'll do that.  Worst case, can
> "prettify" it when reading/editing, then rejoin to one single Uberline
> when I'm done.

For all the human-readability, I think no one likes reading XML. It is
unpleasant for the eye. Even when you've learned the toolset.

For real readability, nothing beats plain text limited to a maximum of
80 columns.

Michael (my name - as another equivalent of a last line of text)

> >>I go as far as to explicitly add an extra blank line at the end of any
> >>text file (source code, html, batch files, email, config files, etc.),
> >>so my text doesn't end up "missing" its last line.
>
> >So "G" conveniently lands you on an empty line from where to start a
> new
> >paragraph.
>
> There's that, too, but it's a minor issue.
>
> I just like to be able to park on a blank line so I know it actually
> ends there cleanly, vs getting to a line of text that abruptly ends,
> with no EOL sequence, so I'm left guessing as to whether it was
> unexpectedly truncated or actually *does* end there.  A friend was
> sending me email that for some reason would end like that, and I'd
> answer back what I saw, but he'd later ask why I didn't respond to any
> points subsequent to that.  "Huh?  *Where*?"  So he got into the habit
> of adding "---" as the last line of text so I'd know it actually ended
> there.  My blank-line is my own equivalent of that.
>
>

--~--~---------~--~----~------------~-------~--~----~
You received this message from the "vim_use" maillist.
For more information, visit http://www.vim.org/maillist.php
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: noeol noendofline binary fileformat

Tony Mechelynck
In reply to this post by Michael Ludwig-3

Michael Ludwig wrote:

> Tony Mechelynck schrieb am 10.08.2007 um 21:09:51 (+0200):
>> Michael Ludwig wrote:
>>> Ben Schmidt schrieb am 10.08.2007 um 10:25:33 (+1000):
>>>>> First question: Is this the correct behaviour?
>>>> I think so, yes.
>>>>
>>>> Unix lineendings must be used in 'binary' mode, otherwise Vim also has
>>>> to remember whether each line had a CR or not, rather than just whether
>>>> to tack an LF at the end. As you have noticed, though, setting 'binary'
>>>> after loading a file, rather than before, will result in the line
>>>> endings changing.
>>> Maybe I'm on the wrong track here, but I used to think that "binary"
>>> implies there be no concept of a line ending - just bytes, which are to
>>> be kept as they are. Of course, when I hit the enter key to add new
>>> lines, this will have to be interpreted in a certain way, preferably in
>>> any one of the established conventions for line endings on various
>>> systems.
>>>
>>>>> Second question: What can I do to preserve both the "fileformat" and
>>>>> "noeol" properties?
>>>> It's a hack, but the best thing I can think of is to modify the
>>>> autocommands from the tip you found to insert the CRs as well as set
>>>> binary, simulating the effect you get when reading the file with -b:
>>> Your hack works - thanks a lot!
>>>
>>> Cheers, Michael
>> The problem is precisely that: keeping bytes as they are.
>>
>> Vim interprets CRLF or LF (in "dos" fileformat mode), LF (in "unix"
>> fileformat mode) or CR (in "mac" fileformat mode) as a line ending.
>> Notice than in dos mode, reading LF-alone will write CR-LF, making dos
>> mode unsuitable for binary files. Happily, as said under ":help
>> 'binary'", in binary mode the file is always read like in "unix"
>> fileformat mode.
>>
>> In Unix mode, CRLF will be seen as a ^M at the end of a line. This is
>> no probem as long as you keep it unchanged.
>>
>> Binary mode is meant for, for instance, changing a text string in a
>> program, to correct a typo or to translate to a different language.
>> Binary programs almost always have binary pointers (absolute or
>> relative) to other places in the program, so it is of primordial
>> importance not to add or delete anything which could make one part of
>> the program move by respect to another. If you did, you could end up,
>> at best, with an "invalid opcode" exception when running the program.
>
> So let's edit a binary file that happens to contain a CR-LF sequence
> using the -b switch to start VIM. Everything will be fine. But let's do
> the same setting "binary" after opening the file (using ":e test.txt")
> and on writing our CR-LF sequence is reduced to just LF, which might be
> wrong.
>
> What makes me wonder is that the "binary" option apparently does not
> cause VIM to behave the same way as the -b switch.

As stated under 'binary' in the help, the 'binary' option should be set either
before reading a file:

        :set binary
        :edit program.exe

or (recommended, but version 7 only) at the time of reading the file:

        :e ++bin program.exe

but not afterwards. Doing it this way has the same effect as the command-line
switch.

If you set it afterwards, the read-in is done with 'nobinary' and therefore
does not behave as with 'binary'.

>
>> Using binary mode to force Vim to omit the last line's ending is a
>> mistake: by making sure that the last line has an end-of-line, Vim is
>> doing "the right thing", as can be seen by doing
>>
>> (in a Windows shell such as cmd.exe) copy file1.txt+file2.txt
>> result.txt (or, in a Unix shell such as bash) cat file1.txt file2.txt
>>> result.txt
>> If file1.txt had a missing end-of-line at the end, the last line of
>> file1.txt will be run into the first line of file2.txt, making them
>> one long line somewhere near the middle of result.txt.
>
> I agree all text files should have an eol at eof.
>
> Regards, Michael

Best regards,
Tony.
--
God is a comic playing to an audience that's afraid to laugh.

--~--~---------~--~----~------------~-------~--~----~
You received this message from the "vim_use" maillist.
For more information, visit http://www.vim.org/maillist.php
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

Re: noeol noendofline binary fileformat

Ben Schmidt

> If you set it afterwards, the read-in is done with 'nobinary' and therefore
> does not behave as with 'binary'.

Hmm. I can see a lot of people being confused by this, actually,
although it is logical. I don't know if this is one of the times Vim
does such things, but perhaps it would be helpful to make it output a
warning when setting binary if the fileformat is not unix? This could
save people from some pretty nasty accidents due to wrong assumptions.

Ben.



Send instant messages to your online friends http://au.messenger.yahoo.com 


--~--~---------~--~----~------------~-------~--~----~
You received this message from the "vim_use" maillist.
For more information, visit http://www.vim.org/maillist.php
-~----------~----~----~----~------~----~------~--~---

Reply | Threaded
Open this post in threaded view
|

RE: noeol noendofline binary fileformat

JohnBeckett
In reply to this post by Ben Schmidt

Ben Schmidt wrote:
> Further updated, and changed name, which didn't quite work
> the way I thought it would, but there you have it...I presume
> if the result sucks someone can put it back...
>
> http://vim.wikia.com/wiki/Preserve_missing_end-of-line_at_end_
of_text_files

Thanks again. You've fixed the tip, and you've encouraged those working on
the Vim Tips wiki!

I fixed the redirection page so the following now points to your tip:

http://vim.wikia.com/wiki/VimTip1369

If you want to see what I mean, use the above link to open the tip. You will
see a small "(Redirected from VimTip1369)" link at the top of the tip. If
you click the VimTip1369 link you will see the redirect page.

> Gave it the 'fileformat' category but without doing anything
> to that category

That's good. If everyone thought "I can't fix this because I don't know how,
or I don't have time", not much would get done. I noticed that you added the
fileformat category, and I created that category. At the very bottom of the
tip page, you can now click the Fileformat link to see the result.

> I wonder whether that category is a bit too specific, though,
> to be all that useful--there can't be all that many tips
> related to it, surely, and the category namespace will just
> get huge if categories are so specific. At the same time it is
> a really good idea to have categories/keywords/something
> related to the Vim help tags. 'eol' or 'endofline' would also
> be an appropriate category with this in mind. And perhaps
> 'writing'.

Good points. I don't know. We're having a drive to put every tip in a
suitable category, and we may change our minds a few times.

John


--~--~---------~--~----~------------~-------~--~----~
You received this message from the "vim_use" maillist.
For more information, visit http://www.vim.org/maillist.php
-~----------~----~----~----~------~----~------~--~---