Lost file using Vim (first time ever in 15 years)

[Taylor Hedberg]

Toddintr, Sun 2012-05-13 @ 06:59:42-0700:

> Christian:
> > Besides, big red warning messages should always be read, because
> > they don't usually happen.
>
> This is the "Soviet" style of user interface design: it dictates *how*
> people should use a product, and ignores the real-life scenarios.
> (Much like the opening line of your message - "Please don't top poste
> and please wrap to 75 columns.")  If I *really* care about customer
> feedback, I don't care how the customer sends me feedback / it can be
> top-posted, bottom-posted, sideways.  What is important is that I am
> getting feedback about the worst thing that can happen w/ my product.

I agree with you that Vim could be improved in this regard. Certainly,
not losing data at all is preferable to losing data after a warning.

Understand, though, that contrary to what your message suggests, Vim is
not a product and you are not a customer. It is developed entirely by a
community of volunteers, who are not paid anything for their work. (If
you donated, then great! But that money went to charity; it is not
lining the pockets of any developers, not even Bram's.) So in short, the
community on this list does not owe you anything at all, and whatever
help is offered is offered out of good will alone.

So, when you show up on the list and start bashing the design of the
software (perhaps understandable in this case) and admit to ignoring
bold, red error messages which explained exactly what was about to
happen to your data, you aren't likely to win much support. What's more,
you defiantly refuse to adhere to the posting conventions of the mailing
list, which are essentially the same conventions followed by nearly
every FOSS list, despite the fact that "do not top-post" appears in the
signature of every posting here, and even after being explicitly asked
to by another contributor. Exactly what kind of response do you expect
to receive?

The most constructive way to improve open-source software of this type
is to make the improvements yourself and submit patches. That's how the
real work gets done. If no one else is independently motivated to work
on the problematic behavior, and you are unable or unwilling to do it
yourself, then the only way you are going to get support is by
interacting with the community in a way that makes other people want to
help you. Only then is anyone likely to consider spending a portion of
their own personal time and energy addressing your complaint. With a
less entitled attitude, you would probably find that most people here
are very friendly and eager to help.

[Christian Brabandt]

Hi Toddintr!

On So, 13 Mai 2012, Toddintr wrote:

> > The copy is still in the buffer. It is valid. The problem is, that
> > writing failed because converting to your desired fileencoding failed.
> >
> > The problem is probably, that Vim opened the file for writing, truncated
> > it and when writing noticed the error. But at this time, the old
> > contents are already gone.
>
> This explains the current way Vim handles the process.  It does not
> have to be this way.  This is what I mean by bad design.  Loss of data
> should only happen in an unforeseen situation (a crash, etc).  If you
> are truncating a file, especially prior to a conversion that can fail,
> you should take extra precautions.

Yes true. I didn't look it up in the code, may be, Vim can indeed be
improved in this area. Where is your patch?

>
> > Besides, big red warning messages should always be read, because they
> > don't usually happen.
>

> This is the "Soviet" style of user interface design: it dictates *how*
> people should use a product, and ignores the real-life scenarios.

You just have yourself disqualified for a serious discussion. I bet, you
know nothing about the Soviet Union, and I doubt you can compare. BTW: I
used to live in a country, that was occupied by them, so I know how it
was back then.

> (Much like the opening line of your message - "Please don't top poste
> and please wrap to 75 columns.")  If I *really* care about customer
> feedback, I don't care how the customer sends me feedback / it can be
> top-posted, bottom-posted, sideways.  What is important is that I am
> getting feedback about the worst thing that can happen w/ my product.

I am sorry, I must have missed the fact that you are going to pay us.
You are not? Oh, well, than you shouldn't be acting like we owe you
something. Pay me, and I listen to everything you are saying and in any
style. I won't complain.

Even better, donate to the ICCF!

> I am done w/ this issue, thank you all for your responses.

So am I.

regards,
Christian

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

[Paul-425]

On Sunday, 13 May, 2012 at 13:31:38 BST, Christian Brabandt wrote:
>You get a warning, that writing failed and the file may possibly be
>corrupt. Vim even tells you, what you should do to write without
>converting the content. What else should Vim do?
>
>I don't understand, what Vim should possibly restore?

I think you're both right. He should have read the warning message, but also I'm surprised that vim modified the file before being given any instruction to write it. Usually, when you open a file in vim (or any editor) and start making changes, you can cat the file and see that it hasn't been modified - until told to save it (manually or autosave) - so I'm a bit surprised that vim actually modified the file in this scenario. I'd have expected it to read it into memory, convert it (in memory) if necessary, and not save it until told to do so. So it's not a matter of restoring anything, but rather one of not overwriting in the first place, because no write command was issued.

--

.

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

[John Beckett-2]

Paul wrote:
> I think you're both right. He should have read the warning
> message, but also I'm surprised that vim modified the file
> before being given any instruction to write it.

The discussion got very ranty earlier (folks, there is no need
to kick the guy, just be glad it wasn't you), and I ignored it.

Can someone write a clean statement of the problem (the file
contents and the commands that led to the behaviour that was
unexpected). Just the facts, without opinions on Vim or users.

John

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

[Ben Fritz]

On Sunday, May 13, 2012 3:10:33 AM UTC-5, Toddintr wrote:
> For the first time ever since I started using Vim, I lost a file.  I had
> a modeline that specified an "encoding" setting for a Python script.
> Vim complained when opening the file.  I googled for a solution, decided
> to try fileencoding instead of encoding.  When saving the file, I saw
> some error/warning messages, but went ahead with a ZZ or w! (don't quiet
> remember).  Result: File length of zero.  (Had backups disabled.)
>

Later you say you used :q! and lost data. Here I see you actually saved the
file with a "force save and quit" and lost data. That's a big difference,
but I admit somewhat irrelevant to the discussion, which has gotten quite
out of hand. I understand you're frustrated. Let's all settle down, we may
have a real issue here (which may have been fixed since 7.3.0, of course).
Regardless, YOU (the original poster) are not going to get anywhere by
outright insulting long-time and very respected members of the Vim
community.

I think I gather from later posts that the problem occurs when you set
'encoding' but not 'fileencoding' from a modeline.

:help 'encoding' says:

        This option cannot be set from a |modeline|.  It would most likely
        corrupt the text.

:help changed-7.3 indicates this protection was added sometime between 7.2
and 7.3, but I don't see it in the changelog for 7.2 patches, so apparently
sometime in the run-up for 7.3 somebody realized setting Vim's internal
encoding via modeline is a very bad idea and fixed this issue. However, your
followup post with version information indicates that you should not have
been able to set the encoding via modeline.

On my Windows Vim, if I successfully save a file with a modeline containing
"encoding=cp857", I get an error when loading the file:

E520: Not allowed in a madeline: encoding=cp857

Setting fileencoding in a modeline, by the way, will not do what you
probably intend. A modeline is processed only AFTER a file is read into Vim,
so setting it in the modeline can only affect how the file is written, it
will NOT affect how Vim interprets the bytes in the file while reading. To
do this, you would need to properly set 'fileencodings' (note the final 's'
in the option name) to detect your desired encoding, PRIOR to reading the
file. A BufReadPre autocmd is a good place to do this, or just have it
always set properly. Another option, if you know the file's encoding, is to
use a ++enc=cp857 argument when editing the file, e.g.

:edit ++enc=cp857 myfile.txt

Also see the AutoFenc plugin:

http://www.vim.org/scripts/script.php?script_id=2721

Anyway, that was slightly off topic. I tried reproducing the error with your
example file, and could not. No matter what I did, without changing text
within the buffer, I was unable to wind up with an empty file by saving
and/or quitting, with or without a !. :wq! doesn't even work for me with a
character in the file invalid for the current 'fileencoding' (e.g. when I
load the file containing the undotted 'i' in your example, without Vim
detecting the correct encoding, and then try to write in the correct
encoding without correcting the character first), it fails to write and
fails to quit, since Vim doesn't know HOW to write. Can you give the exact
commands you execute to result in an empty file? And, what is the initial
value of 'encoding'? And your 'fileencodings' option? Finally, this is
possibly a stupid question, but are you sure you lost your data in the 7.3.0
version of Vim you say you are using? Maybe you accidentally loaded the file
in an older version you still have installed alongside somewhere?

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

[Sergey Khorev]

> Can someone write a clean statement of the problem (the file
> contents and the commands that led to the behaviour that was
> unexpected). Just the facts, without opinions on Vim or users.

This is how I managed to reproduce. Create a file:
----------- snip -----------
# vim: set fileencoding=cp857:

ı
----------- snip -----------

Open it in Vim, edit, then
:set nobackup nowritebackup
:w!

Get the warning and then check the file got truncated.

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

[Christian Brabandt]

Bram,
On Mo, 14 Mai 2012, Sergey Khorev wrote:

> > Can someone write a clean statement of the problem (the file
> > contents and the commands that led to the behaviour that was
> > unexpected). Just the facts, without opinions on Vim or users.
>
> This is how I managed to reproduce. Create a file:
> ----------- snip -----------
> # vim: set fileencoding=cp857:
>
> ı
> ----------- snip -----------
>
> Open it in Vim, edit, then
> :set nobackup nowritebackup
> :w!
>
> Get the warning and then check the file got truncated.

To make this reproducible, I needed to set fenc=ascii

This patch fixes a data conversion error, that leads to a data loss.
Problem is, that open() is done, even though it is not clear, conversion
will work. So this patch runs buf_write_bytes() twice, first time is to
check that each character can be converted successfully and only if this
does not return, does it really open the file and write the result.

Problem is, this will probably slow down writing speed, as
buf_write_bytes needs to be called twice.

I have tested it with the given sample and it works here.

regards,
Christian

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

[Gary Johnson-4]

On 2012-05-14, Christian Brabandt wrote:

> This patch fixes a data conversion error, that leads to a data loss.
> Problem is, that open() is done, even though it is not clear, conversion
> will work. So this patch runs buf_write_bytes() twice, first time is to
> check that each character can be converted successfully and only if this
> does not return, does it really open the file and write the result.
>
> Problem is, this will probably slow down writing speed, as
> buf_write_bytes needs to be called twice.
>
> I have tested it with the given sample and it works here.

How much slower is it for a file large enough that the write times
are more than a second?

I don't think we want to noticeably slow down Vim for everyone all
the time in order to more-conveniently avoid an error that is
observed once in 15 years.

Regards,
Gary

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

[Ben Fritz]

On Monday, May 14, 2012 1:34:52 PM UTC-5, Gary Johnson wrote:
>
> How much slower is it for a file large enough that the write times
> are more than a second?
>
> I don't think we want to noticeably slow down Vim for everyone all
> the time in order to more-conveniently avoid an error that is
> observed once in 15 years.
>

We have this situation:

1. user opens a file
2. (user does some edits)
3. user tries to write the file, gets a conversion error
4. user uses :q!, attempting to abandon all changes
5. user loses ENTIRE CONTENTS of file

I think I'd accept a slow-down in this case!

Granted, the user should have paid attention to the big message saying "don't quit the editor until the file is successfully written!", however, it may not be clear in the slightest how to successfully write the file. What if somebody is editing a huge file (and thus has undolevel set really low), pastes several hundred lines of text from outside Vim, within which a single character is invalid for their current encoding, and then realizes they cannot write the buffer? My first instinct would be to abandon all changes, but apparently that's not going to work here.

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

[Andy Wokula]

Am 14.05.2012 20:21, schrieb Christian Brabandt:

> Bram,
> On Mo, 14 Mai 2012, Sergey Khorev wrote:
>>> Can someone write a clean statement of the problem (the file
>>> contents and the commands that led to the behaviour that was
>>> unexpected). Just the facts, without opinions on Vim or users.
>>
>> This is how I managed to reproduce. Create a file:
>> ----------- snip -----------
>> # vim: set fileencoding=cp857:
>>
>> i
>> ----------- snip -----------
>>
>> Open it in Vim, edit, then
>> :set nobackup nowritebackup
>> :w!
>>
>> Get the warning and then check the file got truncated.

Thanks, was hard enough to get this error message working ^^

btw, s/warning/error/

> To make this reproducible, I needed to set fenc=ascii
>
> This patch fixes a data conversion error, that leads to a data loss.
> Problem is, that open() is done, even though it is not clear, conversion
> will work. So this patch runs buf_write_bytes() twice, first time is to
> check that each character can be converted successfully and only if this
> does not return, does it really open the file and write the result.
>
> Problem is, this will probably slow down writing speed, as
> buf_write_bytes needs to be called twice.
>
> I have tested it with the given sample and it works here.
>
> regards,
> Christian

Setting 'writebackup' (enabled per default) indeed protects the file
(except in the temp folder, where 'backupskip' applies).

This option creates temporary files only, thus doesn't waste disk space.

Isn't this safe enough?

Am 13.05.2012 10:10, schrieb Toddintr:
>   (Had backups disabled.)

Bad luck ... next time, keep 'writebackup' at the default and all should
be fine.

--
Andy

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

[Gary Johnson-4]

On 2012-05-14, Ben Fritz wrote:

> On Monday, May 14, 2012 1:34:52 PM UTC-5, Gary Johnson wrote:
> >
> > How much slower is it for a file large enough that the write times
> > are more than a second?
> >
> > I don't think we want to noticeably slow down Vim for everyone all
> > the time in order to more-conveniently avoid an error that is
> > observed once in 15 years.
> >
>
> We have this situation:
>
> 1. user opens a file
> 2. (user does some edits)
> 3. user tries to write the file, gets a conversion error
> 4. user uses :q!, attempting to abandon all changes
> 5. user loses ENTIRE CONTENTS of file
>
> I think I'd accept a slow-down in this case!

If a slow-down was the only solution, sure.  I had the impression
from the earlier discussion that the problem was avoidable by
reading the error message and being careful.  Hence my description
of the fix as one that would allow the error to be more conveniently
avoided.

> Granted, the user should have paid attention to the big message
> saying "don't quit the editor until the file is successfully
> written!", however, it may not be clear in the slightest how to
> successfully write the file. What if somebody is editing a huge
> file (and thus has undolevel set really low), pastes several
> hundred lines of text from outside Vim, within which a single
> character is invalid for their current encoding, and then realizes
> they cannot write the buffer? My first instinct would be to
> abandon all changes, but apparently that's not going to work here.

As I understand the problem, they also had to have 'nobackup' and
'nowritebackup' set, the latter which is not the default.  So the
user threw away the safety net, disregarded the warning sign, and
because of this I have to wait longer each time I write a file?

Of course, if the slowdown is negligible, then there's probably no
reason not to go with Christian's change.

Regards,
Gary

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

[Christian Brabandt]

Hi Gary!

On Mo, 14 Mai 2012, Gary Johnson wrote:

> On 2012-05-14, Christian Brabandt wrote:
>
> > This patch fixes a data conversion error, that leads to a data loss.
> > Problem is, that open() is done, even though it is not clear, conversion
> > will work. So this patch runs buf_write_bytes() twice, first time is to
> > check that each character can be converted successfully and only if this
> > does not return, does it really open the file and write the result.
> >
> > Problem is, this will probably slow down writing speed, as
> > buf_write_bytes needs to be called twice.
> >
> > I have tested it with the given sample and it works here.
>
> How much slower is it for a file large enough that the write times
> are more than a second?
>
> I don't think we want to noticeably slow down Vim for everyone all
> the time in order to more-conveniently avoid an error that is
> observed once in 15 years.

A small test suggests, that for a 15M file, writing happens here within
0,44s while the patched version takes 1,08s. For smaller files, the time
is negligible, but I fear, for larger files, it is quite noticeable.

Here is another test with a 40B, 1K, 4K, 50K, 500K, 1M, 15M, 70M files:

Size Run Patched Non-patched
40B 1 0,096s 0,089s
        2 0,084s 0,114s
        3 0,108s 0,099s
        Ø 0,096s 0,101s
1K 1 0,085s 0,080s
        2 0,108s 0,079s
        3 0,107s 0,099s
        Ø 0,100s 0,086s
4K 1 0,096s 0,072s
        2 0,096s 0,089s
        3 0,087s 0,089s
        Ø 0,093s 0,083s
50K 1 0,117s 0,082s
        2 0,148s 0,114s
        3 0,164s 0,088s
        Ø 0,143s 0,095s
100K 1 0,138s 0,102s
        2 0,160s 0,079s
        3 0,137s 0,090s
        Ø 0,145s 0,090s
500K 1 0,159s 0,102s
        2 0,185s 0,083s
        3 0,186s 0,113s
        Ø 0,177s 0,099s
1M 1 0,216s 0,089s
        2 0,233s 0,108s
        3 0,206s 0,113s
        Ø 0,218s 0,103s
15M 1 1,098s 0,212s
        2 1,089s 0,110s
        3 1,081s 0,097s
        Ø 1,089s 0,140s
70M 1 6,809s 0,097s
        2 7,160s 0,193s
        3 0,699s 0,097s
        Ø 4,889s 0,129s

Here the non-patched versions are a lot faster, because the bug is
triggered at the beginning of the filewrite, and writing is aborted
afterwards, While the patched version try twice.

Here is a small update to the patch: break in case of errors.
But perhaps, it isn't worth the trouble, since 'writebackup' prevents
that bug.

regards,
Christian

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

[John Beckett-2]

To reproduce problem:

1. Create file bad.tmp by executing the following line.
:call writefile(["Very important file \x8D"], 'bad.tmp')

2. Start Vim editing the file and enter commands, as below.
vim -N -u NONE bad.tmp
:set fenc=cp857
:set nobackup nowritebackup
:w

3. Observe message which you have probably never seen before:
"bad.tmp" E513: write error, conversion failed
    (make 'fenc' empty to override)
WARNING: Original file may be lost or damaged
don't quit the editor until the file is successfully written!

4. Panic and do what has always worked in the past:
:q!

Result:
- Step 2 replaces bad.tmp with an empty file.
- The original contents have been lost.

The "no backup" options cause Vim to overwrite the existing
file. However, the write aborts when the conversion fails,
resulting in lost data.

While a user should not turn off backups on an important file,
it is a horrible trap that one incorrect byte can effectively
delete a file.

A message like the following might help:

    "bad.tmp" E513: write error, conversion failed
    WARNING: The file on disk has probably been corrupted.
    Do not quit until the file is successfully written!
    Try saving it again after entering (nothing after "="):
    :set fenc=

I haven't done more than glance at Christian Brabandt's patch
so I don't know what is achievable. Would something as simple
as checking the number of bytes in the file on disk be
sufficient to decide if corrective action is needed?

John

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

[Ben Fritz]

On Tuesday, May 15, 2012 2:49:38 AM UTC-5, JohnBeckett wrote:
> To reproduce problem:
>
> 1. Create file bad.tmp by executing the following line.
> :call writefile(["Very important file \x8D"], 'bad.tmp')
>
> 2. Start Vim editing the file and enter commands, as below.
> vim -N -u NONE bad.tmp
> :set fenc=cp857

What if, right here, Vim were to check the buffer contents to see if all characters in the file can actually be converted to the new fenc setting, and issue a warning if not? It's not something Vim does currently...but I don't see why it couldn't. And I'd like to know if I do something stupid *when I do it*, not when I try to write the file and end up killing my file.

> :set nobackup nowritebackup
> :w
>
> 3. Observe message which you have probably never seen before:
> "bad.tmp" E513: write error, conversion failed
>     (make 'fenc' empty to override)
> WARNING: Original file may be lost or damaged
> don't quit the editor until the file is successfully written!
>
> 4. Panic and do what has always worked in the past:
> :q!
>

Right, to me, :q! means "oh crap I didn't mean to do that, GET ME OUT!"

> Result:
> - Step 2 replaces bad.tmp with an empty file.
> - The original contents have been lost.
>
> The "no backup" options cause Vim to overwrite the existing
> file. However, the write aborts when the conversion fails,
> resulting in lost data.
>
> While a user should not turn off backups on an important file,
> it is a horrible trap that one incorrect byte can effectively
> delete a file.
>
> A message like the following might help:
>
>     "bad.tmp" E513: write error, conversion failed
>     WARNING: The file on disk has probably been corrupted.
>     Do not quit until the file is successfully written!
>     Try saving it again after entering (nothing after "="):
>     :set fenc=
>
> I haven't done more than glance at Christian Brabandt's patch
> so I don't know what is achievable. Would something as simple
> as checking the number of bytes in the file on disk be
> sufficient to decide if corrective action is needed?
>
> John

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

[Ben Fritz]

On Monday, May 14, 2012 2:39:54 PM UTC-5, Gary Johnson wrote:
>
> As I understand the problem, they also had to have 'nobackup' and
> 'nowritebackup' set, the latter which is not the default.  So the
> user threw away the safety net, disregarded the warning sign, and
> because of this I have to wait longer each time I write a file?
>
> Of course, if the slowdown is negligible, then there's probably no
> reason not to go with Christian's change.
>

Since the error only occurs if 'backup' and 'writebackup' are both off, maybe we could only do the "dry run" conversion in this situation? I.e., if either 'backup' or 'writebackup' are set, do what Vim has always done; but if neither are set, do the extra check first. We could even put a nice notice in :help 'writebackup' saying "if this option is off, writing the file will take longer due to extra safety checks" to discourage turning it off in the first place.

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

[shawn wilson]

This sounds sane.

On May 15, 2012 11:20 AM, "Ben Fritz" <[hidden email]> wrote:

On Tuesday, May 15, 2012 2:49:38 AM UTC-5, JohnBeckett wrote:
> To reproduce problem:
>
> 1. Create file bad.tmp by executing the following line.
> :call writefile(["Very important file \x8D"], 'bad.tmp')
>
> 2. Start Vim editing the file and enter commands, as below.
> vim -N -u NONE bad.tmp
> :set fenc=cp857

What if, right here, Vim were to check the buffer contents to see if all characters in the file can actually be converted to the new fenc setting, and issue a warning if not? It's not something Vim does currently...but I don't see why it couldn't. And I'd like to know if I do something stupid *when I do it*, not when I try to write the file and end up killing my file.

> :set nobackup nowritebackup
> :w
>
> 3. Observe message which you have probably never seen before:
> "bad.tmp" E513: write error, conversion failed
>     (make 'fenc' empty to override)
> WARNING: Original file may be lost or damaged
> don't quit the editor until the file is successfully written!
>
> 4. Panic and do what has always worked in the past:
> :q!
>

Right, to me, :q! means "oh crap I didn't mean to do that, GET ME OUT!"

> Result:
> - Step 2 replaces bad.tmp with an empty file.
> - The original contents have been lost.
>
> The "no backup" options cause Vim to overwrite the existing
> file. However, the write aborts when the conversion fails,
> resulting in lost data.
>
> While a user should not turn off backups on an important file,
> it is a horrible trap that one incorrect byte can effectively
> delete a file.
>
> A message like the following might help:
>
>     "bad.tmp" E513: write error, conversion failed
>     WARNING: The file on disk has probably been corrupted.
>     Do not quit until the file is successfully written!
>     Try saving it again after entering (nothing after "="):
>     :set fenc=
>
> I haven't done more than glance at Christian Brabandt's patch
> so I don't know what is achievable. Would something as simple
> as checking the number of bytes in the file on disk be
> sufficient to decide if corrective action is needed?
>
> John

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

[Tim Chase]

On 05/15/12 10:20, Ben Fritz wrote:

> On Tuesday, May 15, 2012 2:49:38 AM UTC-5, JohnBeckett wrote:
>> 1. Create file bad.tmp by executing the following line.
>> :call writefile(["Very important file \x8D"], 'bad.tmp')
>>
>> 2. Start Vim editing the file and enter commands, as below.
>> vim -N -u NONE bad.tmp
>> :set fenc=cp857
>
> What if, right here, Vim were to check the buffer contents to see
> if all characters in the file can actually be converted to the
> new fenc setting, and issue a warning if not? It's not something
> Vim does currently...but I don't see why it couldn't. And I'd
> like to know if I do something stupid *when I do it*, not when I
> try to write the file and end up killing my file.

I like Ben's suggestion on this one.  Concerns that come to mind:

- warnings might interrupt scripts that run

- how it would interact with the "let &setting=" syntax

- would this just be a message (that gets ignored?) or a prompted
"do you want to undo this catastrophically dangerous action?"


>> 4. Panic and do what has always worked in the past:
>> :q!
>>
>
> Right, to me, :q! means "oh crap I didn't mean to do that, GET ME OUT!"

The caveat here is that, if you have performed an action that might
have fouled up the file-on-disk, your only valid source might be
what is in Vim's buffer.  So if you get a warning, think hard before
attempting to quit.  E.g.

  :e important.txt
  :!rm %

get "E211: File "important.txt" no longer available" message.  Panic and

  :q!

Clearly, vim shouldn't try to override what you expressly did (I
wouldn't want Vim to re-save the file to the disk).  In this case,
even ":q" abandon the content without complaint.

-tim





--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

[Christian Brabandt]

On Tue, May 15, 2012 17:20, Ben Fritz wrote:
> What if, right here, Vim were to check the buffer contents to see if all
> characters in the file can actually be converted to the new fenc setting,
> and issue a warning if not? It's not something Vim does currently...but I
> don't see why it couldn't. And I'd like to know if I do something stupid
> *when I do it*, not when I try to write the file and end up killing my
> file.

That doesn't work, you could still afterwards enter glyphs in utf8, that
can't be converted into the fileencoding.

regards,
Christian

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

[Ben Fritz]

On Tue, May 15, 2012 at 10:50 AM, Christian Brabandt <[hidden email]> wrote:

> On Tue, May 15, 2012 17:20, Ben Fritz wrote:
>> What if, right here, Vim were to check the buffer contents to see if all
>> characters in the file can actually be converted to the new fenc setting,
>> and issue a warning if not? It's not something Vim does currently...but I
>> don't see why it couldn't. And I'd like to know if I do something stupid
>> *when I do it*, not when I try to write the file and end up killing my
>> file.
>
> That doesn't work, you could still afterwards enter glyphs in utf8, that
> can't be converted into the fileencoding.
>

This is true, but I'm more likely to remember not to enter glyphs
which can't be encoded in the current fenc, than I am to know that
these glyphs already exist in the buffer when changing the encoding. I
think I'd still like to be warned when I change file encoding.

Then again...since you point this out, it won't help in preventing the
error that started this thread, if the "user does something" step is
not to change file encoding, but rather to insert characters which
cannot be encoded.

I think the better idea is to do what your patch does, but only if
'writebackup' is disabled to prevent a slow-down when there is already
an existing mechanism to prevent data loss.

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

[Chris Jones-44]

On Tue, May 15, 2012 at 11:23:57AM EDT, Ben Fritz wrote:

> Since the error only occurs if 'backup' and 'writebackup' are both
> off, maybe we could only do the "dry run" conversion in this
> situation? I.e., if either 'backup' or 'writebackup' are set, do what
> Vim has always done; but if neither are set, do the extra check first.
> We could even put a nice notice in :help 'writebackup' saying "if this
> option is off, writing the file will take longer due to extra safety
> checks" to discourage turning it off in the first place.

So you would penalize those of us who turned off this backup/writebackup
bloat..?

Or would you force us to reinstate it so as not to be penalized..?

Aha... you turned off backup and writebackup.. you wicked little
so-and-so.. You seriously thought you were going get away with it..?

I made the decision to turn off this stuff years ago because when I'm
bleary-eyed not wide-awake or fully in control getting back to what
I was doing the night before, the very last thing I want to see when
I fire up Vim is a confusing message about some .swp file and do I want
to recover (r).. etc. The _only_ time I came close to losing _anything_
in Vim was when I almost gave the wrong answer.

I don't believe in a text editor backing up the the data that it
manipulates anyway. I prefer relying on important data being backed up
by a program whose job it is to back up data. Not perfect I grant you,
but it does have the merit of doing its stuff while I sleep and let me
do my editing unmolested.

Now suppose for a minute Christians's (or s/o else's..) patch causes
another data loss issue down the road.. Or any other problem.? Is the
next guy in line going to add another defensive patch on top of it..?
and another.. and another..? Where does it stop..?

No big deal anyway.. What does bother me is what on earth was in that
precious file that the OP (obnoxious poster) lost..?

AFAICT, World War III hasn't broken out yet but I'm holding my breath.

CJ

--
Focus follow mouse users will burn in hell!!!

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php