Vim surprisingly slow?

classic Classic list List threaded Threaded
28 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: Vim surprisingly slow?

Dominique Pellé
 Christian Brabandt <[hidden email]> wrote:

> On Mi, 15 Nov 2017, Dominique Pellé wrote:
>
> > As highlighted in red, notice that we only write 179,281 bytes to file
> > "20150425.txt" but we open/write/close/fsync 5004 times.
> > That repeated I/O patterns kills performance, especially because
> > of the frequent fsync().
> >
> > We also read 1,612,897 bytes from data.json, but that should be
> > fast since we do only 27 read and opened that file only twice
> > according to output of io.pl. Vim reads file deta.json by chunks
> > of 64 KB. 1,612,897 bytes is exactly the file size of data.json.
> >
> > Maybe we can change vim to avoid the constant open/write/close/fsync
> > on file "20150425.txt" which should speed it up greatly.
>
> Thanks for this nice analysis. It is probably the fsync call that is
> slowing Vim down. We can avoid the fsync by disabling the 'fsync'
> option. Also we might want to disable the writebackup option just in
> case it matters. And finally, when a file is written in a different
> encoding, Vim will try to check if conversion is safe by first trying to
> convert the buffer contents to the fileencoding before writing the file.
> That might make also a difference when encoding differs from the file
> encoding.
>
> Perhaps it makes sense to skip fsyncing while being busy in a `:g`
> command. Tim, does this patch make a difference?
>
> diff --git a/src/fileio.c b/src/fileio.c
> index d991822c3..eb626a675 100644
> --- a/src/fileio.c
> +++ b/src/fileio.c
> @@ -4731,7 +4731,7 @@ restore_backup:
>          * work (could be a pipe).
>          * If the 'fsync' option is FALSE, don't fsync().  Useful for laptops.
>          */
> -       if (p_fs && fsync(fd) != 0 && !device)
> +       if (p_fs && !global_busy && fsync(fd) != 0 && !device)
>         {
>             errmsg = (char_u *)_("E667: Fsync failed");
>             end = 0;


Hi Christian

I measured the same command twice before and after your patch:

Before patch:

$ time ./vim  -u NONE data.json  -S new_engine.vim
real    0m15.533s
user    0m1.648s
sys    0m0.824s

$ time ./vim  -u NONE data.json  -S new_engine.vim

real    0m15.444s
user    0m1.636s
sys    0m0.820s

After patch:

$ time ./vim  -u NONE data.json  -S new_engine.vim

real    0m0.741s
user    0m0.536s
sys    0m0.200s

$ time ./vim  -u NONE data.json  -S new_engine.vim

real    0m0.731s
user    0m0.548s
sys    0m0.184s

So your patch makes it more than 20 times faster
on my Linux machine. Nice! Hopefully it's safe to
avoid fsync() in this case.

Regards
Dominique

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Vim surprisingly slow?

Nikolay Aleksandrovich Pavlov
In reply to this post by Tim Chase
2017-11-15 16:57 GMT+03:00 Tim Chase <[hidden email]>:

> On 2017-11-15 08:25, Christian Brabandt wrote:
>> On Mi, 15 Nov 2017, Dominique Pellé wrote:
>> > As highlighted in red, notice that we only write 179,281 bytes to
>> > file "20150425.txt" but we open/write/close/fsync 5004 times.
>> > That repeated I/O patterns kills performance, especially because
>> > of the frequent fsync().
> [snip]
>> > Maybe we can change vim to avoid the constant
>> > open/write/close/fsync on file "20150425.txt" which should speed
>> > it up greatly.
>
> However, my understanding is that the writefile() method should be
> doing the same thing (open, append, close) and that's as fast as I
> expect.  Though I could be wrong and writefile() might internally
> cache the file handle.
>
>> Thanks for this nice analysis. It is probably the fsync call that
>> is slowing Vim down. We can avoid the fsync by disabling the
>> 'fsync' option.
>
> Does the writefile() method does an fsync() too?  If not, it's a
> strong indicator that the fsync() may be part of the guilty slowness.

Does not, not in Vim. In Neovim this is governed by both &fsync option
(by default it is respected) and additional s/S flags pair which
allows one to select a bit safer option vs faster option.

>
> If the writefile() version does an fsync() we may be barking up the
> wrong tree.
>
> I'll test the patch later today when I'm not about to run out the
> door.
>
> Thanks!
>
> -tim
>
>
>
> --
> --
> You received this message from the "vim_use" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> ---
> You received this message because you are subscribed to the Google Groups "vim_use" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> For more options, visit https://groups.google.com/d/optout.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Vim surprisingly slow?

Dominique Pellé
In reply to this post by Dominique Pellé
Dominique Pellé <[hidden email]> wrote:

>  Christian Brabandt <[hidden email]> wrote:
>
>> On Mi, 15 Nov 2017, Dominique Pellé wrote:
>>
>> > As highlighted in red, notice that we only write 179,281 bytes to file
>> > "20150425.txt" but we open/write/close/fsync 5004 times.
>> > That repeated I/O patterns kills performance, especially because
>> > of the frequent fsync().
>> >
>> > We also read 1,612,897 bytes from data.json, but that should be
>> > fast since we do only 27 read and opened that file only twice
>> > according to output of io.pl. Vim reads file deta.json by chunks
>> > of 64 KB. 1,612,897 bytes is exactly the file size of data.json.
>> >
>> > Maybe we can change vim to avoid the constant open/write/close/fsync
>> > on file "20150425.txt" which should speed it up greatly.
>>
>> Thanks for this nice analysis. It is probably the fsync call that is
>> slowing Vim down. We can avoid the fsync by disabling the 'fsync'
>> option. Also we might want to disable the writebackup option just in
>> case it matters. And finally, when a file is written in a different
>> encoding, Vim will try to check if conversion is safe by first trying to
>> convert the buffer contents to the fileencoding before writing the file.
>> That might make also a difference when encoding differs from the file
>> encoding.
>>
>> Perhaps it makes sense to skip fsyncing while being busy in a `:g`
>> command. Tim, does this patch make a difference?
>>
>> diff --git a/src/fileio.c b/src/fileio.c
>> index d991822c3..eb626a675 100644
>> --- a/src/fileio.c
>> +++ b/src/fileio.c
>> @@ -4731,7 +4731,7 @@ restore_backup:
>>          * work (could be a pipe).
>>          * If the 'fsync' option is FALSE, don't fsync().  Useful for laptops.
>>          */
>> -       if (p_fs && fsync(fd) != 0 && !device)
>> +       if (p_fs && !global_busy && fsync(fd) != 0 && !device)
>>         {
>>             errmsg = (char_u *)_("E667: Fsync failed");
>>             end = 0;
>
>
> Hi Christian
>
> I measured the same command twice before and after your patch:
>
> Before patch:
>
> $ time ./vim  -u NONE data.json  -S new_engine.vim
> real    0m15.533s
> user    0m1.648s
> sys    0m0.824s
>
> $ time ./vim  -u NONE data.json  -S new_engine.vim
>
> real    0m15.444s
> user    0m1.636s
> sys    0m0.820s
>
> After patch:
>
> $ time ./vim  -u NONE data.json  -S new_engine.vim
>
> real    0m0.741s
> user    0m0.536s
> sys    0m0.200s
>
> $ time ./vim  -u NONE data.json  -S new_engine.vim
>
> real    0m0.731s
> user    0m0.548s
> sys    0m0.184s
>
> So your patch makes it more than 20 times faster
> on my Linux machine. Nice! Hopefully it's safe to
> avoid fsync() in this case.

I have not had the time to look in details, but the
patch is wrong.  After patch, running the command with
strace, I do not see any fsync() calls anymore.  I thought
that the idea of the patch was to avoid intermediate fsync()
but still fsync() at the end of the command.  But since
there is no fsync() call anymore, it's not safe.

Regards
Dominique

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Vim surprisingly slow?

Nikolay Aleksandrovich Pavlov
2017-11-16 8:43 GMT+03:00 Dominique Pellé <[hidden email]>:

> Dominique Pellé <[hidden email]> wrote:
>
>>  Christian Brabandt <[hidden email]> wrote:
>>
>>> On Mi, 15 Nov 2017, Dominique Pellé wrote:
>>>
>>> > As highlighted in red, notice that we only write 179,281 bytes to file
>>> > "20150425.txt" but we open/write/close/fsync 5004 times.
>>> > That repeated I/O patterns kills performance, especially because
>>> > of the frequent fsync().
>>> >
>>> > We also read 1,612,897 bytes from data.json, but that should be
>>> > fast since we do only 27 read and opened that file only twice
>>> > according to output of io.pl. Vim reads file deta.json by chunks
>>> > of 64 KB. 1,612,897 bytes is exactly the file size of data.json.
>>> >
>>> > Maybe we can change vim to avoid the constant open/write/close/fsync
>>> > on file "20150425.txt" which should speed it up greatly.
>>>
>>> Thanks for this nice analysis. It is probably the fsync call that is
>>> slowing Vim down. We can avoid the fsync by disabling the 'fsync'
>>> option. Also we might want to disable the writebackup option just in
>>> case it matters. And finally, when a file is written in a different
>>> encoding, Vim will try to check if conversion is safe by first trying to
>>> convert the buffer contents to the fileencoding before writing the file.
>>> That might make also a difference when encoding differs from the file
>>> encoding.
>>>
>>> Perhaps it makes sense to skip fsyncing while being busy in a `:g`
>>> command. Tim, does this patch make a difference?
>>>
>>> diff --git a/src/fileio.c b/src/fileio.c
>>> index d991822c3..eb626a675 100644
>>> --- a/src/fileio.c
>>> +++ b/src/fileio.c
>>> @@ -4731,7 +4731,7 @@ restore_backup:
>>>          * work (could be a pipe).
>>>          * If the 'fsync' option is FALSE, don't fsync().  Useful for laptops.
>>>          */
>>> -       if (p_fs && fsync(fd) != 0 && !device)
>>> +       if (p_fs && !global_busy && fsync(fd) != 0 && !device)
>>>         {
>>>             errmsg = (char_u *)_("E667: Fsync failed");
>>>             end = 0;
>>
>>
>> Hi Christian
>>
>> I measured the same command twice before and after your patch:
>>
>> Before patch:
>>
>> $ time ./vim  -u NONE data.json  -S new_engine.vim
>> real    0m15.533s
>> user    0m1.648s
>> sys    0m0.824s
>>
>> $ time ./vim  -u NONE data.json  -S new_engine.vim
>>
>> real    0m15.444s
>> user    0m1.636s
>> sys    0m0.820s
>>
>> After patch:
>>
>> $ time ./vim  -u NONE data.json  -S new_engine.vim
>>
>> real    0m0.741s
>> user    0m0.536s
>> sys    0m0.200s
>>
>> $ time ./vim  -u NONE data.json  -S new_engine.vim
>>
>> real    0m0.731s
>> user    0m0.548s
>> sys    0m0.184s
>>
>> So your patch makes it more than 20 times faster
>> on my Linux machine. Nice! Hopefully it's safe to
>> avoid fsync() in this case.
>
> I have not had the time to look in details, but the
> patch is wrong.  After patch, running the command with
> strace, I do not see any fsync() calls anymore.  I thought
> that the idea of the patch was to avoid intermediate fsync()
> but still fsync() at the end of the command.  But since

No, this is hard (impossible if you consider that command may modify
following text so that it will not match) to determine in the middle
of the global command whether it is going to run once more, so the
patch which intends to do something like this will have to do
something like caching file descriptors and emptying cache at the end,
additionally calling fsync().

> there is no fsync() call anymore, it's not safe.
>
> Regards
> Dominique
>
> --
> --
> You received this message from the "vim_use" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> ---
> You received this message because you are subscribed to the Google Groups "vim_use" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> For more options, visit https://groups.google.com/d/optout.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Vim surprisingly slow?

Christian Brabandt
In reply to this post by Dominique Pellé

On Do, 16 Nov 2017, Dominique Pellé wrote:

> Dominique Pellé <[hidden email]> wrote:
>
> >  Christian Brabandt <[hidden email]> wrote:
> >
> >> On Mi, 15 Nov 2017, Dominique Pellé wrote:
> >>
> >> > As highlighted in red, notice that we only write 179,281 bytes to file
> >> > "20150425.txt" but we open/write/close/fsync 5004 times.
> >> > That repeated I/O patterns kills performance, especially because
> >> > of the frequent fsync().
> >> >
> >> > We also read 1,612,897 bytes from data.json, but that should be
> >> > fast since we do only 27 read and opened that file only twice
> >> > according to output of io.pl. Vim reads file deta.json by chunks
> >> > of 64 KB. 1,612,897 bytes is exactly the file size of data.json.
> >> >
> >> > Maybe we can change vim to avoid the constant open/write/close/fsync
> >> > on file "20150425.txt" which should speed it up greatly.
> >>
> >> Thanks for this nice analysis. It is probably the fsync call that is
> >> slowing Vim down. We can avoid the fsync by disabling the 'fsync'
> >> option. Also we might want to disable the writebackup option just in
> >> case it matters. And finally, when a file is written in a different
> >> encoding, Vim will try to check if conversion is safe by first trying to
> >> convert the buffer contents to the fileencoding before writing the file.
> >> That might make also a difference when encoding differs from the file
> >> encoding.
> >>
> >> Perhaps it makes sense to skip fsyncing while being busy in a `:g`
> >> command. Tim, does this patch make a difference?
> >>
> >> diff --git a/src/fileio.c b/src/fileio.c
> >> index d991822c3..eb626a675 100644
> >> --- a/src/fileio.c
> >> +++ b/src/fileio.c
> >> @@ -4731,7 +4731,7 @@ restore_backup:
> >>          * work (could be a pipe).
> >>          * If the 'fsync' option is FALSE, don't fsync().  Useful for laptops.
> >>          */
> >> -       if (p_fs && fsync(fd) != 0 && !device)
> >> +       if (p_fs && !global_busy && fsync(fd) != 0 && !device)
> >>         {
> >>             errmsg = (char_u *)_("E667: Fsync failed");
> >>             end = 0;
> >
> >
> > Hi Christian
> >
> > I measured the same command twice before and after your patch:
> >
> > Before patch:
> >
> > $ time ./vim  -u NONE data.json  -S new_engine.vim
> > real    0m15.533s
> > user    0m1.648s
> > sys    0m0.824s
> >
> > $ time ./vim  -u NONE data.json  -S new_engine.vim
> >
> > real    0m15.444s
> > user    0m1.636s
> > sys    0m0.820s
> >
> > After patch:
> >
> > $ time ./vim  -u NONE data.json  -S new_engine.vim
> >
> > real    0m0.741s
> > user    0m0.536s
> > sys    0m0.200s
> >
> > $ time ./vim  -u NONE data.json  -S new_engine.vim
> >
> > real    0m0.731s
> > user    0m0.548s
> > sys    0m0.184s
> >
> > So your patch makes it more than 20 times faster
> > on my Linux machine. Nice! Hopefully it's safe to
> > avoid fsync() in this case.
>
> I have not had the time to look in details, but the
> patch is wrong.  After patch, running the command with
> strace, I do not see any fsync() calls anymore.  I thought
> that the idea of the patch was to avoid intermediate fsync()
> but still fsync() at the end of the command.  But since
> there is no fsync() call anymore, it's not safe.

Yeah, the patch skips fsyncing completly for :g commands. However, I
think this might be acceptable for several reasons:

First, if you disable the 'fsync' option (or HAVE_FSYNC is not defined),
Vim works perfectly fine. Also fsync() is not used for the writfile()
function, so it seems to be acceptable)

Second, when using `:g` it is more likely that only a small part of the
buffer will be written several times in relatively short times. So it
does not make sense to have to wait for the filesystem until the write
was complete and has reached the disk. This only causes many unnecessary
waits.

However I would agree, that a final fsync() at the end might be nice.
However we don't really know, when this is the case and even if we did
which is the last run of the `:g` command, it wouldn't be safe, because
perhaps in the last iteration nothing would be written to disk.

On the other hand, we can not fsync() at the end of the `:g` command,
because at that time, we would not have a filehandle to fsync().

I suppose skipping the fsync() would be okay in this edge case for :g
command and trust, that the kernel will write the changes to disk. The
only problem would be a system crash or power failure but I believe this
is relatively unlikely.

Christian
--
Der Philosoph hat wie der Hausbesitzer immer Reparaturen.
                -- Wilhelm Busch

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Vim surprisingly slow?

Bram Moolenaar
In reply to this post by Nikolay Aleksandrovich Pavlov

Nikolay Pavlov wrote:

> 2017-11-16 8:43 GMT+03:00 Dominique Pellé <[hidden email]>:
> > Dominique Pellé <[hidden email]> wrote:
> >
> >>  Christian Brabandt <[hidden email]> wrote:
> >>
> >>> On Mi, 15 Nov 2017, Dominique Pellé wrote:
> >>>
> >>> > As highlighted in red, notice that we only write 179,281 bytes to file
> >>> > "20150425.txt" but we open/write/close/fsync 5004 times.
> >>> > That repeated I/O patterns kills performance, especially because
> >>> > of the frequent fsync().
> >>> >
> >>> > We also read 1,612,897 bytes from data.json, but that should be
> >>> > fast since we do only 27 read and opened that file only twice
> >>> > according to output of io.pl. Vim reads file deta.json by chunks
> >>> > of 64 KB. 1,612,897 bytes is exactly the file size of data.json.
> >>> >
> >>> > Maybe we can change vim to avoid the constant open/write/close/fsync
> >>> > on file "20150425.txt" which should speed it up greatly.
> >>>
> >>> Thanks for this nice analysis. It is probably the fsync call that is
> >>> slowing Vim down. We can avoid the fsync by disabling the 'fsync'
> >>> option. Also we might want to disable the writebackup option just in
> >>> case it matters. And finally, when a file is written in a different
> >>> encoding, Vim will try to check if conversion is safe by first trying to
> >>> convert the buffer contents to the fileencoding before writing the file.
> >>> That might make also a difference when encoding differs from the file
> >>> encoding.
> >>>
> >>> Perhaps it makes sense to skip fsyncing while being busy in a `:g`
> >>> command. Tim, does this patch make a difference?
> >>>
> >>> diff --git a/src/fileio.c b/src/fileio.c
> >>> index d991822c3..eb626a675 100644
> >>> --- a/src/fileio.c
> >>> +++ b/src/fileio.c
> >>> @@ -4731,7 +4731,7 @@ restore_backup:
> >>>          * work (could be a pipe).
> >>>          * If the 'fsync' option is FALSE, don't fsync().  Useful for laptops.
> >>>          */
> >>> -       if (p_fs && fsync(fd) != 0 && !device)
> >>> +       if (p_fs && !global_busy && fsync(fd) != 0 && !device)
> >>>         {
> >>>             errmsg = (char_u *)_("E667: Fsync failed");
> >>>             end = 0;
> >>
> >>
> >> Hi Christian
> >>
> >> I measured the same command twice before and after your patch:
> >>
> >> Before patch:
> >>
> >> $ time ./vim  -u NONE data.json  -S new_engine.vim
> >> real    0m15.533s
> >> user    0m1.648s
> >> sys    0m0.824s
> >>
> >> $ time ./vim  -u NONE data.json  -S new_engine.vim
> >>
> >> real    0m15.444s
> >> user    0m1.636s
> >> sys    0m0.820s
> >>
> >> After patch:
> >>
> >> $ time ./vim  -u NONE data.json  -S new_engine.vim
> >>
> >> real    0m0.741s
> >> user    0m0.536s
> >> sys    0m0.200s
> >>
> >> $ time ./vim  -u NONE data.json  -S new_engine.vim
> >>
> >> real    0m0.731s
> >> user    0m0.548s
> >> sys    0m0.184s
> >>
> >> So your patch makes it more than 20 times faster
> >> on my Linux machine. Nice! Hopefully it's safe to
> >> avoid fsync() in this case.
> >
> > I have not had the time to look in details, but the
> > patch is wrong.  After patch, running the command with
> > strace, I do not see any fsync() calls anymore.  I thought
> > that the idea of the patch was to avoid intermediate fsync()
> > but still fsync() at the end of the command.  But since
>
> No, this is hard (impossible if you consider that command may modify
> following text so that it will not match) to determine in the middle
> of the global command whether it is going to run once more, so the
> patch which intends to do something like this will have to do
> something like caching file descriptors and emptying cache at the end,
> additionally calling fsync().

Assuming the script knows what the last append is going to be, it can
set 'fsync' off for all but the last one.  Or write an extra (empty)
line at the end with 'fsync' set.

> > there is no fsync() call anymore, it's not safe.

Correct, you may lose your work on a system crash.

--
While it's true that many normal people whould prefer not to _date_ an
engineer, most normal people harbor an intense desire to _mate_ with them,
thus producing engineerlike children who will have high-paying jobs long
before losing their virginity.
                                (Scott Adams - The Dilbert principle)

 /// Bram Moolenaar -- [hidden email] -- http://www.Moolenaar.net   \\\
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\  an exciting new programming language -- http://www.Zimbu.org        ///
 \\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Vim surprisingly slow?

Tim Chase
In reply to this post by Christian Brabandt
On 2017-11-16 08:03, Christian Brabandt wrote:
> On the other hand, we can not fsync() at the end of the `:g`
> command, because at that time, we would not have a filehandle to
> fsync().

I suppose one could gather the file-handles used during the :g
command and then issue one fsync() for each of them when finished.

Pseudocode something like

  filehandle_set = set()

  def begin_global_command():
    filehandle_set.clear()
   

  def deferrable_fsync(fp):
    if in_a_global_command():
      filehandle_set.add(fp)
    else:
      fsync(fp)

  def end_global_command():
    for fp in filehandle_set:
      fsync(fp)
    filehandle_set.clear()

> I suppose skipping the fsync() would be okay in this edge case
> for :g command and trust, that the kernel will write the changes to
> disk. The only problem would be a system crash or power failure but
> I believe this is relatively unlikely.

Alternatively, it might just be enough to document that ":w" also
does an fsync() and might be slow, while writefile() doesn't issue
fsync() and could be faster. For me it was as much the surprise as
anything else.

-tim



--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: Vim surprisingly slow?

Nikolay Aleksandrovich Pavlov
2017-11-16 16:35 GMT+03:00 Tim Chase <[hidden email]>:

> On 2017-11-16 08:03, Christian Brabandt wrote:
>> On the other hand, we can not fsync() at the end of the `:g`
>> command, because at that time, we would not have a filehandle to
>> fsync().
>
> I suppose one could gather the file-handles used during the :g
> command and then issue one fsync() for each of them when finished.
>
> Pseudocode something like
>
>   filehandle_set = set()
>
>   def begin_global_command():
>     filehandle_set.clear()
>
>
>   def deferrable_fsync(fp):
>     if in_a_global_command():
>       filehandle_set.add(fp)
>     else:
>       fsync(fp)
>
>   def end_global_command():
>     for fp in filehandle_set:
>       fsync(fp)
>     filehandle_set.clear()
>
>> I suppose skipping the fsync() would be okay in this edge case
>> for :g command and trust, that the kernel will write the changes to
>> disk. The only problem would be a system crash or power failure but
>> I believe this is relatively unlikely.
>
> Alternatively, it might just be enough to document that ":w" also
> does an fsync() and might be slow, while writefile() doesn't issue
> fsync() and could be faster. For me it was as much the surprise as
> anything else.

I do not much like that, when I found that &fsync does not apply to
writefile() I wrote a patch (for Neovim). Had to write another one
shortly after though: with a new flags that unconditionally
disable/enable fsync() call to fight a slowdown for a few plugins
where it does matter.

>
> -tim
>
>
>
> --
> --
> You received this message from the "vim_use" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> ---
> You received this message because you are subscribed to the Google Groups "vim_use" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> For more options, visit https://groups.google.com/d/optout.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
12