Progress indicator for :TOhtml command

classic Classic list List threaded Threaded
28 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Nikolay Aleksandrovich Pavlov
Ответ на сообщение «Re: Progress indicator for :TOhtml command»,
присланное в 19:39:15 05 июня 2010, Суббота,
отправитель Benjamin Fritz:

A small benchmark for your and mine scripts:
                              user    system  cpu  total      Relative
mine, no progress              93,07  10,82   99%  1:44,06    + 5%
mine, only per cents and bar   95,77  10,92   99%  1:46,94    + 8%
mine, %, bar and lines        125,59  14,80   99%  2:20,83    +43%
2html from vim-7.2.303         97,34   1,16   99%  1:38,64    + 0%
your 2html, no progress        77,31   0,99   99%  1:18,55    -20%
your 2html, with progress     100,57   1,20   99%  1:42,76    + 4%

Commands:
# mine, no progress
time vim messages -c 'set ft=messages | execute "%FormatCommand format html" | qa!'
# mine, only % and bar
time vim messages -c 'set ft=messages | let g:formatOptions={"ShowProgress": 1} | execute "%FormatCommand format
html" | qa!'
# mine, %, bar and lines
time vim messages -c 'set ft=messages | let g:formatOptions={"ShowProgress": 2} | execute "%FormatCommand format
html" | qa!'
# 2html, your with progress and 2html from vim-7.2.303
# Difference is that in second case I created a symlink to your file in
# .vim/syntax directory
time vim messages -c 'set ft=messages | execute "TOhtml" | qa!'
# your 2html, with progress
time vim messages -c 'set ft=messages | let g:html_no_progress=1 | execute "TOhtml" | qa!'

You can download messages file here: (1,1M uncompressed, 77K compressed):
http://kp-pav.narod.ru/messages.xz
My script is located here (see version 1.2):
http://www.vim.org/scripts/script.php?script_id=3113

I do not know what exactly is a problem, but your progress is too slow.


Текст сообщения:

> On Thu, Jun 3, 2010 at 10:03 PM, Benjamin Fritz <[hidden email]> wrote:
> > This version is still not fast enough though. It is about 30% slower
> > when the progress bar is enabled than when it is disabled. While I
> > consider it a good tradeoff in most cases, we could certainly make it
> > better.
> >
> > It would probably be faster to pre-calculate the line numbers needed
> > to advance the progress bar rather than doing a bunch of
> > floating-point math every cycle.
>
> I've attached a new version which pre-calculates the (integer) line
> numbers needed to advance the progress bar. Now all the floating point
> math is done once, up front.
>
> The difference is not really very perceptible. I timed the execution
> on two files. First, I did the 5148-line autoload/phpcomplete.vim
> script. Timings were as follows on my laptop:
>
> progress disabled:
> average: 46 seconds
>
> floating-point progress:
> average: 61 seconds
> slowdown: 15 seconds longer than without progress bar
> percentage: 33% longer than without progress bar
>
> precalculated progress:
> average: 62 seconds
> slowdown: 16 seconds longer than without progress bar
> percentage: 35% longer than without progress bar
>
> Next I did a 33258-line C code file:
>
> progress disabled:
> average: 691 seconds
>
> floating-point progress:
> average: 716 seconds
> slowdown: 25 seconds longer than without progress bar
> percentage: 4% longer than without progress bar
>
> precalculated progress:
> average: 711 seconds
> slowdown: 20 seconds longer than without progress bar
> percentage: 3% longer than without progress bar
>
> I also did a number of very small sections of files (my usual use case
> for 2html) and did not notice any significant slowing; it only takes
> 1-2 seconds longer for a 100 or 200 line selection.
>
> I take a few things from this.
>
> First of all, I don't think we'll get much performance improvement
> with this method. I do not know whether it is setting the status line
> and redrawing it, or whether it is the use of the object-oriented
> style functions, but it would probably require a different approach to
> get a significant speedup. I certainly like the look a lot better than
> the echo method, even if we could get echon working. Is a 10-20 second
> slow-down acceptable on large numbers of lines, if the normal
> execution time is measured in minutes anyway? To me, it certainly is.
> If something is going to be taking more than a few minutes, I want a
> progress bar to tell me whether it's worth letting it continue. Since
> the slow-down can be significant for midsize files, we will certainly
> need to mention in the :help that disabling the progress bar will make
> the conversion faster. Maybe we should only show the progress bar
> after some amount of time has elapsed? We could suppress the
> redrawstatus until 10 seconds have passed, or something like that. Any
> thoughts?
>
> Secondly, the precalculated version is not really any faster than the
> full floating-point calculation every cycle. I don't really have an
> opinion of which method gives more readable code. Does anyone else
> have any opinions on which version to keep? I think it would be
> possible to do away with floating point calculations entirely using
> the precalculated version; currently floating point is only used in
> the calculate_ticks function. This might be desireable so that we can
> remove the dependence on the +float feature, which is not marked with
> a "smallest version" indicator in :help +feature-list. This apparently
> means it is "system dependent". Does this mean float is pretty much
> always included, unless it is explicitly removed? How common are Vims
> without floating-point support? I already added use of the split()
> function, which was added in version 7, so this won't work on really
> old Vims...but do we want to support Vim 7.1 and earlier?
>

signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Nikolay Aleksandrovich Pavlov
In reply to this post by Benjamin Fritz
Ответ на сообщение «Re: Progress indicator for :TOhtml command»,
присланное в 19:39:15 05 июня 2010, Суббота,
отправитель Benjamin Fritz:

It occures that the problem is not floating-point math: the attached patch
removes this math but does not add any perfomance. It also removes recalculating
progress bar width (you just used used some generic progress bar?) and
needs_redraw. Also, why you forbid profiling progress bar functions? It is also
fixed.

Текст сообщения:
> On Thu, Jun 3, 2010 at 10:03 PM, Benjamin Fritz <[hidden email]>
wrote:

> > This version is still not fast enough though. It is about 30% slower
> > when the progress bar is enabled than when it is disabled. While I
> > consider it a good tradeoff in most cases, we could certainly make it
> > better.
> >
> > It would probably be faster to pre-calculate the line numbers needed
> > to advance the progress bar rather than doing a bunch of
> > floating-point math every cycle.
>
> I've attached a new version which pre-calculates the (integer) line
> numbers needed to advance the progress bar. Now all the floating point
> math is done once, up front.
>
> The difference is not really very perceptible. I timed the execution
> on two files. First, I did the 5148-line autoload/phpcomplete.vim
> script. Timings were as follows on my laptop:
>
> progress disabled:
> average: 46 seconds
>
> floating-point progress:
> average: 61 seconds
> slowdown: 15 seconds longer than without progress bar
> percentage: 33% longer than without progress bar
>
> precalculated progress:
> average: 62 seconds
> slowdown: 16 seconds longer than without progress bar
> percentage: 35% longer than without progress bar
>
> Next I did a 33258-line C code file:
>
> progress disabled:
> average: 691 seconds
>
> floating-point progress:
> average: 716 seconds
> slowdown: 25 seconds longer than without progress bar
> percentage: 4% longer than without progress bar
>
> precalculated progress:
> average: 711 seconds
> slowdown: 20 seconds longer than without progress bar
> percentage: 3% longer than without progress bar
>
> I also did a number of very small sections of files (my usual use case
> for 2html) and did not notice any significant slowing; it only takes
> 1-2 seconds longer for a 100 or 200 line selection.
>
> I take a few things from this.
>
> First of all, I don't think we'll get much performance improvement
> with this method. I do not know whether it is setting the status line
> and redrawing it, or whether it is the use of the object-oriented
> style functions, but it would probably require a different approach to
> get a significant speedup. I certainly like the look a lot better than
> the echo method, even if we could get echon working. Is a 10-20 second
> slow-down acceptable on large numbers of lines, if the normal
> execution time is measured in minutes anyway? To me, it certainly is.
> If something is going to be taking more than a few minutes, I want a
> progress bar to tell me whether it's worth letting it continue. Since
> the slow-down can be significant for midsize files, we will certainly
> need to mention in the :help that disabling the progress bar will make
> the conversion faster. Maybe we should only show the progress bar
> after some amount of time has elapsed? We could suppress the
> redrawstatus until 10 seconds have passed, or something like that. Any
> thoughts?
>
> Secondly, the precalculated version is not really any faster than the
> full floating-point calculation every cycle. I don't really have an
> opinion of which method gives more readable code. Does anyone else
> have any opinions on which version to keep? I think it would be
> possible to do away with floating point calculations entirely using
> the precalculated version; currently floating point is only used in
> the calculate_ticks function. This might be desireable so that we can
> remove the dependence on the +float feature, which is not marked with
> a "smallest version" indicator in :help +feature-list. This apparently
> means it is "system dependent". Does this mean float is pretty much
> always included, unless it is explicitly removed? How common are Vims
> without floating-point support? I already added use of the split()
> function, which was added in version 7, so this won't work on really
> old Vims...but do we want to support Vim 7.1 and earlier?
>

2html.patch (5K) Download Attachment
signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Benjamin Fritz
In reply to this post by Nikolay Aleksandrovich Pavlov


On Jun 5, 6:26 pm, ZyX <[hidden email]> wrote:

> A small benchmark for your and mine scripts:
>                               user    system  cpu  total      Relative
> mine, no progress              93,07  10,82   99%  1:44,06    + 5%
> mine, only per cents and bar   95,77  10,92   99%  1:46,94    + 8%
> mine, %, bar and lines        125,59  14,80   99%  2:20,83    +43%
> 2html from vim-7.2.303         97,34   1,16   99%  1:38,64    + 0%
> your 2html, no progress        77,31   0,99   99%  1:18,55    -20%
> your 2html, with progress     100,57   1,20   99%  1:42,76    + 4%
>
> [Snip]
>
> I do not know what exactly is a problem, but your progress is too slow.
>

I'm sorry I don't follow. You're saying that a 4% increase in time for
the progress bar, and a 20% decrease without the progress bar, is "too
slow"?

And you're proposing changes that make it an 8% increase with the
progress bar, or a 5% increase without?

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Benjamin Fritz
In reply to this post by Nikolay Aleksandrovich Pavlov
On Jun 5, 8:10 pm, ZyX <[hidden email]> wrote:
>
> It occures that the problem is not floating-point math: the attached patch
> removes this math but does not add any perfomance.
>

Yes, I did not expect any performance gains from removing the little
bit of remaining floating point, since it is just up to 100
calculations done once at the start and thereafter only when the
window changes size. It is a good idea to remove, because as you point
out, that amount of precision is probably unnecessary, and it would
just introduce another dependency.

> It also removes recalculating
> progress bar width (you just used used some generic progress bar?) and
> needs_redraw.

Yes, we did use a generic progress bar as the starting point for this.
However, I think it IS necessary to recalculate the progress bar
width. This is done so that if the user changes window sizes, the
progress bar will be updated accordingly. We don't want a progress bar
that is too big to fit in the window, or smaller than needed for
decent viewing. With your patch, if you start with the gvim window
maximized, then restore the window to a smaller size, Vim goes blank
until the next progress bar update, and then the progress bar is too
large to fit on the screen and is truncated. This is not desirable,
but perhaps it would acceptable if the performance gains are great
enough. This does not seem to be the case, because I added back in the
size recalculation with no noticeable performance hit.

The needs_redraw was done in order to allow us to call redrawstatus on
the correct window. :help redrawstatus says that it redraws the status
line for the *current window* only unless you use redrawstatus! which
redraws all windows. In practice, however, it does not seem to matter
which window we use it in. Why is this?

> Also, why you forbid profiling progress bar functions? It is also
> fixed.
>

Good catch, that's certainly something to include going forward.

There is a slight speed gain from your patch, however there is a
mistaken assumption in the way you update the progress bar. Your code
assumes that the progress bar will only ever update by one tick at a
time. Updating the progress bar without your patch calculates the
entire string every time, using repeat(). Your update simply adds one
to the colored string of spaces, and subtracts one from the uncolored.
This does not work if the user folds away some text and does not use
dynamic folding, it does not work when there are fewer than 100 lines
in the text to convert, and it does not work for the second use of the
progress bar, where there are usually fewer that 100 highlight groups
to process.

I corrected this problem and initially, the performance still seemed
to be improved over the previous version. However, I noticed afterward
that part of the patch removes the "sleep 100m" from the "processing
classes" step. I took this line out of the original script for a fair
comparison, and got the following timings, converting
autoload/netrw.vim (7764 lines) with dynamic folding enabled:

Before patch: 50 seconds
Patch from ZyX: 49 seconds
Fixed patch: 51 seconds

So, it looks like the patch is actually no faster, and potentially
slightly slower than the precalculated version.

I have therefore attached an updated version of my last submission,
which removes floating point from the calculate_ticks function, and
incorporates some of the other improvements from ZyX.

This version takes 50 seconds to convert netrw, if I comment out the
sleep 100 line. Do we want this line in the code? Without it, if there
are not very many highlight groups to process, the "processing
classes" bar flashes by without being seen. This happens anyway for
very small selections. I don't know how I feel about deliberately
slowing down the execution. I have left it commented out for now.

I am very curious about this:

" Note that you must use len(split) instead of len() if you want to use
" unicode in title
let self.pb_len = max_len-len(split(self.title, '\zs'))-3-4-2

Can someone explain the problem described in the comment a little
better? And why does the split on '\zs' work to fix the problem?

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

2html.vim (53K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Nikolay Aleksandrovich Pavlov
Ответ на сообщение «Re: Progress indicator for :TOhtml command»,
присланное в 10:59:42 06 июня 2010, Воскресенье,
отправитель Benjamin Fritz:

The reason why I say that progress bar is too slow is that my script does not
suffer from performance decrease unless you make it redraw on each line. I will
add size recalculation for my script too (I removed it from your script because
I did not realize that while user can do nothing in vim he still can resize the
terminal), but I do not think that this will add any performance penalty.

> I am very curious about this:
>
> " Note that you must use len(split) instead of len() if you want to use
> " unicode in title
> let self.pb_len = max_len-len(split(self.title, '\zs'))-3-4-2
>
> Can someone explain the problem described in the comment a little
> better? And why does the split on '\zs' work to fix the problem?
That is because len(str) measures byte length of C string, while len(split) first
splits the string into a list of characters and then measures the length of
character list. If there are non-latin1 Unicode symbols and encoding is a
multibyte one then length of character list is not equal to bytes count of C
string.

Текст сообщения:

> On Jun 5, 8:10 pm, ZyX <[hidden email]> wrote:
> > It occures that the problem is not floating-point math: the attached
> > patch removes this math but does not add any perfomance.
>
> Yes, I did not expect any performance gains from removing the little
> bit of remaining floating point, since it is just up to 100
> calculations done once at the start and thereafter only when the
> window changes size. It is a good idea to remove, because as you point
> out, that amount of precision is probably unnecessary, and it would
> just introduce another dependency.
>
> > It also removes recalculating
> > progress bar width (you just used used some generic progress bar?) and
> > needs_redraw.
>
> Yes, we did use a generic progress bar as the starting point for this.
> However, I think it IS necessary to recalculate the progress bar
> width. This is done so that if the user changes window sizes, the
> progress bar will be updated accordingly. We don't want a progress bar
> that is too big to fit in the window, or smaller than needed for
> decent viewing. With your patch, if you start with the gvim window
> maximized, then restore the window to a smaller size, Vim goes blank
> until the next progress bar update, and then the progress bar is too
> large to fit on the screen and is truncated. This is not desirable,
> but perhaps it would acceptable if the performance gains are great
> enough. This does not seem to be the case, because I added back in the
> size recalculation with no noticeable performance hit.
>
> The needs_redraw was done in order to allow us to call redrawstatus on
> the correct window. :help redrawstatus says that it redraws the status
> line for the *current window* only unless you use redrawstatus! which
> redraws all windows. In practice, however, it does not seem to matter
> which window we use it in. Why is this?
>
> > Also, why you forbid profiling progress bar functions? It is also
> > fixed.
>
> Good catch, that's certainly something to include going forward.
>
> There is a slight speed gain from your patch, however there is a
> mistaken assumption in the way you update the progress bar. Your code
> assumes that the progress bar will only ever update by one tick at a
> time. Updating the progress bar without your patch calculates the
> entire string every time, using repeat(). Your update simply adds one
> to the colored string of spaces, and subtracts one from the uncolored.
> This does not work if the user folds away some text and does not use
> dynamic folding, it does not work when there are fewer than 100 lines
> in the text to convert, and it does not work for the second use of the
> progress bar, where there are usually fewer that 100 highlight groups
> to process.
>
> I corrected this problem and initially, the performance still seemed
> to be improved over the previous version. However, I noticed afterward
> that part of the patch removes the "sleep 100m" from the "processing
> classes" step. I took this line out of the original script for a fair
> comparison, and got the following timings, converting
> autoload/netrw.vim (7764 lines) with dynamic folding enabled:
>
> Before patch: 50 seconds
> Patch from ZyX: 49 seconds
> Fixed patch: 51 seconds
>
> So, it looks like the patch is actually no faster, and potentially
> slightly slower than the precalculated version.
>
> I have therefore attached an updated version of my last submission,
> which removes floating point from the calculate_ticks function, and
> incorporates some of the other improvements from ZyX.
>
> This version takes 50 seconds to convert netrw, if I comment out the
> sleep 100 line. Do we want this line in the code? Without it, if there
> are not very many highlight groups to process, the "processing
> classes" bar flashes by without being seen. This happens anyway for
> very small selections. I don't know how I feel about deliberately
> slowing down the execution. I have left it commented out for now.
>
> I am very curious about this:
>
> " Note that you must use len(split) instead of len() if you want to use
> " unicode in title
> let self.pb_len = max_len-len(split(self.title, '\zs'))-3-4-2
>
> Can someone explain the problem described in the comment a little
> better? And why does the split on '\zs' work to fix the problem?
>

signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Nikolay Aleksandrovich Pavlov
In reply to this post by Benjamin Fritz
Ответ на сообщение «Re: Progress indicator for :TOhtml command»,
присланное в 10:59:42 06 июня 2010, Воскресенье,
отправитель Benjamin Fritz:

It is odd: the only problem in your script is redrawstatus which is called only
100 times (without styles, 109 with) (>21 seconds), while in my script
redrawstatus called 328 times takes less than a second.

Second problem with the whole 2html is buffer switching, I think you should
consider instead of doing constant switches, save every line in a List and only
after everything is finished create a new buffer and call setline(1, s:list).
Note that new versions of my script are faster (but not much) then your 2html
because I use this technique.

And, why do you calculate length of the title at each progressbarupdate?
Attached patch fixes this and the case when there is no space for progress bar.

Текст сообщения:

> On Jun 5, 8:10 pm, ZyX <[hidden email]> wrote:
> > It occures that the problem is not floating-point math: the attached
> > patch removes this math but does not add any perfomance.
>
> Yes, I did not expect any performance gains from removing the little
> bit of remaining floating point, since it is just up to 100
> calculations done once at the start and thereafter only when the
> window changes size. It is a good idea to remove, because as you point
> out, that amount of precision is probably unnecessary, and it would
> just introduce another dependency.
>
> > It also removes recalculating
> > progress bar width (you just used used some generic progress bar?) and
> > needs_redraw.
>
> Yes, we did use a generic progress bar as the starting point for this.
> However, I think it IS necessary to recalculate the progress bar
> width. This is done so that if the user changes window sizes, the
> progress bar will be updated accordingly. We don't want a progress bar
> that is too big to fit in the window, or smaller than needed for
> decent viewing. With your patch, if you start with the gvim window
> maximized, then restore the window to a smaller size, Vim goes blank
> until the next progress bar update, and then the progress bar is too
> large to fit on the screen and is truncated. This is not desirable,
> but perhaps it would acceptable if the performance gains are great
> enough. This does not seem to be the case, because I added back in the
> size recalculation with no noticeable performance hit.
>
> The needs_redraw was done in order to allow us to call redrawstatus on
> the correct window. :help redrawstatus says that it redraws the status
> line for the *current window* only unless you use redrawstatus! which
> redraws all windows. In practice, however, it does not seem to matter
> which window we use it in. Why is this?
>
> > Also, why you forbid profiling progress bar functions? It is also
> > fixed.
>
> Good catch, that's certainly something to include going forward.
>
> There is a slight speed gain from your patch, however there is a
> mistaken assumption in the way you update the progress bar. Your code
> assumes that the progress bar will only ever update by one tick at a
> time. Updating the progress bar without your patch calculates the
> entire string every time, using repeat(). Your update simply adds one
> to the colored string of spaces, and subtracts one from the uncolored.
> This does not work if the user folds away some text and does not use
> dynamic folding, it does not work when there are fewer than 100 lines
> in the text to convert, and it does not work for the second use of the
> progress bar, where there are usually fewer that 100 highlight groups
> to process.
>
> I corrected this problem and initially, the performance still seemed
> to be improved over the previous version. However, I noticed afterward
> that part of the patch removes the "sleep 100m" from the "processing
> classes" step. I took this line out of the original script for a fair
> comparison, and got the following timings, converting
> autoload/netrw.vim (7764 lines) with dynamic folding enabled:
>
> Before patch: 50 seconds
> Patch from ZyX: 49 seconds
> Fixed patch: 51 seconds
>
> So, it looks like the patch is actually no faster, and potentially
> slightly slower than the precalculated version.
>
> I have therefore attached an updated version of my last submission,
> which removes floating point from the calculate_ticks function, and
> incorporates some of the other improvements from ZyX.
>
> This version takes 50 seconds to convert netrw, if I comment out the
> sleep 100 line. Do we want this line in the code? Without it, if there
> are not very many highlight groups to process, the "processing
> classes" bar flashes by without being seen. This happens anyway for
> very small selections. I don't know how I feel about deliberately
> slowing down the execution. I have left it commented out for now.
>
> I am very curious about this:
>
> " Note that you must use len(split) instead of len() if you want to use
> " unicode in title
> let self.pb_len = max_len-len(split(self.title, '\zs'))-3-4-2
>
> Can someone explain the problem described in the comment a little
> better? And why does the split on '\zs' work to fix the problem?
>

2html.patch (2K) Download Attachment
signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Nikolay Aleksandrovich Pavlov
Ответ на сообщение «Re: Progress indicator for :TOhtml command»,
присланное в 13:03:23 06 июня 2010, Воскресенье,
отправитель ZyX:

Yes, buffer switching is the problem: attached patch uses my technique (save
everything in a list, not in a buffer) and here are the results:

My script:
1:05,09 w/o progress
1:08,40 ShowProgress=1
1:20,59 ShowProgress=2
Your 2html:
1:19,67 w/o progress
1:44,74 with progress
Patched 2html:
1:03,51 w/o progress
1:05,08 with progress

Apply patch to your 2html, not to previously patched version.

Текст сообщения:

> Ответ на сообщение «Re: Progress indicator for :TOhtml command»,
> присланное в 10:59:42 06 июня 2010, Воскресенье,
> отправитель Benjamin Fritz:
>
> It is odd: the only problem in your script is redrawstatus which is called
>  only 100 times (without styles, 109 with) (>21 seconds), while in my
>  script redrawstatus called 328 times takes less than a second.
>
> Second problem with the whole 2html is buffer switching, I think you should
> consider instead of doing constant switches, save every line in a List and
>  only after everything is finished create a new buffer and call setline(1,
>  s:list). Note that new versions of my script are faster (but not much)
>  then your 2html because I use this technique.
>
> And, why do you calculate length of the title at each progressbarupdate?
> Attached patch fixes this and the case when there is no space for progress
>  bar.
>
> Текст сообщения:
> > On Jun 5, 8:10 pm, ZyX <[hidden email]> wrote:
> > > It occures that the problem is not floating-point math: the attached
> > > patch removes this math but does not add any perfomance.
> >
> > Yes, I did not expect any performance gains from removing the little
> > bit of remaining floating point, since it is just up to 100
> > calculations done once at the start and thereafter only when the
> > window changes size. It is a good idea to remove, because as you point
> > out, that amount of precision is probably unnecessary, and it would
> > just introduce another dependency.
> >
> > > It also removes recalculating
> > > progress bar width (you just used used some generic progress bar?) and
> > > needs_redraw.
> >
> > Yes, we did use a generic progress bar as the starting point for this.
> > However, I think it IS necessary to recalculate the progress bar
> > width. This is done so that if the user changes window sizes, the
> > progress bar will be updated accordingly. We don't want a progress bar
> > that is too big to fit in the window, or smaller than needed for
> > decent viewing. With your patch, if you start with the gvim window
> > maximized, then restore the window to a smaller size, Vim goes blank
> > until the next progress bar update, and then the progress bar is too
> > large to fit on the screen and is truncated. This is not desirable,
> > but perhaps it would acceptable if the performance gains are great
> > enough. This does not seem to be the case, because I added back in the
> > size recalculation with no noticeable performance hit.
> >
> > The needs_redraw was done in order to allow us to call redrawstatus on
> > the correct window. :help redrawstatus says that it redraws the status
> > line for the *current window* only unless you use redrawstatus! which
> > redraws all windows. In practice, however, it does not seem to matter
> > which window we use it in. Why is this?
> >
> > > Also, why you forbid profiling progress bar functions? It is also
> > > fixed.
> >
> > Good catch, that's certainly something to include going forward.
> >
> > There is a slight speed gain from your patch, however there is a
> > mistaken assumption in the way you update the progress bar. Your code
> > assumes that the progress bar will only ever update by one tick at a
> > time. Updating the progress bar without your patch calculates the
> > entire string every time, using repeat(). Your update simply adds one
> > to the colored string of spaces, and subtracts one from the uncolored.
> > This does not work if the user folds away some text and does not use
> > dynamic folding, it does not work when there are fewer than 100 lines
> > in the text to convert, and it does not work for the second use of the
> > progress bar, where there are usually fewer that 100 highlight groups
> > to process.
> >
> > I corrected this problem and initially, the performance still seemed
> > to be improved over the previous version. However, I noticed afterward
> > that part of the patch removes the "sleep 100m" from the "processing
> > classes" step. I took this line out of the original script for a fair
> > comparison, and got the following timings, converting
> > autoload/netrw.vim (7764 lines) with dynamic folding enabled:
> >
> > Before patch: 50 seconds
> > Patch from ZyX: 49 seconds
> > Fixed patch: 51 seconds
> >
> > So, it looks like the patch is actually no faster, and potentially
> > slightly slower than the precalculated version.
> >
> > I have therefore attached an updated version of my last submission,
> > which removes floating point from the calculate_ticks function, and
> > incorporates some of the other improvements from ZyX.
> >
> > This version takes 50 seconds to convert netrw, if I comment out the
> > sleep 100 line. Do we want this line in the code? Without it, if there
> > are not very many highlight groups to process, the "processing
> > classes" bar flashes by without being seen. This happens anyway for
> > very small selections. I don't know how I feel about deliberately
> > slowing down the execution. I have left it commented out for now.
> >
> > I am very curious about this:
> >
> > " Note that you must use len(split) instead of len() if you want to use
> > " unicode in title
> > let self.pb_len = max_len-len(split(self.title, '\zs'))-3-4-2
> >
> > Can someone explain the problem described in the comment a little
> > better? And why does the split on '\zs' work to fix the problem?
>

2html.patch (12K) Download Attachment
signature.asc (205 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Benjamin Fritz
On Sun, Jun 6, 2010 at 5:10 AM, ZyX <[hidden email]> wrote:

>
> Yes, buffer switching is the problem: attached patch uses my technique (save
> everything in a list, not in a buffer) and here are the results:
>
> My script:
> 1:05,09 w/o progress
> 1:08,40 ShowProgress=1
> 1:20,59 ShowProgress=2
> Your 2html:
> 1:19,67 w/o progress
> 1:44,74 with progress
> Patched 2html:
> 1:03,51 w/o progress
> 1:05,08 with progress
>
Very nice. This is a huge performance boost, and the times are similar
with and without the progress bar even with my big 33000 line C file
which I used previously.

I think it's about ready now. I've added another progress bar for the
time taken to collect fold information for dynamic folding, and
corrected a few minor bugs in the patch related to dynamic folding. I
did end up adding back in a :sleep to the class processing loop, but I
reduced the time it sleeps. I'm certainly open to removing this.

I've attached the whole file so we don't get into a "which patches do
I need?" quagmire.

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

2html.vim (55K) Download Attachment
12