Progress indicator for :TOhtml command

classic Classic list List threaded Threaded
28 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Progress indicator for :TOhtml command

Benjamin Fritz
I love using the :TOhtml command, and I keep finding more ways to use
it. Recently, I had a large-ish log file (several thousand lines), in
which I wanted to call attention to a few groups of lines, but I
figured people may want the context as well. So, I set up some folds
and some quick syntax highlighting, and went to go create an html copy
of it using the "dynamic folding" feature of the command.

Unfortunately, I discovered that processing such a large file, even
with no syntax highlighting, takes a *very* long time. I probably
should have selected just a smaller area of interest but...

I waited quite a while, and finally hit CTRL-C to stop it. Luckily it
hadn't actually gotten that far (probably about 30%), but I was
worried that it may have been almost done, and all I needed to do was
wait a bit longer.

So anyway, for future use, I wanted to be able to see quickly whether
the conversion was worth waiting for. Therefore, I have written a
patch to add a progress indicator. When I opened the file, I noticed
that indentation is quite a mess (a mix of tabs and spaces, sometimes
in the same line), so I also fixed that up (by using gg=G with noet,
sw=2, sts=2, and ts=2). For clarity, I ran the diff on the file
*after* fixing the ident.

Since the indentation patch is actually *larger* than the file itself,
I have just attached the fully patched file instead of a patch. This
will also make it easier for Windows users to try :-)

Please comment...especially if you know of a better way to accomplish
something similar. I tried using :echon to draw the progress bar, but
the echo was being cleared each time through the main loop, so I
switched to using :echo with a string that is gradually built as the
script processes.

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

2html.vim (49K) Download Attachment
2html.vim.progress.patch (10K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Yongwei Wu
On 1 June 2010 04:47, Benjamin Fritz <[hidden email]> wrote:

> I love using the :TOhtml command, and I keep finding more ways to use
> it. Recently, I had a large-ish log file (several thousand lines), in
> which I wanted to call attention to a few groups of lines, but I
> figured people may want the context as well. So, I set up some folds
> and some quick syntax highlighting, and went to go create an html copy
> of it using the "dynamic folding" feature of the command.
>
> Unfortunately, I discovered that processing such a large file, even
> with no syntax highlighting, takes a *very* long time. I probably
> should have selected just a smaller area of interest but...
>
> I waited quite a while, and finally hit CTRL-C to stop it. Luckily it
> hadn't actually gotten that far (probably about 30%), but I was
> worried that it may have been almost done, and all I needed to do was
> wait a bit longer.
>
> So anyway, for future use, I wanted to be able to see quickly whether
> the conversion was worth waiting for. Therefore, I have written a
> patch to add a progress indicator. When I opened the file, I noticed
> that indentation is quite a mess (a mix of tabs and spaces, sometimes
> in the same line), so I also fixed that up (by using gg=G with noet,
> sw=2, sts=2, and ts=2). For clarity, I ran the diff on the file
> *after* fixing the ident.
>
> Since the indentation patch is actually *larger* than the file itself,
> I have just attached the fully patched file instead of a patch. This
> will also make it easier for Windows users to try :-)
>
> Please comment...especially if you know of a better way to accomplish
> something similar. I tried using :echon to draw the progress bar, but
> the echo was being cleared each time through the main loop, so I
> switched to using :echo with a string that is gradually built as the
> script processes.

I tried it on Windows, and the display was too flashy and intrusive.
I can't say I like it.

And there was nothing to "fix" about the tabs.  They were perfectly
fine.  Tabstop was set to the default (8), and only shiftwidth was set
to 2 (along with sts, which does not affect the saved file).  You
seemed to have confused them.  The original version is easier to view
(using type/cat/more/less or nearly anything).

--
Wu Yongwei
URL: http://wyw.dcweb.cn/

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Christian J. Robinson-2
On Tue, 1 Jun 2010, Yongwei Wu wrote:

> I tried it on Windows, and the display was too flashy and intrusive.
> I can't say I like it.

I tried it too--on Windows and on Linux--and it was flashy and
unusable on both, plus about 15 to 20 percent slower than the original
on both.

I would be for having a progress meter, if it could be implemented in
a better way without slowing the script down.

My best suggestion is just to output one line every 5 to 10 percent
through processing with the current progress, e.g.:

  0%
  5%
  10%
  15%
  [...]
  95%
  Done

- Christian

--
                         Abdominos... sit-ups & pizza.
Christian J. Robinson <[hidden email]>      http://christianrobinson.name/

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Andy Wokula
In reply to this post by Benjamin Fritz
Am 31.05.2010 22:47, schrieb Benjamin Fritz:

> I love using the :TOhtml command, and I keep finding more ways to use
> it. Recently, I had a large-ish log file (several thousand lines), in
> which I wanted to call attention to a few groups of lines, but I
> figured people may want the context as well. So, I set up some folds
> and some quick syntax highlighting, and went to go create an html copy
> of it using the "dynamic folding" feature of the command.
>
> Unfortunately, I discovered that processing such a large file, even
> with no syntax highlighting, takes a *very* long time. I probably
> should have selected just a smaller area of interest but...
>
> I waited quite a while, and finally hit CTRL-C to stop it. Luckily it
> hadn't actually gotten that far (probably about 30%), but I was
> worried that it may have been almost done, and all I needed to do was
> wait a bit longer.
>
> So anyway, for future use, I wanted to be able to see quickly whether
> the conversion was worth waiting for. Therefore, I have written a
> patch to add a progress indicator.

I'd suggest to try one of the existing progress bars (sorry,  didn't try
them myself yet):

http://github.com/tomtom/vimtlib/blob/master/autoload/tlib/progressbar.vim
http://www.vim.org/scripts/script.php?script_id=2006

--
Andy

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Benjamin Fritz
In reply to this post by Yongwei Wu


On May 31, 10:50 pm, Yongwei Wu <[hidden email]> wrote:
> On 1 June 2010 04:47, Benjamin Fritz <[hidden email]> wrote:
>
>
> And there was nothing to "fix" about the tabs.  They were perfectly
> fine.  Tabstop was set to the default (8), and only shiftwidth was set
> to 2 (along with sts, which does not affect the saved file).  You
> seemed to have confused them.  The original version is easier to view
> (using type/cat/more/less or nearly anything).
>

I am fine with tabs or spaces for indentation. I am NOT fine with a
mixture of the two. From the previous modeline, I assumed that the
intent was a tab-based indent. You are free to change the tabstop to
view the file, but if you edit the file using a 'shiftwidth' and
'tabstop' value that differ, the indentation will become a mixture of
tabs and spaces, so that changing tabstop no longer looks right.

I did not modify the value of 'sts', that was already there.

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Benjamin Fritz
In reply to this post by Christian J. Robinson-2


On May 31, 11:11 pm, "Christian J. Robinson" <[hidden email]>
wrote:
> On Tue, 1 Jun 2010, Yongwei Wu wrote:
> > I tried it on Windows, and the display was too flashy and intrusive.
> > I can't say I like it.


On May 31, 11:11 pm, "Christian J. Robinson" <[hidden email]>
wrote:

> I would be for having a progress meter, if it could be implemented in
> a better way without slowing the script down.
>
> My best suggestion is just to output one line every 5 to 10 percent
> through processing with the current progress, e.g.:
>
>   0%
>   5%
>   10%
>   15%
>   [...]
>   95%
>   Done
>

Hmm, strange. I did not notice a slow-down from the original script,
but then I also set 'nomore' and fdm=manual as a way to speed things
up a little in addition to the progress bar changes. On my dinosaur of
a computer, processing 2html.vim took about 40 seconds LESS with the
patched version that with the unpatched version, according to the
before/after times returned by localtime(). What did you do to see the
slowdown?

I also dislike the flashiness, but couldn't find a way around it. The
echo'd text gets cleared every pass through the main processing loop.
For this reason, a periodic (5, 10, 20%, etc.) message wasn't going to
work. I would certainly prefer this method...my first attempt used
echon to display one "tick" every few lines processed (a tick being
calculated to almost fill the current &columns setting). Does someone
have an idea of how I might accomplish this?

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Benjamin Fritz
In reply to this post by Andy Wokula


On Jun 1, 7:21 am, Andy Wokula <[hidden email]> wrote:
>
> I'd suggest to try one of the existing progress bars (sorry,  didn't try
> them myself yet):
>
> http://github.com/tomtom/vimtlib/blob/master/autoload/tlib/progressba...http://www.vim.org/scripts/script.php?script_id=2006
>

Hmm, interesting...and thanks! These two plugins seem to use the
statusline for the progress indicator. I thought of that, and
abandoned the idea when I noticed that the status lines weren't being
populated all the way during processing...but later investigation
showed me that's because 'ruler' gets reset from one of the speed-up
options set by the plugin (I don't remember which). At the time, I was
still hoping to use :echon to output the progress indicator, which I
also had to abandon. My eventual goal is to use the full screen width
(or pretty close to it) for the progress bar. Is there a good way to
get this for an individual status line? I didn't think so, whereas
using an ":echo" type command, &columns basically does the trick.

I'll give it a shot using the status line (maybe making use of a
plugin, but I'm hoping for eventual inclusion into the upstream, so
maybe just pulling code from them). Hopefully it will be more usable.
For me, the screen flashes quite a bit *without* the progress
indicator, so I'm not certain it will be, but perhaps the somewhat
static status line will be more readable than a quickly flashing
message at the bottom of the window.

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Ingo Karkat
In reply to this post by Benjamin Fritz
On 01-Jun-2010 16:43, Ben Fritz wrote:
> I am fine with tabs or spaces for indentation. I am NOT fine with a
> mixture of the two. From the previous modeline, I assumed that the
> intent was a tab-based indent. You are free to change the tabstop to
> view the file, but if you edit the file using a 'shiftwidth' and
> 'tabstop' value that differ, the indentation will become a mixture of
> tabs and spaces, so that changing tabstop no longer looks right.

Personal preferences aside, the syntax/2html.vim looks like a correct
softtabstop=2 to me; I use this setting all the time, and Vim supports it well.
(Cp. :help 'softtabstop'). The only downside is that you should keep ts=8, and
not mess with 'shiftwidth' and related settings while editing. In that regard,
it's less flexible than a pure tab or space-indented format, you're right.

-- regards, ingo

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Christian J. Robinson-2
In reply to this post by Benjamin Fritz
On Tue, 1 Jun 2010, Ben Fritz wrote:

> What did you do to see the slowdown?

In two separate Vim instances ("vim 2html.vim"--not at the same time):

  :let timestart=join(reltime(), ' ') | exe 'so ' . expand('%') | let timefinish=join(reltime(), ' ')
  :echo timestart | echo timefinish

Time elapsed about 39 seconds.

  :let timestart=join(reltime(), ' ') | exe 'TOhtml' | let timefinish=join(reltime(), ' ')
  :echo timestart | echo timefinish

Time elapsed about 31 seconds.

So the first is over 20% slower.

This morning I tried it under a "vim -u NONE --noplugin 2html.vim"
instance and your version was actually four seconds faster than the
original.  Odd.

I do have a fairly complex vimrc, as well as a number of plugins
installed.

- Christian

--
Christian J. Robinson <[hidden email]>

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Christian Brabandt
In reply to this post by Andy Wokula
On Tue, June 1, 2010 2:21 pm, Andy Wokula wrote:
> I'd suggest to try one of the existing progress bars (sorry,  didn't try
> them myself yet):
>
> http://github.com/tomtom/vimtlib/blob/master/autoload/tlib/progressbar.vim
> http://www.vim.org/scripts/script.php?script_id=2006

Here is an experimental version, in which I included the essential part
from the mentioned vim.org/script into the script from Ben.

Bugs: - currently only works well with a big enough window.
      - progressbar requires redrawing the window, while processing
        the 2html.vim script. This means, it slows it down.
      - requires a +float (no check yet)

regards,
Christian

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

2html.vim (52K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Benjamin Fritz
I've done some updates...more to follow. I forgot to hit the "email
updates to me" button on the google groups interface so that I can
upload attachments on a reply.

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Benjamin Fritz
>
> Here is an experimental version, in which I included the essential part
> from the mentioned vim.org/script into the script from Ben.
>
> Bugs: - currently only works well with a big enough window.
>       - progressbar requires redrawing the window, while processing
>         the 2html.vim script. This means, it slows it down.
>       - requires a +float (no check yet)
>

I like this look a lot better! I made a few improvements:

- Change the color of StatusLineNC to match StatusLine (and restore it at the
  end) to reduce flashing of the statusline.
- Draw the progress bar in the new window instead of the original window, to
  reduce the amout of time from the screen clear to the redraw when redrawing
  the windows, reducing the flashing effect.
- Set the size of the new window to only 2 lines, so that only 1 line of the
  buffer actually appears. The intent of this was to further reduce the flashing
  effect, but it also has the nice side affect of making the conversion MUCH
  faster, since less of the expensive html syntax highlighting needs to be
  completed every redraw.
- Added a second progress bar for the attributes processing (previously I was
  doing this with a %d/%d printf).
- Fixed bug in progress bar incr function that prevented incrementing by large
  amounts, that are still less than the maximum value.
- Fixed bug where progress bar would not fill all the way when doing non-dynamic
  folds.
- Fixed off-by-one error in progress bar initialization.

I also removed the "redrawstatus" commands. They did not seem to be necessary,
and only served to increase the processing time. Did you have a particular
reason to include them?

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

2html.vim (51K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Benjamin Fritz
On Tue, Jun 1, 2010 at 6:04 PM, Benjamin Fritz <[hidden email]> wrote:
>
> I like this look a lot better! I made a few improvements:
>

And one more bugfix, to prevent throwing errors in the cleanup
commands at the end.

With the attached script on fairly recent hardware, I get the
following timings (in seconds) when converting the 2html.vim script to
html, several times and discarding the first few trials (these were
outliers...around 100 seconds, probably while getting the cache all
ironed out or something):

with progress bar:
34
34
35
32
32
without progress bar (using let html_no_progress=1):
33
33
33
33
32
32
33
30
unmodified "official" script (with autocmds to set winsize and
foldmethod to prevent syntax highlighting to screw with the results):
31
32
31
31

For me anyway, the difference made by the progress bar, with the
optimizations in the attached version of 2html.vim, is negligible.

So...is the screen flashing still too distracting? I think the version
attached handles it well enough, at least for my purposes.

Any additional comments or fixes?

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

2html.vim (51K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Benjamin Fritz
In reply to this post by Benjamin Fritz


On Jun 2, 3:13 am, "Christian Brabandt" <[hidden email]> wrote:

> On Wed, June 2, 2010 1:04 am, Benjamin Fritz wrote:
> > - Added a second progress bar for the attributes processing (previously I
> > was
> >   doing this with a %d/%d printf).
> > I also removed the "redrawstatus" commands. They did not seem to be
> > necessary,
> > and only served to increase the processing time. Did you have a particular
> > reason to include them?
>
> Yes. I can't see the progressbar otherwise. It is not updating for me. So
> I suggest, enabling it again.
>

Interesting. Is this true, even with the progress bar on the top
window as in the latest version? If it is on the bottom, the screen
clears again for a redraw immediately after the bottom status line is
drawn, making it very hard to see the progress bar...which is why I
put it on the top.

I see from :help :redrawstatus,

                        Useful to update the status line(s) when 'statusline'
                        includes an item that doesn't cause automatic
                        updating.

But we're setting the statusline to a new (static) value...surely the
statusline is redrawn when 'statusline' is set to something new?

If we leave the progress bar on the top window (which I like...any
disagreements?) we need to find the appropriate place to put
the :redrawstatus command, so that we redraw the correct window's
status line. I don't think it's a great idea to use redrawstatus!,
although a "normal" use case is probably just the two windows, so
maybe it's not that much more expensive?

I toyed with the idea of only updating the 'statusline' option and
calling :redrawstatus if the progress bar has changed position in a
visible way, but on my system (Windows XP, gvim 7.2.437) this wasn't
needed.

On an unrelated note...should we continue this on vim_use, vim_dev, or
both? I original posted to both groups so that I could potentially get
more comments, but now it seems we're deep in developer mode.

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Benjamin Fritz


On Jun 2, 8:43 am, Ben Fritz <[hidden email]> wrote:

> On Jun 2, 3:13 am, "Christian Brabandt" <[hidden email]> wrote:
>
> > On Wed, June 2, 2010 1:04 am, Benjamin Fritz wrote:
> > > - Added a second progress bar for the attributes processing (previously I
> > > was
> > >   doing this with a %d/%d printf).
> > > I also removed the "redrawstatus" commands. They did not seem to be
> > > necessary,
> > > and only served to increase the processing time. Did you have a particular
> > > reason to include them?
>
> > Yes. I can't see the progressbar otherwise. It is not updating for me. So
> > I suggest, enabling it again.
>
> Interesting. Is this true, even with the progress bar on the top
> window as in the latest version? If it is on the bottom, the screen
> clears again for a redraw immediately after the bottom status line is
> drawn, making it very hard to see the progress bar...which is why I
> put it on the top.
>

This got me thinking...so tried the following:

gvim -N -u NONE -i NONE
filetype indent plugin on
syntax on
:help intro
:source $HOME/vimfiles/syntax/2html.vim

I do not see the progress bars! The status lines never get drawn at
all (and the conversion is lightning fast). It appears I have
something in my Vim setup that slows things down enough, or redraws
the window, or something, that causes the status lines to appear when
normally they would not.

Any ideas? I don't see anything suspicious in my usual Buf/Win Enter/
Leave autocmds:


--- Auto-Commands ---
style_highlight  WinEnter
    *         if !exists('w:created') |   call
matchadd('WhitespaceError','\S\@<=\s\+\%#\@<!$') | endif
              if &filetype=~'\v<%(c|vim|dosbatch)>' && !
exists('w:tabs_are_bad') |   let w:tabs_are_bad =
matchadd('WhitespaceError',"\t") | endif
misc  WinEnter
    *         if !exists('w:created') && &ft!='qf' |   setlocal nowrap
| endif
matchparen  WinEnter
    *         call s:Highlight_Matching_Pair()
misc  WinEnter
    *         let w:created=1

--- Auto-Commands ---
insertmode  WinLeave
    *         if exists('w:last_fdm') | let &l:foldmethod=w:last_fdm |
unlet w:last_fdm | endif

--- Auto-Commands ---
filetypedetect  BufEnter
    *.xpm     if getline(1) =~ "XPM2" |   setf xpm2 | else |   setf
xpm | endif
    *.xpm2    setf xpm2
misc  BufEnter
    *         if (&ft=='qf' || &previewwindow || bufname('%') ==#
"__Tag_List__") && !exists('s:scrolloff_sav') |   let
s:scrolloff_sav=&scrolloff |   set scrolloff=0 | endif
repeatPlugin  BufEnter
    *         if g:repeat_tick == 0|let g:repeat_tick = b:changedtick|
endif
FileExplorer  BufEnter
    *         silent! call s:LocalBrowse(expand("<amatch>"))
    .*        silent! call s:LocalBrowse(expand("<amatch>"))
BufEnter
    *.vba     setlocal bt=nofile fmr=[[[,]]] fdm=marker|if &ff !=
'unix'| setlocal ma ff=unix noma |endif|call
vimball#ShowMesg(0,"Source this file to extract it! (:so %)")
    *.vba.gz  setlocal bt=nofile fmr=[[[,]]] fdm=marker|if &ff !=
'unix'| setlocal ma ff=unix noma |endif|call
vimball#ShowMesg(0,"Source this file to extract it! (:so %)")
    *.vba.bz2 setlocal bt=nofile fmr=[[[,]]] fdm=marker|if &ff !=
'unix'| setlocal ma ff=unix noma |endif|call
vimball#ShowMesg(0,"Source this file to extract it! (:so %)")
    *.vba.zip setlocal bt=nofile fmr=[[[,]]] fdm=marker|if &ff !=
'unix'| setlocal ma ff=unix noma |endif|call
vimball#ShowMesg(0,"Source this file to extract it! (:so %)")

--- Auto-Commands ---
misc  BufLeave
    *         if (&ft=='qf' || &previewwindow || bufname('%') ==#
"__Tag_List__") && exists('s:scrolloff_sav') |   let
&scrolloff=s:scrolloff_sav |   unlet s:scrolloff_sav | endif
repeatPlugin  BufLeave
    *         let g:repeat_tick = (g:repeat_tick == b:changedtick ||
g:repeat_tick == 0) ? 0 : -1

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Benjamin Fritz
On Wed, Jun 2, 2010 at 8:55 AM, Ben Fritz <[hidden email]> wrote:
> It appears I have
> something in my Vim setup that slows things down enough, or redraws
> the window, or something, that causes the status lines to appear when
> normally they would not.
>
> Any ideas? I don't see anything suspicious in my usual Buf/Win Enter/
> Leave autocmds:
>

Found it. I have the following in my .vimrc:

        " Highlight trailing whitespace, unless it is being entered now
        autocmd WinEnter,VimEnter *
              \ if !exists('w:created') |
              \   call matchadd('WhitespaceError','\S\@<=\s\+\%#\@<!$') |
              \ endif
        " necessary to have it highlight a just-entered trailing space
        autocmd InsertLeave * redraw!

The InsertLeave command is triggering because 2html works using
normal! a... commands. I have the autocmd because of the following
text in :help /\%# :

        WARNING: When the cursor is moved after the pattern was used, the
        result becomes invalid.  Vim doesn't automatically update the matches.
        This is especially relevant for syntax highlighting and 'hlsearch'.
        In other words: When the cursor moves the display isn't updated for
        this change.  An update is done for lines which are changed (the whole
        line is updated) or when using the |CTRL-L| command (the whole screen
        is updated).

So anyway, it looks like we need to do the redrawstatus. It would be
easiest to just use redrawstatus! right after calling the incr
function. I wonder how much of a performance impact this has? If it is
significant, we should probably call it only sometimes, perhaps after
a progress bar position update (which would mean the number of
processed lines would not be updated in real-time on large files). Any
thoughts/input?

It appears we also need to ignore some events; at least BufEnter,
WinEnter, InsertEnter, BufLeave, WinLeave, and InsertLeave. Syntax
seems like a good idea as well, but then we'd want to do the syntax
highlight when done, so perhaps we can leave this one in. Thoughts on
this? Is this really needed as part of the same patch or would this be
a future, performance-enhancing-only, patch?

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Christian Brabandt
In reply to this post by Benjamin Fritz
On Wed, June 2, 2010 4:36 am, Benjamin Fritz wrote:

> Any additional comments or fixes?

Try the attached version:

- Check for +float
- Should work better with smaller window sizes
- Make the progressbar for the attribute processing slightly slower
  (it was too fast, to notice it)
- small enhancements to how the progressbar works and how it displays.
- don't show any content from the html window

regards,
Christian

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

2html.vim (39K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Benjamin Fritz
On Thu, Jun 3, 2010 at 3:18 PM, Christian Brabandt <[hidden email]> wrote:

>
> Try the attached version:
>
> - Check for +float
> - Should work better with smaller window sizes
> - Make the progressbar for the attribute processing slightly slower
>  (it was too fast, to notice it)
> - small enhancements to how the progressbar works and how it displays.
> - don't show any content from the html window
>
More tweaks. This one is about twice as fast as Christian's, which it
accomplishes by only redrawing when the progress bar has changed
position.

Question: Christian's version calls :redrawstatus on the original
window, but the new window is updated perfectly fine. :help
:redrawstatus seems to indicate that only the current window will be
redrawn unless the ! is given. What gives?

Regardless, I have fixed the above issue and made a couple more minor
fixes, including getting the entire title to display on my teensy
laptop screen.

This version is still not fast enough though. It is about 30% slower
when the progress bar is enabled than when it is disabled. While I
consider it a good tradeoff in most cases, we could certainly make it
better.

It would probably be faster to pre-calculate the line numbers needed
to advance the progress bar rather than doing a bunch of
floating-point math every cycle.

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

2html.vim (52K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Benjamin Fritz
On Thu, Jun 3, 2010 at 10:03 PM, Benjamin Fritz <[hidden email]> wrote:
> This version is still not fast enough though. It is about 30% slower
> when the progress bar is enabled than when it is disabled. While I
> consider it a good tradeoff in most cases, we could certainly make it
> better.
>
> It would probably be faster to pre-calculate the line numbers needed
> to advance the progress bar rather than doing a bunch of
> floating-point math every cycle.
>

I've attached a new version which pre-calculates the (integer) line
numbers needed to advance the progress bar. Now all the floating point
math is done once, up front.

The difference is not really very perceptible. I timed the execution
on two files. First, I did the 5148-line autoload/phpcomplete.vim
script. Timings were as follows on my laptop:

progress disabled:
average: 46 seconds

floating-point progress:
average: 61 seconds
slowdown: 15 seconds longer than without progress bar
percentage: 33% longer than without progress bar

precalculated progress:
average: 62 seconds
slowdown: 16 seconds longer than without progress bar
percentage: 35% longer than without progress bar

Next I did a 33258-line C code file:

progress disabled:
average: 691 seconds

floating-point progress:
average: 716 seconds
slowdown: 25 seconds longer than without progress bar
percentage: 4% longer than without progress bar

precalculated progress:
average: 711 seconds
slowdown: 20 seconds longer than without progress bar
percentage: 3% longer than without progress bar

I also did a number of very small sections of files (my usual use case
for 2html) and did not notice any significant slowing; it only takes
1-2 seconds longer for a 100 or 200 line selection.

I take a few things from this.

First of all, I don't think we'll get much performance improvement
with this method. I do not know whether it is setting the status line
and redrawing it, or whether it is the use of the object-oriented
style functions, but it would probably require a different approach to
get a significant speedup. I certainly like the look a lot better than
the echo method, even if we could get echon working. Is a 10-20 second
slow-down acceptable on large numbers of lines, if the normal
execution time is measured in minutes anyway? To me, it certainly is.
If something is going to be taking more than a few minutes, I want a
progress bar to tell me whether it's worth letting it continue. Since
the slow-down can be significant for midsize files, we will certainly
need to mention in the :help that disabling the progress bar will make
the conversion faster. Maybe we should only show the progress bar
after some amount of time has elapsed? We could suppress the
redrawstatus until 10 seconds have passed, or something like that. Any
thoughts?

Secondly, the precalculated version is not really any faster than the
full floating-point calculation every cycle. I don't really have an
opinion of which method gives more readable code. Does anyone else
have any opinions on which version to keep? I think it would be
possible to do away with floating point calculations entirely using
the precalculated version; currently floating point is only used in
the calculate_ticks function. This might be desireable so that we can
remove the dependence on the +float feature, which is not marked with
a "smallest version" indicator in :help +feature-list. This apparently
means it is "system dependent". Does this mean float is pretty much
always included, unless it is explicitly removed? How common are Vims
without floating-point support? I already added use of the split()
function, which was added in version 7, so this won't work on really
old Vims...but do we want to support Vim 7.1 and earlier?

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

2html.vim (53K) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Progress indicator for :TOhtml command

Nikolay Aleksandrovich Pavlov
Ответ на сообщение «Re: Progress indicator for :TOhtml command»,
присланное в 19:39:15 05 июня 2010, Суббота,
отправитель Benjamin Fritz:

Why do you need float math for progress bar? Integer division is enough unless
you are going to show progress in %.1f or even more precious format. %3.d format
that you are using is integer and you obviously can't show half of a character.

Текст сообщения:
> On Thu, Jun 3, 2010 at 10:03 PM, Benjamin Fritz <[hidden email]>
wrote:

> > This version is still not fast enough though. It is about 30% slower
> > when the progress bar is enabled than when it is disabled. While I
> > consider it a good tradeoff in most cases, we could certainly make it
> > better.
> >
> > It would probably be faster to pre-calculate the line numbers needed
> > to advance the progress bar rather than doing a bunch of
> > floating-point math every cycle.
>
> I've attached a new version which pre-calculates the (integer) line
> numbers needed to advance the progress bar. Now all the floating point
> math is done once, up front.
>
> The difference is not really very perceptible. I timed the execution
> on two files. First, I did the 5148-line autoload/phpcomplete.vim
> script. Timings were as follows on my laptop:
>
> progress disabled:
> average: 46 seconds
>
> floating-point progress:
> average: 61 seconds
> slowdown: 15 seconds longer than without progress bar
> percentage: 33% longer than without progress bar
>
> precalculated progress:
> average: 62 seconds
> slowdown: 16 seconds longer than without progress bar
> percentage: 35% longer than without progress bar
>
> Next I did a 33258-line C code file:
>
> progress disabled:
> average: 691 seconds
>
> floating-point progress:
> average: 716 seconds
> slowdown: 25 seconds longer than without progress bar
> percentage: 4% longer than without progress bar
>
> precalculated progress:
> average: 711 seconds
> slowdown: 20 seconds longer than without progress bar
> percentage: 3% longer than without progress bar
>
> I also did a number of very small sections of files (my usual use case
> for 2html) and did not notice any significant slowing; it only takes
> 1-2 seconds longer for a 100 or 200 line selection.
>
> I take a few things from this.
>
> First of all, I don't think we'll get much performance improvement
> with this method. I do not know whether it is setting the status line
> and redrawing it, or whether it is the use of the object-oriented
> style functions, but it would probably require a different approach to
> get a significant speedup. I certainly like the look a lot better than
> the echo method, even if we could get echon working. Is a 10-20 second
> slow-down acceptable on large numbers of lines, if the normal
> execution time is measured in minutes anyway? To me, it certainly is.
> If something is going to be taking more than a few minutes, I want a
> progress bar to tell me whether it's worth letting it continue. Since
> the slow-down can be significant for midsize files, we will certainly
> need to mention in the :help that disabling the progress bar will make
> the conversion faster. Maybe we should only show the progress bar
> after some amount of time has elapsed? We could suppress the
> redrawstatus until 10 seconds have passed, or something like that. Any
> thoughts?
>
> Secondly, the precalculated version is not really any faster than the
> full floating-point calculation every cycle. I don't really have an
> opinion of which method gives more readable code. Does anyone else
> have any opinions on which version to keep? I think it would be
> possible to do away with floating point calculations entirely using
> the precalculated version; currently floating point is only used in
> the calculate_ticks function. This might be desireable so that we can
> remove the dependence on the +float feature, which is not marked with
> a "smallest version" indicator in :help +feature-list. This apparently
> means it is "system dependent". Does this mean float is pretty much
> always included, unless it is explicitly removed? How common are Vims
> without floating-point support? I already added use of the split()
> function, which was added in version 7, so this won't work on really
> old Vims...but do we want to support Vim 7.1 and earlier?
>

signature.asc (205 bytes) Download Attachment
12