vimdiff: ignore linebreaks

classic Classic list List threaded Threaded
21 messages Options
12
Reply | Threaded
Open this post in threaded view
|

vimdiff: ignore linebreaks

Claus Atzenbeck
Hello:

I want to use vimdiff for comparing LaTeX files. For that, changes in white spaces as well as line breaks should be ignored.

For example, the following two paragraphs should be considered as the same, as the TeX output would be exactly the same:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The  quick      brown
fox      jumps   over
the lazy          dog.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The quick brown fox jumps over
the lazy dog.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

Multiple white spaces may be ignored via "diffopt+=iwhite", however, what about line breaks?

Is there a way to teach vimdiff to ignore any line break within a paragraph?

Thanks for any hint.
Claus

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: vimdiff: ignore linebreaks

Tim Chase
> For example, the following two paragraphs should be considered as the same, as the TeX output would be exactly the same:
>
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> The  quick      brown
> fox      jumps   over
> the lazy          dog.
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
> The quick brown fox jumps over
> the lazy dog.
> ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
>
> Multiple white spaces may be ignored via "diffopt+=iwhite", however, what about line breaks?
>
> Is there a way to teach vimdiff to ignore any line break within a paragraph?

Not that I know of off the top of my head.  However, at least on
my Linux boxes, I have a "wdiff" tool available which could be
used to show where differences occur with cross-line granularity.
  I don't know how well it could be made to integrate with Vim,
but I use it occasionally for diffing flowing text.  wdiff allows
you to specify the start/end strings for marking
insertions/deletions if that's of any help.

-tim



--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: vimdiff: ignore linebreaks

MarcWeber
In reply to this post by Claus Atzenbeck
Excerpts from Claus Atzenbeck's message of Thu Jan 12 22:32:16 +0100 2012:
> Is there a way to teach vimdiff to ignore any line break within a paragraph?
AFAIK text diff tools are always line based. Thus teh quickest way is to
replace all \n  and ' ' chars by the same character such as ' '
Then you'll get:

~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The quick brown fox jumps over the lazy dog.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
The quick brown fox jumps over the lazy dog.
~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~

If you have long paragraphs it may not help much.

If you replace '.' by ".\n" manually then you'll get something which might
be comparable. Thus this is an attempt to normalize the output.

This is only for finding differences. You still have to edit the
original file :(

Marc Weber

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: vimdiff: ignore linebreaks

Claus Atzenbeck
In reply to this post by Tim Chase
Am 12.01.2012 um 23:54 schrieb Tim Chase:

> Not that I know of off the top of my head.  However, at least on my Linux boxes, I have a "wdiff" tool available which could be used to show where differences occur with cross-line granularity.  I don't know how well it could be made to integrate with Vim, but I use it occasionally for diffing flowing text.  wdiff allows you to specify the start/end strings for marking insertions/deletions if that's of any help.

Thanks, Tim. (Also thanks to Marc for the quick reply.)

I installed wdiff and dwdiff. Both can do diffs at word granularity level. Perfect. I could put it into vim's "diffexpr".

However, how do I get wdiff or dwdiff to produce an output that vim can deal with?

Thanks so much for any pointer.
Claus

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: vimdiff: ignore linebreaks

Taylor Hedberg
Claus Atzenbeck, Fri 2012-01-13 @ 01:01:59+0100:
> However, how do I get wdiff or dwdiff to produce an output that vim
> can deal with?

I'm not sure you can. According to `:help diff-diffexpr`, Vim requires
the output of the diff program to be an "ed-style" diff, which is
fundamentally linewise. Maybe you could come up with a wrapper script
that would mangle wdiff's output into ed format, but even if possible,
that seems like it would be error-prone and probably more trouble than
it's worth.

You'd have to patch Vim itself to add support for this, it seems.

attachment0 (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: vimdiff: ignore linebreaks

Claus Atzenbeck
Am 13.01.2012 um 15:46 schrieb Taylor Hedberg:

> I'm not sure you can. According to `:help diff-diffexpr`, Vim requires
> the output of the diff program to be an "ed-style" diff, which is
> fundamentally linewise. Maybe you could come up with a wrapper script
> that would mangle wdiff's output into ed format, but even if possible,
> that seems like it would be error-prone and probably more trouble than
> it's worth.

I see your point.

Interestingly, it seems that word wise diffs are not that much needed/wanted for Vim.

Anyway, thanks for your help.

Best,
Claus

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: vimdiff: ignore linebreaks

Marcin Szamotulski
On 16:04 Fri 13 Jan     , Claus Atzenbeck wrote:

> Am 13.01.2012 um 15:46 schrieb Taylor Hedberg:
>
> > I'm not sure you can. According to `:help diff-diffexpr`, Vim requires
> > the output of the diff program to be an "ed-style" diff, which is
> > fundamentally linewise. Maybe you could come up with a wrapper script
> > that would mangle wdiff's output into ed format, but even if possible,
> > that seems like it would be error-prone and probably more trouble than
> > it's worth.
>
> I see your point.
>
> Interestingly, it seems that word wise diffs are not that much needed/wanted for Vim.
They are!, especially for people using LaTeX with vim like me, though I don't
know a solution. LaTeX has latexdiff to make nice diffs, but using it is not
easy and usually takes some time to make a diff. Since it produces pdf output
it serves for different purposes.

Best,
Marcin

>
> Anyway, thanks for your help.
>
> Best,
> Claus
>
> --
> You received this message from the "vim_use" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: vimdiff: ignore linebreaks

Claus Atzenbeck
Am 13.01.2012 um 17:38 schrieb Marcin Szamotulski:

>> Interestingly, it seems that word wise diffs are not that much needed/wanted for Vim.
> They are!, especially for people using LaTeX with vim like me, though I don't
> know a solution. LaTeX has latexdiff to make nice diffs, but using it is not
> easy and usually takes some time to make a diff. Since it produces pdf output
> it serves for different purposes.

I used latexdiff in the past. Nice, but not what I want now. I want to use a version control system to ease collaborative working on (LaTeX) documents in my team.

Diff granularity at word level (instead of lines) would be essential to me for that. I quickly looked at tools like Araxis.com. They look great, however, as Vim has been my editor of choice for years, I rather would stick with it for the whole work, including diff, merge, and edits.

Best,
Claus

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: vimdiff: ignore linebreaks

Tim Chase
On 01/13/12 11:18, Claus Atzenbeck wrote:
>>> Interestingly, it seems that word wise diffs are not that
>>> much needed/wanted for Vim.
>>
>> They are!
>
> I used latexdiff in the past. Nice, but not what I want now. I
> want to use a version control system to ease collaborative
> working on (LaTeX) documents in my team.

I agree that word-level diffs in Vim would be nice.  For my
purposes, I tend to either

1) not insert hard line-breaks in my text, so that it's one whole
line (which Vim correctly diffs), or

2) temporarily issue

   :%s/\W\+/\r/g
   :v/\w/d

in both windows to put EVERY word on its own line (including
markup), and then diff the results of that.  Usually it's
informative.  I then just undo in both windows and have my
original documents back.

-tim


--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: vimdiff: ignore linebreaks

Claus Atzenbeck
Am 13.01.2012 um 19:13 schrieb Tim Chase:

> I agree that word-level diffs in Vim would be nice.  For my purposes, I tend to either
>
> 1) not insert hard line-breaks in my text, so that it's one whole line (which Vim correctly diffs), or

This is not a good option, as it is hard to move around with the cursor keys "hjkl". Furthermore, it takes an additional step to remove hard line breaks coming in from others.

Hard line breaks usually would not be an issue if nobody starts reformatting paragraphs, which could easily the case.

> 2) temporarily issue
>
>  :%s/\W\+/\r/g
>  :v/\w/d
>
> in both windows to put EVERY word on its own line (including markup), and then diff the results of that.  Usually it's informative.  I then just undo in both windows and have my original documents back.

Undo is not an option if I start merging or modifying in diff mode.

Unfortunately, vim does not seem to be a good solution for word-granularity diffing. *sigh*

Anyway, thanks for your feedback.

Best,
Claus

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: vimdiff: ignore linebreaks

Chris Lott
On Fri, Jan 13, 2012 at 12:10 PM, Claus Atzenbeck
<[hidden email]> wrote:
>
> This is not a good option, as it is hard to move around with the cursor keys "hjkl". Furthermore, it takes an additional step to remove hard line breaks coming in from others.
>
> Hard line breaks usually would not be an issue if nobody starts reformatting paragraphs, which could easily the case.

One can remap hjkl to go line by line visually, which is what I do as
I write primarily prose and for various reasons choose to use one line
per paragraph.

Bouncing back and forth between soft and hard breaks is simple enough.

c
--
Chris Lott <[hidden email]>

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: vimdiff: ignore linebreaks

Claus Atzenbeck
Am 13.01.2012 um 22:39 schrieb Chris Lott:

> One can remap hjkl to go line by line visually, which is what I do as
> I write primarily prose and for various reasons choose to use one line
> per paragraph.

Something to be tested. I remapped hjkl to go by visual lines. Let's see how it goes.

Best,
Claus

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: vimdiff: ignore linebreaks

Taylor Hedberg
In reply to this post by Chris Lott
Chris Lott, Fri 2012-01-13 @ 12:39:22-0900:
> One can remap hjkl to go line by line visually, which is what I do as
> I write primarily prose and for various reasons choose to use one line
> per paragraph.

I have the following in my .vimrc, which lets me toggle between "screen
line" and "buffer line" movement by pressing <F7>:

    function! ToggleScreenLineMovementMode()
        if !exists("g:screen_line_movement_mode")
            let g:screen_line_movement_mode = 0
        endif
        if !g:screen_line_movement_mode
            echomsg 'Moving by screen lines'
            let g:screen_line_movement_mode = 1
            nnoremap j gj
            nnoremap k gk
            nnoremap $ g$
            nnoremap 0 g0
            nnoremap ^ g^
        else
            echomsg 'Moving by buffer lines'
            let g:screen_line_movement_mode = 0
            nunmap j
            nunmap k
            nunmap $
            nunmap 0
            nunmap ^
        endif
    endfunction
    nnoremap <silent> <F7> :call ToggleScreenLineMovementMode()<CR>

The only caveat is that it will clobber any pre-existing mappings you
have for j, k, $, 0, and ^. I don't have any, so it's no problem for me,
but that may not be the case for everyone.

attachment0 (853 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

RE: vimdiff: ignore linebreaks

JohnBeckett
In reply to this post by Claus Atzenbeck
Claus Atzenbeck wrote:
>> One can remap hjkl to go line by line visually
>
> Something to be tested. I remapped hjkl to go by visual
> lines. Let's see how it goes.

You can make hjkl work well, but the diffs still suck.

Vim shows diffs on a line by highlighting all text from the
first change on the line to the last. When you have long lines
that is pretty useless (far too much unchanged text is
highlighted).

I doubt if I'll ever get around to it, but investigating a
solution is something I'd like to do. I was thinking (without
having looked at the relevant source) that the best attack might
be to use long lines, but patch Vim's highlighting of the diffs
by inserting code to do diffs between two long lines (which is
essentially duplicating what diff does for lines...bit ugly and
potentially slow).

One reason I haven't looked at it is that my guess is that there
would be a lot of hurdles.

John

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: vimdiff: ignore linebreaks

Bram Moolenaar
In reply to this post by Claus Atzenbeck

Claus Atzenbeck wrote:

> Am 13.01.2012 um 15:46 schrieb Taylor Hedberg:
>
> > I'm not sure you can. According to `:help diff-diffexpr`, Vim requires
> > the output of the diff program to be an "ed-style" diff, which is
> > fundamentally linewise. Maybe you could come up with a wrapper script
> > that would mangle wdiff's output into ed format, but even if possible,
> > that seems like it would be error-prone and probably more trouble than
> > it's worth.
>
> I see your point.
>
> Interestingly, it seems that word wise diffs are not that much
> needed/wanted for Vim.

It's not that they are not wanted, it's that there is no standard syntax
for an external diff program output unless it's by lines.

I would love to include a better diff library in Vim, but still haven't
found a good one that could be included with Vim.

--
The MS-Windows registry is no more hostile than any other bunch of state
information... that is held in a binary format... a format that nobody
understands... and is replicated and cached in a complex and largely
undocumented way... and contains large amounts of duplicate and obfuscated
information...  (Ben Peterson)

 /// Bram Moolenaar -- [hidden email] -- http://www.Moolenaar.net   \\\
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\  an exciting new programming language -- http://www.Zimbu.org        ///
 \\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: vimdiff: ignore linebreaks

Tim Chase
On 01/15/12 08:03, Bram Moolenaar wrote:

>
> Claus Atzenbeck wrote:
>
>> Am 13.01.2012 um 15:46 schrieb Taylor Hedberg:
>>
>>> I'm not sure you can. According to `:help diff-diffexpr`, Vim requires
>>> the output of the diff program to be an "ed-style" diff, which is
>>> fundamentally linewise. Maybe you could come up with a wrapper script
>>> that would mangle wdiff's output into ed format, but even if possible,
>>> that seems like it would be error-prone and probably more trouble than
>>> it's worth.
>>
>> I see your point.
>>
>> Interestingly, it seems that word wise diffs are not that much
>> needed/wanted for Vim.
>
> It's not that they are not wanted, it's that there is no standard syntax
> for an external diff program output unless it's by lines.

Fortunately at least with wdiff, there's an option to change the
pre/post markers for the text added/deleted via the
--{start,end}-{delete,insert} command-line flags, so the diff
could accommodate a fairly arbitrary syntax.  When I've wanted to
see a word-diff, I usually use the default wdiff tags and then do
something like

   wdiff 1.txt 2.txt | vi -c 'match DiffDelete /\[-.\{-}-]/' -c
'2match DiffAdd /{+.\{-}+}/' -

which visually highlights the changes.  This could be wrapped up
in a pretty simple shell/batch script.

-tim


--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: vimdiff: ignore linebreaks

Claus Atzenbeck
Am 15.01.2012 um 15:44 schrieb Tim Chase:

> Fortunately at least with wdiff, there's an option to change the pre/post markers for the text added/deleted via the --{start,end}-{delete,insert} command-line flags, so the diff could accommodate a fairly arbitrary syntax.  When I've wanted to see a word-diff, I usually use the default wdiff tags and then do something like
>
>  wdiff 1.txt 2.txt | vi -c 'match DiffDelete /\[-.\{-}-]/' -c '2match DiffAdd /{+.\{-}+}/' -
>
> which visually highlights the changes.  This could be wrapped up in a pretty simple shell/batch script.

Thanks Tim. This, however, would only produce a 3rd file showing the diffs. I'd like to see both diffing files in one window with the option to modify the one or the other.

Surprising that there is not a good library for that to be included in Vim, as Bram mentioned.

Best,
Claus

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: vimdiff: ignore linebreaks

Christian Brabandt
In reply to this post by Bram Moolenaar
Hallo Bram!

Bram Moolenaar schrieb am Sonntag, den 15. Januar 2012:

>
> Claus Atzenbeck wrote:
>
> > Am 13.01.2012 um 15:46 schrieb Taylor Hedberg:
> >
> > > I'm not sure you can. According to `:help diff-diffexpr`, Vim requires
> > > the output of the diff program to be an "ed-style" diff, which is
> > > fundamentally linewise. Maybe you could come up with a wrapper script
> > > that would mangle wdiff's output into ed format, but even if possible,
> > > that seems like it would be error-prone and probably more trouble than
> > > it's worth.
> >
> > I see your point.
> >
> > Interestingly, it seems that word wise diffs are not that much
> > needed/wanted for Vim.
>
> It's not that they are not wanted, it's that there is no standard syntax
> for an external diff program output unless it's by lines.
>
> I would love to include a better diff library in Vim, but still haven't
> found a good one that could be included with Vim.

I thought, I mention the diff-match-patch by Neil Fraser¹, that has been
mentioned the last time² this topic has been discussed here.

That looks like a nice addition, if something like this can be included
into Vim.

¹http://code.google.com/p/google-diff-match-patch/
²http://groups.google.com/group/vim_use/msg/8a588d748b2d7d60

regards,
Christian
--

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: vimdiff: ignore linebreaks

Bram Moolenaar

Christian Brabandt wrote:

> Bram Moolenaar schrieb am Sonntag, den 15. Januar 2012:
>
> >
> > Claus Atzenbeck wrote:
> >
> > > Am 13.01.2012 um 15:46 schrieb Taylor Hedberg:
> > >
> > > > I'm not sure you can. According to `:help diff-diffexpr`, Vim requires
> > > > the output of the diff program to be an "ed-style" diff, which is
> > > > fundamentally linewise. Maybe you could come up with a wrapper script
> > > > that would mangle wdiff's output into ed format, but even if possible,
> > > > that seems like it would be error-prone and probably more trouble than
> > > > it's worth.
> > >
> > > I see your point.
> > >
> > > Interestingly, it seems that word wise diffs are not that much
> > > needed/wanted for Vim.
> >
> > It's not that they are not wanted, it's that there is no standard syntax
> > for an external diff program output unless it's by lines.
> >
> > I would love to include a better diff library in Vim, but still haven't
> > found a good one that could be included with Vim.
>
> I thought, I mention the diff-match-patch by Neil Fraser¹, that has been
> mentioned the last time² this topic has been discussed here.
>
> That looks like a nice addition, if something like this can be included
> into Vim.
>
> ¹http://code.google.com/p/google-diff-match-patch/
> ²http://groups.google.com/group/vim_use/msg/8a588d748b2d7d60

Looks like there is a C++ implementation, no C implementation.

--
We apologise again for the fault in the subtitles.  Those responsible for
sacking the people who have just been sacked have been sacked.
                 "Monty Python and the Holy Grail" PYTHON (MONTY) PICTURES LTD

 /// Bram Moolenaar -- [hidden email] -- http://www.Moolenaar.net   \\\
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\  an exciting new programming language -- http://www.Zimbu.org        ///
 \\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: vimdiff: ignore linebreaks

L. A. Walsh
In reply to this post by Bram Moolenaar
Bram Moolenaar wrote:
> I would love
> to include a better diff library in Vim, but still haven't
> found a good one that could be included with Vim.
---

When it comes to diff improvements, how does one simply 'ignore all
white space' ala, "diff -w" (which should, IMO, work for the original
writer, (but doesn't... due to what I'd call a bug in 'diff'...)...

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
12