compressed undofiles

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

compressed undofiles

Adam Monsen
I use undofiles heavily. I often copy a file along with its undofile when creating a new similar file. I notice (a) the undofiles can get quite large and (b) they compress very efficiently. I tested compressing a few ".test.txt.un~" files with gzip and saw 90% compression.

Anyone interested in compressed undofiles?

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: compressed undofiles

Dominique Pellé
Adam Monsen <[hidden email]> wrote:

> I use undofiles heavily. I often copy a file along with its
> undofile when creating a new similar file. I notice (a)
> the undofiles can get quite large and (b) they compress
> very efficiently. I tested compressing a few ".test.txt.un~"
> files with gzip and saw 90% compression.
>
> Anyone interested in compressed undofiles?

Indeed, I just checked and I have 200 MiB of files in
my 'undodir' directory with ~4700 files there. So it's big.

Compression and decompression may slow down Vim
So lz4 lib [1] may be the way to go as it's very fast.
zstd lib [2] has a good compromise between speed
and compression ratio. Both libs are now widely used
and available, but not as much as zlib.

[1] https://github.com/lz4/lz4
[2] https://github.com/facebook/zstd

Dominique

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: compressed undofiles

Bram Moolenaar
In reply to this post by Adam Monsen

Adam Monsen wrote:

> I use undofiles heavily. I often copy a file along with its undofile when
> creating a new similar file. I notice (a) the undofiles can get quite large
> and (b) they compress very efficiently. I tested compressing a few
> ".test.txt.un~" files with gzip and saw 90% compression.
>
> Anyone interested in compressed undofiles?

That sounds like a useful option.  It could be implemented like a
filter, using gzip to read from stdin or write to stdout.

The largest proglem I see is to detect that an undo file is compressed,
know that the filter needs to be used.  Perhaps we could read some
number of bytes from the file and match a regexp on it?

--
hundred-and-one symptoms of being an internet addict:
25. You believe nothing looks sexier than a man in boxer shorts illuminated
    only by a 17" inch svga monitor.

 /// Bram Moolenaar -- [hidden email] -- http://www.Moolenaar.net   \\\
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\  an exciting new programming language -- http://www.Zimbu.org        ///
 \\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: compressed undofiles

Erik Christiansen
On 03.06.18 09:46, Bram Moolenaar wrote:

>
> Adam Monsen wrote:
>
> > I use undofiles heavily. I often copy a file along with its undofile when
> > creating a new similar file. I notice (a) the undofiles can get quite large
> > and (b) they compress very efficiently. I tested compressing a few
> > ".test.txt.un~" files with gzip and saw 90% compression.
> >
> > Anyone interested in compressed undofiles?
>
> That sounds like a useful option.  It could be implemented like a
> filter, using gzip to read from stdin or write to stdout.
>
> The largest proglem I see is to detect that an undo file is compressed,
> know that the filter needs to be used.  Perhaps we could read some
> number of bytes from the file and match a regexp on it?

If compression produces a binary file, i.e. contains bytes not found in
the character sets used by Vim, then that ought to be definitive, I
figure. If not, some "magic" could perhaps be prefixed, and removed on
decompression.

In any event, the distinguishing test ought to be replicable by the
"file" command. So, if "file" can already identify e.g. an lz4 file,
then its test would seem to suffice.

Erik

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|

Re: compressed undofiles

John Little-4
In reply to this post by Adam Monsen
On Sunday, June 3, 2018 at 3:48:59 PM UTC+12, Adam Monsen wrote:
> I use undofiles heavily.

> Anyone interested in compressed undofiles?

If you use a file system that supports transparent compression by directory, like btrfs or NTFS, there'd be no need for support in vim.  (Historically people were warned off such compression because of the overhead, but you're implicitly accepting that overhead, and with lots of cores these days...)

So, if you're on Windows, you're good to go.  If on Linux, btrfs is a good idea anyway IMO.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.