editing large file

classic Classic list List threaded Threaded
10 messages Options
Reply | Threaded
Open this post in threaded view
|

editing large file

luke-44
Hi.

I need to edit a large file of around 300M
Will vi be able to do this?
What are the restrictions on file size?

Kind regards.
Luke.
--
............._..
.|  .| |.|/.|_ .
.|__.|_|.|\.|_ .
:61 421 276 282:
Reply | Threaded
Open this post in threaded view
|

Re: editing large file

Pete Johns
On Thu, 2006-03-30 at 08:58:21 +0930, luke@c... sent:
>Hi.
>
G'day!


>I need to edit a large file of around 300M Will vi be able to do
>this?  What are the restrictions on file size?
>
Vim should be able to handle this.

See :help limits for more information related to your particular
setup.

--paj
--
Pete Johns                                   <http://johnsy.com/>
Tel/Fax numbers and IM information:  <http://johnsy.com/contact/>
Flickr             <http://johnsy.com/blog/?entry=20060328203020>
Reply | Threaded
Open this post in threaded view
|

Re: editing large file

Bugzilla from diwaker.lists@gmail.com
In reply to this post by luke-44
> I need to edit a large file of around 300M
> Will vi be able to do this?

Incidentally, yesterday night I had the need to edit a file that was >
1G in size. And I did manage to edit it with Vim, though it was, err,
slow (to be fair, my machine only has 1.5G ram).

Vim rocks! I can't even imagine opening that file with any other editor.

> What are the restrictions on file size?
>
> Kind regards.
> Luke.
> --
> ............._..
> .|  .| |.|/.|_ .
> .|__.|_|.|\.|_ .
> :61 421 276 282:
>


--
Web/Blog/Gallery: http://floatingsun.net/blog
Reply | Threaded
Open this post in threaded view
|

Re: editing large file

John Chapman-2
In reply to this post by luke-44
[hidden email] wrote:

>Hi.
>
>I need to edit a large file of around 300M
>Will vi be able to do this?
>What are the restrictions on file size?
>
>Kind regards.
>Luke.
>  
>
vim can handle it.

Just be aware that it will load the entire file, which will slow it down
a bit.

If you can, you may wish to consider using sed and other unix tools to
edit a text _stream_, as follows:
cat large_file | stream_editor > resultant_large_file

(Where stream_editor could be, eg, sed)

Reply | Threaded
Open this post in threaded view
|

Re: editing large file

panshizhu
In reply to this post by luke-44
[hidden email] wrote on 2006.03.30 07:28:21:

> Hi.
>
> I need to edit a large file of around 300M
> Will vi be able to do this?
> What are the restrictions on file size?
>
> Kind regards.
> Luke.
> --
> ............._..
> .|  .| |.|/.|_ .
> .|__.|_|.|\.|_ .
> :61 421 276 282:

If you have to open very large files, you probably should never rely on
vanilla vi.

Frankly speaking, this is the reason I switch from vi to vim, since it will
take so long to open a huge file in vi.

--
Sincerely
Pan, Shizhu. ext: 2221

Reply | Threaded
Open this post in threaded view
|

Re: editing large file

Tim Chase-2
In reply to this post by luke-44
> I need to edit a large file of around 300M
> Will vi be able to do this?
> What are the restrictions on file size?

In addition to the other comments here, that *yes*, you can
edit such large files in Vim, you may want to turn off some
features that are nice-to-haves on small files, but grind
your system to the pace of molassass running uphill in
winter.  Digging in my vim-list archive, William Natter
posted this section from his vimrc:


let g:SaveUndoLevels = &undolevels
let g:BufSizeThreshold = 1000000
if has("autocmd")
au VimEnter * let g:SaveUndoLevels = &undolevels
au BufReadPre * if getfsize(expand("%")) >=
g:BufSizeThreshold | setlocal noswapfile | endif
au BufEnter * if getfsize(expand("%")) < g:BufSizeThreshold
| let &undolevels=g:SaveUndoLevels | hi Cursor
endif

One of the biggest impacts would be cranking down your
'undolevels' (to 0 or -1) setting.  Additionally, syntax
highlighting can slow you down (especially if your syntax
file is specified to synch from the beginning of the file).
  I don't know if monkeying with the swapfile is for better
or for worse.

Just a few tips for editing humongo files.

-tim



Reply | Threaded
Open this post in threaded view
|

Re: editing large file

Hari Krishna Dara
In reply to this post by luke-44

On Wed, 29 Mar 2006 at 8:06pm, Tim Chase wrote:

> > I need to edit a large file of around 300M
> > Will vi be able to do this?
> > What are the restrictions on file size?
>
> In addition to the other comments here, that *yes*, you can
> edit such large files in Vim, you may want to turn off some
> features that are nice-to-haves on small files, but grind
> your system to the pace of molassass running uphill in
> winter.  Digging in my vim-list archive, William Natter
> posted this section from his vimrc:
>
>
> let g:SaveUndoLevels = &undolevels
> let g:BufSizeThreshold = 1000000
> if has("autocmd")
> au VimEnter * let g:SaveUndoLevels = &undolevels
> au BufReadPre * if getfsize(expand("%")) >=
> g:BufSizeThreshold | setlocal noswapfile | endif
> au BufEnter * if getfsize(expand("%")) < g:BufSizeThreshold
> | let &undolevels=g:SaveUndoLevels | hi Cursor
> endif
>
> One of the biggest impacts would be cranking down your
> 'undolevels' (to 0 or -1) setting.  Additionally, syntax
> highlighting can slow you down (especially if your syntax
> file is specified to synch from the beginning of the file).
>   I don't know if monkeying with the swapfile is for better
> or for worse.
>
> Just a few tips for editing humongo files.
>
> -tim

Here is a more complete version:

" Based on a post by Dr. Chip Campbell
" Augroup LargeFile: for large files: turn undo off, etc (based on vim tip
#611)
" {{{2
let g:LargeFileThreshold = 10*1024*1024 " in megabytes
augroup LargeFile
  au BufReadPre * call <SID>SetLargeFileSettings()
augroup END

function! s:SetLargeFileSettings()
  let f = expand('<afile>')
  if getfsize(f) < g:LargeFileThreshold
    return
  endif

  let b:eikeep= &ei
  let b:ulkeep= &ul
  set ei=FileType
  setlocal noswf bh=unload
  let f=escape(substitute(f,'','/','g'),' ')
  exe "au LargeFile BufEnter ".f." set ul=-1"
  exe "au LargeFile BufLeave ".f." let &ul=".b:ulkeep."|set ei=".b:eikeep
  exe "au LargeFile BufUnload ".f." au! LargeFile * ". f
  echomsg "***note*** handling a large file"
endfunction
" }}}2

--
HTH,
Hari

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around
http://mail.yahoo.com 
Reply | Threaded
Open this post in threaded view
|

Re: editing large file

Benji Fisher
On Wed, Mar 29, 2006 at 11:10:21PM -0800, Hari Krishna Dara wrote:
>
> Here is a more complete version:
>
> " Based on a post by Dr. Chip Campbell
> " Augroup LargeFile: for large files: turn undo off, etc (based on vim tip
> #611)
[snip]

     Has anyone posted this as a vim tip?

http://www.vim.org/tips/index.php

                                        --Benji Fisher
Reply | Threaded
Open this post in threaded view
|

Re: editing large file

Hari Krishna Dara
In reply to this post by luke-44

On Thu, 30 Mar 2006 at 11:20am, Benji Fisher wrote:

> On Wed, Mar 29, 2006 at 11:10:21PM -0800, Hari Krishna Dara wrote:
> >
> > Here is a more complete version:
> >
> > " Based on a post by Dr. Chip Campbell
> > " Augroup LargeFile: for large files: turn undo off, etc (based on vim tip
> > #611)
> [snip]
>
>      Has anyone posted this as a vim tip?
>
> http://www.vim.org/tips/index.php
>

As the comment says, it was posted as tip 611, but if you are asking
about my modification, no I haven't posted it back, but I think my
modifications were merely cosmetic.

--
Thanks,
Hari

__________________________________________________
Do You Yahoo!?
Tired of spam?  Yahoo! Mail has the best spam protection around
http://mail.yahoo.com 
Reply | Threaded
Open this post in threaded view
|

Re: editing large file

Randall Rice

On Mar 30, 2006, at 11:22 PM, Hari Krishna Dara wrote:

>
> On Thu, 30 Mar 2006 at 11:20am, Benji Fisher wrote:
>
>> On Wed, Mar 29, 2006 at 11:10:21PM -0800, Hari Krishna Dara wrote:
>>>
>>> Here is a more complete version:
>>>
>>> " Based on a post by Dr. Chip Campbell
>>> " Augroup LargeFile: for large files: turn undo off, etc (based  
>>> on vim tip
>>> #611)
>> [snip]
>>
>>      Has anyone posted this as a vim tip?
>>
>> http://www.vim.org/tips/index.php
>>
>
> As the comment says, it was posted as tip 611, but if you are asking
> about my modification, no I haven't posted it back, but I think my
> modifications were merely cosmetic.


Dr. Chip actually conveniently posted his version as a script today.

http://www.vim.org/scripts/script.php?script_id=1506


Brent