Internal error: hash_add()

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Internal error: hash_add()

Peter Odding-3
Hi all. I wrote a Vim plug-in* that automatically keeps my global tags
file up-to-date by executing Exuberant Ctags from a CursorHold autocmd.
I've been happily using the plug-in for months now and today it started
throwing the following error which I've never seen before:

easytags.vim: Vim(let):E685: Internal error: hash_add() (at function
easytags#autoload..easytags#highlight_cmd, line 4)

When I looked up :help E685 I found the following:

This is an internal error.  If you can reproduce it, please send in a
bug report.

The good news is that reproducing the E685 error is a matter of calling
taglist('.') using my tags file but the bad news is that the relevant
tags file contains 5810 lines (649K) of personal information (e.g.
extracts from several LaTeX documents) and I've never debugged Vim (nor
any other binary for that matter). I don't really mind providing the
tags file to someone who's willing to analyze this problem but I'm not
happy with posting the file on a public mailing list either :-)

Does anyone have suggestions?

Thanks in advance,

  - Peter Odding

PS. This is with a freshly compiled 7.2 with patches <= 442.

* http://www.vim.org/scripts/script.php?script_id=3114

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: Internal error: hash_add()

Bram Moolenaar

Peter Odding wrote:

> Hi all. I wrote a Vim plug-in* that automatically keeps my global tags
> file up-to-date by executing Exuberant Ctags from a CursorHold autocmd.
> I've been happily using the plug-in for months now and today it started
> throwing the following error which I've never seen before:
>
> easytags.vim: Vim(let):E685: Internal error: hash_add() (at function
> easytags#autoload..easytags#highlight_cmd, line 4)
>
> When I looked up :help E685 I found the following:
>
> This is an internal error.  If you can reproduce it, please send in a
> bug report.
>
> The good news is that reproducing the E685 error is a matter of calling
> taglist('.') using my tags file but the bad news is that the relevant
> tags file contains 5810 lines (649K) of personal information (e.g.
> extracts from several LaTeX documents) and I've never debugged Vim (nor
> any other binary for that matter). I don't really mind providing the
> tags file to someone who's willing to analyze this problem but I'm not
> happy with posting the file on a public mailing list either :-)
>
> Does anyone have suggestions?

Well, first save that tags file, generating it again may get rid of the
problem and we won't be able to reproduce it.

It's trange, hash tables are used a lot, it's unlikely that hash_add()
itself is wrong.  Perhaps memory got corrupted somehow?

Can you reproduce the problem without loading scripts, just calling
taglist('.') before doing anything else?

I'm afraid that anonymyzing your tags file may make the problem go
away...  Perhaps you can try ":%s/[a-zA-Z]/x/g" on it?  Make a copy
first.

Ah! when I do this on the Vim source tags file I can actually reproduce
it!  I'll put this in the todo list.

--
If Microsoft would build a car...
... You'd have to press the "Start" button to turn the engine off.

 /// Bram Moolenaar -- [hidden email] -- http://www.Moolenaar.net   \\\
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\        download, build and distribute -- http://www.A-A-P.org        ///
 \\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: Internal error: hash_add()

Lech Lorens
On 08-Jun-2010 Bram Moolenaar <[hidden email]> wrote:

>
> Well, first save that tags file, generating it again may get rid of the
> problem and we won't be able to reproduce it.
>
> It's trange, hash tables are used a lot, it's unlikely that hash_add()
> itself is wrong.  Perhaps memory got corrupted somehow?
>
> Can you reproduce the problem without loading scripts, just calling
> taglist('.') before doing anything else?
>
> I'm afraid that anonymyzing your tags file may make the problem go
> away...  Perhaps you can try ":%s/[a-zA-Z]/x/g" on it?  Make a copy
> first.
>
> Ah! when I do this on the Vim source tags file I can actually reproduce
> it!  I'll put this in the todo list.
I simplified the tags file to a single entry making the problem appear
(find the file attached).
I might be able to look at the problem tomorrow.

--
Cheers,
Lech

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

tags (409 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Internal error: hash_add()

Peter Odding-3
In reply to this post by Bram Moolenaar
Bram Moolenaar wrote:
> Well, first save that tags file, generating it again may get rid of the
> problem and we won't be able to reproduce it.

I already put the buggy tags file in a TAR to be sure I wouldn't
accidentally modify it. And the problem did indeed go away after
modifying the tags file. The modification that solved the problem was
:g/\.tex\t/d so the problem was with one of the tags entries for a LaTeX
document, which are very long (e.g. paragraph length) and contain spaces
and other weird characters in the tag name field. I don't use tags in
LaTeX documents anyway so I've since disabled all tag generation for
*.tex files.

> It's trange, hash tables are used a lot, it's unlikely that hash_add()
> itself is wrong.  Perhaps memory got corrupted somehow?
>
> Can you reproduce the problem without loading scripts, just calling
> taglist('.') before doing anything else?

Yes I tested that using the following command before sending the initial
message:

gvim -u NONE --noplugin -c ':set tags=~/.vimtags | call taglist(".")'

> I'm afraid that anonymyzing your tags file may make the problem go
> away...  Perhaps you can try ":%s/[a-zA-Z]/x/g" on it?  Make a copy
> first.
>
> Ah! when I do this on the Vim source tags file I can actually reproduce
> it!  I'll put this in the todo list.

Well that :substitute command replaces all extra flags in the tags file
as well. When I actually try it though, I get the same error message as
reported before, but now repeated a dozen times all over my screen :-)

Since you mentioned you can reproduce the problem (and I just saw a mail
by Lech Lorens stating the same) I guess there's no point in polluting
vim_dev with a 900K attachment.

Thanks for your work on Vim!

  - Peter Odding

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: Internal error: hash_add()

Lech Lorens
On 09-Jun-2010 Peter Odding <[hidden email]> wrote:
> Bram Moolenaar wrote:
> >Ah! when I do this on the Vim source tags file I can actually reproduce
> >it!  I'll put this in the todo list.
[...]
> Since you mentioned you can reproduce the problem (and I just saw a
> mail by Lech Lorens stating the same) I guess there's no point in
> polluting vim_dev with a 900K attachment.

I looked into this internal error that I was able to reproduce and found
out that it was caused by the fact that I messed up the tags file. Could
it be that your tags file is malformed? More precisely: is it possible
that when you perform a tag search, there are some entries matching that
have multiple values for the same key (entries in tags files occupy one
line each, tags' properties are stored as "key:value", each pair
separated from the previous one with a tab character)?

In the scenario I investigated nothing bad aside from the error message
seems to be taking place. I'll prepare a patch soon.

Cheers,
Lech

--
Cheers,
Lech

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: Internal error: hash_add()

Peter Odding-3
(forgot to CC vim_dev when replying to Lech Lorens :-S)

Hi Lech,

> I looked into this internal error that I was able to reproduce and found
> out that it was caused by the fact that I messed up the tags file. Could
> it be that your tags file is malformed? More precisely: is it possible
> that when you perform a tag search, there are some entries matching that
> have multiple values for the same key (entries in tags files occupy one
> line each, tags' properties are stored as "key:value", each pair
> separated from the previous one with a tab character)?

If I understood you correctly you're referring to the list of optional
key/value pairs after the ;" sequence? I checked the buggy tags file in
Vim using the following pattern and found a line such as you described.

        ^.*;".*\s\(\w\+\):\S\+.*\s\(\1\)

The (only) match was on the last line of the tags file, which appears to
have been concatenated with a random other line from the tags file. This
meant that the "language:" field occurred twice on this line. The
relevant line is (everything between ---8<--- and --->8---):

---8<---
ystep /usr/include/python2.6/Imaging.h /^    int ystep;$/;" m
language:C struct:ImagingCodecStateInstance access:publicCheckOptions
/home/peter/.vim/plugin/pyref.vim /^function! s:CheckOptions()$/;" f
language:Vim
--->8---

I stumbled upon this problem when calling taglist('.') so I guess that
makes every entry in the tags file a "matching entry" which means both
your assumptions hold.

> In the scenario I investigated nothing bad aside from the error message
> seems to be taking place. I'll prepare a patch soon.

The only complication is that I use exceptions for error handling in my
plug-ins so that Vim stops processing my scripts on the first error
message or exception.

Because my plug-ins use a :try/:catch block and one of the first steps
of some of my plug-ins is to get a list of all tags by calling
taglist('.'), the E685 error message is converted to an exception, which
means the result of the taglist() call is never assigned to a variable.
In other words, once a plug-in uses exceptions it becomes impossible to
read the valid entries from a corrupted tags file :-(

:set tags=~/.vimtags-buggy
:try
: echomsg "calling taglist()"
: let results = taglist('.')
: echomsg "called taglist()"
:catch
: echomsg "catched" v:exception
: " The {results} are never assigned.
:endtry

calling taglist()
catched Vim(call):E685: Internal error: hash_add()

Or did I miss another way to do this? By the way, thanks for the time
and effort you're putting into this problem and Vim in general :-)

  - Peter Odding

--
You received this message from the "vim_dev" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php