The X bug with Gvim

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

The X bug with Gvim

Jacky Liu
Hi everyone

I need multi-threading in Vim because I need some work being done
under the desk without turning Vim into a "zombie", now I know Vim is
single-threaded, so I decided to give Python a try, demonstration:

        " x-bug.vim

        python3 << EOF

        import threading
        import time

        print('thread 1')

        def print_to_vim():
                print('thread 2')

        threading.Thread(name='test', target=print_to_vim).start()

        print('thread 1 again')

        time.sleep(3)

        EOF

The above code works fine with console Vim as it prints the intended
message while everything else look normal, but it would make Gvim
collapse every time it's been sourced, leaving this message:

        gvim: Fatal IO error 11 (Resource temporarily unavailable) on X
server :0.0.

The problem seems to be with the X system. I did some more test and it
seems ok to have multiple Python threads in Gvim, but only one of
these threads is permitted to interact with the GUI (means to cause
any change of display within the Gvim program window)

Since I'm an ordinary user to both Vim and Python and don't know much
about the Unix System, can anyone please tell me:

1. Should this be considered a bug?

2. Is there any way to get around this, thus making it safe to have
several Python threads interacting with Gvim?

Thanks

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: The X bug with Gvim

Jacky Liu


On Sep 6, 1:40 am, Jacky Liu <[hidden email]> wrote:

> Hi everyone
>
> I need multi-threading in Vim because I need some work being done
> under the desk without turning Vim into a "zombie", now I know Vim is
> single-threaded, so I decided to give Python a try, demonstration:
>
>         " x-bug.vim
>
>         python3 << EOF
>
>         import threading
>         import time
>
>         print('thread 1')
>
>         def print_to_vim():
>                 print('thread 2')
>
>         threading.Thread(name='test', target=print_to_vim).start()
>
>         print('thread 1 again')
>
>         time.sleep(3)
>
>         EOF
>
> The above code works fine with console Vim as it prints the intended
> message while everything else look normal, but it would make Gvim
> collapse every time it's been sourced, leaving this message:
>
>         gvim: Fatal IO error 11 (Resource temporarily unavailable) on X
> server :0.0.
>
> The problem seems to be with the X system. I did some more test and it
> seems ok to have multiple Python threads in Gvim, but only one of
> these threads is permitted to interact with the GUI (means to cause
> any change of display within the Gvim program window)
>
> Since I'm an ordinary user to both Vim and Python and don't know much
> about the Unix System, can anyone please tell me:
>
> 1. Should this be considered a bug?
>
> 2. Is there any way to get around this, thus making it safe to have
> several Python threads interacting with Gvim?
>
> Thanks


Please, isn't there anyone interested in this topic?

I think doing multi-threading in Vim would find great use, like in
Conque -- its current implementation is far from optimal. Follow this
way down we may use Vim as an excellent UI for almost any text-based
program there is (or going even further with non-text based programs
if you like), and now this X-bug or whatsoever seems to be the biggest
(if not last) obstacle.

Anyone? Please tell me what you think.

Besides, the vim doc had nothing on this topic, it didn't say that
Python shouldn't go multi-threaded with Gvim. Should I try to post
this in the vim_dev group?

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: The X bug with Gvim

Andrei Kulakov
On 09/07/2011 07:31 PM, Jacky Liu wrote:

>
>
> On Sep 6, 1:40 am, Jacky Liu<[hidden email]>  wrote:
>> Hi everyone
>>
>> I need multi-threading in Vim because I need some work being done
>> under the desk without turning Vim into a "zombie", now I know Vim is
>> single-threaded, so I decided to give Python a try, demonstration:
>>
>>          " x-bug.vim
>>
>>          python3<<  EOF
>>
>>          import threading
>>          import time
>>
>>          print('thread 1')
>>
>>          def print_to_vim():
>>                  print('thread 2')
>>
>>          threading.Thread(name='test', target=print_to_vim).start()
>>
>>          print('thread 1 again')
>>
>>          time.sleep(3)
>>
>>          EOF
>>
>> The above code works fine with console Vim as it prints the intended
>> message while everything else look normal, but it would make Gvim
>> collapse every time it's been sourced, leaving this message:
>>
>>          gvim: Fatal IO error 11 (Resource temporarily unavailable) on X
>> server :0.0.
>>
>> The problem seems to be with the X system. I did some more test and it
>> seems ok to have multiple Python threads in Gvim, but only one of
>> these threads is permitted to interact with the GUI (means to cause
>> any change of display within the Gvim program window)
>>
>> Since I'm an ordinary user to both Vim and Python and don't know much
>> about the Unix System, can anyone please tell me:
>>
>> 1. Should this be considered a bug?
>>
>> 2. Is there any way to get around this, thus making it safe to have
>> several Python threads interacting with Gvim?
>>
>> Thanks
>
>
> Please, isn't there anyone interested in this topic?
>
> I think doing multi-threading in Vim would find great use, like in
> Conque -- its current implementation is far from optimal. Follow this
> way down we may use Vim as an excellent UI for almost any text-based
> program there is (or going even further with non-text based programs
> if you like), and now this X-bug or whatsoever seems to be the biggest
> (if not last) obstacle.
>
> Anyone? Please tell me what you think.
>
> Besides, the vim doc had nothing on this topic, it didn't say that
> Python shouldn't go multi-threaded with Gvim. Should I try to post
> this in the vim_dev group?
>


I agree that better integration with Python, including multithreading,
would be the best new feature for Vim. I think the issue is that
it's not easy to do? At least, it ranks #2 in this list:

http://www.vim.org/sponsor/vote_results.php

And so far I don't think much was done in this respect, so here's
hoping for the future...

  -ak

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: The X bug with Gvim

Benjamin Fritz
In reply to this post by Jacky Liu


On Sep 5, 12:40 pm, Jacky Liu <[hidden email]> wrote:
> Hi everyone
>
> I need multi-threading in Vim because I need some work being done
> under the desk without turning Vim into a "zombie"

What sort of work? Perhaps it's something you could use the client
server interface for, to start an external background process and then
send the results back into Vim when complete, like this:

http://vim.wikia.com/wiki/Execute_external_programs_asynchronously_under_Windows
(the tip is for Windows but the techniques should work with minor
modifications on Unix).

The background process could probably even be another Vim if needed,
but that just seems a bit strange.

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: The X bug with Gvim

Jacky Liu
In reply to this post by Jacky Liu

Finally, Thanks to AK and Ben. I've been through this topic for a
while, let me explain what I was having in mind:

Basically, I want to use Vim as an UI to my own program, or as many
programs as I can make it. This shouldn't be what Vim stands against I
suppose, for Vim already has built-in support for programs like gdb, I
was just trying to extend it a little. My objective was to send
messages/commands to an external process using key mappings/user
commands, and monitor the output through Vim buffers.

To accomplish this I'd like to use a dedicated thread(preferably a
Python thread) to keep polling from the external process in question.

I had considered using server-client feature, but from my
understanding this far, that would require the external process to
follow Vim's client-server protocol to be able to communicate with
Vim, which most programs out there certainly don't. In contrast, use
polling is much more universal, this way Vim would theoretically be
able to let the user operate any program within its own interface.

Taking Conque as an example -- as an emulator it has to deal with the
user and the Shell process simultaneously -- a typical task for multi-
threading I suppose, but thanks to Vim's single thread-iness (and
probably this bug too), it now has to uses an auto-sequence like this
to make it through:

let s:count= 0
function! s:PollFromShell()
        echo 'Counting: ' . s:count " stands for polling in the real code
        let s:count= s:count + 1
        call feedkeys("\<Right>\<Left>", 't')
endfunction
set updatetime=1000 " once per second
autocmd! CursorHold *    call s:PollFromShell()

The real code is much more complicated, this is for showing the key
mechanism.

This works, as Conque has already been used by many, yet it has
serious drawbacks: you see feedkeys() here would interfere with the
user input -- you can't use multi-keystroke commands like 'gt' for the
upcoming 't' could have been flushed away by feedkeys(), neither can
you use any Ex commands any more because they all need multiple key-
strokes. Yet Conque's way to work around this, is to literally map
*every* key on the keyboard(and many others) to its internal function,
which would send them to the Shell process momentarily, only in this
way can it tell the user apart from feedkeys(). Thus the result, no
Vim operations, no commands, just an ordinary(and much slower) Shell
inside a Vim window, that's all.

This is the reason why I think multi-threading would greatly enhance
the ability of Vim. Since Conque has been there for a while with
fairly frequent updating, yet it still use this approach, I suppose
there really isn't any better workaround.

If you have any thoughts please let me know. Now I'm using a less-than-
optimal approach to get around this bug -- I have a second thread
polling the external process but only change the underlying data
structure, it must be careful not to emit any visible output to the
GUI. Meanwhile in the main thread, I have to manually refresh the vim
buffer(through key mappings) to see what happened in the other
program. This is not much of a UI but it seems to be the best I can
get right now.

Thanks


--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: The X bug with Gvim

Jacky Liu

>
> To accomplish this I'd like to use a dedicated thread(preferably a
> Python thread) to keep polling from the external process in question.
>

lith mentioned using a separate python/ruby process for listening in
another thread,

http://groups.google.com/group/vim_use/browse_thread/thread/ae32d6ebc9f70ff4

I haven't thought about this, maybe it would work.

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: The X bug with Gvim

H Xu
In reply to this post by Jacky Liu
On 09/06/2011 01:40 AM, Jacky Liu wrote:

> " x-bug.vim
>
> python3<<  EOF
>
> import threading
> import time
>
> print('thread 1')
>
> def print_to_vim():
> print('thread 2')
>
> threading.Thread(name='test', target=print_to_vim).start()
>
> print('thread 1 again')
>
> time.sleep(3)
>
> EOF

It's a little weird, since I'm using python to do multithread things and
it works very well:

http://www.vim.org/scripts/script.php?script_id=3219

I'll explore this later.

-- Hong

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: The X bug with Gvim

H Xu
In reply to this post by Jacky Liu
On 09/06/2011 01:40 AM, Jacky Liu wrote:

> Hi everyone
>
> I need multi-threading in Vim because I need some work being done
> under the desk without turning Vim into a "zombie", now I know Vim is
> single-threaded, so I decided to give Python a try, demonstration:
>
> " x-bug.vim
>
> python3<<  EOF
>
> import threading
> import time
>
> print('thread 1')
>
> def print_to_vim():
> print('thread 2')
>
> threading.Thread(name='test', target=print_to_vim).start()
>
> print('thread 1 again')
>
> time.sleep(3)
>
> EOF
>

I know the problem. The following code works well:

" x-bug.vim

python << EOF

import threading
import time

print('thread 1')

def print_to_vim():
     print('thread 2')

a = threading.Thread(target=print_to_vim, args=())

a.daemon = False

print('thread 1 again')

time.sleep(3)

EOF


-- Hong

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: The X bug with Gvim

Jacky Liu

>
> I know the problem. The following code works well:
>
> " x-bug.vim
>
> python << EOF
>
> import threading
> import time
>
> print('thread 1')
>
> def print_to_vim():
>      print('thread 2')
>
> a = threading.Thread(target=print_to_vim, args=())
>
> a.daemon = False
>
> print('thread 1 again')
>
> time.sleep(3)
>
> EOF
>
> -- Hong



Thanks, but I think you've forgotten to start() that second thread?

Thanks for the link though, I'll have a look at how multi-threading
works in your plugin. As I mentioned above, multiple threads through
the Py Interface was actually OK, just that only one of them can
output through Gvim.


--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: The X bug with Gvim

Xavier de Gaye-2
In reply to this post by Jacky Liu
On Thu, Sep 8, 2011 at 1:33 PM, Jacky Liu wrote:
> ...
> Basically, I want to use Vim as an UI to my own program, or as many
> programs as I can make it. This shouldn't be what Vim stands against I
> suppose, for Vim already has built-in support for programs like gdb, I
> was just trying to extend it a little. My objective was to send
> messages/commands to an external process using key mappings/user
> commands, and monitor the output through Vim buffers.
> ...


You could use the Vim netbeans interface. The Vim netbeans interface
allows editing or/and monitoring Vim buffers. The ":nbkey" command can
be used to send any text (no key mapping is required) over the
netbeans socket as a "keyCommand" event and as a "keyAtPos" event.


--
Xavier

Les Chemins de Lokoti: http://lokoti.alwaysdata.net

--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: The X bug with Gvim

Jacky Liu
In reply to this post by Jacky Liu

Found this interesting thread back in 2007, Bram Moolenaar himself had
commented too:

http://vim.1045645.n5.nabble.com/Python-crash-td1160989.html

the same exact question, saving mentioning that it's about the X
system. The test code in that post gives the following message on my
computer:

RenderBadPicture (invalid Picture parameter)
Vim: Got X error
Vim: Finished.
gvim73: Fatal IO error 11 (Resource temporarily unavailable) on X
server :0.0.

And now it's been 4 years. I better save myself any more tries and
stay with what I have in hands right now, hopefully this will be
resolved in the future.


--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
Reply | Threaded
Open this post in threaded view
|

Re: The X bug with Gvim

Jacky Liu
In reply to this post by Xavier de Gaye-2

>
> You could use the Vim netbeans interface. The Vim netbeans interface
> allows editing or/and monitoring Vim buffers. The ":nbkey" command can
> be used to send any text (no key mapping is required) over the
> netbeans socket as a "keyCommand" event and as a "keyAtPos" event.
>



Think I've been through this feature before, seemingly it's for
integrating Vim into a bigger environment with another program as "the
driving party", not sure whether it would fit the bill, I'll have a
recheck. Thanks.


--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php