Escaping for system()

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Escaping for system()

Andrew Stewart
Hello!

I have a plugin which uses `system(command, input)` to run some external commands.  There seem to be a few escaping problems (mostly on Windows).

So I did some research and wrote it all up here: https://github.com/airblade/vim-system-escape

In a nutshell I'd like to write a VimL `shellescape(str)` function which gets the escaping right for all Vim versions greater than, say, 7.0.  I.e. implement the current built-in escaping logic in VimL.

And maybe have a similar function which escapes the whole command passed to `system(command)`...though I'm not sure yet whether that's necessary.

Anyway, I'd be most grateful for suggestions and/or feedback on my notes at the link above.

Thanks in advance,

Andy Stewart

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: Escaping for system()

tyru-2

Hi.

Passing arguments to programs on Windows is difficult at several points,
especially when you pass multiple arguments or *special character* at many many sides...

Vim side:
on Windows, when 'shellslash' option is set, shellescape() returns a string wrapped by single quotes which cmd.exe doesn't recognize.
So I implements a shellescape() function which is not affected by 'shellslash' option value.
https://github.com/vim-jp/vital.vim/blob/9159770df1ba4b7b63da7a20fa7906bf66f6ebe/autoload/vital/__latest__/Process.vim#L140-L148

Windows side:
Windows doesn't support multiple arguments by Win32API-level.
A system call to run a new process(CreateProcess()) only has one command-line parameter including a program name and arguments.
So a program must parse one concatenated argument to arguments.
But this process was done by startup routine automatically generated by compiler thus usually you don't concern about it (in C, startup routine is called before main()).

But there are two problems here:

1. There are programs that implements a startup routine by itself.
For example, cygwin's binary executables implements a startup routine by itself to parse UNIXy special characters like glob pattern ("*").

2. cmd.exe and CreateProcess()'s source code is closed.
Thus we can't see precise parsing process.

There are 3 important points to doubt when you debug:
cmd.exe -> CreateProcess() -> startup routine -> [a program accesses arguments ...]
Each processes parses arguments and may get wrong recognition there.

Conclusion:
Passing arguments to programs on Windows is too difficult ;(
There are no *right* way to pass arguments.
We always have to choose *mostly right* way...

2014/01/31 0:22 "Andrew Stewart" <[hidden email]>:
Hello!

I have a plugin which uses `system(command, input)` to run some external commands.  There seem to be a few escaping problems (mostly on Windows).

So I did some research and wrote it all up here: https://github.com/airblade/vim-system-escape

In a nutshell I'd like to write a VimL `shellescape(str)` function which gets the escaping right for all Vim versions greater than, say, 7.0.  I.e. implement the current built-in escaping logic in VimL.

And maybe have a similar function which escapes the whole command passed to `system(command)`...though I'm not sure yet whether that's necessary.

Anyway, I'd be most grateful for suggestions and/or feedback on my notes at the link above.

Thanks in advance,

Andy Stewart

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
 
---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: Escaping for system()

tyru-2


2014/01/31 2:34 "tyru" <[hidden email]>:
>
> Hi.
>
> Passing arguments to programs on Windows is difficult at several points,
> especially when you pass multiple arguments or *special character* at many many sides...
>
> Vim side:
> on Windows, when 'shellslash' option is set, shellescape() returns a string wrapped by single quotes which cmd.exe doesn't recognize.
> So I implements a shellescape() function which is not affected by 'shellslash' option value.
> https://github.com/vim-jp/vital.vim/blob/9159770df1ba4b7b63da7a20fa7906bf66f6ebe/autoload/vital/__latest__/Process.vim#L140-L148
>
> Windows side:
> Windows doesn't support multiple arguments by Win32API-level.
> A system call to run a new process(CreateProcess()) only has one command-line parameter including a program name and arguments.
> So a program must parse one concatenated argument to arguments.
> But this process was done by startup routine automatically generated by compiler thus usually you don't concern about it (in C, startup routine is called before main()).
>
> But there are two problems here:
>
> 1. There are programs that implements a startup routine by itself.
> For example, cygwin's binary executables implements a startup routine by itself to parse UNIXy special characters like glob pattern ("*").
>
> 2. cmd.exe and CreateProcess()'s source code is closed.
> Thus we can't see precise parsing process.
>
> There are 3 important points to doubt when you debug:
> cmd.exe -> CreateProcess() -> startup routine -> [a program accesses arguments ...]
> Each processes parses arguments and may get wrong recognition there.

And you can skip cmd.exe if you use :!start (:help :!start).
But :!start doesn't wait a process until exit.
I hope system()'s first argument could take a List value and skip cmd.exe...

>
> Conclusion:
> Passing arguments to programs on Windows is too difficult ;(
> There are no *right* way to pass arguments.
> We always have to choose *mostly right* way...
>
> 2014/01/31 0:22 "Andrew Stewart" <[hidden email]>:
>
>> Hello!
>>
>> I have a plugin which uses `system(command, input)` to run some external commands.  There seem to be a few escaping problems (mostly on Windows).
>>
>> So I did some research and wrote it all up here: https://github.com/airblade/vim-system-escape
>>
>> In a nutshell I'd like to write a VimL `shellescape(str)` function which gets the escaping right for all Vim versions greater than, say, 7.0.  I.e. implement the current built-in escaping logic in VimL.
>>
>> And maybe have a similar function which escapes the whole command passed to `system(command)`...though I'm not sure yet whether that's necessary.
>>
>> Anyway, I'd be most grateful for suggestions and/or feedback on my notes at the link above.
>>
>> Thanks in advance,
>>
>> Andy Stewart
>>
>> --
>> --
>> You received this message from the "vim_use" maillist.
>> Do not top-post! Type your reply below the text you are replying to.
>> For more information, visit http://www.vim.org/maillist.php
>>
>> ---
>> You received this message because you are subscribed to the Google Groups "vim_use" group.
>> To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
>> For more options, visit https://groups.google.com/groups/opt_out.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php
 
---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.
Reply | Threaded
Open this post in threaded view
|

Re: Escaping for system()

Benjamin Fritz
In reply to this post by Andrew Stewart
On Thursday, January 30, 2014 9:22:15 AM UTC-6, Andrew Stewart wrote:

> Hello!
>
> I have a plugin which uses `system(command, input)` to run some external commands.  There seem to be a few escaping problems (mostly on Windows).
>
> So I did some research and wrote it all up here: https://github.com/airblade/vim-system-escape
>
> In a nutshell I'd like to write a VimL `shellescape(str)` function which gets the escaping right for all Vim versions greater than, say, 7.0.  I.e. implement the current built-in escaping logic in VimL.
>
> And maybe have a similar function which escapes the whole command passed to `system(command)`...though I'm not sure yet whether that's necessary.
>
> Anyway, I'd be most grateful for suggestions and/or feedback on my notes at the link above.
>
> Thanks in advance,
>
> Andy Stewart

A novel approach I've seen in a plugin (I don't remember which one), is to write all the desired shell commands to a temporary file, write it as a .bat file, and then execute THAT file in the shell instead of the original commands. Then very little escaping is needed.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/groups/opt_out.