Quantcast

'maxfuncdepth' ignored

Previous Topic Next Topic
 
classic Classic list List threaded Threaded
9 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

'maxfuncdepth' ignored

brettstahlman
Consider the following recursive user function...

fu! Fun(count)
    if a:count > 0
        call Fun(a:count - 1)
    endif
endfu

:h 'maxfuncdepth' describes the option's purpose as follows:

Maximum depth of function calls for user functions.  This normally
catches endless recursion.  When using a recursive function with
more depth, set 'maxfuncdepth' to a bigger number.

So I would expect to be able to make the following recursive call with no error:
    :set maxfuncdepth=1000
    :call Fun(500)

But I get the following error after slightly less than 200 recursive calls:
E169: Command too recursive

The documentation for E169 states the following:

This happens when an Ex command executes an Ex command that executes an Ex
command, etc.  This is only allowed 200 times.  When it's more there probably
is an endless loop.  Probably a |:execute| or |:source| command is involved.

It's as though the :call (Ex command) is triggering the error long
before the number of calls to the user function Fun() has reached
'maxfuncdepth'. But if this is the way it's supposed to work, what's
the point of 'maxfuncdepth'? Don't all calls to user functions involve
an Ex command (since both `call' and `let' are Ex commands)? Is there
a way to permit more than 200 recursive calls to Fun() without
triggering the error?

Thanks,
Brett Stahlman

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 'maxfuncdepth' ignored

Nikolay Aleksandrovich Pavlov
2016-12-30 20:09 GMT+03:00 Brett Stahlman <[hidden email]>:

> Consider the following recursive user function...
>
> fu! Fun(count)
>     if a:count > 0
>         call Fun(a:count - 1)
>     endif
> endfu
>
> :h 'maxfuncdepth' describes the option's purpose as follows:
>
> Maximum depth of function calls for user functions.  This normally
> catches endless recursion.  When using a recursive function with
> more depth, set 'maxfuncdepth' to a bigger number.
>
> So I would expect to be able to make the following recursive call with no error:
>     :set maxfuncdepth=1000
>     :call Fun(500)
>
> But I get the following error after slightly less than 200 recursive calls:
> E169: Command too recursive
>
> The documentation for E169 states the following:
>
> This happens when an Ex command executes an Ex command that executes an Ex
> command, etc.  This is only allowed 200 times.  When it's more there probably
> is an endless loop.  Probably a |:execute| or |:source| command is involved.
>
> It's as though the :call (Ex command) is triggering the error long
> before the number of calls to the user function Fun() has reached
> 'maxfuncdepth'. But if this is the way it's supposed to work, what's
> the point of 'maxfuncdepth'? Don't all calls to user functions involve
> an Ex command (since both `call' and `let' are Ex commands)? Is there
> a way to permit more than 200 recursive calls to Fun() without
> triggering the error?

I tried lambdas, but they also catch this error due to the way they
are implemented. Unlike (until you consider their internal
implementation) lambdas regular functions are lists of Ex commands, so
this is not surprising. Note that by default &maxfuncdepth is 100
which is lesser then 200.

>
> Thanks,
> Brett Stahlman
>
> --
> --
> You received this message from the "vim_use" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> ---
> You received this message because you are subscribed to the Google Groups "vim_use" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> For more options, visit https://groups.google.com/d/optout.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 'maxfuncdepth' ignored

brettstahlman
On Saturday, December 31, 2016 at 12:23:10 PM UTC-6, ZyX wrote:

> 2016-12-30 20:09 GMT+03:00 Brett Stahlman <[hidden email]>:
> > Consider the following recursive user function...
> >
> > fu! Fun(count)
> >     if a:count > 0
> >         call Fun(a:count - 1)
> >     endif
> > endfu
> >
> > :h 'maxfuncdepth' describes the option's purpose as follows:
> >
> > Maximum depth of function calls for user functions.  This normally
> > catches endless recursion.  When using a recursive function with
> > more depth, set 'maxfuncdepth' to a bigger number.
> >
> > So I would expect to be able to make the following recursive call with no error:
> >     :set maxfuncdepth=1000
> >     :call Fun(500)
> >
> > But I get the following error after slightly less than 200 recursive calls:
> > E169: Command too recursive
> >
> > The documentation for E169 states the following:
> >
> > This happens when an Ex command executes an Ex command that executes an Ex
> > command, etc.  This is only allowed 200 times.  When it's more there probably
> > is an endless loop.  Probably a |:execute| or |:source| command is involved.
> >
> > It's as though the :call (Ex command) is triggering the error long
> > before the number of calls to the user function Fun() has reached
> > 'maxfuncdepth'. But if this is the way it's supposed to work, what's
> > the point of 'maxfuncdepth'? Don't all calls to user functions involve
> > an Ex command (since both `call' and `let' are Ex commands)? Is there
> > a way to permit more than 200 recursive calls to Fun() without
> > triggering the error?
>
> I tried lambdas, but they also catch this error due to the way they
> are implemented. Unlike (until you consider their internal
> implementation) lambdas regular functions are lists of Ex commands, so
> this is not surprising. Note that by default &maxfuncdepth is 100
> which is lesser then 200.
Hmm... Perhaps Bram will weigh in on this, but effectively limiting 'maxfuncdepth' to 200 feels like an unintended consequence, rather than design intent - especially since the help on 'maxfuncdepth' makes no mention of the limit. The documentation on E169 suggests that the purpose of the 200 limit is to detect certain types of recursion involving :source and :execute commands. If it was meant to apply to function calls generally, why even have a separate option for function calls, especially if you can't increase its value to something that would permit meaningful recursion?

Since there's no option governing the E169 limit, perhaps it could be changed to the maximum of 200 and 'maxfuncdepth'. Or perhaps it could take into account the type of Ex command (i.e., source/execute vs call/let). Or perhaps there could be a 'maxmemfunc' option (analogous to 'maxmempattern'), which would limit function call recursion by stack space consumed (or some rough approximation thereof) rather than # of calls.

I noticed this because I'm running a tree processing algorithm that is inherently recursive. I had intended to compute 'maxfuncdepth' as a function of another option, but discovered that my choice was silently ignored for anything over 200. Although the depth of the trees can exceed 200 in extreme cases, the depth is bounded and known, so it made sense simply to boost 'maxfuncdepth' long enough to recurse the tree. If there's no way around the 200 maximum, I'll probably have to rewrite the algorithm to use breadth-first traversals, rather than the much more natural (and simple) tree recursion.

Thanks,
Brett Stahlman

>
> >
> > Thanks,
> > Brett Stahlman
> >
> > --
> > --
> > You received this message from the "vim_use" maillist.
> > Do not top-post! Type your reply below the text you are replying to.
> > For more information, visit http://www.vim.org/maillist.php
> >
> > ---
> > You received this message because you are subscribed to the Google Groups "vim_use" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> > For more options, visit https://groups.google.com/d/optout.
--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 'maxfuncdepth' ignored

Bram Moolenaar

Brett Stahlman wrote:

> On Saturday, December 31, 2016 at 12:23:10 PM UTC-6, ZyX wrote:
> > 2016-12-30 20:09 GMT+03:00 Brett Stahlman <[hidden email]>:
> > > Consider the following recursive user function...
> > >
> > > fu! Fun(count)
> > >     if a:count > 0
> > >         call Fun(a:count - 1)
> > >     endif
> > > endfu
> > >
> > > :h 'maxfuncdepth' describes the option's purpose as follows:
> > >
> > > Maximum depth of function calls for user functions.  This normally
> > > catches endless recursion.  When using a recursive function with
> > > more depth, set 'maxfuncdepth' to a bigger number.
> > >
> > > So I would expect to be able to make the following recursive call with no error:
> > >     :set maxfuncdepth=1000
> > >     :call Fun(500)
> > >
> > > But I get the following error after slightly less than 200 recursive calls:
> > > E169: Command too recursive
> > >
> > > The documentation for E169 states the following:
> > >
> > > This happens when an Ex command executes an Ex command that executes an Ex
> > > command, etc.  This is only allowed 200 times.  When it's more there probably
> > > is an endless loop.  Probably a |:execute| or |:source| command is involved.
> > >
> > > It's as though the :call (Ex command) is triggering the error long
> > > before the number of calls to the user function Fun() has reached
> > > 'maxfuncdepth'. But if this is the way it's supposed to work, what's
> > > the point of 'maxfuncdepth'? Don't all calls to user functions involve
> > > an Ex command (since both `call' and `let' are Ex commands)? Is there
> > > a way to permit more than 200 recursive calls to Fun() without
> > > triggering the error?
> >
> > I tried lambdas, but they also catch this error due to the way they
> > are implemented. Unlike (until you consider their internal
> > implementation) lambdas regular functions are lists of Ex commands, so
> > this is not surprising. Note that by default &maxfuncdepth is 100
> > which is lesser then 200.
>
> Hmm... Perhaps Bram will weigh in on this, but effectively limiting 'maxfuncdepth' to 200 feels like an unintended consequence, rather than design intent - especially since the help on 'maxfuncdepth' makes no mention of the limit. The documentation on E169 suggests that the purpose of the 200 limit is to detect certain types of recursion involving :source and :execute commands. If it was meant to apply to function calls generally, why even have a separate option for function calls, especially if you can't increase its value to something that would permit meaningful recursion?
>
> Since there's no option governing the E169 limit, perhaps it could be changed to the maximum of 200 and 'maxfuncdepth'. Or perhaps it could take into account the type of Ex command (i.e., source/execute vs call/let). Or perhaps there could be a 'maxmemfunc' option (analogous to 'maxmempattern'), which would limit function call recursion by stack space consumed (or some rough approximation thereof) rather than # of calls.
>
> I noticed this because I'm running a tree processing algorithm that is inherently recursive. I had intended to compute 'maxfuncdepth' as a function of another option, but discovered that my choice was silently ignored for anything over 200. Although the depth of the trees can exceed 200 in extreme cases, the depth is bounded and known, so it made sense simply to boost 'maxfuncdepth' long enough to recurse the tree. If there's no way around the 200 maximum, I'll probably have to rewrite the algorithm to use breadth-first traversals, rather than the much more natural (and simple) tree recursion.

I think that we can also used 'maxfuncdepth' for the Ex command
recursion.  They have similar functionality, even though the semantics
are a bit different.  It's not worth adding another option for this.

So you can set 'maxfuncdepth' to 1000 and have a call depth of 1000,
minus any other Ex commands that are involved, such as :execute.

--
hundred-and-one symptoms of being an internet addict:
172. You join listservers just for the extra e-mail.

 /// Bram Moolenaar -- [hidden email] -- http://www.Moolenaar.net   \\\
///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
\\\  an exciting new programming language -- http://www.Zimbu.org        ///
 \\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 'maxfuncdepth' ignored

brettstahlman
On Monday, January 2, 2017 at 8:26:52 AM UTC-6, Bram Moolenaar wrote:

> Brett Stahlman wrote:
>
> > On Saturday, December 31, 2016 at 12:23:10 PM UTC-6, ZyX wrote:
> > > 2016-12-30 20:09 GMT+03:00 Brett Stahlman <[hidden email]>:
> > > > Consider the following recursive user function...
> > > >
> > > > fu! Fun(count)
> > > >     if a:count > 0
> > > >         call Fun(a:count - 1)
> > > >     endif
> > > > endfu
> > > >
> > > > :h 'maxfuncdepth' describes the option's purpose as follows:
> > > >
> > > > Maximum depth of function calls for user functions.  This normally
> > > > catches endless recursion.  When using a recursive function with
> > > > more depth, set 'maxfuncdepth' to a bigger number.
> > > >
> > > > So I would expect to be able to make the following recursive call with no error:
> > > >     :set maxfuncdepth=1000
> > > >     :call Fun(500)
> > > >
> > > > But I get the following error after slightly less than 200 recursive calls:
> > > > E169: Command too recursive
> > > >
> > > > The documentation for E169 states the following:
> > > >
> > > > This happens when an Ex command executes an Ex command that executes an Ex
> > > > command, etc.  This is only allowed 200 times.  When it's more there probably
> > > > is an endless loop.  Probably a |:execute| or |:source| command is involved.
> > > >
> > > > It's as though the :call (Ex command) is triggering the error long
> > > > before the number of calls to the user function Fun() has reached
> > > > 'maxfuncdepth'. But if this is the way it's supposed to work, what's
> > > > the point of 'maxfuncdepth'? Don't all calls to user functions involve
> > > > an Ex command (since both `call' and `let' are Ex commands)? Is there
> > > > a way to permit more than 200 recursive calls to Fun() without
> > > > triggering the error?
> > >
> > > I tried lambdas, but they also catch this error due to the way they
> > > are implemented. Unlike (until you consider their internal
> > > implementation) lambdas regular functions are lists of Ex commands, so
> > > this is not surprising. Note that by default &maxfuncdepth is 100
> > > which is lesser then 200.
> >
> > Hmm... Perhaps Bram will weigh in on this, but effectively limiting 'maxfuncdepth' to 200 feels like an unintended consequence, rather than design intent - especially since the help on 'maxfuncdepth' makes no mention of the limit. The documentation on E169 suggests that the purpose of the 200 limit is to detect certain types of recursion involving :source and :execute commands. If it was meant to apply to function calls generally, why even have a separate option for function calls, especially if you can't increase its value to something that would permit meaningful recursion?
> >
> > Since there's no option governing the E169 limit, perhaps it could be changed to the maximum of 200 and 'maxfuncdepth'. Or perhaps it could take into account the type of Ex command (i.e., source/execute vs call/let). Or perhaps there could be a 'maxmemfunc' option (analogous to 'maxmempattern'), which would limit function call recursion by stack space consumed (or some rough approximation thereof) rather than # of calls.
> >
> > I noticed this because I'm running a tree processing algorithm that is inherently recursive. I had intended to compute 'maxfuncdepth' as a function of another option, but discovered that my choice was silently ignored for anything over 200. Although the depth of the trees can exceed 200 in extreme cases, the depth is bounded and known, so it made sense simply to boost 'maxfuncdepth' long enough to recurse the tree. If there's no way around the 200 maximum, I'll probably have to rewrite the algorithm to use breadth-first traversals, rather than the much more natural (and simple) tree recursion.
>
> I think that we can also used 'maxfuncdepth' for the Ex command
> recursion.  They have similar functionality, even though the semantics
> are a bit different.  It's not worth adding another option for this.
>
> So you can set 'maxfuncdepth' to 1000 and have a call depth of 1000,
> minus any other Ex commands that are involved, such as :execute.
Sounds great!
Thanks,
Brett Stahlman

>
> --
> hundred-and-one symptoms of being an internet addict:
> 172. You join listservers just for the extra e-mail.
>
>  /// Bram Moolenaar -- [hidden email] -- http://www.Moolenaar.net   \\\
> ///        sponsor Vim, vote for features -- http://www.Vim.org/sponsor/ \\\
> \\\  an exciting new programming language -- http://www.Zimbu.org        ///
>  \\\            help me help AIDS victims -- http://ICCF-Holland.org    ///

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 'maxfuncdepth' ignored

Nikolay Aleksandrovich Pavlov
In reply to this post by brettstahlman
2017-01-01 0:40 GMT+03:00 Brett Stahlman <[hidden email]>:

> On Saturday, December 31, 2016 at 12:23:10 PM UTC-6, ZyX wrote:
>> 2016-12-30 20:09 GMT+03:00 Brett Stahlman <[hidden email]>:
>> > Consider the following recursive user function...
>> >
>> > fu! Fun(count)
>> >     if a:count > 0
>> >         call Fun(a:count - 1)
>> >     endif
>> > endfu
>> >
>> > :h 'maxfuncdepth' describes the option's purpose as follows:
>> >
>> > Maximum depth of function calls for user functions.  This normally
>> > catches endless recursion.  When using a recursive function with
>> > more depth, set 'maxfuncdepth' to a bigger number.
>> >
>> > So I would expect to be able to make the following recursive call with no error:
>> >     :set maxfuncdepth=1000
>> >     :call Fun(500)
>> >
>> > But I get the following error after slightly less than 200 recursive calls:
>> > E169: Command too recursive
>> >
>> > The documentation for E169 states the following:
>> >
>> > This happens when an Ex command executes an Ex command that executes an Ex
>> > command, etc.  This is only allowed 200 times.  When it's more there probably
>> > is an endless loop.  Probably a |:execute| or |:source| command is involved.
>> >
>> > It's as though the :call (Ex command) is triggering the error long
>> > before the number of calls to the user function Fun() has reached
>> > 'maxfuncdepth'. But if this is the way it's supposed to work, what's
>> > the point of 'maxfuncdepth'? Don't all calls to user functions involve
>> > an Ex command (since both `call' and `let' are Ex commands)? Is there
>> > a way to permit more than 200 recursive calls to Fun() without
>> > triggering the error?
>>
>> I tried lambdas, but they also catch this error due to the way they
>> are implemented. Unlike (until you consider their internal
>> implementation) lambdas regular functions are lists of Ex commands, so
>> this is not surprising. Note that by default &maxfuncdepth is 100
>> which is lesser then 200.
>
> Hmm... Perhaps Bram will weigh in on this, but effectively limiting 'maxfuncdepth' to 200 feels like an unintended consequence, rather than design intent - especially since the help on 'maxfuncdepth' makes no mention of the limit. The documentation on E169 suggests that the purpose of the 200 limit is to detect certain types of recursion involving :source and :execute commands. If it was meant to apply to function calls generally, why even have a separate option for function calls, especially if you can't increase its value to something that would permit meaningful recursion?
>
> Since there's no option governing the E169 limit, perhaps it could be changed to the maximum of 200 and 'maxfuncdepth'. Or perhaps it could take into account the type of Ex command (i.e., source/execute vs call/let). Or perhaps there could be a 'maxmemfunc' option (analogous to 'maxmempattern'), which would limit function call recursion by stack space consumed (or some rough approximation thereof) rather than # of calls.

Limiting stack space consumed should be cleaner and more in line with
the purpose of the limit (recursive nature of VimL executor and a
number of different functions requires either imposing such limits or
catching stack overflows). I am not sure though whether it is possible
to get stack space consumption rate on any of the platforms Vim
supports: after some searching I found only a number of dirty hacks
like in http://stackoverflow.com/questions/53827/checking-available-stack-size-in-c.
&maxmemfunc would be possible if VimL executor was not recursive and
reimplemented stack based on malloc() like some other interpreted
languages do, but it is recursive and uses system stack instead.

>
> I noticed this because I'm running a tree processing algorithm that is inherently recursive. I had intended to compute 'maxfuncdepth' as a function of another option, but discovered that my choice was silently ignored for anything over 200. Although the depth of the trees can exceed 200 in extreme cases, the depth is bounded and known, so it made sense simply to boost 'maxfuncdepth' long enough to recurse the tree. If there's no way around the 200 maximum, I'll probably have to rewrite the algorithm to use breadth-first traversals, rather than the much more natural (and simple) tree recursion.
>
> Thanks,
> Brett Stahlman
>
>>
>> >
>> > Thanks,
>> > Brett Stahlman
>> >
>> > --
>> > --
>> > You received this message from the "vim_use" maillist.
>> > Do not top-post! Type your reply below the text you are replying to.
>> > For more information, visit http://www.vim.org/maillist.php
>> >
>> > ---
>> > You received this message because you are subscribed to the Google Groups "vim_use" group.
>> > To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
>> > For more options, visit https://groups.google.com/d/optout.
>
> --
> --
> You received this message from the "vim_use" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> ---
> You received this message because you are subscribed to the Google Groups "vim_use" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> For more options, visit https://groups.google.com/d/optout.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 'maxfuncdepth' ignored

brettstahlman
On Monday, January 2, 2017 at 11:40:32 AM UTC-6, ZyX wrote:

> 2017-01-01 0:40 GMT+03:00 Brett Stahlman <[hidden email]>:
> > On Saturday, December 31, 2016 at 12:23:10 PM UTC-6, ZyX wrote:
> >> 2016-12-30 20:09 GMT+03:00 Brett Stahlman <[hidden email]>:
> >> > Consider the following recursive user function...
> >> >
> >> > fu! Fun(count)
> >> >     if a:count > 0
> >> >         call Fun(a:count - 1)
> >> >     endif
> >> > endfu
> >> >
> >> > :h 'maxfuncdepth' describes the option's purpose as follows:
> >> >
> >> > Maximum depth of function calls for user functions.  This normally
> >> > catches endless recursion.  When using a recursive function with
> >> > more depth, set 'maxfuncdepth' to a bigger number.
> >> >
> >> > So I would expect to be able to make the following recursive call with no error:
> >> >     :set maxfuncdepth=1000
> >> >     :call Fun(500)
> >> >
> >> > But I get the following error after slightly less than 200 recursive calls:
> >> > E169: Command too recursive
> >> >
> >> > The documentation for E169 states the following:
> >> >
> >> > This happens when an Ex command executes an Ex command that executes an Ex
> >> > command, etc.  This is only allowed 200 times.  When it's more there probably
> >> > is an endless loop.  Probably a |:execute| or |:source| command is involved.
> >> >
> >> > It's as though the :call (Ex command) is triggering the error long
> >> > before the number of calls to the user function Fun() has reached
> >> > 'maxfuncdepth'. But if this is the way it's supposed to work, what's
> >> > the point of 'maxfuncdepth'? Don't all calls to user functions involve
> >> > an Ex command (since both `call' and `let' are Ex commands)? Is there
> >> > a way to permit more than 200 recursive calls to Fun() without
> >> > triggering the error?
> >>
> >> I tried lambdas, but they also catch this error due to the way they
> >> are implemented. Unlike (until you consider their internal
> >> implementation) lambdas regular functions are lists of Ex commands, so
> >> this is not surprising. Note that by default &maxfuncdepth is 100
> >> which is lesser then 200.
> >
> > Hmm... Perhaps Bram will weigh in on this, but effectively limiting 'maxfuncdepth' to 200 feels like an unintended consequence, rather than design intent - especially since the help on 'maxfuncdepth' makes no mention of the limit. The documentation on E169 suggests that the purpose of the 200 limit is to detect certain types of recursion involving :source and :execute commands. If it was meant to apply to function calls generally, why even have a separate option for function calls, especially if you can't increase its value to something that would permit meaningful recursion?
> >
> > Since there's no option governing the E169 limit, perhaps it could be changed to the maximum of 200 and 'maxfuncdepth'. Or perhaps it could take into account the type of Ex command (i.e., source/execute vs call/let). Or perhaps there could be a 'maxmemfunc' option (analogous to 'maxmempattern'), which would limit function call recursion by stack space consumed (or some rough approximation thereof) rather than # of calls.
>
> Limiting stack space consumed should be cleaner and more in line with
> the purpose of the limit (recursive nature of VimL executor and a
> number of different functions requires either imposing such limits or
> catching stack overflows). I am not sure though whether it is possible
> to get stack space consumption rate on any of the platforms Vim
> supports: after some searching I found only a number of dirty hacks
> like in http://stackoverflow.com/questions/53827/checking-available-stack-size-in-c.
> &maxmemfunc would be possible if VimL executor was not recursive and
> reimplemented stack based on malloc() like some other interpreted
> languages do, but it is recursive and uses system stack instead.
I can definitely see advantages to a limit based on memory usage, but I haven't looked at the VimL implementation to see what sort of added complexity that would entail. In addition to the overhead of the function calls themselves, there's also the memory allocated (dynamically) by each function invocation to consider. I'm guessing that's stored on some sort of Vim heap (not on the system stack).

Example:
fu! Func()
    let big_data = Build_big_data_structure()
    .
    .
    call Func()
endfu

Using 'maxfuncdepth' for E169 is probably the simplest approach, and the only downside is that if the user does something stupid, he could exhaust process memory. But he could already do that: consider that Build_big_data_structure in the example above could build an arbitrarily large structure on each recursive invocation, such that overflow could occur even within 200 calls.

Brett Stahlman.


>
> >
> > I noticed this because I'm running a tree processing algorithm that is inherently recursive. I had intended to compute 'maxfuncdepth' as a function of another option, but discovered that my choice was silently ignored for anything over 200. Although the depth of the trees can exceed 200 in extreme cases, the depth is bounded and known, so it made sense simply to boost 'maxfuncdepth' long enough to recurse the tree. If there's no way around the 200 maximum, I'll probably have to rewrite the algorithm to use breadth-first traversals, rather than the much more natural (and simple) tree recursion.
> >
> > Thanks,
> > Brett Stahlman
> >
> >>
> >> >
> >> > Thanks,
> >> > Brett Stahlman
> >> >
> >> > --
> >> > --
> >> > You received this message from the "vim_use" maillist.
> >> > Do not top-post! Type your reply below the text you are replying to.
> >> > For more information, visit http://www.vim.org/maillist.php
> >> >
> >> > ---
> >> > You received this message because you are subscribed to the Google Groups "vim_use" group.
> >> > To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> >> > For more options, visit https://groups.google.com/d/optout.
> >
> > --
> > --
> > You received this message from the "vim_use" maillist.
> > Do not top-post! Type your reply below the text you are replying to.
> > For more information, visit http://www.vim.org/maillist.php
> >
> > ---
> > You received this message because you are subscribed to the Google Groups "vim_use" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> > For more options, visit https://groups.google.com/d/optout.
--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 'maxfuncdepth' ignored

Nikolay Aleksandrovich Pavlov
2017-01-02 23:36 GMT+03:00 Brett Stahlman <[hidden email]>:

> On Monday, January 2, 2017 at 11:40:32 AM UTC-6, ZyX wrote:
>> 2017-01-01 0:40 GMT+03:00 Brett Stahlman <[hidden email]>:
>> > On Saturday, December 31, 2016 at 12:23:10 PM UTC-6, ZyX wrote:
>> >> 2016-12-30 20:09 GMT+03:00 Brett Stahlman <[hidden email]>:
>> >> > Consider the following recursive user function...
>> >> >
>> >> > fu! Fun(count)
>> >> >     if a:count > 0
>> >> >         call Fun(a:count - 1)
>> >> >     endif
>> >> > endfu
>> >> >
>> >> > :h 'maxfuncdepth' describes the option's purpose as follows:
>> >> >
>> >> > Maximum depth of function calls for user functions.  This normally
>> >> > catches endless recursion.  When using a recursive function with
>> >> > more depth, set 'maxfuncdepth' to a bigger number.
>> >> >
>> >> > So I would expect to be able to make the following recursive call with no error:
>> >> >     :set maxfuncdepth=1000
>> >> >     :call Fun(500)
>> >> >
>> >> > But I get the following error after slightly less than 200 recursive calls:
>> >> > E169: Command too recursive
>> >> >
>> >> > The documentation for E169 states the following:
>> >> >
>> >> > This happens when an Ex command executes an Ex command that executes an Ex
>> >> > command, etc.  This is only allowed 200 times.  When it's more there probably
>> >> > is an endless loop.  Probably a |:execute| or |:source| command is involved.
>> >> >
>> >> > It's as though the :call (Ex command) is triggering the error long
>> >> > before the number of calls to the user function Fun() has reached
>> >> > 'maxfuncdepth'. But if this is the way it's supposed to work, what's
>> >> > the point of 'maxfuncdepth'? Don't all calls to user functions involve
>> >> > an Ex command (since both `call' and `let' are Ex commands)? Is there
>> >> > a way to permit more than 200 recursive calls to Fun() without
>> >> > triggering the error?
>> >>
>> >> I tried lambdas, but they also catch this error due to the way they
>> >> are implemented. Unlike (until you consider their internal
>> >> implementation) lambdas regular functions are lists of Ex commands, so
>> >> this is not surprising. Note that by default &maxfuncdepth is 100
>> >> which is lesser then 200.
>> >
>> > Hmm... Perhaps Bram will weigh in on this, but effectively limiting 'maxfuncdepth' to 200 feels like an unintended consequence, rather than design intent - especially since the help on 'maxfuncdepth' makes no mention of the limit. The documentation on E169 suggests that the purpose of the 200 limit is to detect certain types of recursion involving :source and :execute commands. If it was meant to apply to function calls generally, why even have a separate option for function calls, especially if you can't increase its value to something that would permit meaningful recursion?
>> >
>> > Since there's no option governing the E169 limit, perhaps it could be changed to the maximum of 200 and 'maxfuncdepth'. Or perhaps it could take into account the type of Ex command (i.e., source/execute vs call/let). Or perhaps there could be a 'maxmemfunc' option (analogous to 'maxmempattern'), which would limit function call recursion by stack space consumed (or some rough approximation thereof) rather than # of calls.
>>
>> Limiting stack space consumed should be cleaner and more in line with
>> the purpose of the limit (recursive nature of VimL executor and a
>> number of different functions requires either imposing such limits or
>> catching stack overflows). I am not sure though whether it is possible
>> to get stack space consumption rate on any of the platforms Vim
>> supports: after some searching I found only a number of dirty hacks
>> like in http://stackoverflow.com/questions/53827/checking-available-stack-size-in-c.
>> &maxmemfunc would be possible if VimL executor was not recursive and
>> reimplemented stack based on malloc() like some other interpreted
>> languages do, but it is recursive and uses system stack instead.
>
> I can definitely see advantages to a limit based on memory usage, but I haven't looked at the VimL implementation to see what sort of added complexity that would entail. In addition to the overhead of the function calls themselves, there's also the memory allocated (dynamically) by each function invocation to consider. I'm guessing that's stored on some sort of Vim heap (not on the system stack).
>
> Example:
> fu! Func()
>     let big_data = Build_big_data_structure()
>     .
>     .
>     call Func()
> endfu
>
> Using 'maxfuncdepth' for E169 is probably the simplest approach, and the only downside is that if the user does something stupid, he could exhaust process memory. But he could already do that: consider that Build_big_data_structure in the example above could build an arbitrarily large structure on each recursive invocation, such that overflow could occur even within 200 calls.

Vim does not have anything like “Vim heap”, it either uses system
stack or system (libc) allocator. Unlike Neovim, Vim intends to handle
`malloc()` failures gracefully (i.e. handle NULL returns), but this
obviously not possible with stack exhaustion: no standard way to get
stack consumption, no errors (i.e. NULL returns) from `alloca()` on
most systems, no standard way to get stack size limit (though unlike
stack consumption there is a number of system-specific ways),
generally you don’t even know in which direction stack grows and
whether the whole stack is in one continuous block of memory
addresses.

So in Vim in most cases where you may see recursion there is recursion
depth counter.

About NULL returns: Neovim authors decided that this does not worth
the hassle: e.g. it is completely possible that Neovim will get killed
because `malloc()` returned something non-NULL without actually
allocating memory (search for “overcommit”) and memory really was
exhausted. Also Vim only now added some functions to test memory
allocation failures and they are not used yet for most cases, so
nobody knows whether handling memory allocation failures properly
works at all.

>
> Brett Stahlman.
>
>
>>
>> >
>> > I noticed this because I'm running a tree processing algorithm that is inherently recursive. I had intended to compute 'maxfuncdepth' as a function of another option, but discovered that my choice was silently ignored for anything over 200. Although the depth of the trees can exceed 200 in extreme cases, the depth is bounded and known, so it made sense simply to boost 'maxfuncdepth' long enough to recurse the tree. If there's no way around the 200 maximum, I'll probably have to rewrite the algorithm to use breadth-first traversals, rather than the much more natural (and simple) tree recursion.
>> >
>> > Thanks,
>> > Brett Stahlman
>> >
>> >>
>> >> >
>> >> > Thanks,
>> >> > Brett Stahlman
>> >> >
>> >> > --
>> >> > --
>> >> > You received this message from the "vim_use" maillist.
>> >> > Do not top-post! Type your reply below the text you are replying to.
>> >> > For more information, visit http://www.vim.org/maillist.php
>> >> >
>> >> > ---
>> >> > You received this message because you are subscribed to the Google Groups "vim_use" group.
>> >> > To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
>> >> > For more options, visit https://groups.google.com/d/optout.
>> >
>> > --
>> > --
>> > You received this message from the "vim_use" maillist.
>> > Do not top-post! Type your reply below the text you are replying to.
>> > For more information, visit http://www.vim.org/maillist.php
>> >
>> > ---
>> > You received this message because you are subscribed to the Google Groups "vim_use" group.
>> > To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
>> > For more options, visit https://groups.google.com/d/optout.
>
> --
> --
> You received this message from the "vim_use" maillist.
> Do not top-post! Type your reply below the text you are replying to.
> For more information, visit http://www.vim.org/maillist.php
>
> ---
> You received this message because you are subscribed to the Google Groups "vim_use" group.
> To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> For more options, visit https://groups.google.com/d/optout.

--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: 'maxfuncdepth' ignored

h_east
Hi neovim guy,

2017-1-3(Tue) 7:58:59 UTC+9 ZyX:

> 2017-01-02 23:36 GMT+03:00 Brett Stahlman <[hidden email]>:
> > On Monday, January 2, 2017 at 11:40:32 AM UTC-6, ZyX wrote:
> >> 2017-01-01 0:40 GMT+03:00 Brett Stahlman <[hidden email]>:
> >> > On Saturday, December 31, 2016 at 12:23:10 PM UTC-6, ZyX wrote:
> >> >> 2016-12-30 20:09 GMT+03:00 Brett Stahlman <[hidden email]>:
> >> >> > Consider the following recursive user function...
> >> >> >
> >> >> > fu! Fun(count)
> >> >> >     if a:count > 0
> >> >> >         call Fun(a:count - 1)
> >> >> >     endif
> >> >> > endfu
> >> >> >
> >> >> > :h 'maxfuncdepth' describes the option's purpose as follows:
> >> >> >
> >> >> > Maximum depth of function calls for user functions.  This normally
> >> >> > catches endless recursion.  When using a recursive function with
> >> >> > more depth, set 'maxfuncdepth' to a bigger number.
> >> >> >
> >> >> > So I would expect to be able to make the following recursive call with no error:
> >> >> >     :set maxfuncdepth=1000
> >> >> >     :call Fun(500)
> >> >> >
> >> >> > But I get the following error after slightly less than 200 recursive calls:
> >> >> > E169: Command too recursive
> >> >> >
> >> >> > The documentation for E169 states the following:
> >> >> >
> >> >> > This happens when an Ex command executes an Ex command that executes an Ex
> >> >> > command, etc.  This is only allowed 200 times.  When it's more there probably
> >> >> > is an endless loop.  Probably a |:execute| or |:source| command is involved.
> >> >> >
> >> >> > It's as though the :call (Ex command) is triggering the error long
> >> >> > before the number of calls to the user function Fun() has reached
> >> >> > 'maxfuncdepth'. But if this is the way it's supposed to work, what's
> >> >> > the point of 'maxfuncdepth'? Don't all calls to user functions involve
> >> >> > an Ex command (since both `call' and `let' are Ex commands)? Is there
> >> >> > a way to permit more than 200 recursive calls to Fun() without
> >> >> > triggering the error?
> >> >>
> >> >> I tried lambdas, but they also catch this error due to the way they
> >> >> are implemented. Unlike (until you consider their internal
> >> >> implementation) lambdas regular functions are lists of Ex commands, so
> >> >> this is not surprising. Note that by default &maxfuncdepth is 100
> >> >> which is lesser then 200.
> >> >
> >> > Hmm... Perhaps Bram will weigh in on this, but effectively limiting 'maxfuncdepth' to 200 feels like an unintended consequence, rather than design intent - especially since the help on 'maxfuncdepth' makes no mention of the limit. The documentation on E169 suggests that the purpose of the 200 limit is to detect certain types of recursion involving :source and :execute commands. If it was meant to apply to function calls generally, why even have a separate option for function calls, especially if you can't increase its value to something that would permit meaningful recursion?
> >> >
> >> > Since there's no option governing the E169 limit, perhaps it could be changed to the maximum of 200 and 'maxfuncdepth'. Or perhaps it could take into account the type of Ex command (i.e., source/execute vs call/let). Or perhaps there could be a 'maxmemfunc' option (analogous to 'maxmempattern'), which would limit function call recursion by stack space consumed (or some rough approximation thereof) rather than # of calls.
> >>
> >> Limiting stack space consumed should be cleaner and more in line with
> >> the purpose of the limit (recursive nature of VimL executor and a
> >> number of different functions requires either imposing such limits or
> >> catching stack overflows). I am not sure though whether it is possible
> >> to get stack space consumption rate on any of the platforms Vim
> >> supports: after some searching I found only a number of dirty hacks
> >> like in http://stackoverflow.com/questions/53827/checking-available-stack-size-in-c.
> >> &maxmemfunc would be possible if VimL executor was not recursive and
> >> reimplemented stack based on malloc() like some other interpreted
> >> languages do, but it is recursive and uses system stack instead.
> >
> > I can definitely see advantages to a limit based on memory usage, but I haven't looked at the VimL implementation to see what sort of added complexity that would entail. In addition to the overhead of the function calls themselves, there's also the memory allocated (dynamically) by each function invocation to consider. I'm guessing that's stored on some sort of Vim heap (not on the system stack).
> >
> > Example:
> > fu! Func()
> >     let big_data = Build_big_data_structure()
> >     .
> >     .
> >     call Func()
> > endfu
> >
> > Using 'maxfuncdepth' for E169 is probably the simplest approach, and the only downside is that if the user does something stupid, he could exhaust process memory. But he could already do that: consider that Build_big_data_structure in the example above could build an arbitrarily large structure on each recursive invocation, such that overflow could occur even within 200 calls.
>
> Vim does not have anything like “Vim heap”, it either uses system
> stack or system (libc) allocator. Unlike Neovim, Vim intends to handle
> `malloc()` failures gracefully (i.e. handle NULL returns), but this
> obviously not possible with stack exhaustion: no standard way to get
> stack consumption, no errors (i.e. NULL returns) from `alloca()` on
> most systems, no standard way to get stack size limit (though unlike
> stack consumption there is a number of system-specific ways),
> generally you don’t even know in which direction stack grows and
> whether the whole stack is in one continuous block of memory
> addresses.
>
> So in Vim in most cases where you may see recursion there is recursion
> depth counter.
>
> About NULL returns: Neovim authors decided that this does not worth
> the hassle: e.g. it is completely possible that Neovim will get killed
> because `malloc()` returned something non-NULL without actually
> allocating memory (search for “overcommit”) and memory really was
> exhausted. Also Vim only now added some functions to test memory
> allocation failures and they are not used yet for most cases, so
> nobody knows whether handling memory allocation failures properly
> works at all.
Is it Linux only, right?
It is OOM Killer(Out of Memory Killer).
Are other OS's (Windows, FreeBSD, etc...) so?
Please find out more and I want you to tell me :-)

P.S.
Good luck with Partial porting.

>
> >
> > Brett Stahlman.
> >
> >
> >>
> >> >
> >> > I noticed this because I'm running a tree processing algorithm that is inherently recursive. I had intended to compute 'maxfuncdepth' as a function of another option, but discovered that my choice was silently ignored for anything over 200. Although the depth of the trees can exceed 200 in extreme cases, the depth is bounded and known, so it made sense simply to boost 'maxfuncdepth' long enough to recurse the tree. If there's no way around the 200 maximum, I'll probably have to rewrite the algorithm to use breadth-first traversals, rather than the much more natural (and simple) tree recursion.
> >> >
> >> > Thanks,
> >> > Brett Stahlman
> >> >
> >> >>
> >> >> >
> >> >> > Thanks,
> >> >> > Brett Stahlman
> >> >> >
> >> >> > --
> >> >> > --
> >> >> > You received this message from the "vim_use" maillist.
> >> >> > Do not top-post! Type your reply below the text you are replying to.
> >> >> > For more information, visit http://www.vim.org/maillist.php
> >> >> >
> >> >> > ---
> >> >> > You received this message because you are subscribed to the Google Groups "vim_use" group.
> >> >> > To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> >> >> > For more options, visit https://groups.google.com/d/optout.
> >> >
> >> > --
> >> > --
> >> > You received this message from the "vim_use" maillist.
> >> > Do not top-post! Type your reply below the text you are replying to.
> >> > For more information, visit http://www.vim.org/maillist.php
> >> >
> >> > ---
> >> > You received this message because you are subscribed to the Google Groups "vim_use" group.
> >> > To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> >> > For more options, visit https://groups.google.com/d/optout.
> >
> > --
> > --
> > You received this message from the "vim_use" maillist.
> > Do not top-post! Type your reply below the text you are replying to.
> > For more information, visit http://www.vim.org/maillist.php
> >
> > ---
> > You received this message because you are subscribed to the Google Groups "vim_use" group.
> > To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
> > For more options, visit https://groups.google.com/d/optout.
--
--
You received this message from the "vim_use" maillist.
Do not top-post! Type your reply below the text you are replying to.
For more information, visit http://www.vim.org/maillist.php

---
You received this message because you are subscribed to the Google Groups "vim_use" group.
To unsubscribe from this group and stop receiving emails from it, send an email to [hidden email].
For more options, visit https://groups.google.com/d/optout.
Loading...