Flattening syntactic tail nests (paren-free, continued)

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

Flattening syntactic tail nests (paren-free, continued)

Claus Reinke
[this has grown a bit long, because in addition to concrete
 suggestions, it discusses the design problems leading up
 to these suggestions and includes an extended example;
 I hope that makes for an easier read overall, but you
 might want to allocate a few minutes to read it in one go]

Summary:

    Callback nesting is an important programming pattern,
    not just in asynchronous programming. The pattern is
    just barely useable in current Javascript (so bad that most
    attempts to address the problem try to avoid the pattern,
    making do with non-nested callbacks or resorting to
    pre-processors that suggest language extensions going
    beyond mere syntax).

    Now, Javascript can handle callback nesting just fine[*],
    semantically, it is only the syntax that gets in the way,
    and it is those syntax issues that are addressed here.

Context:

Parens and braces indicate grouping. If the intended grouping
is unambiguous, it should be possible to omit the redundant
parens/braces, to avoid distracting syntax noise. This has been
discussed for Javascript's control structures [0], but there is
another source of nested syntax that has been driving coders
up the walls[1]: the nesting that follows function definitions
and applications.

Problems:

1 Javascript's grammar requires parens around arguments
    in function definitions and applications, and braces around
    function bodies (expression closures experiment with skipping
    the latter [2,3])

2 Javascript function definitions do not determine their
    number of arguments (length is a "typical" number, 15.3.5.1,
    but that is an approximation, and functions can be called with
    fewer or with more arguments)

3 The number of function arguments is fixed (independently)
    at each call site, by the placement of parens (making it more
    difficult to remove those parens).

We can't just relax the grammar to solve (1), because the
parens serve a purpose separate from grouping (3), and that
has its cause deep in the language definition (2). With lots of
code relying on optional arguments with default values or
using arguments as unbounded lists, eliminating (2) sounds
unlikely (Harmony's rest parameters and parameter default
values not withstanding).

A general solution to excessive parens and braces surrounding
function definitions and calls remains elusive, but tail nests turn
out to be a useful special case: they characterize many of the
frequent problem cases (such as callbacks as final arguments
in asynchronous APIs [1]) and their tail position provides
additional leverage against the arity issue noted above (3).

Definition

    Tail nests are to syntactic nesting depth as
    tail calls are to runtime call stack depth.

    In terms of parens and braces, that is code somewhat like

        (... ( .. ( .. ))) or { .. { .. { .. }}}

    or mixtures of these (possibly with semicolons thrown in).

The idea for reducing nesting depth in the context of tail nests
translates directly from tail call optimization:

    - if the start of a nest can be determined by other means
        (redundant start marker)
    - and the end of that nest coincides with the end of the
        enclosing nest (tail nest; implies redundant end marker)

    + then the wrapping of that nest is redundant and can
        be omitted (the tail nest need not add its own parens/
        braces, keeping the nesting depth constant no matter
        how many tail nests are added)

The question is:

Which tail nest can we identify in real-world Javascript code,
and can we find a way to use the redundancy, so that we can
remove some parens/braces without confusing things?

Running example and suggestions:

My prime example would be callback nesting. Here is some
code straight from an appropriately named nodejs group
thread["I love async, but I can't code like this" 1]:

mainWindow.menu("File", function(err, file) {
  if(err) throw err;
  file.openMenu(function(err, menu) {
    if(err) throw err;
    menu.item("Open", function(err, item) {
      if(err) throw err;
      item.click(function(err) {
        if(err) throw err;
        mainWindow.getChild(type('Window'), function(err, dialog) {
          if(err) throw err;
          ...
        });
      });
    });
  });
});

Getting rid of the template error handling is straightforward,
so I'll ignore that for the rest of this discussion, just keeping
the syntactically problematic part of the code structure:

mainWindow.menu("File", function(file) {
  file.openMenu(function(menu) {
    menu.item("Open", function(item) {
      item.click(function() {
        mainWindow.getChild(type('Window'), function(dialog) {
          ...
        });
      });
    });
  });
});

which leaves us with the problem of closing all those tail nests
(the semicolons before '}' could be supplied by ASI, I assume,
but we need to track what happens once we remove braces,
so I leave them in for now).

The start of a function body is redundantly marked by the end
of its formal parameter list, and the end of the callback bodies
in tail nests like these is redundantly marked by the end of
their enclosing calls' actual parameter lists. This redundancy
motivates the first suggestion:

Suggestion 1: make braces surrounding function bodies in
    function definitions optional

    (function bodies without explicit braces extend as far as
    possible, but their extent can be limited from the outside)

Something similar was suggested for expression closures[2,3],
replacing inner braces by outer parens (is that part included
in #-functions?). The example changes as follows:

mainWindow.menu("File", function(file)
  file.openMenu(function(menu)
    menu.item("Open", function(item)
      item.click(function()
        mainWindow.getChild(type('Window'), function(dialog)
          ...
        );
      );
    );
  );
);

Since {}-blocks can now be implicit, we should inform ASI.

Suggestion 1a: augment ASI to treat an implicit block ending
    the same way as an explicit closing brace '}' (insert missing
    semicolon at block end)

That means we can still drop most of those semicolons:

mainWindow.menu("File", function(file)
  file.openMenu(function(menu)
    menu.item("Open", function(item)
      item.click(function()
        mainWindow.getChild(type('Window'), function(dialog)
          ...
        )
      )
    )
  )
);

Now for the parens that specify argument lists of function
calls. We can't just remove them, because they carry
information about how many arguments to pass (neither
passing the arguments one-by-one nor passing too many
arguments at once is equivalent, thanks to problem (2)).

But perhaps we can use the same trick we used for function
definitions in suggestion 1: let the context tell us where the
function application ends. For instance, if we have

    (f(x,z,y))

then the inner parens are redundant, as long as we assume
that a function application without explicit parens extends
as far to the right as possible

Suggestion 2': [to avoid ambiguities, this will be refined below]
    make parens surrounding argument lists of function
    applications optional

    (function applications without explicit parens extend as far
    as possible, but their extent can be limited from the outside)

Using this, we can rewrite our example code by moving the
function application parens one level out, but..

mainWindow.menu("File", function(file)
  (file.openMenu function(menu)
    (menu.item "Open", function(item)
      (item.click function()
        (mainWindow.getChild type('Window'), function(dialog)
          ...
        )
      )
    )
  )
);

"Argh!", that didn't really help, did it?-) Well, actually it did,
but the syntax is a little broken, so the improvement isn't
obvious.

For the moment, we remove the ambiguities in the simplest
way: assume a new infix operator '@' (for application, [@]),
whose only purpose is to help us (parsers) to identify
function applications in spite of optional parens:

    '(f @ a1, .. , an)' is equivalent to 'f(a1,..,an)'

    [in the former, the parens are optional, delimiting the
     application from the outside; in the latter, the parens
     are part of the language construct]

Parser experts might be able to come up with something
nicer, the idea here is simply that 'f @ ' starts a function
application the same way that 'f(' does, but without the
expectation of a closing ')' to match.

Suggestion 2:
    make parens surrounding argument lists of function
    applications optional

    (function applications without explicit parens extend as far
    as possible, but their extent can be limited from the outside)

    To avoid ambiguities, function applications with optional
    parens retain an explicit start marker for their argument
    list: '@' instead of '(', with no matching end marker.

    '(f @ a1, .. , an)' is equivalent to 'f(a1,..,an)'

    [in the former, the parens are optional, supplied by the
     usage context; in the latter, the parens are a required
     part of the language construct]

Our example code, with this suggestion:

mainWindow.menu("File", function(file)
  (file.openMenu @ function(menu)
    (menu.item @ "Open", function(item)
      (item.click @ function()
        (mainWindow.getChild @ type('Window'), function(dialog)
          ...
        )
      )
    )
  )
);

Now, the reason that this is any better, in spite of the same
number of parens, is that the parens are one level up, where
they can be made redundant by other nesting constructs at
that level (remember the tail call analogy: we don't avoid
nesting entirely, we just reuse outer nests, to keep the level
of nestings constant).

To begin with, the 'mainWindow.getChild' call ends where
the 'item.click' callback ends

mainWindow.menu("File", function(file)
  (file.openMenu @ function(menu)
    (menu.item @ "Open", function(item)
      (item.click @ function()
        mainWindow.getChild @ type('Window'), function(dialog)
          ...
      )
    )
  )
);

The 'item.click' call ends where the 'menu.item' callback ends:

mainWindow.menu("File", function(file)
  (file.openMenu @ function(menu)
    (menu.item @ "Open", function(item)
      item.click @ function()
        mainWindow.getChild @ type('Window'), function(dialog)
          ...
    )
  )
);

and so on, until we are left with:

mainWindow.menu("File", function(file)
  file.openMenu @ function(menu)
    menu.item @ "Open", function(item)
      item.click @ function()
        mainWindow.getChild @ type('Window'), function(dialog)
          ...
);

Which is means that we have managed to get rid of
that tail nest of parens and braces, without any semantics/
runtime changes, just by changing the syntax. In theory,
at least!-)

In practice, I'd be very interested to hear whether
these suggestions (or more suitable variants) would
be implementable in your Javascript frontends. If we
could get rid of the '@' and commas as well, that would
be even better, but that appears to conflict with curried
function application.

Claus

PS. In the running example, callbacks indicated asynchronous
    code, and there have been lots of attacks on that problem,
    ranging from libraries over preprocessors to language
    extensions. But callback nesting is typical of a wider range
    of programming patterns, to which most of the async-
    specific proposals do not apply. In contrast, the suggestions
    here address some async-independent and purely syntactic
    shortcomings in Javascript's core functionality.

[*] ok, tail call optimization would help, but that seems to be
    on the way in?
[@] '@' is traditional for function application. In Javascript,
    it seems to be used only for conditional compilation in
    JScript, where its use seems limited to prefix namespacing
    (also recommended only in comments)

[0] http://brendaneich.com/2010/11/paren-free/
[1] "I love async, but I can't code like this"
http://groups.google.com/group/nodejs/browse_thread/thread/c334947643c80968/e216892d168a7584
[2]
https://developer.mozilla.org/en/New_in_JavaScript_1.8#Expression_closures_(Merge_into_own_page.2fsection)
[3] http://perfectionkills.com/a-closer-look-at-expression-closures/

 

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Flattening syntactic tail nests (paren-free, continued)

Claus Reinke
In my previous post, I suggested tail nests as a profitable target
for reducing reducing parens and braces that distract from
common code patterns. I made two concrete suggestions, to
make both braces around function bodies and parens around
function applications optional.

The combination of both suggestions would allow to reduce
the nesting of parens and braces in tail nests, such as nested
callbacks (function definition nested in function application
argument nested in ..). For the running example, this was
successful, but rather odd looking - it should be possible
to do better.

So I would like to modify those suggestions, to bring them
more in line with Javascript practice and Harmony directions.
As a bonus, the modified suggestions simplify another common
case of nesting, namely curried function definitions.

>[mnemonic summary of old suggestions, to be modified]
>
>Suggestion 1 (optional braces around function bodies):
>
>    '(function (..) ..)' may be used instead of 'function (..) {..}'
>
>Suggestion 2 (optional parens around function applications):
>
>    '(f @ a1, .. , an)' may be used instead of  'f(a1,..,an)'

Suggestion 2 is especially problematic if the argument list
has more than one component: the hope was that we
might later get rid of the commas as well, assuming that
argument lists are argument tuples, which could become
individual arguments by currying. However, I've come to
think that this underlying assumption does not fit Javascript:

With optional arguments and unbounded lengths, argument
lists are really argument arrays - they do not stand for multiple
arguments (Javascript has curried functions for that), they each
stand for a single argument with an unspecified number of
components. We just use them for multiple arguments because
curried function definitions are so awkward.

Harmony changes, such as formal parameter destructuring
and spreads, will remove the differences in feature sets between
argument lists and arrays, making it possible to replace

    'f(a1,..,an)' and 'function f(a1,..,am) {..}'

with

    'f([a1,..,an])' and 'function f([a1,..,am]) {..}'

at which point the parens will be redundant and each argument
list can be a proper array (dear arguments: I want my syntax
back!-). So this part is well in hand already, and trying to break
up those argument list arrays in other ways would go against
the direction Harmony is taking.

Also, the language already provides for curried function
definitions and applications, they just don't get used much
yet. Curried applications are easy, but curried definitions
happen to be very awkward, syntactically.

My modified suggestions take both curried functions and
argument lists as arrays into account, so they do a little less
work on applications, making better use of existing syntax,
and a little more work on definitions, hoping to give curried
definitions a lift.

What I'd like is to be able to replace the horrible

    function (..) { return function (..) { .. }}

with the shorthand notation

    function (..)(..) { .. }

(and, similarly, '#(..)(..){..}' instead of '#(..){ #(..){..}}')

giving curried function definitions the same short syntax
as curried function applications. This shorthand makes it
obvious that each argument list really is just a single,
complex argument to the function. There are a few obvious
problems which would have made this difficult before
Harmony:

    - using the outer function's arguments pseudo-array
        in the inner function's body;
       
        Harmony's spreads avoid that problem entirely.

    - using the outer function's 'this' in the inner function's
        body (the shorthand notation no longer has an outer
        function body in which to rename 'this' to 'self');

        this is being addressed in other threads here, so
        Harmony is likely to offer at least one solution
        (optional naming for the implicit argument 'this').

    - if the formal parameters can consist of multiple
        argument lists, the beginning of the function body
        is no longer unambiguous without those braces;
        to address this, we need an explicit syntactic marker
        at the boundary between arguments and body;

        from the archives, it seems that '=>' has been a
        fairly popular suggestion (usually instead of a prefix
        marker), so I'll adopt that, but without removing the
        prefix marker, be it 'function' or '#'.

Which results in the following modified suggestions

// --------------------------------------------------
// Modified suggestion 1 (function definitions):

    1a (curried definition shorthand)  

            function (..)(..) { .. }

        may be used instead of

            function (..) { return function (..) { .. }}

        for arbitrary nestings.

    1b (function bodies)

            (function (..)..(..) => ..)

        may be used instead of

            function (..)..(..) {..}

        (the outer parens are _not_ part of the function definition
         syntax - they indicate that the source context delimits the
         function definition from the outside: if the construct in
         which the definition is nested ends, so does the definition)

    1c (ASI needs to know about 1b's implicit block endings)

// --------------------------------------------------
// Modified suggestion 2 (function applications):

        (f @ x)

    may be used instead of

        f(x)

    (the outer parens are _not_ part of the function application
     syntax - they indicate that the source context delimits the
     function application from the outside: if the construct in
     which the application is nested ends, so does the application)

    '@' is now just a left-associative infix operator (same
    precedence as function application), so

        (f @ x @ y @ z)

    is

        (((f @ x) @ y) @ z)

    which is

        f (x) (y) (z)

    (ideally, infix function application would just be juxtaposition,
      ie, the parens around single-component argument lists
      would be optional, without requiring an explicit operator)

// --------------------------------------------------

The effect on our running example, in terms of removing
redundant parens and braces, can be similar, but that
now needs support from the libraries: they would need to
curry their API functions so that the callback becomes a
second, separate argument (instead of the last component
in a single complex argument list).

(mainWindow.menu("File") @ function(file) =>
  file.openMenu @ function(menu) =>
    menu.item("Open") @  function(item) =>
      item.click @ function() =>
        mainWindow.getChild(type('Window')) @
            function(dialog) =>
              ...
);

// Note: since function definitions extend as far as possible,
// to the closing paren here, the inner '@' are unambiguosly
// nested

These modified suggestions are simpler, more in line with
Harmony's directions, and help to reduce the syntactic noise
for two common sources of nesting: curried definitions and
definitions as callback arguments.

Comments, please?-)
Claus
 
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Flattening syntactic tail nests (paren-free, continued)

Dave Herman
Claus,

Thanks for the suggestions. Let me see if I can summarize them and respond briefly:

* functions with expression bodies

This was proposed for ES4 and implemented in SpiderMonkey. I believe there are some unfortunate ambiguities in the grammar that came up, and I think they've been treated as moribund. But I love them, and would love it if we could find a way to bring them back in a way that didn't break the grammar.

* using @ for infix function application

I'm pretty skeptical. I don't see it solving any major problems. The @ sigil is wanted for many other purposes, so I'm loath to give it up for something that isn't incredibly compelling.

* curried function definitions

This seems potentially do-able. I don't think there's necessarily huge demand for it, but I'm cautiously open to the idea. We'd have to think through the consequences for the grammar pretty carefully.

* functions with expression bodies using => as a separator

Now we're in bikeshedding territory. At the risk of upsetting people, I just have to say that I don't have enough time in my life to participate in more es-discuss mega-threads on surface syntax.

* making nested callbacks better via all of the above

I think this is a deeper problem that can't be solved with cosmetics. This is why I've worked on generators as a good solution for the nested-callback problem.

Dave

On Apr 8, 2011, at 3:42 AM, Claus Reinke wrote:

> In my previous post, I suggested tail nests as a profitable target for reducing reducing parens and braces that distract from common code patterns. I made two concrete suggestions, to
> make both braces around function bodies and parens around
> function applications optional.
>
> The combination of both suggestions would allow to reduce
> the nesting of parens and braces in tail nests, such as nested callbacks (function definition nested in function application
> argument nested in ..). For the running example, this was
> successful, but rather odd looking - it should be possible
> to do better.
>
> So I would like to modify those suggestions, to bring them more in line with Javascript practice and Harmony directions.
> As a bonus, the modified suggestions simplify another common
> case of nesting, namely curried function definitions.
>
>> [mnemonic summary of old suggestions, to be modified]
>>
>> Suggestion 1 (optional braces around function bodies):
>>   '(function (..) ..)' may be used instead of 'function (..) {..}'
>>
>> Suggestion 2 (optional parens around function applications):
>>
>>   '(f @ a1, .. , an)' may be used instead of  'f(a1,..,an)'
>
> Suggestion 2 is especially problematic if the argument list
> has more than one component: the hope was that we might later get rid of the commas as well, assuming that
> argument lists are argument tuples, which could become individual arguments by currying. However, I've come to think that this underlying assumption does not fit Javascript:
> With optional arguments and unbounded lengths, argument lists are really argument arrays - they do not stand for multiple arguments (Javascript has curried functions for that), they each stand for a single argument with an unspecified number of components. We just use them for multiple arguments because
> curried function definitions are so awkward.
>
> Harmony changes, such as formal parameter destructuring and spreads, will remove the differences in feature sets between
> argument lists and arrays, making it possible to replace
>
>   'f(a1,..,an)' and 'function f(a1,..,am) {..}'
>
> with
>   'f([a1,..,an])' and 'function f([a1,..,am]) {..}'
>
> at which point the parens will be redundant and each argument
> list can be a proper array (dear arguments: I want my syntax back!-). So this part is well in hand already, and trying to break up those argument list arrays in other ways would go against the direction Harmony is taking.
>
> Also, the language already provides for curried function definitions and applications, they just don't get used much
> yet. Curried applications are easy, but curried definitions
> happen to be very awkward, syntactically.
>
> My modified suggestions take both curried functions and
> argument lists as arrays into account, so they do a little less work on applications, making better use of existing syntax, and a little more work on definitions, hoping to give curried definitions a lift.
>
> What I'd like is to be able to replace the horrible
>
>   function (..) { return function (..) { .. }}
>
> with the shorthand notation
>
>   function (..)(..) { .. }
>
> (and, similarly, '#(..)(..){..}' instead of '#(..){ #(..){..}}')
>
> giving curried function definitions the same short syntax
> as curried function applications. This shorthand makes it
> obvious that each argument list really is just a single, complex argument to the function. There are a few obvious problems which would have made this difficult before
> Harmony:
>
>   - using the outer function's arguments pseudo-array
>       in the inner function's body;
>              Harmony's spreads avoid that problem entirely.
>
>   - using the outer function's 'this' in the inner function's
>       body (the shorthand notation no longer has an outer
>       function body in which to rename 'this' to 'self');
>       this is being addressed in other threads here, so        Harmony is likely to offer at least one solution
>       (optional naming for the implicit argument 'this').
>
>   - if the formal parameters can consist of multiple
>       argument lists, the beginning of the function body
>       is no longer unambiguous without those braces;
>       to address this, we need an explicit syntactic marker
>       at the boundary between arguments and body;
>
>       from the archives, it seems that '=>' has been a
>       fairly popular suggestion (usually instead of a prefix
>       marker), so I'll adopt that, but without removing the
>       prefix marker, be it 'function' or '#'.
>
> Which results in the following modified suggestions
>
> // --------------------------------------------------
> // Modified suggestion 1 (function definitions):
>   1a (curried definition shorthand)  
>           function (..)(..) { .. }
>
>       may be used instead of
>
>           function (..) { return function (..) { .. }}
>
>       for arbitrary nestings.
>
>   1b (function bodies)
>
>           (function (..)..(..) => ..)
>
>       may be used instead of
>           function (..)..(..) {..}
>
>       (the outer parens are _not_ part of the function definition
>        syntax - they indicate that the source context delimits the
>        function definition from the outside: if the construct in
>        which the definition is nested ends, so does the definition)
>   1c (ASI needs to know about 1b's implicit block endings)
>
> // --------------------------------------------------
> // Modified suggestion 2 (function applications):
>
>       (f @ x)
>
>   may be used instead of
>
>       f(x)
>
>   (the outer parens are _not_ part of the function application
>    syntax - they indicate that the source context delimits the
>    function application from the outside: if the construct in
>    which the application is nested ends, so does the application)
>   '@' is now just a left-associative infix operator (same    precedence as function application), so
>
>       (f @ x @ y @ z)
>
>   is
>
>       (((f @ x) @ y) @ z)
>
>   which is
>
>       f (x) (y) (z)
>
>   (ideally, infix function application would just be juxtaposition,
>     ie, the parens around single-component argument lists
>     would be optional, without requiring an explicit operator)
>
> // --------------------------------------------------
>
> The effect on our running example, in terms of removing
> redundant parens and braces, can be similar, but that
> now needs support from the libraries: they would need to
> curry their API functions so that the callback becomes a
> second, separate argument (instead of the last component
> in a single complex argument list).
> (mainWindow.menu("File") @ function(file) =>
> file.openMenu @ function(menu) =>
>   menu.item("Open") @  function(item) =>
>     item.click @ function() =>
>       mainWindow.getChild(type('Window')) @            function(dialog) =>
>             ...
> );
>
> // Note: since function definitions extend as far as possible,
> // to the closing paren here, the inner '@' are unambiguosly
> // nested
>
> These modified suggestions are simpler, more in line with
> Harmony's directions, and help to reduce the syntactic noise
> for two common sources of nesting: curried definitions and
> definitions as callback arguments.
>
> Comments, please?-)
> Claus
> _______________________________________________
> es-discuss mailing list
> [hidden email]
> https://mail.mozilla.org/listinfo/es-discuss

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Flattening syntactic tail nests (paren-free, continued)

Claus Reinke
Hi Dave,

thanks for your reply. I was beginning to fear that these
suggestions had been missed. Let me start directly with
your main point, because it biases the detailed evaluation:

> * making nested callbacks better via all of the above
>
> I think this is a deeper problem that can't be solved with
> cosmetics. This is why I've worked on generators as a
> good solution for the nested-callback problem.

Fortunately, this is a point were I can argue with some
confidence. I like generators and your usage of generators
(it would be nice to have some more written examples of
what one will be able to do, btw), but until you've got yield*
(delegated yield), they won't address the same problem
range. Even then, it depends on the usage.

The general usage scenario I am thinking of is derived
from monadic programming. Depending on where you're
coming from, this is lambda-calculus with added effects
(such as exceptions, threaded state, non-determinism), or
it is simply a refinement of continuation-based programming
(separating the basic operations from composition with
callbacks, by introducing a separate interface for callback
chaining). Either way, we know that the programming
patterns are expressive enough to cover a wide range of
sequential, and some concurrent programming idioms.

This knowledge is based on nearly 20 years of programming
practice, and if you count the use of continuations and monads
in language semantics, you can easily add a decade or two [1].

Since you are working at the borderline toward concurrent
programming (coroutines, asynchronous i/o, non-preemptive,
cooperative scheduling), you think that something more
than syntax is involved. And you're right, in a way, but even
in your usage scenarios, your code will be helped by better
syntax. And you really do not want to emulate monads by
hand-coding yield-based trampolining, even though that
seems useful for async coding and coroutines [4].

The reason why cosmetics will help is that the semantic
problem is already solved, or is in the process of being
solved separately, in your case. Callback nesting does not
only occur in async coding, but arises generally in continuation
based coding styles, including monadic programming.

What all those applications of callbacks have in common
is that they emulate a language that looks like let declarations
(actually let expressions or let statements) but does something
more behind the scenes

    let(a = operation1(..)) {
    let(b = operation2(..)) {
        operation3(a,b);
    }
    }

In plain code, this does no error handling, no backtracking,
etc, just bindings followed by evaluating a block. In monadic
code, this could do error handling behind the scenes (if the
handling is fairly standard), or backtracking (eg, if that code
was part of a parser, it might try operation2 for every possible
parse of operation1, until both succeed), or some other effect,
depending on the monad in use. There are whole cottage
industries around monadic programming in various languages,
some aware that they are using monads, others not.

Languages like Haskell and F# take this programming style
seriously and introduce special syntax for it [2,3] (do-notation
in Haskell, since Haskell 1.3; computation expressions in F#).

But the desugaring of the special syntax is based on
callback nesting. Leaving out some details, the let statements
in the example could be desugared to

    (function(a) {
        (function(b) {
            operation3(a,b);
        }(operation2(..)));
    }(operation1(..)));

Apart from possible side-effects, this is standard lambda-calculus,
or correspondence between parametric and declarative forms, if
you will;-) The nesting is important as it allows us to build up
scopes in which all intermediate results are available.

Callback nesting uses that desugaring, but takes control of the
function applications (they become callbacks, which I'll write as
curried parameters, for clarity)

    operation1(..)(function(a){
        operation2(..)(function(b){
            operation3(a,b);
        });
    });

This is a basic continuation-passing style, which leads to nested
callbacks, and depends on tail-call optimization to work (without
tco, one has to introduce an additional layer of interpretation, to
keep stack usage low by trampolining).

Monadic style simply says that the composition with callbacks
should be separated from the basic operations, so each operation
would implement a simple interface for callback composition.

For the purposes of this overview, there will be some way to
inject values into the monad (Monad.value(val)) and some way
to pass the result of a monadic computation to a monadic
continuation (Monad.then(continuation)). The example changes
little

    operation1(..).then(function(a){
        operation2(..).then(function(b){
            operation3(a,b);
        });
    });

but most code can now be written to use the interface (value/then)
instead of any concrete implementation of that interface. In other
words, one can now write reusable code that will work with any
monad. As a special case, one can define monads that combine
effects - for instance, one could have a monad for async i/o and
combine it with a monad for boilerplate error handling. More
generally, the generic monadic library code will include control
structures (such as loops).

Anyway, the semantic basis is there in Javascript, so we mostly
need to clean up the syntax to make these programming patterns
useable. My suggestions were minimalistic, just removing those
unnecessary and distracting nested parens and braces.

Alternatively, one could go the way of Haskell and F#, and
introduce more syntactic sugar, perhaps in the form of monadic
let bindings. Then our example could be written as

    letM(a = operation1(..)) {
    letM(b = operation2(..)) {
        operation3(a,b);
    }
    }

(desugaring to the previous form) and we'd also need some
way to state which monad this code is to use.

I just thought it would be appropriate to take the smaller
steps first, just removing the parens and braces, to make
the code structure stand out better

    (operation1(..).then @ function(a) =>
     operation2(..).then @ function(b) =>
     operation3(a,b);
    );

Assuming the grammar extensions can be worked out,
this looks like a clear improvement, and it is entirely
syntactic. It doesn't go as far as full monadic syntax
sugar, but is more widely applicable.

Monadic programming is one of the programming idioms
I'd not like to lose, and it is awkward in current Javascript.
Blogs on async coding, error handling, parser combinator
libraries, McCarthy's amb, etc, suggest that many other
Javascript coders are already playing with monadic ideas
(without necessary linking their experiments to monads).

Perhaps ES/next/next will have monadic syntax sugar
as well, following Haskell and F#, but for ES/next I wanted
to start with something less ambitious that will do the job
for a while.

Here are the suggestions in again, mnemonically:

>> 1a (curried definition shorthand)
>>     function (..)(..) { .. } -->  function (..) { return function (..)
>> { .. }}
>>
>> 1b (function bodies, optional braces)
>>     (function (..)..(..) => ..) --> (function (..)..(..) {..})
>>
>> 2 (function applications, optional parens):
>>     (f @ x) --> f(x)

To address your other comments:

These suggestions are not limited to functions with expression
bodies (though they'd apply there, too, and 1b has been tried
there already) - they simply shift the responsibility for delimiting
the function body or function application one level up, where it
is possible to remove nested parens and braces, because one
ending paren/brace can end several constructs at once.

The use of '@' is unfortunate, but I'm just not sure whether it
will be possible to use just juxtaposition instead (even if it
should turn out to be unambiguous in theory, coders will
find it confusing if there is no explicit delimiter).

The use of '=>' is not bike-shedding - as with '@', it simply
replaces hard-coded parens/braces with an infix marker,
allowing me to make the parens/braces optional in the
grammar, and to remove nested parens/braces in code.

The specific symbols were chosen because the use of '@'
for application is traditional in implementation papers, and
the use of '=>' was the most popular in the list archives.

Claus

[1] Reynolds, "The discoveries of continuations", 1993.
    Lisp and Symbolic Computation 6 (3/4): 233-248
    ftp://ftp.cs.cmu.edu/user/jcr/histcont.pdf

[2] Section 7.2 ("Monads") in: Hudak, Hughes, Peyton
    Jones, Wadler, "A History of Haskell: being lazy with
    class", The Third ACM SIGPLAN History of Programming
    Languages Conference (HOPL-III) San Diego, California,
    June 9-10, 2007.
    http://research.microsoft.com/en-us/um/people/simonpj/papers/history-of-haskell/

[3] Syme, "Some Details on F# Computation Expressions",
    http://blogs.msdn.com/b/dsyme/archive/2007/09/22/some-details-on-f-computation-expressions-aka-monadic-or-workflow-syntax.aspx

[4] http://www.neilmix.com/2007/02/07/threading-in-javascript-17/

 

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Flattening syntactic tail nests (paren-free, continued)

Brendan Eich-3
In reply to this post by Dave Herman
On Apr 17, 2011, at 9:09 AM, David Herman wrote:

> * functions with expression bodies
>
> This was proposed for ES4 and implemented in SpiderMonkey. I believe there are some unfortunate ambiguities in the grammar that came up, and I think they've been treated as moribund. But I love them, and would love it if we could find a way to bring them back in a way that didn't break the grammar.

The counterexample Waldemar gave, adjusted to use a function with expression body instead of a let expression, is worth repeating:

  function f() {return "f"}
  var x = 3;
  alert(function (a) a ? f : x++(1));
      /*^^^^^^^^^^^^^^^^^^^^^^^^*/

The alert should show "f". Note that the call grammar:

  CallExpression :
    MemberExpression Arguments
    CallExpression Arguments

does not parse an unparenthesized AssignmentExpression (the function with expression body's body non-terminal, in ES4 and what we implemented) as the "callee" part of the right-hand side (underlined with ^^^ above). But it does parse a PrimaryExpression, which includes function expressions, which loops back to the AssignmentExpression at the end.

The rule is "You can't have an AssignmentExpression terminating a PrimaryExpression". Note that this has nothing to do with ASI.

As I wrote at the time, this problem could be patched by making the body expression nonterminal be high-precedence, or else require parens.

That seemed not worth the trouble to me at the time, and now I really would rather work on sharp functions, or arrow functions, to relieve the cost of 'return' as well as 'function'.

At the last TC39 meeting, Waldemar objected to implicit return of a (reformed) completion value. I have an idea to address that, which I'll post in a new thread.

/be
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Flattening syntactic tail nests (paren-free, continued)

Brendan Eich-3
On Apr 18, 2011, at 12:14 PM, Brendan Eich wrote:

> On Apr 17, 2011, at 9:09 AM, David Herman wrote:
>
>> * functions with expression bodies
>>
>> This was proposed for ES4 and implemented in SpiderMonkey. I believe there are some unfortunate ambiguities in the grammar that came up, and I think they've been treated as moribund. But I love them, and would love it if we could find a way to bring them back in a way that didn't break the grammar.
>
> The counterexample Waldemar gave, adjusted to use a function with expression body instead of a let expression, is worth repeating:
>
>  function f() {return "f"}
>  var x = 3;
>  alert(function (a) a ? f : x++(1));
>      /*^^^^^^^^^^^^^^^^^^^^^^^^*/
>
> The alert should show "f". Note that the call grammar:
>
>  CallExpression :
>    MemberExpression Arguments
>    CallExpression Arguments
>
> does not parse an unparenthesized AssignmentExpression (the function with expression body's body non-terminal, in ES4 and what we implemented) as the "callee" part of the right-hand side (underlined with ^^^ above). But it does parse a PrimaryExpression, which includes function expressions, which loops back to the AssignmentExpression at the end.

To be super-clear, in case anyone thought the x++(1) was a call expression, the grammar forbids PostfixExpression (a RHS of UnaryExpression) being unparenthesized as a callee in a call expression.

But by making functions with expression bodies be PrimaryExpressions ending with AssignmentExpressions, we have created a bad loop up the grammar that should allow the example with ^^^ under its callee part to parse.

It doesn't work in Firefox (SpiderMonkey), as Waldemar pointed out. We don't use a bottom-up parser.

Ruby has a flatter grammar-graph with lots of loops through its big expression nonterminal. Not so C-like languages including JS. Making bad loops at the end of right-hand sides is bad. Hope this helps the problem "stick" in folks minds.

/be

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss