Proposal: `await.all {...}` for parallelism

classic Classic list List threaded Threaded
48 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: `await.all {...}` for parallelism

Jacob Bloom
>>This [current] structure is also just fundamentally different from working
>>serially in async/await and it forces you to reason about the problem in a
>>specific way. This doesn't appear to be a conscious decision to force good
>>code practices
>
>Actually I'd argue that it is. Doing stuff concurrently is fundamentally
>different from doing it serially, and should be reasoned about every time you
>use it.

I agree that parallelism is different and should be handled with care,
but I don't think it follows that the best way to reason about
parallelism is the way that `Promise.all` encourages. Making something
more complicated doesn't necessarily mean you'll do a better job of
reasoning about it.

If you think the proposed syntax encourages poorly-reasoned-about
code, I'm open to iterating on it to find a syntax that works with the
developer to handle parallelism in a safe way, and also doesn't
require them to write too much boilerplate code.

On Thu, Nov 21, 2019 at 3:16 PM Naveen Chawla <[hidden email]> wrote:

>
> Yes of course, I was responding to your proposal and the subsequent email about it being incompatible with existing JavaScript because "await" on its own accepts non-promises, so wouldn't return an array of results from an array of promises, hence why I proposed await.all etc.
>
> On Thu, 21 Nov 2019 at 18:29, manuelbarzi <[hidden email]> wrote:
>>>
>>> I have a solution for that:
>>>
>>> const promises = [...]
>>> await.all promises //returns an array of results
>>> await.race promises //returns a single result
>>
>>
>> well, my proposal is exactly that, but doing `await.all` by default with just `await`.
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: `await.all {...}` for parallelism

Naveen Chawla
I do find the pattern of promise "all" combined with destructuring the easiest way to handle parallelism. I think it's the only "deterministic" parallel pattern code wise.

I think non determinism in code increases the probability of bugs.

On Thu, 21 Nov 2019, 23:42 Jacob Bloom, <[hidden email]> wrote:
>>This [current] structure is also just fundamentally different from working
>>serially in async/await and it forces you to reason about the problem in a
>>specific way. This doesn't appear to be a conscious decision to force good
>>code practices
>
>Actually I'd argue that it is. Doing stuff concurrently is fundamentally
>different from doing it serially, and should be reasoned about every time you
>use it.

I agree that parallelism is different and should be handled with care,
but I don't think it follows that the best way to reason about
parallelism is the way that `Promise.all` encourages. Making something
more complicated doesn't necessarily mean you'll do a better job of
reasoning about it.

If you think the proposed syntax encourages poorly-reasoned-about
code, I'm open to iterating on it to find a syntax that works with the
developer to handle parallelism in a safe way, and also doesn't
require them to write too much boilerplate code.

On Thu, Nov 21, 2019 at 3:16 PM Naveen Chawla <[hidden email]> wrote:
>
> Yes of course, I was responding to your proposal and the subsequent email about it being incompatible with existing JavaScript because "await" on its own accepts non-promises, so wouldn't return an array of results from an array of promises, hence why I proposed await.all etc.
>
> On Thu, 21 Nov 2019 at 18:29, manuelbarzi <[hidden email]> wrote:
>>>
>>> I have a solution for that:
>>>
>>> const promises = [...]
>>> await.all promises //returns an array of results
>>> await.race promises //returns a single result
>>
>>
>> well, my proposal is exactly that, but doing `await.all` by default with just `await`.

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Re: Proposal: `await.all {...}` for parallelism

Tom Boutell
In reply to this post by Jacob Bloom
I am very sympathetic to pitches to allow more common cases for promise libraries to be written in an "awaitful" syntax without thinking explicitly about promises.

Howeever I think that changing the meaning of the semicolon in a particular context has too much potential for confusion. As others have said, parallel execution is different, and it should look and feel different. The most basic assumption a developer makes (consecutive lines of code run consecutively) is difficult to get away from; that's why we introduced "await" in the first place, to bring back the ability to write deterministic code with consecutive statements. Which sounds like a reasonable ask, when it's put that way. (:

I did propose this recently:

for (const item of items concurrency 5) {
  await  doTheThing(item);
}

However in this case I'm not talking about consecutive statements, I'm only talking about rules for simultaneously (in the sense of async, not threads) running more than one instance of the block. So I'm not proposing that we change the meaning of the semicolon(s) *within* the block in a way that could mean that if you're looking at half the code in the middle you would be likely to fundamentally misunderstand its operation.

I think that risk - that you can't tell what a semicolon means without reference to the outer context - is what makes your proposal a bridge too far for me.


--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Re: Proposal: `await.all {...}` for parallelism

Naveen Chawla
However, if `await.all { ... }` were to mean "wait for all non-awaited async function calls made within this block to complete before proceeding", as I suggested earlier, I think that could satisfy determinism for "await" wherever it is used, and satisfy the original motivation:

```
await.all {
    for (const item of items) {
        doTheThingAsync(item);
    }
}
```

Notice I have omitted `await` inside the loop. Like current JavaScript, that causes parallel execution, so no change on that front, from a determinism perspective. So determinism is not hurt by `await.all`. Rather, it guarantees completion before going further.

In an earlier example (paraphrase-coded as I forgot the names):

```
let x, y;

await.all {
   x = getXAsync();
   y = getYAsync();
}

processXAndY(x, y);
```

I think the benefit of this syntax appears more stark with the looped (first) example, as current JavaScript requires building an array in the loop to subsequently pass to `Promise.all`, which I think is a little more difficult to conceptualize than the `await.all { ... }` way of doing it. The 2nd example is arguably better than current JavaScript too, particularly because the coder doesn't have to be very smart with destructuring in light of understanding the "Promise.all" return type, etc. In other words, less cognitive overhead, which I think is a net positive.

On Fri, 22 Nov 2019 at 13:44, Tom Boutell <[hidden email]> wrote:
I am very sympathetic to pitches to allow more common cases for promise libraries to be written in an "awaitful" syntax without thinking explicitly about promises.

Howeever I think that changing the meaning of the semicolon in a particular context has too much potential for confusion. As others have said, parallel execution is different, and it should look and feel different. The most basic assumption a developer makes (consecutive lines of code run consecutively) is difficult to get away from; that's why we introduced "await" in the first place, to bring back the ability to write deterministic code with consecutive statements. Which sounds like a reasonable ask, when it's put that way. (:

I did propose this recently:

for (const item of items concurrency 5) {
  await  doTheThing(item);
}

However in this case I'm not talking about consecutive statements, I'm only talking about rules for simultaneously (in the sense of async, not threads) running more than one instance of the block. So I'm not proposing that we change the meaning of the semicolon(s) *within* the block in a way that could mean that if you're looking at half the code in the middle you would be likely to fundamentally misunderstand its operation.

I think that risk - that you can't tell what a semicolon means without reference to the outer context - is what makes your proposal a bridge too far for me.


--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Re: Proposal: `await.all {...}` for parallelism

Tom Boutell
This is very interesting, but this code:

await.all {
   x = getXAsync();
   y = getYAsync();
}

processXAndY(x, y);

Still carries within it the problem that if I'm looking at just the middle of the { ... } block — if "await.all" has scrolled offscreen — I'll be completely wrong about what ";" means. I think that's too much magic.

Also, in the case of the "for" loop, this doesn't address managing the level of concurrency. Although it could in theory with a syntax like await.all({ concurrency: 5 }), I'm not sure if it's practical to implement that for your general case.

Actually I'm curious about what the implementation would look like in general.  If it were babel compiling this, I guess it would have to wrap every statement not preceded by "await" with a check for whether it returns a thenable and add it to an array if it does. But with the concurrency feature it would also have to defer executing the code at all until the right time as otherwise we're still starting zillions of "processes" at once.


On Sat, Nov 23, 2019 at 5:08 AM Naveen Chawla <[hidden email]> wrote:
However, if `await.all { ... }` were to mean "wait for all non-awaited async function calls made within this block to complete before proceeding", as I suggested earlier, I think that could satisfy determinism for "await" wherever it is used, and satisfy the original motivation:

```
await.all {
    for (const item of items) {
        doTheThingAsync(item);
    }
}
```

Notice I have omitted `await` inside the loop. Like current JavaScript, that causes parallel execution, so no change on that front, from a determinism perspective. So determinism is not hurt by `await.all`. Rather, it guarantees completion before going further.

In an earlier example (paraphrase-coded as I forgot the names):

```
let x, y;

await.all {
   x = getXAsync();
   y = getYAsync();
}

processXAndY(x, y);
```

I think the benefit of this syntax appears more stark with the looped (first) example, as current JavaScript requires building an array in the loop to subsequently pass to `Promise.all`, which I think is a little more difficult to conceptualize than the `await.all { ... }` way of doing it. The 2nd example is arguably better than current JavaScript too, particularly because the coder doesn't have to be very smart with destructuring in light of understanding the "Promise.all" return type, etc. In other words, less cognitive overhead, which I think is a net positive.

On Fri, 22 Nov 2019 at 13:44, Tom Boutell <[hidden email]> wrote:
I am very sympathetic to pitches to allow more common cases for promise libraries to be written in an "awaitful" syntax without thinking explicitly about promises.

Howeever I think that changing the meaning of the semicolon in a particular context has too much potential for confusion. As others have said, parallel execution is different, and it should look and feel different. The most basic assumption a developer makes (consecutive lines of code run consecutively) is difficult to get away from; that's why we introduced "await" in the first place, to bring back the ability to write deterministic code with consecutive statements. Which sounds like a reasonable ask, when it's put that way. (:

I did propose this recently:

for (const item of items concurrency 5) {
  await  doTheThing(item);
}

However in this case I'm not talking about consecutive statements, I'm only talking about rules for simultaneously (in the sense of async, not threads) running more than one instance of the block. So I'm not proposing that we change the meaning of the semicolon(s) *within* the block in a way that could mean that if you're looking at half the code in the middle you would be likely to fundamentally misunderstand its operation.

I think that risk - that you can't tell what a semicolon means without reference to the outer context - is what makes your proposal a bridge too far for me.


--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss


--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Re: Proposal: `await.all {...}` for parallelism

Naveen Chawla
Hi! It does not change the meaning of the ";" at all. As you may already know, omitting `await` already invokes multiple async function calls in parallel, in current JavaScript, so absolutely no change in that respect.

The only thing this `await.all` suggestion does, is ensure that all non-awaited async function calls are completed before proceeding beyond the end of the block.

i.e. it adds fairly straightforward and terse deterministic control to otherwise non-deterministic code, without requiring knowledge of destructuring or `Promise.all`.

On Sat, 23 Nov 2019 at 13:25, Tom Boutell <[hidden email]> wrote:
This is very interesting, but this code:

await.all {
   x = getXAsync();
   y = getYAsync();
}

processXAndY(x, y);

Still carries within it the problem that if I'm looking at just the middle of the { ... } block — if "await.all" has scrolled offscreen — I'll be completely wrong about what ";" means. I think that's too much magic.

Also, in the case of the "for" loop, this doesn't address managing the level of concurrency. Although it could in theory with a syntax like await.all({ concurrency: 5 }), I'm not sure if it's practical to implement that for your general case.

Actually I'm curious about what the implementation would look like in general.  If it were babel compiling this, I guess it would have to wrap every statement not preceded by "await" with a check for whether it returns a thenable and add it to an array if it does. But with the concurrency feature it would also have to defer executing the code at all until the right time as otherwise we're still starting zillions of "processes" at once.


On Sat, Nov 23, 2019 at 5:08 AM Naveen Chawla <[hidden email]> wrote:
However, if `await.all { ... }` were to mean "wait for all non-awaited async function calls made within this block to complete before proceeding", as I suggested earlier, I think that could satisfy determinism for "await" wherever it is used, and satisfy the original motivation:

```
await.all {
    for (const item of items) {
        doTheThingAsync(item);
    }
}
```

Notice I have omitted `await` inside the loop. Like current JavaScript, that causes parallel execution, so no change on that front, from a determinism perspective. So determinism is not hurt by `await.all`. Rather, it guarantees completion before going further.

In an earlier example (paraphrase-coded as I forgot the names):

```
let x, y;

await.all {
   x = getXAsync();
   y = getYAsync();
}

processXAndY(x, y);
```

I think the benefit of this syntax appears more stark with the looped (first) example, as current JavaScript requires building an array in the loop to subsequently pass to `Promise.all`, which I think is a little more difficult to conceptualize than the `await.all { ... }` way of doing it. The 2nd example is arguably better than current JavaScript too, particularly because the coder doesn't have to be very smart with destructuring in light of understanding the "Promise.all" return type, etc. In other words, less cognitive overhead, which I think is a net positive.

On Fri, 22 Nov 2019 at 13:44, Tom Boutell <[hidden email]> wrote:
I am very sympathetic to pitches to allow more common cases for promise libraries to be written in an "awaitful" syntax without thinking explicitly about promises.

Howeever I think that changing the meaning of the semicolon in a particular context has too much potential for confusion. As others have said, parallel execution is different, and it should look and feel different. The most basic assumption a developer makes (consecutive lines of code run consecutively) is difficult to get away from; that's why we introduced "await" in the first place, to bring back the ability to write deterministic code with consecutive statements. Which sounds like a reasonable ask, when it's put that way. (:

I did propose this recently:

for (const item of items concurrency 5) {
  await  doTheThing(item);
}

However in this case I'm not talking about consecutive statements, I'm only talking about rules for simultaneously (in the sense of async, not threads) running more than one instance of the block. So I'm not proposing that we change the meaning of the semicolon(s) *within* the block in a way that could mean that if you're looking at half the code in the middle you would be likely to fundamentally misunderstand its operation.

I think that risk - that you can't tell what a semicolon means without reference to the outer context - is what makes your proposal a bridge too far for me.


--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss


--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Re: Proposal: `await.all {...}` for parallelism

Tom Boutell
Hey, you're absolutely right! It's OK because it just means things are more deterministic before the block exits. It doesn't impact any reasonable expectations *during* the block.

I am convinced that your syntax is useful and does not introduce any new confusion.

I wonder, then, if it is also possible to implement concurrency limits for this properly?

await.all({ concurrency: 5 }) {
  for (const item of items) {
    // returns promise
    item.process();
  }
}

This is more challenging because, in our transpilation, we can't just bottle up all the promises and call Promise.all at the end. It would be too late to manage how many are in process at once, bashing on various API limits (:

On Sun, Nov 24, 2019 at 7:43 PM Naveen Chawla <[hidden email]> wrote:
Hi! It does not change the meaning of the ";" at all. As you may already know, omitting `await` already invokes multiple async function calls in parallel, in current JavaScript, so absolutely no change in that respect.

The only thing this `await.all` suggestion does, is ensure that all non-awaited async function calls are completed before proceeding beyond the end of the block.

i.e. it adds fairly straightforward and terse deterministic control to otherwise non-deterministic code, without requiring knowledge of destructuring or `Promise.all`.

On Sat, 23 Nov 2019 at 13:25, Tom Boutell <[hidden email]> wrote:
This is very interesting, but this code:

await.all {
   x = getXAsync();
   y = getYAsync();
}

processXAndY(x, y);

Still carries within it the problem that if I'm looking at just the middle of the { ... } block — if "await.all" has scrolled offscreen — I'll be completely wrong about what ";" means. I think that's too much magic.

Also, in the case of the "for" loop, this doesn't address managing the level of concurrency. Although it could in theory with a syntax like await.all({ concurrency: 5 }), I'm not sure if it's practical to implement that for your general case.

Actually I'm curious about what the implementation would look like in general.  If it were babel compiling this, I guess it would have to wrap every statement not preceded by "await" with a check for whether it returns a thenable and add it to an array if it does. But with the concurrency feature it would also have to defer executing the code at all until the right time as otherwise we're still starting zillions of "processes" at once.


On Sat, Nov 23, 2019 at 5:08 AM Naveen Chawla <[hidden email]> wrote:
However, if `await.all { ... }` were to mean "wait for all non-awaited async function calls made within this block to complete before proceeding", as I suggested earlier, I think that could satisfy determinism for "await" wherever it is used, and satisfy the original motivation:

```
await.all {
    for (const item of items) {
        doTheThingAsync(item);
    }
}
```

Notice I have omitted `await` inside the loop. Like current JavaScript, that causes parallel execution, so no change on that front, from a determinism perspective. So determinism is not hurt by `await.all`. Rather, it guarantees completion before going further.

In an earlier example (paraphrase-coded as I forgot the names):

```
let x, y;

await.all {
   x = getXAsync();
   y = getYAsync();
}

processXAndY(x, y);
```

I think the benefit of this syntax appears more stark with the looped (first) example, as current JavaScript requires building an array in the loop to subsequently pass to `Promise.all`, which I think is a little more difficult to conceptualize than the `await.all { ... }` way of doing it. The 2nd example is arguably better than current JavaScript too, particularly because the coder doesn't have to be very smart with destructuring in light of understanding the "Promise.all" return type, etc. In other words, less cognitive overhead, which I think is a net positive.

On Fri, 22 Nov 2019 at 13:44, Tom Boutell <[hidden email]> wrote:
I am very sympathetic to pitches to allow more common cases for promise libraries to be written in an "awaitful" syntax without thinking explicitly about promises.

Howeever I think that changing the meaning of the semicolon in a particular context has too much potential for confusion. As others have said, parallel execution is different, and it should look and feel different. The most basic assumption a developer makes (consecutive lines of code run consecutively) is difficult to get away from; that's why we introduced "await" in the first place, to bring back the ability to write deterministic code with consecutive statements. Which sounds like a reasonable ask, when it's put that way. (:

I did propose this recently:

for (const item of items concurrency 5) {
  await  doTheThing(item);
}

However in this case I'm not talking about consecutive statements, I'm only talking about rules for simultaneously (in the sense of async, not threads) running more than one instance of the block. So I'm not proposing that we change the meaning of the semicolon(s) *within* the block in a way that could mean that if you're looking at half the code in the middle you would be likely to fundamentally misunderstand its operation.

I think that risk - that you can't tell what a semicolon means without reference to the outer context - is what makes your proposal a bridge too far for me.


--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss


--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his


--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Re: Proposal: `await.all {...}` for parallelism

Tom Boutell
There is however a performance concern with your code that we should talk about.

If I write this:

await.all {
  returnsAPromise();
  for (let i = 0; (i < 100000); i++) {
    doesSimpleFastSynchronousMath();
  }
  returnsAnotherPromise();
}

Then Babel will have no choice but to compile this to:

{
  const promises = [];
  {
    const maybeThenable = returnsAPromise();
    if (maybeThenable && maybeThenable.then) {
      promises.push(maybeThenable);
    }
  }
  promises.push(returnsAPromise());
  for (let i = 0; (i < 100000); i++) {
    const maybeThenable = doesSimpleFastSynchronousMath();
    if (maybeThenable && maybeThenable.then) {
      promises.push(maybeThenable);
    }
  }
  const maybeThenable = returnsAnotherPromise();
  if (maybeThenable && maybeThenable.then) {
    promises.push(maybeThenable);
  }
}
await Promise.all(promises);

Which could have a significant performance impact on that synchronous inner loop.



On Mon, Nov 25, 2019 at 7:55 AM Tom Boutell <[hidden email]> wrote:
Hey, you're absolutely right! It's OK because it just means things are more deterministic before the block exits. It doesn't impact any reasonable expectations *during* the block.

I am convinced that your syntax is useful and does not introduce any new confusion.

I wonder, then, if it is also possible to implement concurrency limits for this properly?

await.all({ concurrency: 5 }) {
  for (const item of items) {
    // returns promise
    item.process();
  }
}

This is more challenging because, in our transpilation, we can't just bottle up all the promises and call Promise.all at the end. It would be too late to manage how many are in process at once, bashing on various API limits (:

On Sun, Nov 24, 2019 at 7:43 PM Naveen Chawla <[hidden email]> wrote:
Hi! It does not change the meaning of the ";" at all. As you may already know, omitting `await` already invokes multiple async function calls in parallel, in current JavaScript, so absolutely no change in that respect.

The only thing this `await.all` suggestion does, is ensure that all non-awaited async function calls are completed before proceeding beyond the end of the block.

i.e. it adds fairly straightforward and terse deterministic control to otherwise non-deterministic code, without requiring knowledge of destructuring or `Promise.all`.

On Sat, 23 Nov 2019 at 13:25, Tom Boutell <[hidden email]> wrote:
This is very interesting, but this code:

await.all {
   x = getXAsync();
   y = getYAsync();
}

processXAndY(x, y);

Still carries within it the problem that if I'm looking at just the middle of the { ... } block — if "await.all" has scrolled offscreen — I'll be completely wrong about what ";" means. I think that's too much magic.

Also, in the case of the "for" loop, this doesn't address managing the level of concurrency. Although it could in theory with a syntax like await.all({ concurrency: 5 }), I'm not sure if it's practical to implement that for your general case.

Actually I'm curious about what the implementation would look like in general.  If it were babel compiling this, I guess it would have to wrap every statement not preceded by "await" with a check for whether it returns a thenable and add it to an array if it does. But with the concurrency feature it would also have to defer executing the code at all until the right time as otherwise we're still starting zillions of "processes" at once.


On Sat, Nov 23, 2019 at 5:08 AM Naveen Chawla <[hidden email]> wrote:
However, if `await.all { ... }` were to mean "wait for all non-awaited async function calls made within this block to complete before proceeding", as I suggested earlier, I think that could satisfy determinism for "await" wherever it is used, and satisfy the original motivation:

```
await.all {
    for (const item of items) {
        doTheThingAsync(item);
    }
}
```

Notice I have omitted `await` inside the loop. Like current JavaScript, that causes parallel execution, so no change on that front, from a determinism perspective. So determinism is not hurt by `await.all`. Rather, it guarantees completion before going further.

In an earlier example (paraphrase-coded as I forgot the names):

```
let x, y;

await.all {
   x = getXAsync();
   y = getYAsync();
}

processXAndY(x, y);
```

I think the benefit of this syntax appears more stark with the looped (first) example, as current JavaScript requires building an array in the loop to subsequently pass to `Promise.all`, which I think is a little more difficult to conceptualize than the `await.all { ... }` way of doing it. The 2nd example is arguably better than current JavaScript too, particularly because the coder doesn't have to be very smart with destructuring in light of understanding the "Promise.all" return type, etc. In other words, less cognitive overhead, which I think is a net positive.

On Fri, 22 Nov 2019 at 13:44, Tom Boutell <[hidden email]> wrote:
I am very sympathetic to pitches to allow more common cases for promise libraries to be written in an "awaitful" syntax without thinking explicitly about promises.

Howeever I think that changing the meaning of the semicolon in a particular context has too much potential for confusion. As others have said, parallel execution is different, and it should look and feel different. The most basic assumption a developer makes (consecutive lines of code run consecutively) is difficult to get away from; that's why we introduced "await" in the first place, to bring back the ability to write deterministic code with consecutive statements. Which sounds like a reasonable ask, when it's put that way. (:

I did propose this recently:

for (const item of items concurrency 5) {
  await  doTheThing(item);
}

However in this case I'm not talking about consecutive statements, I'm only talking about rules for simultaneously (in the sense of async, not threads) running more than one instance of the block. So I'm not proposing that we change the meaning of the semicolon(s) *within* the block in a way that could mean that if you're looking at half the code in the middle you would be likely to fundamentally misunderstand its operation.

I think that risk - that you can't tell what a semicolon means without reference to the outer context - is what makes your proposal a bridge too far for me.


--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss


--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his


--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his


--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Re: Proposal: `await.all {...}` for parallelism

manuelbarzi
why not making it work with the addition of a new keyword suffix for parallel awaits (for example, `||`), grouping the mid blocks that require parallelism. playing with your example and going a bit further:

```
async {
  const v0 = await|| returnsAPromise(); // to be grouped in parallel
  for (let i = 0; (i < 100000); i++) {
    doesSimpleFastSynchronousMath();
  }
  const v1 = await|| returnsAnotherPromise(); // to be grouped in parallel
  async {
      await returnsAnotherPromise1();
      const v2 = await|| returnsAnotherPromise2(); // to be grouped in parallel
      const v3 = await|| returnsAnotherPromise3(); // to be grouped in parallel
      await returnsAnotherPromise4(v2, v3);
      const v4 = await returnsAnotherPromise5();
  }
  await returnsAnotherPromiseX(v0, v1);
}
```

On Mon, Nov 25, 2019 at 2:06 PM Tom Boutell <[hidden email]> wrote:
There is however a performance concern with your code that we should talk about.

If I write this:

await.all {
  returnsAPromise();
  for (let i = 0; (i < 100000); i++) {
    doesSimpleFastSynchronousMath();
  }
  returnsAnotherPromise();
}

Then Babel will have no choice but to compile this to:

{
  const promises = [];
  {
    const maybeThenable = returnsAPromise();
    if (maybeThenable && maybeThenable.then) {
      promises.push(maybeThenable);
    }
  }
  promises.push(returnsAPromise());
  for (let i = 0; (i < 100000); i++) {
    const maybeThenable = doesSimpleFastSynchronousMath();
    if (maybeThenable && maybeThenable.then) {
      promises.push(maybeThenable);
    }
  }
  const maybeThenable = returnsAnotherPromise();
  if (maybeThenable && maybeThenable.then) {
    promises.push(maybeThenable);
  }
}
await Promise.all(promises);

Which could have a significant performance impact on that synchronous inner loop.



On Mon, Nov 25, 2019 at 7:55 AM Tom Boutell <[hidden email]> wrote:
Hey, you're absolutely right! It's OK because it just means things are more deterministic before the block exits. It doesn't impact any reasonable expectations *during* the block.

I am convinced that your syntax is useful and does not introduce any new confusion.

I wonder, then, if it is also possible to implement concurrency limits for this properly?

await.all({ concurrency: 5 }) {
  for (const item of items) {
    // returns promise
    item.process();
  }
}

This is more challenging because, in our transpilation, we can't just bottle up all the promises and call Promise.all at the end. It would be too late to manage how many are in process at once, bashing on various API limits (:

On Sun, Nov 24, 2019 at 7:43 PM Naveen Chawla <[hidden email]> wrote:
Hi! It does not change the meaning of the ";" at all. As you may already know, omitting `await` already invokes multiple async function calls in parallel, in current JavaScript, so absolutely no change in that respect.

The only thing this `await.all` suggestion does, is ensure that all non-awaited async function calls are completed before proceeding beyond the end of the block.

i.e. it adds fairly straightforward and terse deterministic control to otherwise non-deterministic code, without requiring knowledge of destructuring or `Promise.all`.

On Sat, 23 Nov 2019 at 13:25, Tom Boutell <[hidden email]> wrote:
This is very interesting, but this code:

await.all {
   x = getXAsync();
   y = getYAsync();
}

processXAndY(x, y);

Still carries within it the problem that if I'm looking at just the middle of the { ... } block — if "await.all" has scrolled offscreen — I'll be completely wrong about what ";" means. I think that's too much magic.

Also, in the case of the "for" loop, this doesn't address managing the level of concurrency. Although it could in theory with a syntax like await.all({ concurrency: 5 }), I'm not sure if it's practical to implement that for your general case.

Actually I'm curious about what the implementation would look like in general.  If it were babel compiling this, I guess it would have to wrap every statement not preceded by "await" with a check for whether it returns a thenable and add it to an array if it does. But with the concurrency feature it would also have to defer executing the code at all until the right time as otherwise we're still starting zillions of "processes" at once.


On Sat, Nov 23, 2019 at 5:08 AM Naveen Chawla <[hidden email]> wrote:
However, if `await.all { ... }` were to mean "wait for all non-awaited async function calls made within this block to complete before proceeding", as I suggested earlier, I think that could satisfy determinism for "await" wherever it is used, and satisfy the original motivation:

```
await.all {
    for (const item of items) {
        doTheThingAsync(item);
    }
}
```

Notice I have omitted `await` inside the loop. Like current JavaScript, that causes parallel execution, so no change on that front, from a determinism perspective. So determinism is not hurt by `await.all`. Rather, it guarantees completion before going further.

In an earlier example (paraphrase-coded as I forgot the names):

```
let x, y;

await.all {
   x = getXAsync();
   y = getYAsync();
}

processXAndY(x, y);
```

I think the benefit of this syntax appears more stark with the looped (first) example, as current JavaScript requires building an array in the loop to subsequently pass to `Promise.all`, which I think is a little more difficult to conceptualize than the `await.all { ... }` way of doing it. The 2nd example is arguably better than current JavaScript too, particularly because the coder doesn't have to be very smart with destructuring in light of understanding the "Promise.all" return type, etc. In other words, less cognitive overhead, which I think is a net positive.

On Fri, 22 Nov 2019 at 13:44, Tom Boutell <[hidden email]> wrote:
I am very sympathetic to pitches to allow more common cases for promise libraries to be written in an "awaitful" syntax without thinking explicitly about promises.

Howeever I think that changing the meaning of the semicolon in a particular context has too much potential for confusion. As others have said, parallel execution is different, and it should look and feel different. The most basic assumption a developer makes (consecutive lines of code run consecutively) is difficult to get away from; that's why we introduced "await" in the first place, to bring back the ability to write deterministic code with consecutive statements. Which sounds like a reasonable ask, when it's put that way. (:

I did propose this recently:

for (const item of items concurrency 5) {
  await  doTheThing(item);
}

However in this case I'm not talking about consecutive statements, I'm only talking about rules for simultaneously (in the sense of async, not threads) running more than one instance of the block. So I'm not proposing that we change the meaning of the semicolon(s) *within* the block in a way that could mean that if you're looking at half the code in the middle you would be likely to fundamentally misunderstand its operation.

I think that risk - that you can't tell what a semicolon means without reference to the outer context - is what makes your proposal a bridge too far for me.


--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss


--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his


--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his


--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Re: Proposal: `await.all {...}` for parallelism

manuelbarzi
another example, that may "normalize" it a bit more:

```
async { // ... = returns a promise
    const x1 = await|| ...
    const x2 = await ... (x1)
    const x3 = await|| ...
    const x10 = await ... (x2x3)
    
    let x4x5x6
    
    async {
        x4 = await|| ... (x1x2)
        x5 = await|| ... (x2x3)
        x6 = await ... (x4x5x10)
    }

    let x7x8x9

    async {
        x7 = await|| ... (x4x6)
        x8 = await ... (x6x7)
        x9 = await|| ... (x5x6)
    }

    await ... (x8x9)
}
```

On Mon, Nov 25, 2019 at 3:19 PM manuelbarzi <[hidden email]> wrote:
why not making it work with the addition of a new keyword suffix for parallel awaits (for example, `||`), grouping the mid blocks that require parallelism. playing with your example and going a bit further:

```
async {
  const v0 = await|| returnsAPromise(); // to be grouped in parallel
  for (let i = 0; (i < 100000); i++) {
    doesSimpleFastSynchronousMath();
  }
  const v1 = await|| returnsAnotherPromise(); // to be grouped in parallel
  async {
      await returnsAnotherPromise1();
      const v2 = await|| returnsAnotherPromise2(); // to be grouped in parallel
      const v3 = await|| returnsAnotherPromise3(); // to be grouped in parallel
      await returnsAnotherPromise4(v2, v3);
      const v4 = await returnsAnotherPromise5();
  }
  await returnsAnotherPromiseX(v0, v1);
}
```

On Mon, Nov 25, 2019 at 2:06 PM Tom Boutell <[hidden email]> wrote:
There is however a performance concern with your code that we should talk about.

If I write this:

await.all {
  returnsAPromise();
  for (let i = 0; (i < 100000); i++) {
    doesSimpleFastSynchronousMath();
  }
  returnsAnotherPromise();
}

Then Babel will have no choice but to compile this to:

{
  const promises = [];
  {
    const maybeThenable = returnsAPromise();
    if (maybeThenable && maybeThenable.then) {
      promises.push(maybeThenable);
    }
  }
  promises.push(returnsAPromise());
  for (let i = 0; (i < 100000); i++) {
    const maybeThenable = doesSimpleFastSynchronousMath();
    if (maybeThenable && maybeThenable.then) {
      promises.push(maybeThenable);
    }
  }
  const maybeThenable = returnsAnotherPromise();
  if (maybeThenable && maybeThenable.then) {
    promises.push(maybeThenable);
  }
}
await Promise.all(promises);

Which could have a significant performance impact on that synchronous inner loop.



On Mon, Nov 25, 2019 at 7:55 AM Tom Boutell <[hidden email]> wrote:
Hey, you're absolutely right! It's OK because it just means things are more deterministic before the block exits. It doesn't impact any reasonable expectations *during* the block.

I am convinced that your syntax is useful and does not introduce any new confusion.

I wonder, then, if it is also possible to implement concurrency limits for this properly?

await.all({ concurrency: 5 }) {
  for (const item of items) {
    // returns promise
    item.process();
  }
}

This is more challenging because, in our transpilation, we can't just bottle up all the promises and call Promise.all at the end. It would be too late to manage how many are in process at once, bashing on various API limits (:

On Sun, Nov 24, 2019 at 7:43 PM Naveen Chawla <[hidden email]> wrote:
Hi! It does not change the meaning of the ";" at all. As you may already know, omitting `await` already invokes multiple async function calls in parallel, in current JavaScript, so absolutely no change in that respect.

The only thing this `await.all` suggestion does, is ensure that all non-awaited async function calls are completed before proceeding beyond the end of the block.

i.e. it adds fairly straightforward and terse deterministic control to otherwise non-deterministic code, without requiring knowledge of destructuring or `Promise.all`.

On Sat, 23 Nov 2019 at 13:25, Tom Boutell <[hidden email]> wrote:
This is very interesting, but this code:

await.all {
   x = getXAsync();
   y = getYAsync();
}

processXAndY(x, y);

Still carries within it the problem that if I'm looking at just the middle of the { ... } block — if "await.all" has scrolled offscreen — I'll be completely wrong about what ";" means. I think that's too much magic.

Also, in the case of the "for" loop, this doesn't address managing the level of concurrency. Although it could in theory with a syntax like await.all({ concurrency: 5 }), I'm not sure if it's practical to implement that for your general case.

Actually I'm curious about what the implementation would look like in general.  If it were babel compiling this, I guess it would have to wrap every statement not preceded by "await" with a check for whether it returns a thenable and add it to an array if it does. But with the concurrency feature it would also have to defer executing the code at all until the right time as otherwise we're still starting zillions of "processes" at once.


On Sat, Nov 23, 2019 at 5:08 AM Naveen Chawla <[hidden email]> wrote:
However, if `await.all { ... }` were to mean "wait for all non-awaited async function calls made within this block to complete before proceeding", as I suggested earlier, I think that could satisfy determinism for "await" wherever it is used, and satisfy the original motivation:

```
await.all {
    for (const item of items) {
        doTheThingAsync(item);
    }
}
```

Notice I have omitted `await` inside the loop. Like current JavaScript, that causes parallel execution, so no change on that front, from a determinism perspective. So determinism is not hurt by `await.all`. Rather, it guarantees completion before going further.

In an earlier example (paraphrase-coded as I forgot the names):

```
let x, y;

await.all {
   x = getXAsync();
   y = getYAsync();
}

processXAndY(x, y);
```

I think the benefit of this syntax appears more stark with the looped (first) example, as current JavaScript requires building an array in the loop to subsequently pass to `Promise.all`, which I think is a little more difficult to conceptualize than the `await.all { ... }` way of doing it. The 2nd example is arguably better than current JavaScript too, particularly because the coder doesn't have to be very smart with destructuring in light of understanding the "Promise.all" return type, etc. In other words, less cognitive overhead, which I think is a net positive.

On Fri, 22 Nov 2019 at 13:44, Tom Boutell <[hidden email]> wrote:
I am very sympathetic to pitches to allow more common cases for promise libraries to be written in an "awaitful" syntax without thinking explicitly about promises.

Howeever I think that changing the meaning of the semicolon in a particular context has too much potential for confusion. As others have said, parallel execution is different, and it should look and feel different. The most basic assumption a developer makes (consecutive lines of code run consecutively) is difficult to get away from; that's why we introduced "await" in the first place, to bring back the ability to write deterministic code with consecutive statements. Which sounds like a reasonable ask, when it's put that way. (:

I did propose this recently:

for (const item of items concurrency 5) {
  await  doTheThing(item);
}

However in this case I'm not talking about consecutive statements, I'm only talking about rules for simultaneously (in the sense of async, not threads) running more than one instance of the block. So I'm not proposing that we change the meaning of the semicolon(s) *within* the block in a way that could mean that if you're looking at half the code in the middle you would be likely to fundamentally misunderstand its operation.

I think that risk - that you can't tell what a semicolon means without reference to the outer context - is what makes your proposal a bridge too far for me.


--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss


--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his


--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his


--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Re: Proposal: `await.all {...}` for parallelism

Naveen Chawla
Hi Tom! Would you mind clarifying the performance hit you are referring to? I'm seeing that the "synchronous" calls wouldn't be added to the array you used in your example, so it's not clear to me to which performance hit you are referring.

Hi Manuel! Would you mind explaining the added value of your proposal? I am only seeing it being more verbose, but I've not found any added functionality or benefit from it

On Mon, 25 Nov 2019 at 14:57, manuelbarzi <[hidden email]> wrote:
another example, that may "normalize" it a bit more:

```
async { // ... = returns a promise
    const x1 = await|| ...
    const x2 = await ... (x1)
    const x3 = await|| ...
    const x10 = await ... (x2x3)
    
    let x4x5x6
    
    async {
        x4 = await|| ... (x1x2)
        x5 = await|| ... (x2x3)
        x6 = await ... (x4x5x10)
    }

    let x7x8x9

    async {
        x7 = await|| ... (x4x6)
        x8 = await ... (x6x7)
        x9 = await|| ... (x5x6)
    }

    await ... (x8x9)
}
```

On Mon, Nov 25, 2019 at 3:19 PM manuelbarzi <[hidden email]> wrote:
why not making it work with the addition of a new keyword suffix for parallel awaits (for example, `||`), grouping the mid blocks that require parallelism. playing with your example and going a bit further:

```
async {
  const v0 = await|| returnsAPromise(); // to be grouped in parallel
  for (let i = 0; (i < 100000); i++) {
    doesSimpleFastSynchronousMath();
  }
  const v1 = await|| returnsAnotherPromise(); // to be grouped in parallel
  async {
      await returnsAnotherPromise1();
      const v2 = await|| returnsAnotherPromise2(); // to be grouped in parallel
      const v3 = await|| returnsAnotherPromise3(); // to be grouped in parallel
      await returnsAnotherPromise4(v2, v3);
      const v4 = await returnsAnotherPromise5();
  }
  await returnsAnotherPromiseX(v0, v1);
}
```

On Mon, Nov 25, 2019 at 2:06 PM Tom Boutell <[hidden email]> wrote:
There is however a performance concern with your code that we should talk about.

If I write this:

await.all {
  returnsAPromise();
  for (let i = 0; (i < 100000); i++) {
    doesSimpleFastSynchronousMath();
  }
  returnsAnotherPromise();
}

Then Babel will have no choice but to compile this to:

{
  const promises = [];
  {
    const maybeThenable = returnsAPromise();
    if (maybeThenable && maybeThenable.then) {
      promises.push(maybeThenable);
    }
  }
  promises.push(returnsAPromise());
  for (let i = 0; (i < 100000); i++) {
    const maybeThenable = doesSimpleFastSynchronousMath();
    if (maybeThenable && maybeThenable.then) {
      promises.push(maybeThenable);
    }
  }
  const maybeThenable = returnsAnotherPromise();
  if (maybeThenable && maybeThenable.then) {
    promises.push(maybeThenable);
  }
}
await Promise.all(promises);

Which could have a significant performance impact on that synchronous inner loop.



On Mon, Nov 25, 2019 at 7:55 AM Tom Boutell <[hidden email]> wrote:
Hey, you're absolutely right! It's OK because it just means things are more deterministic before the block exits. It doesn't impact any reasonable expectations *during* the block.

I am convinced that your syntax is useful and does not introduce any new confusion.

I wonder, then, if it is also possible to implement concurrency limits for this properly?

await.all({ concurrency: 5 }) {
  for (const item of items) {
    // returns promise
    item.process();
  }
}

This is more challenging because, in our transpilation, we can't just bottle up all the promises and call Promise.all at the end. It would be too late to manage how many are in process at once, bashing on various API limits (:

On Sun, Nov 24, 2019 at 7:43 PM Naveen Chawla <[hidden email]> wrote:
Hi! It does not change the meaning of the ";" at all. As you may already know, omitting `await` already invokes multiple async function calls in parallel, in current JavaScript, so absolutely no change in that respect.

The only thing this `await.all` suggestion does, is ensure that all non-awaited async function calls are completed before proceeding beyond the end of the block.

i.e. it adds fairly straightforward and terse deterministic control to otherwise non-deterministic code, without requiring knowledge of destructuring or `Promise.all`.

On Sat, 23 Nov 2019 at 13:25, Tom Boutell <[hidden email]> wrote:
This is very interesting, but this code:

await.all {
   x = getXAsync();
   y = getYAsync();
}

processXAndY(x, y);

Still carries within it the problem that if I'm looking at just the middle of the { ... } block — if "await.all" has scrolled offscreen — I'll be completely wrong about what ";" means. I think that's too much magic.

Also, in the case of the "for" loop, this doesn't address managing the level of concurrency. Although it could in theory with a syntax like await.all({ concurrency: 5 }), I'm not sure if it's practical to implement that for your general case.

Actually I'm curious about what the implementation would look like in general.  If it were babel compiling this, I guess it would have to wrap every statement not preceded by "await" with a check for whether it returns a thenable and add it to an array if it does. But with the concurrency feature it would also have to defer executing the code at all until the right time as otherwise we're still starting zillions of "processes" at once.


On Sat, Nov 23, 2019 at 5:08 AM Naveen Chawla <[hidden email]> wrote:
However, if `await.all { ... }` were to mean "wait for all non-awaited async function calls made within this block to complete before proceeding", as I suggested earlier, I think that could satisfy determinism for "await" wherever it is used, and satisfy the original motivation:

```
await.all {
    for (const item of items) {
        doTheThingAsync(item);
    }
}
```

Notice I have omitted `await` inside the loop. Like current JavaScript, that causes parallel execution, so no change on that front, from a determinism perspective. So determinism is not hurt by `await.all`. Rather, it guarantees completion before going further.

In an earlier example (paraphrase-coded as I forgot the names):

```
let x, y;

await.all {
   x = getXAsync();
   y = getYAsync();
}

processXAndY(x, y);
```

I think the benefit of this syntax appears more stark with the looped (first) example, as current JavaScript requires building an array in the loop to subsequently pass to `Promise.all`, which I think is a little more difficult to conceptualize than the `await.all { ... }` way of doing it. The 2nd example is arguably better than current JavaScript too, particularly because the coder doesn't have to be very smart with destructuring in light of understanding the "Promise.all" return type, etc. In other words, less cognitive overhead, which I think is a net positive.

On Fri, 22 Nov 2019 at 13:44, Tom Boutell <[hidden email]> wrote:
I am very sympathetic to pitches to allow more common cases for promise libraries to be written in an "awaitful" syntax without thinking explicitly about promises.

Howeever I think that changing the meaning of the semicolon in a particular context has too much potential for confusion. As others have said, parallel execution is different, and it should look and feel different. The most basic assumption a developer makes (consecutive lines of code run consecutively) is difficult to get away from; that's why we introduced "await" in the first place, to bring back the ability to write deterministic code with consecutive statements. Which sounds like a reasonable ask, when it's put that way. (:

I did propose this recently:

for (const item of items concurrency 5) {
  await  doTheThing(item);
}

However in this case I'm not talking about consecutive statements, I'm only talking about rules for simultaneously (in the sense of async, not threads) running more than one instance of the block. So I'm not proposing that we change the meaning of the semicolon(s) *within* the block in a way that could mean that if you're looking at half the code in the middle you would be likely to fundamentally misunderstand its operation.

I think that risk - that you can't tell what a semicolon means without reference to the outer context - is what makes your proposal a bridge too far for me.


--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss


--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his


--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his


--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Re: Proposal: `await.all {...}` for parallelism

manuelbarzi
Hi Manuel! Would you mind explaining the added value of your proposal? I am only seeing it being more verbose, but I've not found any added functionality or benefit from it

the proposal combines parallel and series async / await and could resolve complex trees, like the following example:

```
// NOTE async {} = asynchronous scope (it groups parallel awaits => `await||`)
// NOTE p? = function call that returns a promise (? = just and index)

async { 
    const x1 = await|| p1()
    const x2 = await p2(x1)
    const x3 = await|| p3()
    const x10 = await p10(x2x3)
    
    let x4x5x6
    
    async {
        x4 = await|| p4(x1x2)
        x5 = await|| p5(x2x3)
        x6 = await p6(x4x5x10)
    }

    let x7x8x9

    async {
        x7 = await|| p7(x4x6)
        x8 = await p8(x6x7)
        x9 = await|| p9(x5x6)
    }

    await p11(x8x9)
}

// it would resolve a tree of parallel and series like following with traditional promises

Promise.resolve()
    .then(() => Promise.all([p1p3]))
    .then((x1x3=> 
        p2(x1)
            .then(x2 => 
                p10(x2x3)
                     .then(x10 => {
                        let x4x5x6

                        return Promise.resolve()
                            .then(() => Promise.all([p4(x1x2), p5(x2x3)]))
                            .then(results => [x4x5] = results)
                            .then(() => p6(x4x5x10))
                            .then(x6 => {
                                let x7x8x9

                                return Promise.resolve()
                                    .then(() => Promise.all([p7(x4x6), p9(x5x6)]))
                                    .then(results => [x7x9] = results)
                                    .then(() => p8(x6x7))
                                    .then(_x8 => x8 = _x8)
                                    .then(() => p11(x8x9))
                            })
                     })
            )
    )
```



_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Re: Proposal: `await.all {...}` for parallelism

manuelbarzi
a few little corrections...

// NOTE async {} = asynchronous scope (it groups parallel awaits => `await||`)
// NOTE p? = function call that returns a promise (? = just and index)

async {
    const x1 = await || p1()
    const x2 = await p2(x1)
    const x3 = await || p3()
    const x10 = await p10(x2x3)

    let x4x5x6

    async {
        x4 = await || p4(x1x2)
        x5 = await || p5(x2x3)
        x6 = await p6(x4x5x10)
    }

    let x7x8x9

    async {
        x7 = await || p7(x4x6)
        x8 = await p8(x6x7)
        x9 = await || p9(x5x6)
    }

    await p11(x8x9)
}

Promise.resolve()
    .then(() => Promise.all([p1(), p3()]))
    .then((x1x3=>
        p2(x1)
            .then(x2 =>
                p10(x2x3)
                    .then(x10 => {
                        let x4x5x6

                        return Promise.resolve()
                            .then(() => Promise.all([p4(x1x2), p5(x2x3)]))
                            .then(results => [x4x5] = results)
                            .then(() => p6(x4x5x10))
                            .then(_x6 => x6 = _x6)
                            .then(() => {
                                let x7x8x9
        
                                return Promise.resolve()
                                    .then(() => Promise.all([p7(x4x6), p9(x5x6)]))
                                    .then(results => [x7x9] = results)
                                    .then(() => p8(x6x7))
                                    .then(_x8 => x8 = _x8)
                                    .then(() => p11(x8x9))
                            })
                    })
            )
    )




On Mon, Nov 25, 2019 at 5:41 PM manuelbarzi <[hidden email]> wrote:
Hi Manuel! Would you mind explaining the added value of your proposal? I am only seeing it being more verbose, but I've not found any added functionality or benefit from it

the proposal combines parallel and series async / await and could resolve complex trees, like the following example:

```
// NOTE async {} = asynchronous scope (it groups parallel awaits => `await||`)
// NOTE p? = function call that returns a promise (? = just and index)

async { 
    const x1 = await|| p1()
    const x2 = await p2(x1)
    const x3 = await|| p3()
    const x10 = await p10(x2x3)
    
    let x4x5x6
    
    async {
        x4 = await|| p4(x1x2)
        x5 = await|| p5(x2x3)
        x6 = await p6(x4x5x10)
    }

    let x7x8x9

    async {
        x7 = await|| p7(x4x6)
        x8 = await p8(x6x7)
        x9 = await|| p9(x5x6)
    }

    await p11(x8x9)
}

// it would resolve a tree of parallel and series like following with traditional promises

Promise.resolve()
    .then(() => Promise.all([p1p3]))
    .then((x1x3=> 
        p2(x1)
            .then(x2 => 
                p10(x2x3)
                     .then(x10 => {
                        let x4x5x6

                        return Promise.resolve()
                            .then(() => Promise.all([p4(x1x2), p5(x2x3)]))
                            .then(results => [x4x5] = results)
                            .then(() => p6(x4x5x10))
                            .then(x6 => {
                                let x7x8x9

                                return Promise.resolve()
                                    .then(() => Promise.all([p7(x4x6), p9(x5x6)]))
                                    .then(results => [x7x9] = results)
                                    .then(() => p8(x6x7))
                                    .then(_x8 => x8 = _x8)
                                    .then(() => p11(x8x9))
                            })
                     })
            )
    )
```



_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Re: Proposal: `await.all {...}` for parallelism

manuelbarzi
yet smaller, as Promise.resolve() was totally redundant here...

// NOTE async {} = asynchronous scope (it groups parallel awaits => `await||`)
// NOTE p? = function call that returns a promise (? = just and index)

async {
    const x1 = await || p1()
    const x2 = await p2(x1)
    const x3 = await || p3()
    const x10 = await p10(x2x3)

    let x4x5x6

    async {
        x4 = await || p4(x1x2)
        x5 = await || p5(x2x3)
        x6 = await p6(x4x5x10)
    }

    let x7x8x9

    async {
        x7 = await || p7(x4x6)
        x8 = await p8(x6x7)
        x9 = await || p9(x5x6)
    }

    await p11(x8x9)
}

Promise.all([p1(), p3()])
    .then((x1x3=>
        p2(x1)
            .then(x2 =>
                p10(x2x3)
                    .then(x10 => {
                        let x4x5x6

                        return Promise.all([p4(x1x2), p5(x2x3)])
                            .then(results => [x4x5] = results)
                            .then(() => p6(x4x5x10))
                            .then(_x6 => x6 = _x6)
                            .then(() => {
                                let x7x8x9
        
                                return Promise.all([p7(x4x6), p9(x5x6)])
                                    .then(results => [x7x9] = results)
                                    .then(() => p8(x6x7))
                                    .then(_x8 => x8 = _x8)
                                    .then(() => p11(x8x9))
                            })
                    })
            )
    )

On Mon, Nov 25, 2019 at 5:57 PM manuelbarzi <[hidden email]> wrote:
a few little corrections...

// NOTE async {} = asynchronous scope (it groups parallel awaits => `await||`)
// NOTE p? = function call that returns a promise (? = just and index)

async {
    const x1 = await || p1()
    const x2 = await p2(x1)
    const x3 = await || p3()
    const x10 = await p10(x2x3)

    let x4x5x6

    async {
        x4 = await || p4(x1x2)
        x5 = await || p5(x2x3)
        x6 = await p6(x4x5x10)
    }

    let x7x8x9

    async {
        x7 = await || p7(x4x6)
        x8 = await p8(x6x7)
        x9 = await || p9(x5x6)
    }

    await p11(x8x9)
}

Promise.resolve()
    .then(() => Promise.all([p1(), p3()]))
    .then((x1x3=>
        p2(x1)
            .then(x2 =>
                p10(x2x3)
                    .then(x10 => {
                        let x4x5x6

                        return Promise.resolve()
                            .then(() => Promise.all([p4(x1x2), p5(x2x3)]))
                            .then(results => [x4x5] = results)
                            .then(() => p6(x4x5x10))
                            .then(_x6 => x6 = _x6)
                            .then(() => {
                                let x7x8x9
        
                                return Promise.resolve()
                                    .then(() => Promise.all([p7(x4x6), p9(x5x6)]))
                                    .then(results => [x7x9] = results)
                                    .then(() => p8(x6x7))
                                    .then(_x8 => x8 = _x8)
                                    .then(() => p11(x8x9))
                            })
                    })
            )
    )




On Mon, Nov 25, 2019 at 5:41 PM manuelbarzi <[hidden email]> wrote:
Hi Manuel! Would you mind explaining the added value of your proposal? I am only seeing it being more verbose, but I've not found any added functionality or benefit from it

the proposal combines parallel and series async / await and could resolve complex trees, like the following example:

```
// NOTE async {} = asynchronous scope (it groups parallel awaits => `await||`)
// NOTE p? = function call that returns a promise (? = just and index)

async { 
    const x1 = await|| p1()
    const x2 = await p2(x1)
    const x3 = await|| p3()
    const x10 = await p10(x2x3)
    
    let x4x5x6
    
    async {
        x4 = await|| p4(x1x2)
        x5 = await|| p5(x2x3)
        x6 = await p6(x4x5x10)
    }

    let x7x8x9

    async {
        x7 = await|| p7(x4x6)
        x8 = await p8(x6x7)
        x9 = await|| p9(x5x6)
    }

    await p11(x8x9)
}

// it would resolve a tree of parallel and series like following with traditional promises

Promise.resolve()
    .then(() => Promise.all([p1p3]))
    .then((x1x3=> 
        p2(x1)
            .then(x2 => 
                p10(x2x3)
                     .then(x10 => {
                        let x4x5x6

                        return Promise.resolve()
                            .then(() => Promise.all([p4(x1x2), p5(x2x3)]))
                            .then(results => [x4x5] = results)
                            .then(() => p6(x4x5x10))
                            .then(x6 => {
                                let x7x8x9

                                return Promise.resolve()
                                    .then(() => Promise.all([p7(x4x6), p9(x5x6)]))
                                    .then(results => [x7x9] = results)
                                    .then(() => p8(x6x7))
                                    .then(_x8 => x8 = _x8)
                                    .then(() => p11(x8x9))
                            })
                     })
            )
    )
```



_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Re: Proposal: `await.all {...}` for parallelism

Tom Boutell
I don't think I like "await ||" as a syntax, because it doesn't have anything to do with the "OR" operator.

It does avoid adding a bunch of potentially optimization-breaking if statements to the transpiled output for synchronous code like in my example, though, because you only get the behavior for the promises you actually choose to collect with it.

Manuel, I am not sure I understand your examples. You are consuming the values x1 and x2 in p4 right in the middle of the same async block that contains the "await ||" statements that produce them. They are not guaranteed to resolve until after that block is over, no?

On Mon, Nov 25, 2019 at 12:05 PM manuelbarzi <[hidden email]> wrote:
yet smaller, as Promise.resolve() was totally redundant here...

// NOTE async {} = asynchronous scope (it groups parallel awaits => `await||`)
// NOTE p? = function call that returns a promise (? = just and index)

async {
    const x1 = await || p1()
    const x2 = await p2(x1)
    const x3 = await || p3()
    const x10 = await p10(x2x3)

    let x4x5x6

    async {
        x4 = await || p4(x1x2)
        x5 = await || p5(x2x3)
        x6 = await p6(x4x5x10)
    }

    let x7x8x9

    async {
        x7 = await || p7(x4x6)
        x8 = await p8(x6x7)
        x9 = await || p9(x5x6)
    }

    await p11(x8x9)
}

Promise.all([p1(), p3()])
    .then((x1x3=>
        p2(x1)
            .then(x2 =>
                p10(x2x3)
                    .then(x10 => {
                        let x4x5x6

                        return Promise.all([p4(x1x2), p5(x2x3)])
                            .then(results => [x4x5] = results)
                            .then(() => p6(x4x5x10))
                            .then(_x6 => x6 = _x6)
                            .then(() => {
                                let x7x8x9
        
                                return Promise.all([p7(x4x6), p9(x5x6)])
                                    .then(results => [x7x9] = results)
                                    .then(() => p8(x6x7))
                                    .then(_x8 => x8 = _x8)
                                    .then(() => p11(x8x9))
                            })
                    })
            )
    )

On Mon, Nov 25, 2019 at 5:57 PM manuelbarzi <[hidden email]> wrote:
a few little corrections...

// NOTE async {} = asynchronous scope (it groups parallel awaits => `await||`)
// NOTE p? = function call that returns a promise (? = just and index)

async {
    const x1 = await || p1()
    const x2 = await p2(x1)
    const x3 = await || p3()
    const x10 = await p10(x2x3)

    let x4x5x6

    async {
        x4 = await || p4(x1x2)
        x5 = await || p5(x2x3)
        x6 = await p6(x4x5x10)
    }

    let x7x8x9

    async {
        x7 = await || p7(x4x6)
        x8 = await p8(x6x7)
        x9 = await || p9(x5x6)
    }

    await p11(x8x9)
}

Promise.resolve()
    .then(() => Promise.all([p1(), p3()]))
    .then((x1x3=>
        p2(x1)
            .then(x2 =>
                p10(x2x3)
                    .then(x10 => {
                        let x4x5x6

                        return Promise.resolve()
                            .then(() => Promise.all([p4(x1x2), p5(x2x3)]))
                            .then(results => [x4x5] = results)
                            .then(() => p6(x4x5x10))
                            .then(_x6 => x6 = _x6)
                            .then(() => {
                                let x7x8x9
        
                                return Promise.resolve()
                                    .then(() => Promise.all([p7(x4x6), p9(x5x6)]))
                                    .then(results => [x7x9] = results)
                                    .then(() => p8(x6x7))
                                    .then(_x8 => x8 = _x8)
                                    .then(() => p11(x8x9))
                            })
                    })
            )
    )




On Mon, Nov 25, 2019 at 5:41 PM manuelbarzi <[hidden email]> wrote:
Hi Manuel! Would you mind explaining the added value of your proposal? I am only seeing it being more verbose, but I've not found any added functionality or benefit from it

the proposal combines parallel and series async / await and could resolve complex trees, like the following example:

```
// NOTE async {} = asynchronous scope (it groups parallel awaits => `await||`)
// NOTE p? = function call that returns a promise (? = just and index)

async { 
    const x1 = await|| p1()
    const x2 = await p2(x1)
    const x3 = await|| p3()
    const x10 = await p10(x2x3)
    
    let x4x5x6
    
    async {
        x4 = await|| p4(x1x2)
        x5 = await|| p5(x2x3)
        x6 = await p6(x4x5x10)
    }

    let x7x8x9

    async {
        x7 = await|| p7(x4x6)
        x8 = await p8(x6x7)
        x9 = await|| p9(x5x6)
    }

    await p11(x8x9)
}

// it would resolve a tree of parallel and series like following with traditional promises

Promise.resolve()
    .then(() => Promise.all([p1p3]))
    .then((x1x3=> 
        p2(x1)
            .then(x2 => 
                p10(x2x3)
                     .then(x10 => {
                        let x4x5x6

                        return Promise.resolve()
                            .then(() => Promise.all([p4(x1x2), p5(x2x3)]))
                            .then(results => [x4x5] = results)
                            .then(() => p6(x4x5x10))
                            .then(x6 => {
                                let x7x8x9

                                return Promise.resolve()
                                    .then(() => Promise.all([p7(x4x6), p9(x5x6)]))
                                    .then(results => [x7x9] = results)
                                    .then(() => p8(x6x7))
                                    .then(_x8 => x8 = _x8)
                                    .then(() => p11(x8x9))
                            })
                     })
            )
    )
```




--

THOMAS BOUTELL | CHIEF TECHNOLOGY OFFICER
APOSTROPHECMS | apostrophecms.com | he/him/his

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Re: Proposal: `await.all {...}` for parallelism

manuelbarzi
I don't think I like "await ||" as a syntax, because it doesn't have anything to do with the "OR" operator.

it is just a quick proposal to symbolise parallelism easily (||), but just that, of course it could be any other more convenient symbol
 
Manuel, I am not sure I understand your examples. You are consuming the values x1 and x2 in p4 right in the middle of the same async block that contains the "await ||" statements that produce them. They are not guaranteed to resolve until after that block is over, no?

the `async {}` block would group only the parallel awaits (`await||`) into a `Promise.all` and would chain it in order with the other series awaits inside (normal `await`).

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Fwd: Re: Proposal: `await.all {...}` for parallelism

manuelbarzi
hi Tom

So the parallel awaits would execute before the series awaits?

not exactly. it would group the parallel awaits at the position the first parallel await|| is sentenced.

following the demo i sent:

// NOTE async {} = asynchronous scope (it groups parallel awaits => `await||`)
// NOTE p? = function call that returns a promise (? = just and index)

async {
    const x1 = await || p1()
    const x2 = await p2(x1)
    const x3 = await || p3()
    const x10 = await p10(x2x3)

    let x4x5x6

    async {
        x4 = await || p4(x1x2)
        x5 = await || p5(x2x3)
        x6 = await p6(x4x5x10)
    }

    let x7x8x9

    async {
        x7 = await || p7(x4x6)
        x8 = await p8(x6x7)
        x9 = await || p9(x5x6)
    }

    await p11(x8x9)
}

in the outer block, the first sentence is already parallel, ok, then all parallel awaits in this block will group at first position, siblings of this first one (that is p1 and p3).

in the inner blocks too (just a casualty i created the demos this way, but could be different and still work). in the first inner block, p4 and p5, and in the second inner block, p7 and p9.

"transpiling" it to promises, it would resolve a complex scenario like following (the demo could be improved, of course):

 // it would resolve a tree of parallel and series like following with traditional promises

Promise.all([p1(), p3()])
    .then((x1x3=>
        p2(x1)
            .then(x2 =>
                p10(x2x3)
                    .then(x10 => {
                        let x4x5x6

                        return Promise.all([p4(x1x2), p5(x2x3)])
                            .then(results => [x4x5] = results)
                            .then(() => p6(x4x5x10))
                            .then(_x6 => x6 = _x6)
                            .then(() => {
                                let x7x8x9
        
                                return Promise.all([p7(x4x6), p9(x5x6)])
                                    .then(results => [x7x9] = results)
                                    .then(() => p8(x6x7))
                                    .then(_x8 => x8 = _x8)
                                    .then(() => p11(x8x9))
                            })
                    })
            )
    )

What if there are series awaits before the first parallel await?

no problem, if there was one before, then in it would be chained first in the outer promise, and following it would be the promise.all for the next parallel group.
 
What if the parallel awaits depend on the results of those series awaits? It reads very strangely.

that's totally valid as far as the parallel await is located after the series awaits.

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Re: Proposal: `await.all {...}` for parallelism

manuelbarzi
by the way, i think it can also be improved without the need of `async {}` blocks, just marking together the parallel awaits (`await||`) and write the code like in series, but grouping the parallel awaits when transpiling:

// NOTE async {} = asynchronous scope (it groups parallel awaits => `await||`)
// NOTE p? = function call that returns a promise (? = just and index)

const x0 = await p0()

const x11 = f11() // sync code in-the-middle

const x1 = await || p1(x0)
const x3 = await || p3(x11)
const x2 = await p2(x1)
const x10 = await p10(x2x3)

const x4 = await || p4(x1x2)
const x5 = await || p5(x2x3)
const x6 = await p6(x4x5x10)

const x7 = await || p7(x4x6)
const x9 = await || p9(x5x6)
const x8 = await p8(x6x7)

await p11(x8x9)

// it would resolve a tree of parallel and series like following with traditional promises

p0
    .then(x0 => {
        const x11 = f11()

        return Promise.all([p1(x0), p3(x11)])
            .then((x1x3=>
                p2(x1)
                    .then(x2 =>
                        p10(x2x3)
                            .then(x10 =>
                                Promise.all([p4(x1x2), p5(x2x3)])
                                    .then((x4x5=>
                                        p6(x4x5x10)
                                            .then(x6 => Promise.all([p7(x4x6), p9(x5x6)])
                                                .then((x7x9=> p8(x6x7)
                                                    .then(x8 => p11(x8x9))
                                                )
                                            )
                                    )
                            )
                    )
            )
    })



On Tue, Nov 26, 2019 at 12:21 PM manuelbarzi <[hidden email]> wrote:
hi Tom

So the parallel awaits would execute before the series awaits?

not exactly. it would group the parallel awaits at the position the first parallel await|| is sentenced.

following the demo i sent:

// NOTE async {} = asynchronous scope (it groups parallel awaits => `await||`)
// NOTE p? = function call that returns a promise (? = just and index)

async {
    const x1 = await || p1()
    const x2 = await p2(x1)
    const x3 = await || p3()
    const x10 = await p10(x2x3)

    let x4x5x6

    async {
        x4 = await || p4(x1x2)
        x5 = await || p5(x2x3)
        x6 = await p6(x4x5x10)
    }

    let x7x8x9

    async {
        x7 = await || p7(x4x6)
        x8 = await p8(x6x7)
        x9 = await || p9(x5x6)
    }

    await p11(x8x9)
}

in the outer block, the first sentence is already parallel, ok, then all parallel awaits in this block will group at first position, siblings of this first one (that is p1 and p3).

in the inner blocks too (just a casualty i created the demos this way, but could be different and still work). in the first inner block, p4 and p5, and in the second inner block, p7 and p9.

"transpiling" it to promises, it would resolve a complex scenario like following (the demo could be improved, of course):

 // it would resolve a tree of parallel and series like following with traditional promises

Promise.all([p1(), p3()])
    .then((x1x3=>
        p2(x1)
            .then(x2 =>
                p10(x2x3)
                    .then(x10 => {
                        let x4x5x6

                        return Promise.all([p4(x1x2), p5(x2x3)])
                            .then(results => [x4x5] = results)
                            .then(() => p6(x4x5x10))
                            .then(_x6 => x6 = _x6)
                            .then(() => {
                                let x7x8x9
        
                                return Promise.all([p7(x4x6), p9(x5x6)])
                                    .then(results => [x7x9] = results)
                                    .then(() => p8(x6x7))
                                    .then(_x8 => x8 = _x8)
                                    .then(() => p11(x8x9))
                            })
                    })
            )
    )

What if there are series awaits before the first parallel await?

no problem, if there was one before, then in it would be chained first in the outer promise, and following it would be the promise.all for the next parallel group.
 
What if the parallel awaits depend on the results of those series awaits? It reads very strangely.

that's totally valid as far as the parallel await is located after the series awaits.

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Re: Proposal: `await.all {...}` for parallelism

Naveen Chawla
In reply to this post by manuelbarzi
Hi Manuel, I think your idea is too much magic, in the sense of not being straightforward to understand. It actually executes subsequent code first, e.g.

async {
   x1 = await || doStuffAsync1();
   x2 = await doStuffAsync2();
   x3 = await || doStuffAsync3();
}

In your idea, `doStuffAsync3()` is executed and completed before doStuffAsync2() even begins! I think this would add bugs due to the confusion, more particularly if outside variables were modified and relied upon.

Why not just maximally preserve current JavaScript for parallel execution, just by omitting `await` in multiple async calls, simply wrapping it in an `await.all` block to ensure completion before code continues past the block. This surely is the more straightforward way to satisfy the same goals?

On Tue, 26 Nov 2019 at 11:22, manuelbarzi <[hidden email]> wrote:
hi Tom

So the parallel awaits would execute before the series awaits?

not exactly. it would group the parallel awaits at the position the first parallel await|| is sentenced.

following the demo i sent:

// NOTE async {} = asynchronous scope (it groups parallel awaits => `await||`)
// NOTE p? = function call that returns a promise (? = just and index)

async {
    const x1 = await || p1()
    const x2 = await p2(x1)
    const x3 = await || p3()
    const x10 = await p10(x2x3)

    let x4x5x6

    async {
        x4 = await || p4(x1x2)
        x5 = await || p5(x2x3)
        x6 = await p6(x4x5x10)
    }

    let x7x8x9

    async {
        x7 = await || p7(x4x6)
        x8 = await p8(x6x7)
        x9 = await || p9(x5x6)
    }

    await p11(x8x9)
}

in the outer block, the first sentence is already parallel, ok, then all parallel awaits in this block will group at first position, siblings of this first one (that is p1 and p3).

in the inner blocks too (just a casualty i created the demos this way, but could be different and still work). in the first inner block, p4 and p5, and in the second inner block, p7 and p9.

"transpiling" it to promises, it would resolve a complex scenario like following (the demo could be improved, of course):

 // it would resolve a tree of parallel and series like following with traditional promises

Promise.all([p1(), p3()])
    .then((x1x3=>
        p2(x1)
            .then(x2 =>
                p10(x2x3)
                    .then(x10 => {
                        let x4x5x6

                        return Promise.all([p4(x1x2), p5(x2x3)])
                            .then(results => [x4x5] = results)
                            .then(() => p6(x4x5x10))
                            .then(_x6 => x6 = _x6)
                            .then(() => {
                                let x7x8x9
        
                                return Promise.all([p7(x4x6), p9(x5x6)])
                                    .then(results => [x7x9] = results)
                                    .then(() => p8(x6x7))
                                    .then(_x8 => x8 = _x8)
                                    .then(() => p11(x8x9))
                            })
                    })
            )
    )

What if there are series awaits before the first parallel await?

no problem, if there was one before, then in it would be chained first in the outer promise, and following it would be the promise.all for the next parallel group.
 
What if the parallel awaits depend on the results of those series awaits? It reads very strangely.

that's totally valid as far as the parallel await is located after the series awaits.
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Re: Proposal: `await.all {...}` for parallelism

manuelbarzi
 
Why not just maximally preserve current JavaScript for parallel execution, just by omitting `await` in multiple async calls, simply wrapping it in an `await.all` block to ensure completion before code continues past the block. This surely is the more straightforward way to satisfy the same goals?

because wrapping it an `await.all` on the one hand explicitly groups promises, but brings complexity on returning values assignment, specially when is about constants (`const`). so, if you avoid blocks, just marking parallel awaits with, for example, a suffix `await||`, or whatever other more convenient way, you can just write the code in series as normally, and avoid that complexity. the transpiler would just require to group the consecutive marked parallel awaits (`await||`) into a Promise.all() and that's it. following i reproduce the demo before:

```
// NOTE p? = function call that returns a promise (? = just and index)
// NOTE s? = function call that runs synchronously and returns a value (? = just and index)

const x0 = await p0()

const x11 = s11() // sync code in-the-middle

const x1 = await || p1(x0)
const x3 = await || p3(x11)
const x2 = await p2(x1)
const x10 = await p10(x2x3)

const x12 = s12() // sync code in-the-middle

const x4 = await || p4(x1x2)
const x5 = await || p5(x2x3x12)
const x6 = await p6(x4x5x10)

const x7 = await || p7(x4x6)
const x9 = await || p9(x5x6)
const x8 = await p8(x6x7)

await p11(x8x9)

// it would resolve a tree of parallel and series like following with traditional promises

p0
    .then(x0 => {
        const x11 = f11()

        return Promise.all([p1(x0), p3(x11)])
            .then((x1x3=>
                p2(x1)
                    .then(x2 =>
                        p10(x2x3)
                            .then(x10 => {
                                const x12 = s12()

                                return Promise.all([p4(x1x2), p5(x2x3x12)])
                                    .then((x4x5=>
                                        p6(x4x5x10)
                                            .then(x6 => Promise.all([p7(x4x6), p9(x5x6)])
                                                .then((x7x9=> p8(x6x7)
                                                    .then(x8 => p11(x8x9))
                                                )
                                            )
                                    )
                            })
                    )
            )
    })
```

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
123