Suggestions to triple quoted strings proposal

classic Classic list List threaded Threaded
57 messages Options
123
Reply | Threaded
Open this post in threaded view
|

Re: Immediate closing of iterators

Chris Hansen-5
> >   Also, a generator doesn't have an iterator::get
> > method since that would complicate the question of who "owns" it.
>
> This is not a problem in the ES4 proposal, or in Python.  Ownership
> of storage and close are coupled only to guarantee that close happens
> eventually, even if the client code fails to call gen.close()
> explicitly.  More below.

By the "owner" of a generator I meant the loop responsible for closing
it, not the storage owner.  But maybe there should be an iterator::get
method on generators -- people just have to be aware that loops close
generators, which might be confusing.

> > That only "solves" Jeff's problem by disallowing it (ta-daa! ;-) but
> > it does away with the need for  any kind of finalization, prompt or
> > not.  In my experience (from java) GC finalization is something you
> > want to steer well clear of.
>
> Finalization is definitely two-phase in systems that have to support
> close (which might resurrect the generator) and then release its
> storage.  Those of us burdened with GC-based memory management for ES/
> JS/AS implementations have to dance with the GC here.

I see, you would still need finalization to guarantee that generators
not created by loops are eventually closed.  But is that a guarantee
you actually need to make -- especially if it complicates the
implementation and potentially opens the browser up for a new type of
DOS attacks?  C# doesn't guarantee this.  In java, even though you're
guaranteed that finalizers will be run, they advise people not to rely
on them.  Instead, it could just be part of the contract on Generator:
if you create it, you have to close it (unless you know that close is
a no-op).  It's not something that people are likely to do often and I
think they will close explicitly anyway rather than rely on
finalization, which adds a source of nondeterminism to a program.
Especially if the browser might actually cancel close ops.

Also, having finalization will mandate a non-conservative GC.


-- Chris

Reply | Threaded
Open this post in threaded view
|

Re: Immediate closing of iterators

Brendan Eich-2
On Dec 21, 2006, at 2:43 PM, Chris Hansen wrote:

>> >   Also, a generator doesn't have an iterator::get
>> > method since that would complicate the question of who "owns" it.
>>
>> This is not a problem in the ES4 proposal, or in Python.  Ownership
>> of storage and close are coupled only to guarantee that close happens
>> eventually, even if the client code fails to call gen.close()
>> explicitly.  More below.
>
> By the "owner" of a generator I meant the loop responsible for closing
> it, not the storage owner.

I see.  That makes sense in the context of a loop or comprehension,  
but generators are used otherwise. The "Motivation" lead-in from PEP  
342 (http://www.python.org/dev/peps/pep-0342/) says:

[PEP 255 generators, which lacked close] do not allow execution to be  
paused within the "try" portion of try/finally blocks, and therefore  
make it difficult for an aborted coroutine to clean up after itself.

Automating close only for generators iterated by for-in loops leaves  
non-loop use cases that nevertheless need to clean up after  
themselves out in the cold.

>   But maybe there should be an iterator::get
> method on generators

I think we should stick to the Pythonic rule that iterators return  
themselves from their iterator::get method (Python calls it __iter__  
but we can avoid __ugly__ names with less::ugly namespacing).

> -- people just have to be aware that loops close
> generators, which might be confusing.

Yes.  A Python hacker moving to ES4 might be outraged by any  
deviation, but we've already done away with GeneratorExit in favor of  
a forced return (which PEP 325 suggested as one of two approaches,  
not the one PEP 342 chose; python-dev interactions have led to both  
PEP 342 authors favoring the forced-return approach in a future  
version of Python).  I've argued that we should avoid gratuitous  
differences with Python, given that we are specifying Python-like  
generators and not threads, call/cc, or general coroutines for ES4.  
But clearly we can afford to diverge, or even try to anticipate, at  
the boundary cases.

OTOH, it sounds like C# (and perhaps IronPython?  How would it do  
otherwise given the single GC ruling the CLR?) may set the "close on  
loop exit" expectations for another segment of hackers.

We can document this and stick to it, my gut says.

>> > That only "solves" Jeff's problem by disallowing it (ta-daa! ;-)  
>> but
>> > it does away with the need for  any kind of finalization, prompt or
>> > not.  In my experience (from java) GC finalization is something you
>> > want to steer well clear of.
>>
>> Finalization is definitely two-phase in systems that have to support
>> close (which might resurrect the generator) and then release its
>> storage.  Those of us burdened with GC-based memory management for  
>> ES/
>> JS/AS implementations have to dance with the GC here.
>
> I see, you would still need finalization to guarantee that generators
> not created by loops are eventually closed.  But is that a guarantee
> you actually need to make -- especially if it complicates the
> implementation and potentially opens the browser up for a new type of
> DOS attacks?

As Gosling remarked over a decade ago, "DOS attacks are a dime a  
dozen" (my paraphrase ;-).

The DOS hazard is like setTimeout, and we've dealt with it for  
Firefox 2.  We don't bother closing any generator-iterator whose  
static scope parent (the window in whose scope the generator was  
constructed) is unmarked after the GC's mark phase.  In this case,  
both the generator and its window object are gone (just not yet  
finalized).  If you reparent a generator from window A to window B,  
by creating it in a script loaded in A, storing it in B, unloading or  
closing window A, and then unloading or closing B without manually  
calling the generator's close, again the system won't call close for  
you.

Any long-lived generator whose window affinity changes will have to  
be manually closed.

But consider the general case, outside of the cramped world of web  
page scripts: XUL apps and extensions, or other SpiderMonkey  
embeddings, will use generators in various ways, and not be subject  
to DOS attacks.  These embeddings expose stateful APIs to trusted  
code, and failing to call close may leak an OS resource or fail to  
synchronize important state.

>   C# doesn't guarantee this.  In java, even though you're
> guaranteed that finalizers will be run, they advise people not to rely
> on them.  Instead, it could just be part of the contract on Generator:
> if you create it, you have to close it (unless you know that close is
> a no-op).  It's not something that people are likely to do often and I
> think they will close explicitly anyway rather than rely on
> finalization, which adds a source of nondeterminism to a program.
> Especially if the browser might actually cancel close ops.

Agreed on the wisdom of untimely cleanup from finalization.

However, the try/finally issue still makes us want to follow Python,  
for general purposes (not just browser purposes).  Lacking a DOS  
threat, and assuming the generator is written correctly, we convinced  
ourselves this summer that finally clauses should run from close  
after the last yield in a try has returned a value, and the caller or  
the GC is done with the generator.  If the caller forgets to close,  
finally still should run (says the ES3 spec in all of the cases it  
defines).

You could argue that breaking finally to avoid DOS attacks means no  
one can count on it in a generator that yields from its try block,  
but I think that overstates the case.

We tried that out on some Pythonistas with an earlier variation:  
outer finally should run even if the generator yields when called  
from close.  The response was "the generator's broken, it should get  
an exception immediately rather than a chance to run outer finally  
blocks."  Python is not totally consistent here, since an explicit  
close that yields will fail, aborting the close, and the GC will  
retry, so the misbehaving generator will get another chance to run.  
In Firefox 2, a bad generator of the form

function badsanta() {
   try {
     yield "rock";
   } finally {
     try {
       yield "barf";
     } finally {
       print("closing");
     }
   }
}
var it = badsanta();
it.next();
it.close();

will throw from the 'yield "barf"', and print "closing" from the  
inner finally.  We say if you're bad, you still get finally  
guarantees. :-)

You're right that "timely release" means "don't count on the GC", and  
we've seen bad embeddings of SpiderMonkey go down the path that Java  
warns against, time after time.  We're not trying to facilitate such  
bad programming.  But we do think finally should work in generators  
even when the user forgets to close, in the absence of DOS threats.

> Also, having finalization will mandate a non-conservative GC.

Why?

/be


Reply | Threaded
Open this post in threaded view
|

Re: Immediate closing of iterators

Lars T Hansen-2
In reply to this post by Chris Hansen-5
Chris Hansen writes:

> Also, having finalization will mandate a non-conservative GC.

I assume you're trying to guarantee that the finalizer is run once the
object becomes garbage.  But non-conservative generational GCs in general
do not make any guarantees about the promptness of collection of any
particular object, thus also do not guarantee anything about the running
of finalizers.  A generational GC that maintains an set of independently
collectable generations and guesses which ones of them are the best to
collect may never collect particular generations if it believes those
generations contain very little garbage.  A very quick scan of the work by
Detlefs et al on "Garbage-first" GC suggests that this collector might
behave like that, for example.

--lars


Reply | Threaded
Open this post in threaded view
|

Re: Immediate closing of iterators

Chris Hansen-5
You're right, non-conservative GC is still possible.

I thought the guarantee was that an unreachable generator will
eventually be closed.  If you're using a conservative GC you may not
be able to discover whether or not an object is dead and hence may
keep dead objects alive indefinitely.  With a generational GC you may
keep objects alive long after they're dead but at least you can, if
you want, determine with certainty whether or not an object is dead.
If you had a policy that caused a full collection to be run of all
generations with some regularity (and the spec could mandate that) a
generational GC could offer (what I thought was) the guarantee.

If, on the hand, the guarantee is that a generator will be closed
before its space is reclaimed then using a conservative GC is still
fine because the guarantee doesn't deal with generators that are not
discovered to be garbage.  But then is that a guarantee that is really
useful to anyone?


-- Chris

On 12/22/06, Lars T Hansen <[hidden email]> wrote:

> Chris Hansen writes:
>
> > Also, having finalization will mandate a non-conservative GC.
>
> I assume you're trying to guarantee that the finalizer is run once the
> object becomes garbage.  But non-conservative generational GCs in general
> do not make any guarantees about the promptness of collection of any
> particular object, thus also do not guarantee anything about the running
> of finalizers.  A generational GC that maintains an set of independently
> collectable generations and guesses which ones of them are the best to
> collect may never collect particular generations if it believes those
> generations contain very little garbage.  A very quick scan of the work by
> Detlefs et al on "Garbage-first" GC suggests that this collector might
> behave like that, for example.
>
> --lars
>
>

Reply | Threaded
Open this post in threaded view
|

Re: Immediate closing of iterators

Christian Plesner Hansen-2
What I meant was of course that _conservative_ GC is still possible,
not non-conservative.  D'oh.

On 12/22/06, Chris Hansen <[hidden email]> wrote:

> You're right, non-conservative GC is still possible.
>
> I thought the guarantee was that an unreachable generator will
> eventually be closed.  If you're using a conservative GC you may not
> be able to discover whether or not an object is dead and hence may
> keep dead objects alive indefinitely.  With a generational GC you may
> keep objects alive long after they're dead but at least you can, if
> you want, determine with certainty whether or not an object is dead.
> If you had a policy that caused a full collection to be run of all
> generations with some regularity (and the spec could mandate that) a
> generational GC could offer (what I thought was) the guarantee.
>
> If, on the hand, the guarantee is that a generator will be closed
> before its space is reclaimed then using a conservative GC is still
> fine because the guarantee doesn't deal with generators that are not
> discovered to be garbage.  But then is that a guarantee that is really
> useful to anyone?
>
>
> -- Chris
>
> On 12/22/06, Lars T Hansen <[hidden email]> wrote:
> > Chris Hansen writes:
> >
> > > Also, having finalization will mandate a non-conservative GC.
> >
> > I assume you're trying to guarantee that the finalizer is run once the
> > object becomes garbage.  But non-conservative generational GCs in general
> > do not make any guarantees about the promptness of collection of any
> > particular object, thus also do not guarantee anything about the running
> > of finalizers.  A generational GC that maintains an set of independently
> > collectable generations and guesses which ones of them are the best to
> > collect may never collect particular generations if it believes those
> > generations contain very little garbage.  A very quick scan of the work by
> > Detlefs et al on "Garbage-first" GC suggests that this collector might
> > behave like that, for example.
> >
> > --lars
> >
> >
>

Reply | Threaded
Open this post in threaded view
|

Re: Immediate closing of iterators

Chris Hansen-5
In reply to this post by Brendan Eich-2
> Automating close only for generators iterated by for-in loops leaves
> non-loop use cases that nevertheless need to clean up after
> themselves out in the cold.

I wouldn't call the absence of automatic closure "leaving them out in
the cold".  The coroutine generators described in PEP 342 are similar
to threads, and the analogous situation for threads would be to
guarantee that a thread is automatically "closed" and has all its
pending finally clauses run, even if it never returns.  If a thread
blocks on some call and never runs again I wouldn't consider it "being
left out in the cold" if the underlying system didn't forcefully close
it eventually and run finally clauses.  I would argue that the same
reasoning applies to generators: if you create them and want to make
sure they're closed then you have to close them yourself.  I don't
think people (except pythonians maybe) would consider that
unreasonable.

> But consider the general case, outside of the cramped world of web
> page scripts: XUL apps and extensions, or other SpiderMonkey
> embeddings, will use generators in various ways, and not be subject
> to DOS attacks.  These embeddings expose stateful APIs to trusted
> code, and failing to call close may leak an OS resource or fail to
> synchronize important state.

Since finalization is tied to the GC (which might be conservative) it
cannot, by definition, be relied on to release resources in something
that resembles a timely fashion, or at all.

> However, the try/finally issue still makes us want to follow Python,
> for general purposes (not just browser purposes).  Lacking a DOS
> threat, and assuming the generator is written correctly, we convinced
> ourselves this summer that finally clauses should run from close
> after the last yield in a try has returned a value, and the caller or
> the GC is done with the generator.  If the caller forgets to close,
> finally still should run (says the ES3 spec in all of the cases it
> defines).

I would claim that as soon as generators start to resemble threads or
coroutines the situation changes.  In java, if a thread blocks
indefinitely in a try clause then then the finally clause will never
be executed (at least not as far as I can tell).

> We say if you're bad, you still get finally
> guarantees. :-)

If you do have finally guarantees I think that's the right thing to
do.  Of course I'm arguing for the polar opposite: even if you're good
you don't get any guarantees :-).

There's another issue with finally guarantees that I don't think has
been mentioned: which thread is it that closes the generators?
Consider this example:

let genCount = 0;
function makeGenerator() {
  genCount++;
  return myGenerator();
}

function myGenerator() {
  try {
    yield 1;
    yield 2;
  } finally {
    genCount--;
  }
}

Imagine that the program has been running for a while and the GC
decides to set in right after makeGenerator has read genCount but
before it has incremented it.  If the GC happens to close some of
these generators then genCount will be in an inconsistent state when
execution continues.  If the GC doesn't then who does?

Sorry about the mail address mixup by the way; that's what you get
when you mix private and work mail...  I did think it was odd, though,
that the posting even got through since my work mail address isn't a
subscriber to es4-discuss.


-- Chris

Reply | Threaded
Open this post in threaded view
|

Re: Immediate closing of iterators

Brendan Eich-2
In reply to this post by Chris Hansen-5
On Dec 22, 2006, at 1:23 AM, Chris Hansen wrote:

> I thought the guarantee was that an unreachable generator will
> eventually be closed.  If you're using a conservative GC you may not
> be able to discover whether or not an object is dead and hence may
> keep dead objects alive indefinitely.

This is a "quality of implementation" issue, about which the spec  
will not dictate.

Conservative GC in runtimes I'm familiar with works pretty well,  
trading reduced risk of humans failing to manage the root set  
correctly against risk of false positive.  Such codebases use  
classifying allocators pervasively, putting images and other non-
pointer data in unscanned allocations.  The hard cases are the thread  
stacks managed by good old C and C++, where the GC has no type  
information.  False positives are possible here, and they can cause  
bloat, but it's bounded by the last-in-first-out discipline.  If your  
event loop has a float on the stack that aliases a pointer into the  
heap, though, ....

> With a generational GC you may
> keep objects alive long after they're dead but at least you can, if
> you want, determine with certainty whether or not an object is dead.
> If you had a policy that caused a full collection to be run of all
> generations with some regularity (and the spec could mandate that) a
> generational GC could offer (what I thought was) the guarantee.

The spec is not going to mandate that; it's not the guarantee we're  
looking for.

> If, on the hand, the guarantee is that a generator will be closed
> before its space is reclaimed then using a conservative GC is still
> fine because the guarantee doesn't deal with generators that are not
> discovered to be garbage.  But then is that a guarantee that is really
> useful to anyone?

Good question.  We thought so, more on formal grounds to-do with try/
finally, than on practical grounds that don't hold up as soon as you  
hypothesize code failing to call close on a generator and expecting  
timely GC to call it.  Anyone else care to weigh in?

/be


Reply | Threaded
Open this post in threaded view
|

Re: Immediate closing of iterators

Brendan Eich-2
In reply to this post by Chris Hansen-5
On Dec 22, 2006, at 9:05 AM, Chris Hansen wrote:

> I would argue that the same
> reasoning applies to generators: if you create them and want to make
> sure they're closed then you have to close them yourself.  I don't
> think people (except pythonians maybe) would consider that
> unreasonable.

This is the 64,000 eollar (or euro, stabler currency :-/) question.

> Since finalization is tied to the GC (which might be conservative) it
> cannot, by definition, be relied on to release resources in something
> that resembles a timely fashion, or at all.

If you can't rely on the GC to recover memory in the absence of  
client code bugs that generate uncollectible graphs, you need a new GC.

> I would claim that as soon as generators start to resemble threads or
> coroutines the situation changes.  In java, if a thread blocks
> indefinitely in a try clause then then the finally clause will never
> be executed (at least not as far as I can tell).

It's a good point, I say again I like it. Need to get feedback from  
others.

> There's another issue with finally guarantees that I don't think has
> been mentioned: which thread is it that closes the generators?
> Consider this example:
>
> let genCount = 0;
> function makeGenerator() {
>  genCount++;
>  return myGenerator();
> }
>
> function myGenerator() {
>  try {
>    yield 1;
>    yield 2;
>  } finally {
>    genCount--;
>  }
> }
>
> Imagine that the program has been running for a while and the GC
> decides to set in right after makeGenerator has read genCount but
> before it has incremented it.  If the GC happens to close some of
> these generators then genCount will be in an inconsistent state when
> execution continues.  If the GC doesn't then who does?

ECMA-262 does not define anything to-do with threads, and ES4 won't  
either.

In the browser, the execution model is run-to-completion.  No event  
handler, not even a timeout, may preempt a running script (even if  
the script is flying a modal dialog; contrary behavior including in  
Firefox is a bug).  The GC will not preempt makeGenerator at  
arbitrary points, either.  It might run synchronously (nesting on the  
current thread stack) at a backward branch or return (after the  
result has been evaluated), or from a native method, getter, or  
setter.  It won't run in the midst of genCount++.

Any multi-threaded browser has to preserve the invariants of this  
model, for backward compatibility and developer sanity.

The browser execution rules are not well-specified, but they should  
be, in a browser embedding spec (not directly in ES4).  Fodder for  
the WHATWG or W3C.

/be

Reply | Threaded
Open this post in threaded view
|

Re: Immediate closing of iterators

Chris Hansen-5
> > Since finalization is tied to the GC (which might be conservative) it
> > cannot, by definition, be relied on to release resources in something
> > that resembles a timely fashion, or at all.
>
> If you can't rely on the GC to recover memory in the absence of
> client code bugs that generate uncollectible graphs, you need a new GC.

If the spec is written such that an implementation is allowed to use a
conservative GC then, by the definition of conservative GC, the spec
must allow for implementations to occasionally not reclaim memory and
hence not close generators.  If the spec mandates generators to be
closed eventually then conservative GCs cannot be used to implement
ES4.

If the spec does require implementations to eventually close
generators then, as Lars points out, there are still non-conservative
GCs that keep several generations and very rarely collect older
generations.  In that case, while you can expect the GC to close
generators, you cannot expect it to happen in a timely fashion.  That
makes it problematic to rely on the GC to release external resources.

Finally, the spec might require generators to be closed in a timely
fashion, for some suitable definition of "timely fashion".  I'm sure
you don't want to do that.

In the absence of finalization none of these problems occur because
then the collection or non-collection of an unreachable object cannot
be observed from the program.

> > There's another issue with finally guarantees that I don't think has
> > been mentioned: which thread is it that closes the generators?
> > Consider this example:
> >
> > let genCount = 0;
> > function makeGenerator() {
> >  genCount++;
> >  return myGenerator();
> > }
> >
> > function myGenerator() {
> >  try {
> >    yield 1;
> >    yield 2;
> >  } finally {
> >    genCount--;
> >  }
> > }
> >
> > Imagine that the program has been running for a while and the GC
> > decides to set in right after makeGenerator has read genCount but
> > before it has incremented it.  If the GC happens to close some of
> > these generators then genCount will be in an inconsistent state when
> > execution continues.  If the GC doesn't then who does?
>
> ECMA-262 does not define anything to-do with threads, and ES4 won't
> either.
>
> In the browser, the execution model is run-to-completion.  No event
> handler, not even a timeout, may preempt a running script (even if
> the script is flying a modal dialog; contrary behavior including in
> Firefox is a bug).

So there's actually a "negative" finalization guarantee:  in a
browser, unreachable and unclosed generators are guaranteed not to be
closed until after the script or event has run to completion.
Long-running scripts should not rely on generators being closed for
them I guess...

> The GC will not preempt makeGenerator at
> arbitrary points, either.  It might run synchronously (nesting on the
> current thread stack) at a backward branch or return (after the
> result has been evaluated), or from a native method, getter, or
> setter.  It won't run in the midst of genCount++.
> Any multi-threaded browser has to preserve the invariants of this
> model, for backward compatibility and developer sanity.
>
> The browser execution rules are not well-specified, but they should
> be, in a browser embedding spec (not directly in ES4).  Fodder for
> the WHATWG or W3C.

I used a counter to keep the example simple but you can easily imagine
a more complex example where the critical region contains a backward
branch, return, or one of the other cases.

As for the spec not specifying threading behavior, I would claim that
this issue _forces_ you to specify it.  If my program relies on
nothing but the semantics defined in the spec then the program should
run correctly on all spec-compliant implementations.  If the spec
leaves this question unanswered then I can't hope to write
implementation- or embedding-independent code.  If the spec doesn't
explicitly disallow close methods to be run preemptively then you
have, in effect, introduced multi threading -- at least, if I want to
write an implementation-independent program I have to consider the
possibility that there will be (spec compliant) implementations that
finalize preemptively.

On the other hand, if you do disallow close methods to be run
preemptively then you're more or less guaranteeing that on any
compliant implementation, close methods will _not_ be run in a timely
fashion, since they will have to wait for the script to finish.  That
may not be a problem in browsers but who knows where people might want
to embed ES4 and run long-running scripts


-- Chris

Reply | Threaded
Open this post in threaded view
|

Re: Immediate closing of iterators

Brendan Eich-2
On Jan 1, 2007, at 8:36 AM, Chris Hansen wrote:

> In the absence of finalization none of these problems occur because
> then the collection or non-collection of an unreachable object cannot
> be observed from the program.

Thanks, this is the compelling argument for C#-style close automation  
(i.e., the ES4 spec automates close calling on exit from for-in and  
for-each-in loops only); all other generator use-cases that want  
close must do it themselves.  As I've said several times, I'm in  
agreement.  Is everyone else?

> As for the spec not specifying threading behavior, I would claim that
> this issue _forces_ you to specify it.  If my program relies on
> nothing but the semantics defined in the spec then the program should
> run correctly on all spec-compliant implementations.  If the spec
> leaves this question unanswered then I can't hope to write
> implementation- or embedding-independent code.

It's true that ECMA-262 alone cannot be used to write portable "JS",  
across (e.g. tellme.com's) VXML server, Macromedia server, Web  
server, and Web browser embeddings, just to name some embeddings with  
different execution models.

>   If the spec doesn't
> explicitly disallow close methods to be run preemptively then you
> have, in effect, introduced multi threading -- at least, if I want to
> write an implementation-independent program I have to consider the
> possibility that there will be (spec compliant) implementations that
> finalize preemptively.

The spec should not automate close calling from the GC, we agree, so  
(I hope we agree that) it can go back to sticking its head in the  
sand and pretending the world is single-threaded.  Its SML-NJ  
reference implementation will not use threads in any way that could  
violate the run-to-completion model.

> On the other hand, if you do disallow close methods to be run
> preemptively then you're more or less guaranteeing that on any
> compliant implementation, close methods will _not_ be run in a timely
> fashion, since they will have to wait for the script to finish.

The important timely-close use-case is the for-in loop. I can't think  
of any others that aren't contrived. But we agree, so it would be  
helpful for others on the list who see things differently (including  
you Pythonistas) to speak up.

/be

Reply | Threaded
Open this post in threaded view
|

Re: Immediate closing of iterators

Chris Hansen-5
> > In the absence of finalization none of these problems occur because
> > then the collection or non-collection of an unreachable object cannot
> > be observed from the program.
>
> Thanks, this is the compelling argument for C#-style close automation
> (i.e., the ES4 spec automates close calling on exit from for-in and
> for-each-in loops only); all other generator use-cases that want
> close must do it themselves.  As I've said several times, I'm in
> agreement.  Is everyone else?

We agree about C#-style close and that's not what I'm arguing for.
I'm arguing against having automatically closing of generators that
are not used in for-in loops, for instance ones used as coroutines.
It's not my impression that we agree there.  But in any case I'll give
it a rest and hopefully someone else will have an opinion on this.

> The important timely-close use-case is the for-in loop. I can't think
> of any others that aren't contrived.

You could easily imagine using generators to access files, networks
sockets or other external resources, and I don't think it is contrived
to imagine such a generator used outside a for-in loop.  In that case
you don't need prompt finalization and that's not what I mean when I
say "timely fashion", but you would want it to happen eventually and
you probably don't want too much time to pass before it happens.


-- Chris

Reply | Threaded
Open this post in threaded view
|

Re: Immediate closing of iterators

Brendan Eich-2
On Jan 2, 2007, at 3:23 PM, Chris Hansen wrote:

>> > In the absence of finalization none of these problems occur because
>> > then the collection or non-collection of an unreachable object  
>> cannot
>> > be observed from the program.
>>
>> Thanks, this is the compelling argument for C#-style close automation
>> (i.e., the ES4 spec automates close calling on exit from for-in and
>> for-each-in loops only); all other generator use-cases that want
>> close must do it themselves.  As I've said several times, I'm in
>> agreement.  Is everyone else?
>
> We agree about C#-style close and that's not what I'm arguing for.
> I'm arguing against having automatically closing of generators that
> are not used in for-in loops, for instance ones used as coroutines.
> It's not my impression that we agree there.

No, we agree.  That's what I keep saying ("I'm sold", etc.), and the  
above words are intentionally exhaustive: "all other generator use-
cases [than for-in loops] that want close must do it themselves."

Sorry if I was unclear.  You and I have always agreed that timely  
release of scarce resources should not depend on GC, but for JS1.7  
(in Firefox 2), we followed Python 2.5 closely for the sake of  
guaranteeing finally execution when the last yield is from the  
matching try.  This seems to cater to bad practices; it creates a  
hazard for users, especially those coming from the Python world.

Still troubling, but less so, is the problem for Python people who  
expect close to be automated in non-for-in-loop cases.

>> The important timely-close use-case is the for-in loop. I can't think
>> of any others that aren't contrived.
>
> You could easily imagine using generators to access files, networks
> sockets or other external resources, and I don't think it is contrived
> to imagine such a generator used outside a for-in loop.  In that case
> you don't need prompt finalization and that's not what I mean when I
> say "timely fashion", but you would want it to happen eventually and
> you probably don't want too much time to pass before it happens.

This is a use-case mentioned in the PEPs, but as you've argued it  
limits GC implementation choices if ES4 were to require it.  Since at  
least one ECMA member's ECMA_262 + ES4-like extensions implementation  
(ActionScript 3 in the Flash Player) uses conservative GC, I do not  
believe that TG1 will or should specify any further close automation  
than for for-in loops.  That's my opinion, at any rate.

Expect more responses, I've reminded interested people about this list.

/be


Reply | Threaded
Open this post in threaded view
|

Re: Immediate closing of iterators

Igor Bukanov
On 1/3/07, Brendan Eich <[hidden email]> wrote:
> On Jan 2, 2007, at 3:23 PM, Chris Hansen wrote:
> for JS1.7
> (in Firefox 2), we followed Python 2.5 closely for the sake of
> guaranteeing finally execution when the last yield is from the
> matching try.

A simple way to address it is to require at the runtime that when
yield is executed inside try with finally, the iterator object must be
a part of for-in loop or, potentially, C#-like  autoclose(obj) {}
block. If this is not the case, an exception should be thrown.

Reply | Threaded
Open this post in threaded view
|

Re: Immediate closing of iterators

Peter Hall-2
But that means a programmer wouldn't have the option to close it
manually in that circumstance..

Peter


On 1/4/07, Igor Bukanov <[hidden email]> wrote:

> On 1/3/07, Brendan Eich <[hidden email]> wrote:
> > On Jan 2, 2007, at 3:23 PM, Chris Hansen wrote:
> > for JS1.7
> > (in Firefox 2), we followed Python 2.5 closely for the sake of
> > guaranteeing finally execution when the last yield is from the
> > matching try.
>
> A simple way to address it is to require at the runtime that when
> yield is executed inside try with finally, the iterator object must be
> a part of for-in loop or, potentially, C#-like  autoclose(obj) {}
> block. If this is not the case, an exception should be thrown.
> _______________________________________________
> Es4-discuss mailing list
> [hidden email]
> https://mail.mozilla.org/listinfo/es4-discuss
>

Reply | Threaded
Open this post in threaded view
|

Re: Immediate closing of iterators

P T Withington
In reply to this post by Brendan Eich-2
On 2007-01-02, at 15:30 EST, Brendan Eich wrote:

> On Jan 1, 2007, at 8:36 AM, Chris Hansen wrote:
>
>> In the absence of finalization none of these problems occur because
>> then the collection or non-collection of an unreachable object cannot
>> be observed from the program.
>
> Thanks, this is the compelling argument for C#-style close  
> automation (i.e., the ES4 spec automates close calling on exit from  
> for-in and for-each-in loops only); all other generator use-cases  
> that want close must do it themselves.  As I've said several times,  
> I'm in agreement.  Is everyone else?

Agreed.

My 2p: finalization is the `goto` of GC-ed languages

Reply | Threaded
Open this post in threaded view
|

Re: Immediate closing of iterators

Igor Bukanov
In reply to this post by Peter Hall-2
On 1/4/07, Peter Hall <[hidden email]> wrote:
> But that means a programmer wouldn't have the option to close it
> manually in that circumstance..

You mean that it would not be possible to use outside for-in loop a
generator with yield inside try with finally? Surely calling
generatorInstance.next() will throw an exception in that case but I do
not see what would prevent calling  generatorInstance.close().

For me the guarantee that finally is always executed at the clear
defined moment is worth the restriction. And if some generator want to
allow its usage outside for-in, then it must not use yield with
finally and rely on the explicit close.

Regards, Igor

Reply | Threaded
Open this post in threaded view
|

Re: Immediate closing of iterators

Peter Hall-2
On 1/4/07, Igor Bukanov <[hidden email]> wrote:
> On 1/4/07, Peter Hall <[hidden email]> wrote:
> > But that means a programmer wouldn't have the option to close it
> > manually in that circumstance..
>
> You mean that it would not be possible to use outside for-in loop a
> generator with yield inside try with finally? Surely calling
> generatorInstance.next() will throw an exception in that case but I do
> not see what would prevent calling  generatorInstance.close().
>

Nothing would prevent you calling it. I was rather meaning that, in
cases where the generator cannot be automatically closed, a programmer
should have the option to close it manually, rather than the language
disallowing it and throwing an error.

But this is looking like a non-argument, since a concensus seems to
have been reached already.

Peter

123