Removal of WeakMap/WeakSet clear

classic Classic list List threaded Threaded
61 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

Removal of WeakMap/WeakSet clear

Katelyn Gadd
Is there a detailed rationale for this somewhere? Making typical
applications pay the cost here for a specific security scenario seems
really bizarre to me. Clearing standard library data structures is an
incredibly common operation. If you want to ensure that someone can't
clear the map/set, shouldn't you be handing them an encapsulated
version of the data structure? This seems like a corner case that
shouldn't justify removing an important primitive.

If you have a clear method, the security problem seems solved by
wrapping it in an object or using a proxy to deny the ability to clear
(you hide the actual map/set, so it can't be cleared - you expose only
the operations you want to expose).

If you don't have a clear method, anyone wanting to clear the data
structure has to throw it away and allocate a new one. This has
significant disadvantages:
The new structure starts empty at a default size, so repopulating it
will have to grow the buffer multiple times - this is undesirable for
cases where you are reusing a single data structure to store state for
a long-running application.
The allocation adds to GC and memory pressure for a long-running
application that needs to clear data structures frequently. Were it a
lightweight data type this would matter less, but a typical map
instance with data in it can occupy a considerable amount of space in
the heap.
Being able to clear the structure now requires that all consumers have
support for replacing their reference(s) to the old map with the new
one. This makes it harder to maintain encapsulation because you may
have saved a reference to the map in a private property or within a
closure. Now you need to add accessibility points to everything that
might retain the map so that you can update the reference. Or, you
have to encapsulate maps and sets just to recreate the clear operation
that should have been there to begin with.

In either case, encapsulation or shielding the container behind a
proxy is necessary. I insist that the common case is the one that
shouldn't have to encapsulate, because optimizing for that case will
benefit the vast majority of web applications that use it and the
penalty to security-sensitive cases is small.

-kg
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Removal of WeakMap/WeakSet clear

Claude Pache

The root of the issue, is that WeakMaps are thought as a tool for two unrelated use cases. Quoting [1]:

> (...) WeakMaps were motivate[d] by two distinct use cases.  Soft fields and object-keyed caches.

Now, in short, for the "soft fields" use case, a `.clear()` method is unwanted, but for the "object-keyed caches" use case, a `.clear()` method is welcome.

—Claude

[1]: https://esdiscuss.org/topic/weakmap-clear-and-inverted-implementations


> Le 26 nov. 2014 à 18:33, Katelyn Gadd <[hidden email]> a écrit :
>
> Is there a detailed rationale for this somewhere? Making typical
> applications pay the cost here for a specific security scenario seems
> really bizarre to me. Clearing standard library data structures is an
> incredibly common operation. If you want to ensure that someone can't
> clear the map/set, shouldn't you be handing them an encapsulated
> version of the data structure? This seems like a corner case that
> shouldn't justify removing an important primitive.
>
> If you have a clear method, the security problem seems solved by
> wrapping it in an object or using a proxy to deny the ability to clear
> (you hide the actual map/set, so it can't be cleared - you expose only
> the operations you want to expose).
>
> If you don't have a clear method, anyone wanting to clear the data
> structure has to throw it away and allocate a new one. This has
> significant disadvantages:
> The new structure starts empty at a default size, so repopulating it
> will have to grow the buffer multiple times - this is undesirable for
> cases where you are reusing a single data structure to store state for
> a long-running application.
> The allocation adds to GC and memory pressure for a long-running
> application that needs to clear data structures frequently. Were it a
> lightweight data type this would matter less, but a typical map
> instance with data in it can occupy a considerable amount of space in
> the heap.
> Being able to clear the structure now requires that all consumers have
> support for replacing their reference(s) to the old map with the new
> one. This makes it harder to maintain encapsulation because you may
> have saved a reference to the map in a private property or within a
> closure. Now you need to add accessibility points to everything that
> might retain the map so that you can update the reference. Or, you
> have to encapsulate maps and sets just to recreate the clear operation
> that should have been there to begin with.
>
> In either case, encapsulation or shielding the container behind a
> proxy is necessary. I insist that the common case is the one that
> shouldn't have to encapsulate, because optimizing for that case will
> benefit the vast majority of web applications that use it and the
> penalty to security-sensitive cases is small.
>
> -kg
> _______________________________________________
> es-discuss mailing list
> [hidden email]
> https://mail.mozilla.org/listinfo/es-discuss

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Removal of WeakMap/WeakSet clear

Allen Wirfs-Brock

On Nov 26, 2014, at 10:22 AM, Claude Pache wrote:


The root of the issue, is that WeakMaps are thought as a tool for two unrelated use cases. Quoting [1]:

(...) WeakMaps were motivate[d] by two distinct use cases.  Soft fields and object-keyed caches.

Now, in short, for the "soft fields" use case, a `.clear()` method is unwanted, but for the "object-keyed caches" use case, a `.clear()` method is welcome.

—Claude

[1]: https://esdiscuss.org/topic/weakmap-clear-and-inverted-implementations

Let's ignore security for  the moment.

If  WeakMaps/WeakSets are not inspectable (via iteration) and do not have a clear operation, then the inverted implementation technique can be use used.  This technique eliminates significant GS complexity.

The ability to iterate over the elements of a WeakMap/Set would make implementation specific (and probably non-deterministic) GC timing observable to ES programs.  Non-deterministic and observable implementation specific behavior is bad for interoperability.  That reason alone is enough for TC39 to choose to not make WeakMaps/Sets non-iterable as interoperability is one of our highest priorities. 

That said, there are use cases for iterable weak collections if you are willing to tolerate the non-determinism and observability of the GC.  Implementations supporting situation where those use cases are important might choose to provide a non-standard IterableWeakMap/Set implementation and there would be no particular reason for not including a 'clear' method.   If there of evidence of enough utility for such collections TC39 might even decide to standardize them.  In which case we would probably but them into a standard module that needs to be explicitly imported.  Use of the module would be a flag that the program may have non-deterministic behavior.

Allen




Le 26 nov. 2014 à 18:33, Katelyn Gadd <[hidden email]> a écrit :

Is there a detailed rationale for this somewhere? Making typical
applications pay the cost here for a specific security scenario seems
really bizarre to me. Clearing standard library data structures is an
incredibly common operation. If you want to ensure that someone can't
clear the map/set, shouldn't you be handing them an encapsulated
version of the data structure? This seems like a corner case that
shouldn't justify removing an important primitive.

If you have a clear method, the security problem seems solved by
wrapping it in an object or using a proxy to deny the ability to clear
(you hide the actual map/set, so it can't be cleared - you expose only
the operations you want to expose).

If you don't have a clear method, anyone wanting to clear the data
structure has to throw it away and allocate a new one. This has
significant disadvantages:
The new structure starts empty at a default size, so repopulating it
will have to grow the buffer multiple times - this is undesirable for
cases where you are reusing a single data structure to store state for
a long-running application.
The allocation adds to GC and memory pressure for a long-running
application that needs to clear data structures frequently. Were it a
lightweight data type this would matter less, but a typical map
instance with data in it can occupy a considerable amount of space in
the heap.
Being able to clear the structure now requires that all consumers have
support for replacing their reference(s) to the old map with the new
one. This makes it harder to maintain encapsulation because you may
have saved a reference to the map in a private property or within a
closure. Now you need to add accessibility points to everything that
might retain the map so that you can update the reference. Or, you
have to encapsulate maps and sets just to recreate the clear operation
that should have been there to begin with.

In either case, encapsulation or shielding the container behind a
proxy is necessary. I insist that the common case is the one that
shouldn't have to encapsulate, because optimizing for that case will
benefit the vast majority of web applications that use it and the
penalty to security-sensitive cases is small.

-kg
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss



_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

RE: Removal of WeakMap/WeakSet clear

Andrea Giammarchi-2
In reply to this post by Katelyn Gadd
Agreed, and i believe the only real reason would be the inability to polyfill it properly, in order to discourage its usage.

If this is why, it doesn't justify its drop, imo

From: [hidden email]
Sent: ‎26/‎11/‎2014 17:33
To: [hidden email]
Subject: Removal of WeakMap/WeakSet clear

Is there a detailed rationale for this somewhere? Making typical
applications pay the cost here for a specific security scenario seems
really bizarre to me. Clearing standard library data structures is an
incredibly common operation. If you want to ensure that someone can't
clear the map/set, shouldn't you be handing them an encapsulated
version of the data structure? This seems like a corner case that
shouldn't justify removing an important primitive.

If you have a clear method, the security problem seems solved by
wrapping it in an object or using a proxy to deny the ability to clear
(you hide the actual map/set, so it can't be cleared - you expose only
the operations you want to expose).

If you don't have a clear method, anyone wanting to clear the data
structure has to throw it away and allocate a new one. This has
significant disadvantages:
The new structure starts empty at a default size, so repopulating it
will have to grow the buffer multiple times - this is undesirable for
cases where you are reusing a single data structure to store state for
a long-running application.
The allocation adds to GC and memory pressure for a long-running
application that needs to clear data structures frequently. Were it a
lightweight data type this would matter less, but a typical map
instance with data in it can occupy a considerable amount of space in
the heap.
Being able to clear the structure now requires that all consumers have
support for replacing their reference(s) to the old map with the new
one. This makes it harder to maintain encapsulation because you may
have saved a reference to the map in a private property or within a
closure. Now you need to add accessibility points to everything that
might retain the map so that you can update the reference. Or, you
have to encapsulate maps and sets just to recreate the clear operation
that should have been there to begin with.

In either case, encapsulation or shielding the container behind a
proxy is necessary. I insist that the common case is the one that
shouldn't have to encapsulate, because optimizing for that case will
benefit the vast majority of web applications that use it and the
penalty to security-sensitive cases is small.

-kg
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Removal of WeakMap/WeakSet clear

Mark S. Miller-2
In reply to this post by Katelyn Gadd
On Wed, Nov 26, 2014 at 9:33 AM, Katelyn Gadd <[hidden email]> wrote:
> Is there a detailed rationale for this somewhere?

It is a combination of three issues.

1. The security issue.
2. The implementation issue.
3. The ES process issue.

The implementation issue is that the use cases for WeakMaps basically
divide into the following cases:

a. Those for which we're confident that the map's lifetime outlives
its typical key.
b. Those for which we're confident that the key's lifetime outlives
the typical map it is a key in.
c. Those for which we're not confident which typically lives longer.

For #a, the transposed representation of WeakMaps is strictly better.
The non-transposed implementation would promote ephemeral keys to
later GC generations, which is very expensive. (I believe the expense
of this promotion has been radically underestimated in most prior
discussions.) This is the GC pressure that really matters.

For #b, just use a Map rather than a WeakMap.

For #c, transposed rep or not is a tie. In the interests of minimizing
implementation mechanism, we should just use a transposed rep for this
as well.

Given the transposed representation, the only implementation
techniques for clear are

x. Enumerating all memory
y. Having the WeakMap encapsulate the token used to lookup the value
in the key's hidden map, and have .clear replace this token.

#x is not reasonable.
#y is equivalent to the replacing of the Map that would be the
user-level technique for emulating clear in any case. This is exactly
what the use of WeakMaps for membranes do, when the membrane should be
revoked. If .clear could be implemented for the transposed
representation more efficiently than this, membranes would benefit
from .clear as well. I have never expected they could.


The process issue is that .clear was not in the original proposal (for
all of these reasons), and it never achieved consensus in committee.
It would have violated our process to keep it in the spec. The process
issue is not "Should it be dropped?" since it was never legitimately
added. The only issue is "Should it be added?".



> Making typical
> applications pay the cost here for a specific security scenario seems
> really bizarre to me. Clearing standard library data structures is an
> incredibly common operation. If you want to ensure that someone can't
> clear the map/set, shouldn't you be handing them an encapsulated
> version of the data structure? This seems like a corner case that
> shouldn't justify removing an important primitive.
>
> If you have a clear method, the security problem seems solved by
> wrapping it in an object or using a proxy to deny the ability to clear
> (you hide the actual map/set, so it can't be cleared - you expose only
> the operations you want to expose).
>
> If you don't have a clear method, anyone wanting to clear the data
> structure has to throw it away and allocate a new one. This has
> significant disadvantages:
> The new structure starts empty at a default size, so repopulating it
> will have to grow the buffer multiple times - this is undesirable for
> cases where you are reusing a single data structure to store state for
> a long-running application.
> The allocation adds to GC and memory pressure for a long-running
> application that needs to clear data structures frequently. Were it a
> lightweight data type this would matter less, but a typical map
> instance with data in it can occupy a considerable amount of space in
> the heap.
> Being able to clear the structure now requires that all consumers have
> support for replacing their reference(s) to the old map with the new
> one. This makes it harder to maintain encapsulation because you may
> have saved a reference to the map in a private property or within a
> closure. Now you need to add accessibility points to everything that
> might retain the map so that you can update the reference. Or, you
> have to encapsulate maps and sets just to recreate the clear operation
> that should have been there to begin with.
>
> In either case, encapsulation or shielding the container behind a
> proxy is necessary. I insist that the common case is the one that
> shouldn't have to encapsulate, because optimizing for that case will
> benefit the vast majority of web applications that use it and the
> penalty to security-sensitive cases is small.
>
> -kg
> _______________________________________________
> es-discuss mailing list
> [hidden email]
> https://mail.mozilla.org/listinfo/es-discuss



--
    Cheers,
    --MarkM
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Removal of WeakMap/WeakSet clear

Mark S. Miller-2
In reply to this post by Allen Wirfs-Brock
On Wed, Nov 26, 2014 at 11:09 AM, Allen Wirfs-Brock
<[hidden email]> wrote:

>
> On Nov 26, 2014, at 10:22 AM, Claude Pache wrote:
>
>
> The root of the issue, is that WeakMaps are thought as a tool for two
> unrelated use cases. Quoting [1]:
>
> (...) WeakMaps were motivate[d] by two distinct use cases.  Soft fields and
> object-keyed caches.
>
>
> Now, in short, for the "soft fields" use case, a `.clear()` method is
> unwanted, but for the "object-keyed caches" use case, a `.clear()` method is
> welcome.
>
> —Claude
>
> [1]: https://esdiscuss.org/topic/weakmap-clear-and-inverted-implementations
>
>
> Let's ignore security for  the moment.
>
> If  WeakMaps/WeakSets are not inspectable (via iteration) and do not have a
> clear operation, then the inverted implementation technique can be use used.
> This technique eliminates significant GS complexity.
>
> The ability to iterate over the elements of a WeakMap/Set would make
> implementation specific (and probably non-deterministic) GC timing
> observable to ES programs.  Non-deterministic and observable implementation
> specific behavior is bad for interoperability.

Hi Allen, as much as I'd like to agree with this, I don't see the
observability-of-GC issue. Could you explain more? Thanks.


>  That reason alone is enough
> for TC39 to choose to not make WeakMaps/Sets non-iterable as
> interoperability is one of our highest priorities.
>
> That said, there are use cases for iterable weak collections if you are
> willing to tolerate the non-determinism and observability of the GC.
> Implementations supporting situation where those use cases are important
> might choose to provide a non-standard IterableWeakMap/Set implementation
> and there would be no particular reason for not including a 'clear' method.
> If there of evidence of enough utility for such collections TC39 might even
> decide to standardize them.  In which case we would probably but them into a
> standard module that needs to be explicitly imported.  Use of the module
> would be a flag that the program may have non-deterministic behavior.
>
> Allen
>
>
>
>
> Le 26 nov. 2014 à 18:33, Katelyn Gadd <[hidden email]> a écrit :
>
>
> Is there a detailed rationale for this somewhere? Making typical
>
> applications pay the cost here for a specific security scenario seems
>
> really bizarre to me. Clearing standard library data structures is an
>
> incredibly common operation. If you want to ensure that someone can't
>
> clear the map/set, shouldn't you be handing them an encapsulated
>
> version of the data structure? This seems like a corner case that
>
> shouldn't justify removing an important primitive.
>
>
> If you have a clear method, the security problem seems solved by
>
> wrapping it in an object or using a proxy to deny the ability to clear
>
> (you hide the actual map/set, so it can't be cleared - you expose only
>
> the operations you want to expose).
>
>
> If you don't have a clear method, anyone wanting to clear the data
>
> structure has to throw it away and allocate a new one. This has
>
> significant disadvantages:
>
> The new structure starts empty at a default size, so repopulating it
>
> will have to grow the buffer multiple times - this is undesirable for
>
> cases where you are reusing a single data structure to store state for
>
> a long-running application.
>
> The allocation adds to GC and memory pressure for a long-running
>
> application that needs to clear data structures frequently. Were it a
>
> lightweight data type this would matter less, but a typical map
>
> instance with data in it can occupy a considerable amount of space in
>
> the heap.
>
> Being able to clear the structure now requires that all consumers have
>
> support for replacing their reference(s) to the old map with the new
>
> one. This makes it harder to maintain encapsulation because you may
>
> have saved a reference to the map in a private property or within a
>
> closure. Now you need to add accessibility points to everything that
>
> might retain the map so that you can update the reference. Or, you
>
> have to encapsulate maps and sets just to recreate the clear operation
>
> that should have been there to begin with.
>
>
> In either case, encapsulation or shielding the container behind a
>
> proxy is necessary. I insist that the common case is the one that
>
> shouldn't have to encapsulate, because optimizing for that case will
>
> benefit the vast majority of web applications that use it and the
>
> penalty to security-sensitive cases is small.
>
>
> -kg
>
> _______________________________________________
>
> es-discuss mailing list
>
> [hidden email]
>
> https://mail.mozilla.org/listinfo/es-discuss
>
>
> _______________________________________________
> es-discuss mailing list
> [hidden email]
> https://mail.mozilla.org/listinfo/es-discuss
>
>
>
> _______________________________________________
> es-discuss mailing list
> [hidden email]
> https://mail.mozilla.org/listinfo/es-discuss
>



--
    Cheers,
    --MarkM
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Removal of WeakMap/WeakSet clear

Tab Atkins Jr.
On Wed, Nov 26, 2014 at 11:23 AM, Mark S. Miller <[hidden email]> wrote:

> On Wed, Nov 26, 2014 at 11:09 AM, Allen Wirfs-Brock
> <[hidden email]> wrote:
>>
>> On Nov 26, 2014, at 10:22 AM, Claude Pache wrote:
>>
>>
>> The root of the issue, is that WeakMaps are thought as a tool for two
>> unrelated use cases. Quoting [1]:
>>
>> (...) WeakMaps were motivate[d] by two distinct use cases.  Soft fields and
>> object-keyed caches.
>>
>>
>> Now, in short, for the "soft fields" use case, a `.clear()` method is
>> unwanted, but for the "object-keyed caches" use case, a `.clear()` method is
>> welcome.
>>
>> —Claude
>>
>> [1]: https://esdiscuss.org/topic/weakmap-clear-and-inverted-implementations
>>
>>
>> Let's ignore security for  the moment.
>>
>> If  WeakMaps/WeakSets are not inspectable (via iteration) and do not have a
>> clear operation, then the inverted implementation technique can be use used.
>> This technique eliminates significant GS complexity.
>>
>> The ability to iterate over the elements of a WeakMap/Set would make
>> implementation specific (and probably non-deterministic) GC timing
>> observable to ES programs.  Non-deterministic and observable implementation
>> specific behavior is bad for interoperability.
>
> Hi Allen, as much as I'd like to agree with this, I don't see the
> observability-of-GC issue. Could you explain more? Thanks.

Allen appears to be mixing in an iterability argument.  This isn't
germane to the .clear() discussion, unless you think of .clear() as
being implemented by iterating and removing keys.

~TJ
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Removal of WeakMap/WeakSet clear

Allen Wirfs-Brock

On Nov 26, 2014, at 11:25 AM, Tab Atkins Jr. wrote:

> On Wed, Nov 26, 2014 at 11:23 AM, Mark S. Miller <[hidden email]> wrote:
>> On Wed, Nov 26, 2014 at 11:09 AM, Allen Wirfs-Brock
>> <[hidden email]> wrote:
>>>
>>> On Nov 26, 2014, at 10:22 AM, Claude Pache wrote:
>>>
>>>
>>> The root of the issue, is that WeakMaps are thought as a tool for two
>>> unrelated use cases. Quoting [1]:
>>>
>>> (...) WeakMaps were motivate[d] by two distinct use cases.  Soft fields and
>>> object-keyed caches.
>>>
>>>
>>> Now, in short, for the "soft fields" use case, a `.clear()` method is
>>> unwanted, but for the "object-keyed caches" use case, a `.clear()` method is
>>> welcome.
>>>
>>> —Claude
>>>
>>> [1]: https://esdiscuss.org/topic/weakmap-clear-and-inverted-implementations
>>>
>>>
>>> Let's ignore security for  the moment.
>>>
>>> If  WeakMaps/WeakSets are not inspectable (via iteration) and do not have a
>>> clear operation, then the inverted implementation technique can be use used.
>>> This technique eliminates significant GS complexity.
>>>
>>> The ability to iterate over the elements of a WeakMap/Set would make
>>> implementation specific (and probably non-deterministic) GC timing
>>> observable to ES programs.  Non-deterministic and observable implementation
>>> specific behavior is bad for interoperability.
>>
>> Hi Allen, as much as I'd like to agree with this, I don't see the
>> observability-of-GC issue. Could you explain more? Thanks.
>
> Allen appears to be mixing in an iterability argument.  This isn't
> germane to the .clear() discussion, unless you think of .clear() as
> being implemented by iterating and removing keys.

Right, it's the lack of iterability and clear that makes the  inverted implementation feasible.  We justify lack of iterability because it exposes GC.

Allen
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Removal of WeakMap/WeakSet clear

Katelyn Gadd
The detailed explanations are helpful for understanding this, at least
when it comes to WeakMap. Thanks. I was not aware that clear() was a
tenative addition - MDN
(https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/WeakMap)
lists it as available in all the major browsers. Understanding it as
'should this new thing be added' makes the decision less controversial
for me even if I personally want the primitive to be there.

However: Does this mean that if I implement clear myself by creating
new WeakMaps, i'm inadvertently doing horrible things to the GC? I get
the impression that this would mean every time I throw the old map
away, the data in the keys' hidden maps persists for some unknown
amount of time, and I'm adding to that hidden map. If I destroy and
repopulate these maps on a regular basis that seems like it could get
out of hand quickly. If this usage pattern is fundamentally bad for JS
VMs, it seems like the sort of guidance that probably needs to be
clearly messaged to users - 'clear() isn't available because you
really, really shouldn't do that' vs 'clear() isn't available because
you can write it yourself'. Just my 2c there - very interesting to
know about the reasoning behind this.

On 26 November 2014 at 13:28, Allen Wirfs-Brock <[hidden email]> wrote:

>
> On Nov 26, 2014, at 11:25 AM, Tab Atkins Jr. wrote:
>
>> On Wed, Nov 26, 2014 at 11:23 AM, Mark S. Miller <[hidden email]> wrote:
>>> On Wed, Nov 26, 2014 at 11:09 AM, Allen Wirfs-Brock
>>> <[hidden email]> wrote:
>>>>
>>>> On Nov 26, 2014, at 10:22 AM, Claude Pache wrote:
>>>>
>>>>
>>>> The root of the issue, is that WeakMaps are thought as a tool for two
>>>> unrelated use cases. Quoting [1]:
>>>>
>>>> (...) WeakMaps were motivate[d] by two distinct use cases.  Soft fields and
>>>> object-keyed caches.
>>>>
>>>>
>>>> Now, in short, for the "soft fields" use case, a `.clear()` method is
>>>> unwanted, but for the "object-keyed caches" use case, a `.clear()` method is
>>>> welcome.
>>>>
>>>> —Claude
>>>>
>>>> [1]: https://esdiscuss.org/topic/weakmap-clear-and-inverted-implementations
>>>>
>>>>
>>>> Let's ignore security for  the moment.
>>>>
>>>> If  WeakMaps/WeakSets are not inspectable (via iteration) and do not have a
>>>> clear operation, then the inverted implementation technique can be use used.
>>>> This technique eliminates significant GS complexity.
>>>>
>>>> The ability to iterate over the elements of a WeakMap/Set would make
>>>> implementation specific (and probably non-deterministic) GC timing
>>>> observable to ES programs.  Non-deterministic and observable implementation
>>>> specific behavior is bad for interoperability.
>>>
>>> Hi Allen, as much as I'd like to agree with this, I don't see the
>>> observability-of-GC issue. Could you explain more? Thanks.
>>
>> Allen appears to be mixing in an iterability argument.  This isn't
>> germane to the .clear() discussion, unless you think of .clear() as
>> being implemented by iterating and removing keys.
>
> Right, it's the lack of iterability and clear that makes the  inverted implementation feasible.  We justify lack of iterability because it exposes GC.
>
> Allen
> _______________________________________________
> es-discuss mailing list
> [hidden email]
> https://mail.mozilla.org/listinfo/es-discuss
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Removal of WeakMap/WeakSet clear

Brendan Eich-2
Katelyn Gadd wrote:
> However: Does this mean that if I implement clear myself by creating
> new WeakMaps, i'm inadvertently doing horrible things to the GC?

Assume yes (only defensive assumption). What goes wrong?

Sorry if I missed it: what is your .clear use case, in example code?

/be
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Removal of WeakMap/WeakSet clear

Mark S. Miller-2
In reply to this post by Katelyn Gadd
On Wed, Nov 26, 2014 at 2:49 PM, Katelyn Gadd <[hidden email]> wrote:

> The detailed explanations are helpful for understanding this, at least
> when it comes to WeakMap. Thanks. I was not aware that clear() was a
> tenative addition - MDN
> (https://developer.mozilla.org/en-US/docs/Web/JavaScript/Reference/Global_Objects/WeakMap)
> lists it as available in all the major browsers. Understanding it as
> 'should this new thing be added' makes the decision less controversial
> for me even if I personally want the primitive to be there.
>
> However: Does this mean that if I implement clear myself by creating
> new WeakMaps, i'm inadvertently doing horrible things to the GC?

Well, I'll try to explain how expensive I expect it to be, in
qualitative terms, and leave it to you to judge whether it is
horrible. As I mentioned, the original motivating use case for
WeakMaps -- membranes -- throws WeakMaps away in precisely this manner
when the membrane is revoked. So for that use case at least, I do not
consider it too horrible. Nor do I know of any better alternative.


> I get
> the impression that this would mean every time I throw the old map
> away, the data in the keys' hidden maps persists for some unknown
> amount of time,

Yes.

> and I'm adding to that hidden map.

Yes.

> If I destroy and
> repopulate these maps on a regular basis that seems like it could get
> out of hand quickly.

If you're doing this at greater frequency than the expected lifetime
of the keys, then I need to ask: Why are you using a WeakMap instead
of a Map? (An example would be great)


> If this usage pattern is fundamentally bad for JS
> VMs, it seems like the sort of guidance that probably needs to be
> clearly messaged to users - 'clear() isn't available because you
> really, really shouldn't do that' vs 'clear() isn't available because
> you can write it yourself'. Just my 2c there - very interesting to
> know about the reasoning behind this.

Sure. But this kind of advice is outside the normative spec. Perhaps
it is more appropriate for the various outlets authored by many of the
authors here on es-discuss? (Both those on and not on TC39)


--
    Cheers,
    --MarkM
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Removal of WeakMap/WeakSet clear

Andreas Rossberg-4
In reply to this post by Mark S. Miller-2
The discussion here still makes various untested assumptions about the
transposed representation. With respect to .clear in particular, it
seems to assume that this representation does not normally need the
ability to iterate (internally) over the keys of a weak map. AFAICT,
that is incorrect. Both the implementation we discussed in the V8
team, and from what I heard, the implementation in IE, maintain a list
of all its keys for each weak map. Otherwise, when a map dies, you
couldn't generally kill the entries and the values associated with
them easily and in a timely manner. (And of course, this list means
almost twice the space usage for a map, because you duplicate part of
the information.)

So from that perspective at least, .clear is not an issue.

I also don't see the security issue, to be honest. In any
security-relevant (or abstraction-relevant) scenario you would be
crazy to hand out the (mutable!) weak map to third parties anyway.
They could run just as much havoc by _adding_ random things to the map
(or removing things at guess) as they could by clearing it.

Mark, can you explain the specific scenario you have in mind?

/Andreas


On 26 November 2014 at 20:21, Mark S. Miller <[hidden email]> wrote:

> On Wed, Nov 26, 2014 at 9:33 AM, Katelyn Gadd <[hidden email]> wrote:
>> Is there a detailed rationale for this somewhere?
>
> It is a combination of three issues.
>
> 1. The security issue.
> 2. The implementation issue.
> 3. The ES process issue.
>
> The implementation issue is that the use cases for WeakMaps basically
> divide into the following cases:
>
> a. Those for which we're confident that the map's lifetime outlives
> its typical key.
> b. Those for which we're confident that the key's lifetime outlives
> the typical map it is a key in.
> c. Those for which we're not confident which typically lives longer.
>
> For #a, the transposed representation of WeakMaps is strictly better.
> The non-transposed implementation would promote ephemeral keys to
> later GC generations, which is very expensive. (I believe the expense
> of this promotion has been radically underestimated in most prior
> discussions.) This is the GC pressure that really matters.
>
> For #b, just use a Map rather than a WeakMap.
>
> For #c, transposed rep or not is a tie. In the interests of minimizing
> implementation mechanism, we should just use a transposed rep for this
> as well.
>
> Given the transposed representation, the only implementation
> techniques for clear are
>
> x. Enumerating all memory
> y. Having the WeakMap encapsulate the token used to lookup the value
> in the key's hidden map, and have .clear replace this token.
>
> #x is not reasonable.
> #y is equivalent to the replacing of the Map that would be the
> user-level technique for emulating clear in any case. This is exactly
> what the use of WeakMaps for membranes do, when the membrane should be
> revoked. If .clear could be implemented for the transposed
> representation more efficiently than this, membranes would benefit
> from .clear as well. I have never expected they could.
>
>
> The process issue is that .clear was not in the original proposal (for
> all of these reasons), and it never achieved consensus in committee.
> It would have violated our process to keep it in the spec. The process
> issue is not "Should it be dropped?" since it was never legitimately
> added. The only issue is "Should it be added?".
>
>
>
>> Making typical
>> applications pay the cost here for a specific security scenario seems
>> really bizarre to me. Clearing standard library data structures is an
>> incredibly common operation. If you want to ensure that someone can't
>> clear the map/set, shouldn't you be handing them an encapsulated
>> version of the data structure? This seems like a corner case that
>> shouldn't justify removing an important primitive.
>>
>> If you have a clear method, the security problem seems solved by
>> wrapping it in an object or using a proxy to deny the ability to clear
>> (you hide the actual map/set, so it can't be cleared - you expose only
>> the operations you want to expose).
>>
>> If you don't have a clear method, anyone wanting to clear the data
>> structure has to throw it away and allocate a new one. This has
>> significant disadvantages:
>> The new structure starts empty at a default size, so repopulating it
>> will have to grow the buffer multiple times - this is undesirable for
>> cases where you are reusing a single data structure to store state for
>> a long-running application.
>> The allocation adds to GC and memory pressure for a long-running
>> application that needs to clear data structures frequently. Were it a
>> lightweight data type this would matter less, but a typical map
>> instance with data in it can occupy a considerable amount of space in
>> the heap.
>> Being able to clear the structure now requires that all consumers have
>> support for replacing their reference(s) to the old map with the new
>> one. This makes it harder to maintain encapsulation because you may
>> have saved a reference to the map in a private property or within a
>> closure. Now you need to add accessibility points to everything that
>> might retain the map so that you can update the reference. Or, you
>> have to encapsulate maps and sets just to recreate the clear operation
>> that should have been there to begin with.
>>
>> In either case, encapsulation or shielding the container behind a
>> proxy is necessary. I insist that the common case is the one that
>> shouldn't have to encapsulate, because optimizing for that case will
>> benefit the vast majority of web applications that use it and the
>> penalty to security-sensitive cases is small.
>>
>> -kg
>> _______________________________________________
>> es-discuss mailing list
>> [hidden email]
>> https://mail.mozilla.org/listinfo/es-discuss
>
>
>
> --
>     Cheers,
>     --MarkM
> _______________________________________________
> es-discuss mailing list
> [hidden email]
> https://mail.mozilla.org/listinfo/es-discuss
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Removal of WeakMap/WeakSet clear

Andrea Giammarchi-2
Andreas I think Mark few times already gave for granted WeakMaps are based on transposed representations and it's also partially pointless from a polyfill point of view to not implement them as such but clear makes fast polyfills very pointless with `.clear()` in place ( e.g. [1] )

I've had similar thoughts when I've first realized there was a `delete` method since I thought: if you can delete, you should be able to iterate and get keys ... which wasn't even the initial case but that's how my initial polyfill was implemented, with a real `.clear()` ability that was making the WeakMap itself not weak anymore [2]

I start thinking for performance and polyfill sake *maybe* it's better for everyone to keep `.clear()` out of the equation so there won't be broken polyfill and implementations can decide which implementation is better for their main use cases.

Who wants to expose clear can always do it via wrappers:

```js
// just for example purpose
function CleanableWeakMap() {
  this.clean();
  this.delete = this._wm.delete.bind(this._wm);
  this.get = this._wm.get.bind(this._wm);
  this.has = this._wm.has.bind(this._wm);
  this.set = this._wm.set.bind(this._wm);
}
CleanableWeakMap.prototype.clean = function () {
  this._wm = new WeakMap;
};
```

Above code should make everyone happy except the GC in between a fake `.clean()` and a garbage call but at least it could be sort of consistent across ES5 to ES.next platforms.

Willing also to drop my non WeakMap and bring in either the Polymer polyfill which is already missing `.clear()` but polluting the global scope anyway with this "not really full spec" implementation, or the one in link [1]


Regards



[1] changing the slot without being able to clean up a thing:

[2] not a WeakMap, you should delete things ( aka: sort of pointless WeakMap )



On Thu, Nov 27, 2014 at 9:17 AM, Andreas Rossberg <[hidden email]> wrote:
The discussion here still makes various untested assumptions about the
transposed representation. With respect to .clear in particular, it
seems to assume that this representation does not normally need the
ability to iterate (internally) over the keys of a weak map. AFAICT,
that is incorrect. Both the implementation we discussed in the V8
team, and from what I heard, the implementation in IE, maintain a list
of all its keys for each weak map. Otherwise, when a map dies, you
couldn't generally kill the entries and the values associated with
them easily and in a timely manner. (And of course, this list means
almost twice the space usage for a map, because you duplicate part of
the information.)

So from that perspective at least, .clear is not an issue.

I also don't see the security issue, to be honest. In any
security-relevant (or abstraction-relevant) scenario you would be
crazy to hand out the (mutable!) weak map to third parties anyway.
They could run just as much havoc by _adding_ random things to the map
(or removing things at guess) as they could by clearing it.

Mark, can you explain the specific scenario you have in mind?

/Andreas


On 26 November 2014 at 20:21, Mark S. Miller <[hidden email]> wrote:
> On Wed, Nov 26, 2014 at 9:33 AM, Katelyn Gadd <[hidden email]> wrote:
>> Is there a detailed rationale for this somewhere?
>
> It is a combination of three issues.
>
> 1. The security issue.
> 2. The implementation issue.
> 3. The ES process issue.
>
> The implementation issue is that the use cases for WeakMaps basically
> divide into the following cases:
>
> a. Those for which we're confident that the map's lifetime outlives
> its typical key.
> b. Those for which we're confident that the key's lifetime outlives
> the typical map it is a key in.
> c. Those for which we're not confident which typically lives longer.
>
> For #a, the transposed representation of WeakMaps is strictly better.
> The non-transposed implementation would promote ephemeral keys to
> later GC generations, which is very expensive. (I believe the expense
> of this promotion has been radically underestimated in most prior
> discussions.) This is the GC pressure that really matters.
>
> For #b, just use a Map rather than a WeakMap.
>
> For #c, transposed rep or not is a tie. In the interests of minimizing
> implementation mechanism, we should just use a transposed rep for this
> as well.
>
> Given the transposed representation, the only implementation
> techniques for clear are
>
> x. Enumerating all memory
> y. Having the WeakMap encapsulate the token used to lookup the value
> in the key's hidden map, and have .clear replace this token.
>
> #x is not reasonable.
> #y is equivalent to the replacing of the Map that would be the
> user-level technique for emulating clear in any case. This is exactly
> what the use of WeakMaps for membranes do, when the membrane should be
> revoked. If .clear could be implemented for the transposed
> representation more efficiently than this, membranes would benefit
> from .clear as well. I have never expected they could.
>
>
> The process issue is that .clear was not in the original proposal (for
> all of these reasons), and it never achieved consensus in committee.
> It would have violated our process to keep it in the spec. The process
> issue is not "Should it be dropped?" since it was never legitimately
> added. The only issue is "Should it be added?".
>
>
>
>> Making typical
>> applications pay the cost here for a specific security scenario seems
>> really bizarre to me. Clearing standard library data structures is an
>> incredibly common operation. If you want to ensure that someone can't
>> clear the map/set, shouldn't you be handing them an encapsulated
>> version of the data structure? This seems like a corner case that
>> shouldn't justify removing an important primitive.
>>
>> If you have a clear method, the security problem seems solved by
>> wrapping it in an object or using a proxy to deny the ability to clear
>> (you hide the actual map/set, so it can't be cleared - you expose only
>> the operations you want to expose).
>>
>> If you don't have a clear method, anyone wanting to clear the data
>> structure has to throw it away and allocate a new one. This has
>> significant disadvantages:
>> The new structure starts empty at a default size, so repopulating it
>> will have to grow the buffer multiple times - this is undesirable for
>> cases where you are reusing a single data structure to store state for
>> a long-running application.
>> The allocation adds to GC and memory pressure for a long-running
>> application that needs to clear data structures frequently. Were it a
>> lightweight data type this would matter less, but a typical map
>> instance with data in it can occupy a considerable amount of space in
>> the heap.
>> Being able to clear the structure now requires that all consumers have
>> support for replacing their reference(s) to the old map with the new
>> one. This makes it harder to maintain encapsulation because you may
>> have saved a reference to the map in a private property or within a
>> closure. Now you need to add accessibility points to everything that
>> might retain the map so that you can update the reference. Or, you
>> have to encapsulate maps and sets just to recreate the clear operation
>> that should have been there to begin with.
>>
>> In either case, encapsulation or shielding the container behind a
>> proxy is necessary. I insist that the common case is the one that
>> shouldn't have to encapsulate, because optimizing for that case will
>> benefit the vast majority of web applications that use it and the
>> penalty to security-sensitive cases is small.
>>
>> -kg
>> _______________________________________________
>> es-discuss mailing list
>> [hidden email]
>> https://mail.mozilla.org/listinfo/es-discuss
>
>
>
> --
>     Cheers,
>     --MarkM
> _______________________________________________
> es-discuss mailing list
> [hidden email]
> https://mail.mozilla.org/listinfo/es-discuss
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss


_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Removal of WeakMap/WeakSet clear

Andreas Rossberg-4
On 27 November 2014 at 15:22, Andrea Giammarchi
<[hidden email]> wrote:
> Andreas I think Mark few times already gave for granted WeakMaps are based
> on transposed representations and it's also partially pointless from a
> polyfill point of view to not implement them as such but clear makes fast
> polyfills very pointless with `.clear()` in place ( e.g. [1] )

I think you misunderstood. The argument in this thread that I was
responding to was that .clear is incompatible with a (presumably
desirable) transposed representation. That is not so, at least not for
any realistic implementation of that approach that I am aware of.

Polyfills are a different issue that I haven't considered. However,
the only functionally correct polyfill for WeakMaps is via non-weak
Maps or even arrays, and then there is no problem either.

> I've had similar thoughts when I've first realized there was a `delete`
> method since I thought: if you can delete, you should be able to iterate and
> get keys ... which wasn't even the initial case but that's how my initial
> polyfill was implemented, with a real `.clear()` ability that was making the
> WeakMap itself not weak anymore [2]

Well, there is no functionally correct polyfill for WeakMaps that is
actually weak, regardless of .clear.

> I start thinking for performance and polyfill sake *maybe* it's better for
> everyone to keep `.clear()` out of the equation so there won't be broken
> polyfill and implementations can decide which implementation is better for
> their main use cases.
>
> Who wants to expose clear can always do it via wrappers:
>
> ```js
> // just for example purpose
> function CleanableWeakMap() {
>   this.clean();
>   this.delete = this._wm.delete.bind(this._wm);
>   this.get = this._wm.get.bind(this._wm);
>   this.has = this._wm.has.bind(this._wm);
>   this.set = this._wm.set.bind(this._wm);
> }
> CleanableWeakMap.prototype.clean = function () {
>   this._wm = new WeakMap;
> };
> ```

Sure, but the same is true the other way round. So that argument
doesn't resolve the discussion either way.

/Andreas


> Above code should make everyone happy except the GC in between a fake
> `.clean()` and a garbage call but at least it could be sort of consistent
> across ES5 to ES.next platforms.
>
> Willing also to drop my non WeakMap and bring in either the Polymer polyfill
> which is already missing `.clear()` but polluting the global scope anyway
> with this "not really full spec" implementation, or the one in link [1]
>
>
> Regards
>
>
>
> [1] changing the slot without being able to clean up a thing:
>      https://gist.github.com/WebReflection/5991636#file-weakmap-js-L15-L19
>
> [2] not a WeakMap, you should delete things ( aka: sort of pointless WeakMap
> )
>
> https://github.com/WebReflection/es6-collections#the-weakmap-is-not-weak--and-why
>
>
>
> On Thu, Nov 27, 2014 at 9:17 AM, Andreas Rossberg <[hidden email]>
> wrote:
>>
>> The discussion here still makes various untested assumptions about the
>> transposed representation. With respect to .clear in particular, it
>> seems to assume that this representation does not normally need the
>> ability to iterate (internally) over the keys of a weak map. AFAICT,
>> that is incorrect. Both the implementation we discussed in the V8
>> team, and from what I heard, the implementation in IE, maintain a list
>> of all its keys for each weak map. Otherwise, when a map dies, you
>> couldn't generally kill the entries and the values associated with
>> them easily and in a timely manner. (And of course, this list means
>> almost twice the space usage for a map, because you duplicate part of
>> the information.)
>>
>> So from that perspective at least, .clear is not an issue.
>>
>> I also don't see the security issue, to be honest. In any
>> security-relevant (or abstraction-relevant) scenario you would be
>> crazy to hand out the (mutable!) weak map to third parties anyway.
>> They could run just as much havoc by _adding_ random things to the map
>> (or removing things at guess) as they could by clearing it.
>>
>> Mark, can you explain the specific scenario you have in mind?
>>
>> /Andreas
>>
>>
>> On 26 November 2014 at 20:21, Mark S. Miller <[hidden email]> wrote:
>> > On Wed, Nov 26, 2014 at 9:33 AM, Katelyn Gadd <[hidden email]> wrote:
>> >> Is there a detailed rationale for this somewhere?
>> >
>> > It is a combination of three issues.
>> >
>> > 1. The security issue.
>> > 2. The implementation issue.
>> > 3. The ES process issue.
>> >
>> > The implementation issue is that the use cases for WeakMaps basically
>> > divide into the following cases:
>> >
>> > a. Those for which we're confident that the map's lifetime outlives
>> > its typical key.
>> > b. Those for which we're confident that the key's lifetime outlives
>> > the typical map it is a key in.
>> > c. Those for which we're not confident which typically lives longer.
>> >
>> > For #a, the transposed representation of WeakMaps is strictly better.
>> > The non-transposed implementation would promote ephemeral keys to
>> > later GC generations, which is very expensive. (I believe the expense
>> > of this promotion has been radically underestimated in most prior
>> > discussions.) This is the GC pressure that really matters.
>> >
>> > For #b, just use a Map rather than a WeakMap.
>> >
>> > For #c, transposed rep or not is a tie. In the interests of minimizing
>> > implementation mechanism, we should just use a transposed rep for this
>> > as well.
>> >
>> > Given the transposed representation, the only implementation
>> > techniques for clear are
>> >
>> > x. Enumerating all memory
>> > y. Having the WeakMap encapsulate the token used to lookup the value
>> > in the key's hidden map, and have .clear replace this token.
>> >
>> > #x is not reasonable.
>> > #y is equivalent to the replacing of the Map that would be the
>> > user-level technique for emulating clear in any case. This is exactly
>> > what the use of WeakMaps for membranes do, when the membrane should be
>> > revoked. If .clear could be implemented for the transposed
>> > representation more efficiently than this, membranes would benefit
>> > from .clear as well. I have never expected they could.
>> >
>> >
>> > The process issue is that .clear was not in the original proposal (for
>> > all of these reasons), and it never achieved consensus in committee.
>> > It would have violated our process to keep it in the spec. The process
>> > issue is not "Should it be dropped?" since it was never legitimately
>> > added. The only issue is "Should it be added?".
>> >
>> >
>> >
>> >> Making typical
>> >> applications pay the cost here for a specific security scenario seems
>> >> really bizarre to me. Clearing standard library data structures is an
>> >> incredibly common operation. If you want to ensure that someone can't
>> >> clear the map/set, shouldn't you be handing them an encapsulated
>> >> version of the data structure? This seems like a corner case that
>> >> shouldn't justify removing an important primitive.
>> >>
>> >> If you have a clear method, the security problem seems solved by
>> >> wrapping it in an object or using a proxy to deny the ability to clear
>> >> (you hide the actual map/set, so it can't be cleared - you expose only
>> >> the operations you want to expose).
>> >>
>> >> If you don't have a clear method, anyone wanting to clear the data
>> >> structure has to throw it away and allocate a new one. This has
>> >> significant disadvantages:
>> >> The new structure starts empty at a default size, so repopulating it
>> >> will have to grow the buffer multiple times - this is undesirable for
>> >> cases where you are reusing a single data structure to store state for
>> >> a long-running application.
>> >> The allocation adds to GC and memory pressure for a long-running
>> >> application that needs to clear data structures frequently. Were it a
>> >> lightweight data type this would matter less, but a typical map
>> >> instance with data in it can occupy a considerable amount of space in
>> >> the heap.
>> >> Being able to clear the structure now requires that all consumers have
>> >> support for replacing their reference(s) to the old map with the new
>> >> one. This makes it harder to maintain encapsulation because you may
>> >> have saved a reference to the map in a private property or within a
>> >> closure. Now you need to add accessibility points to everything that
>> >> might retain the map so that you can update the reference. Or, you
>> >> have to encapsulate maps and sets just to recreate the clear operation
>> >> that should have been there to begin with.
>> >>
>> >> In either case, encapsulation or shielding the container behind a
>> >> proxy is necessary. I insist that the common case is the one that
>> >> shouldn't have to encapsulate, because optimizing for that case will
>> >> benefit the vast majority of web applications that use it and the
>> >> penalty to security-sensitive cases is small.
>> >>
>> >> -kg
>> >> _______________________________________________
>> >> es-discuss mailing list
>> >> [hidden email]
>> >> https://mail.mozilla.org/listinfo/es-discuss
>> >
>> >
>> >
>> > --
>> >     Cheers,
>> >     --MarkM
>> > _______________________________________________
>> > es-discuss mailing list
>> > [hidden email]
>> > https://mail.mozilla.org/listinfo/es-discuss
>> _______________________________________________
>> es-discuss mailing list
>> [hidden email]
>> https://mail.mozilla.org/listinfo/es-discuss
>
>
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Removal of WeakMap/WeakSet clear

Andrea Giammarchi-2
On Thu, Nov 27, 2014 at 2:44 PM, Andreas Rossberg <[hidden email]> wrote:

Well, there is no functionally correct polyfill for WeakMaps that is
actually weak, regardless of .clear.


Please bear with me so I understand your point: if you have `o1` and `o2` and `o1.link = o2;`, wouldn't `o2` be "free" once `o1` is not referenced anymore ?



 


_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Removal of WeakMap/WeakSet clear

Andreas Rossberg-4
On 27 November 2014 at 15:58, Andrea Giammarchi
<[hidden email]> wrote:
> On Thu, Nov 27, 2014 at 2:44 PM, Andreas Rossberg <[hidden email]>
> wrote:
>> Well, there is no functionally correct polyfill for WeakMaps that is
>> actually weak, regardless of .clear.
>
> Please bear with me so I understand your point: if you have `o1` and `o2`
> and `o1.link = o2;`, wouldn't `o2` be "free" once `o1` is not referenced
> anymore ?

Yes, but this approach only works for regular, extensible, writable
objects o1, i.e., fails for frozen or sealed objects, proxies, etc.
And of course, as a polyfill for weak maps, it breaks other things,
such as correctly iterating over properties of an object used as a
key.

/Andreas
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Removal of WeakMap/WeakSet clear

Andrea Giammarchi-2
Right, the compromise then is usually to use a descriptor with `enumerable:false` that won't break "the common web" but agreed it's not that reliable or safe ... although developers are interested in the "weak" bit, and most of them most likely won't deal with "new" kind of objects such Proxies and/or frozen/sealed but I understand your point and that's why I've been happily using an Array for keys and one for values that never failed, but also never granted weakness requiring either the usage of clear or delete, which is usually more problematic.

Now I let Mark answer your initial questions and stop talking about polyfills since there's not a clear winner anyway.

Regards


On Thu, Nov 27, 2014 at 3:17 PM, Andreas Rossberg <[hidden email]> wrote:
On 27 November 2014 at 15:58, Andrea Giammarchi
<[hidden email]> wrote:
> On Thu, Nov 27, 2014 at 2:44 PM, Andreas Rossberg <[hidden email]>
> wrote:
>> Well, there is no functionally correct polyfill for WeakMaps that is
>> actually weak, regardless of .clear.
>
> Please bear with me so I understand your point: if you have `o1` and `o2`
> and `o1.link = o2;`, wouldn't `o2` be "free" once `o1` is not referenced
> anymore ?

Yes, but this approach only works for regular, extensible, writable
objects o1, i.e., fails for frozen or sealed objects, proxies, etc.
And of course, as a polyfill for weak maps, it breaks other things,
such as correctly iterating over properties of an object used as a
key.

/Andreas


_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Removal of WeakMap/WeakSet clear

Allen Wirfs-Brock
In reply to this post by Andreas Rossberg-4

On Nov 27, 2014, at 1:17 AM, Andreas Rossberg wrote:

> The discussion here still makes various untested assumptions about the
> transposed representation. With respect to .clear in particular, it
> seems to assume that this representation does not normally need the
> ability to iterate (internally) over the keys of a weak map. AFAICT,
> that is incorrect. Both the implementation we discussed in the V8
> team, and from what I heard, the implementation in IE, maintain a list
> of all its keys for each weak map. Otherwise, when a map dies, you
> couldn't generally kill the entries and the values associated with
> them easily and in a timely manner. (And of course, this list means
> almost twice the space usage for a map, because you duplicate part of
> the information.)
>
> So from that perspective at least, .clear is not an issue.

Since I made this assertion that 'clear' interfere with with an inverted implementation of WeakMap/Set I'll explain what I'm thinking.

First, here is my design for an inverted implementation:

Every object internally maintains a table (probably a hash table if it contains more than a few elements) that is used to implement WeakMap/Set.  Entry in the table is a key/value pair where the key is a WeakMap/Set instance.  Values are arbitrary ES values.  Let's call such a table an "inverted map" and generically refer to such WeakMaps/Sets as WCs.

A WC object has no instance specific state (other than the state required of all objects, such as [[Prototype]] etc.

The WeakMap 'set' method applied to a WeakMap M with arguments k and v works by accessing the inverted map of k and creating an entry in the inverted map with key: M and value: k (or updating the value field with v with an entry for M already exists).

WeakMap get/set/has/delete work similarly by accessing keys' inverted map using M as the key.

Note that since such a WC instance does not contain any references to its keys (or values) it does not prevent an object that is used as one of its keys from being garbage collected.  When an object is GC'd its inverted map is, of course, also discarded.

There is no change in the GC latency of an object uses as a WeakMap key.  If the only reference to an object used as a value in a  Weakmap is from a single inverted map entry, then that value object should be GC's when its key value is GC'ed.

So far great, none of this requires anything special from the GC. But what if there are no references to a WC instance other than from the keys of inverted maps?  In that case the WC instance should be considered garbage, but if the inverted map keys are normal strong object references such should-be-garbage WC instance won't be recognized as such.  (This might not be a very big deal if we were only talking about the WC instances as they are small and presumably not very numerous, but retaining those instances also cause any corresponding inverted map value references to be retained).

We can fix this by having the GC treat all key entries of inverted maps as weak references. Note that this requires weak reference support in the GC (definitely a complication) but does not require that the GC implement ephemeron semantics (a bigger complication).

Note that nothing above requires (or allows) enumerating the entries of a WC.  Explicit enumeration operations or a 'clear' method would require some way to enumerate the objects that are used as keys of a WC.

This is the end of my assumed inverted WC design and why I assert that a clear method is incompatible with it.

Lets explore how we can extend this design to support iteration (of the sort need to implement 'clear').  To do iteration we need to have some way to find all of the objects that are used as keys of a WC. Since were are using an inverted representation (and assuming we don't want to examine all objects to see if the WC is a key in its inverted map). We need a way to efficiently find all objects that are used as key of a particular WC.  The obvious way to do that would be to add to the internal state of a WC a list of all active key values. This would need to be a list represented using weak references.  Otherwise, it would prevent key objected from being properly GC'ed.  So, at the very least, clear or other iteration requires an additional weak reference list for each WC (or perhaps a likely less efficient single global weak list) that would not otherwise be needed.

Allen


_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Removal of WeakMap/WeakSet clear

Brendan Eich-2
(Nit-correcting along with comments and questions in-line, just trying
to parse what you mean, not snarking -- except for the first comment :-D.)

Allen Wirfs-Brock wrote:

> On Nov 27, 2014, at 1:17 AM, Andreas Rossberg wrote:
>
>> >  The discussion here still makes various untested assumptions about the
>> >  transposed representation. With respect to .clear in particular, it
>> >  seems to assume that this representation does not normally need the
>> >  ability to iterate (internally) over the keys of a weak map. AFAICT,
>> >  that is incorrect. Both the implementation we discussed in the V8
>> >  team, and from what I heard, the implementation in IE, maintain a list
>> >  of all its keys for each weak map. Otherwise, when a map dies, you
>> >  couldn't generally kill the entries and the values associated with
>> >  them easily and in a timely manner. (And of course, this list means
>> >  almost twice the space usage for a map, because you duplicate part of
>> >  the information.)
>> >  
>> >  So from that perspective at least, .clear is not an issue.
>
> Since I made this assertion that 'clear' interfere with with an inverted implementation of WeakMap/Set I'll explain what I'm thinking.
>
> First, here is my design for an inverted implementation:
>
> Every object internally maintains a table (probably a hash table if it contains more than a few elements) that is used to implement WeakMap/Set.  Entry in the table is a key/value pair where the key is a WeakMap/Set instance.  Values are arbitrary ES values.  Let's call such a table an "inverted map" and generically refer to such WeakMaps/Sets as WCs.

(Water Closet? Just curious what the C stands for :-P.)

> A WC object has no instance specific state (other than the state required of all objects, such as [[Prototype]] etc.
>
> The WeakMap 'set' method applied to a WeakMap M with arguments k and v works by accessing the inverted map of k and creating an entry in the inverted map with key: M and value: k

value: v, right?

>   (or updating the value field with v with an entry for M already exists).
>
> WeakMap get/set/has/delete work similarly by accessing keys' inverted map using M as the key.

[A] This means M must be mapped to WC, via some internal member or
(side-)side-table.

> Note that since such a WC instance does not contain any references to its keys (or values) it does not prevent an object that is used as one of its keys from being garbage collected.  When an object is GC'd its inverted map is, of course, also discarded.

In this sense the inverted map is atomic, featureless, just an identity
-- like a symbol, but hidden from inspection. It could be a symbol, even.

> There is no change in the GC latency of an object uses

("used")

>   as a WeakMap key.  If the only reference to an object used as a value in a  Weakmap is from a single inverted map entry, then that value object should be GC's

"GC'ed"

>   when its key value is GC'ed.
>
> So far great, none of this requires anything special from the GC. But what if there are no references to a WC instance other than from the keys of inverted maps?

If M is garbage then its connection [A] to its WC or WCs won't be
considered by the GC. Just confirming that you are presuming M is
garbage in this paragraph.

If M is not garbage, then [A] must keep WC alive.

> In that case the WC instance should be considered garbage, but if the inverted map keys are normal strong object references such should-be-garbage WC instance won't be recognized as such.  (This might not be a very big deal if we were only talking about the WC instances as they are small and presumably not very numerous, but retaining those instances also cause any corresponding inverted map value references to be retained).

That's not clear to me: if a WC is implemented using a symbol, and
nothing else references that symbol, then (unlike string-equated
property keys, where a new name could be computed later), the WC-keyed
entries can be purged.

> We can fix this by having the GC treat all key entries of inverted maps as weak references. Note that this requires weak reference support in the GC (definitely a complication) but does not require that the GC implement ephemeron semantics (a bigger complication).

I would appreciate a reply to the previous comment ("That's not clear to
me") before going on. This may be just a clarifying point, or a tangent,
with respect to the feasibility of .clear for maps with inverted
representations, but I hope it will clarify things so no one (including
me!) goes down a bogus path.

/be

>
> Note that nothing above requires (or allows) enumerating the entries of a WC.  Explicit enumeration operations or a 'clear' method would require some way to enumerate the objects that are used as keys of a WC.
>
> This is the end of my assumed inverted WC design and why I assert that a clear method is incompatible with it.
>
> Lets explore how we can extend this design to support iteration (of the sort need to implement 'clear').  To do iteration we need to have some way to find all of the objects that are used as keys of a WC. Since were are using an inverted representation (and assuming we don't want to examine all objects to see if the WC is a key in its inverted map). We need a way to efficiently find all objects that are used as key of a particular WC.  The obvious way to do that would be to add to the internal state of a WC a list of all active key values. This would need to be a list represented using weak references.  Otherwise, it would prevent key objected from being properly GC'ed.  So, at the very least, clear or other iteration requires an additional weak reference list for each WC (or perhaps a likely less efficient single global weak list) that would not otherwise be needed.
>
> Allen
>
>
> _______________________________________________
> es-discuss mailing list
> [hidden email]
> https://mail.mozilla.org/listinfo/es-discuss
>
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: Removal of WeakMap/WeakSet clear

Allen Wirfs-Brock

On Nov 27, 2014, at 11:49 AM, Brendan Eich wrote:

> (Nit-correcting along with comments and questions in-line, just trying to parse what you mean, not snarking -- except for the first comment :-D.)
>
> Allen Wirfs-Brock wrote:
>> On Nov 27, 2014, at 1:17 AM, Andreas Rossberg wrote:
>>
>>> >  The discussion here still makes various untested assumptions about the
>>> >  transposed representation. With respect to .clear in particular, it
>>> >  seems to assume that this representation does not normally need the
>>> >  ability to iterate (internally) over the keys of a weak map. AFAICT,
>>> >  that is incorrect. Both the implementation we discussed in the V8
>>> >  team, and from what I heard, the implementation in IE, maintain a list
>>> >  of all its keys for each weak map. Otherwise, when a map dies, you
>>> >  couldn't generally kill the entries and the values associated with
>>> >  them easily and in a timely manner. (And of course, this list means
>>> >  almost twice the space usage for a map, because you duplicate part of
>>> >  the information.)
>>> >  >  So from that perspective at least, .clear is not an issue.
>>
>> Since I made this assertion that 'clear' interfere with with an inverted implementation of WeakMap/Set I'll explain what I'm thinking.
>>
>> First, here is my design for an inverted implementation:
>>
>> Every object internally maintains a table (probably a hash table if it contains more than a few elements) that is used to implement WeakMap/Set.  Entry in the table is a key/value pair where the key is a WeakMap/Set instance.  Values are arbitrary ES values.  Let's call such a table an "inverted map" and generically refer to such WeakMaps/Sets as WCs.
>
> (Water Closet? Just curious what the C stands for :-P.)

Collection

>
>> A WC object has no instance specific state (other than the state required of all objects, such as [[Prototype]] etc.
>>
>> The WeakMap 'set' method applied to a WeakMap M with arguments k and v works by accessing the inverted map of k and creating an entry in the inverted map with key: M and value: k
>
> value: v, right?

oops, right
>
>>  (or updating the value field with v with an entry for M already exists).
>>
>> WeakMap get/set/has/delete work similarly by accessing keys' inverted map using M as the key.
>
> [A] This means M must be mapped to WC, via some internal member or (side-)side-table.

M is an instance of a WC, eg,  M is a WeakMap which is a kind of WC
>
>> Note that since such a WC instance does not contain any references to its keys (or values) it does not prevent an object that is used as one of its keys from being garbage collected.  When an object is GC'd its inverted map is, of course, also discarded.
>
> In this sense the inverted map is atomic, featureless, just an identity -- like a symbol, but hidden from inspection. It could be a symbol, even.

No, a WC instance is atomic, featureless, etc. internally.  Externally it expose a WeakMap or WeakSet api.
The inverted map is part of the implementation level represent of an Object.  It is a map-like data structure in the same sense that the own roperties of an object might be implemented using a map-like data structure

>
>> There is no change in the GC latency of an object uses
>
> ("used")
>
>>  as a WeakMap key.  If the only reference to an object used as a value in a  Weakmap is from a single inverted map entry, then that value object should be GC's
>
> "GC'ed"
>
>>  when its key value is GC'ed.
>>
>> So far great, none of this requires anything special from the GC. But what if there are no references to a WC instance other than from the keys of inverted maps?
>
> If M is garbage then its connection [A] to its WC or WCs won't be considered by the GC. Just confirming that you are presuming M is garbage in this paragraph.

yes
>
> If M is not garbage, then [A] must keep WC alive.

I think you are seeing an additional pointer that I didn't intend. M is a WC so there is no [A]. There concern is about the reference from the key side of k's inverted map. It should keep M alive.
>
>> In that case the WC instance should be considered garbage, but if the inverted map keys are normal strong object references such should-be-garbage WC instance won't be recognized as such.  (This might not be a very big deal if we were only talking about the WC instances as they are small and presumably not very numerous, but retaining those instances also cause any corresponding inverted map value references to be retained).
>
> That's not clear to me: if a WC is implemented using a symbol, and nothing else references that symbol, then (unlike string-equated property keys, where a new name could be computed later), the WC-keyed entries can be purged.

But WC instances do have mutable state (their [[Prototype]] and [[Exstensible]] internal slots.  Plus you might use a WC as the key of a WC, so that means a WC instance also needs to have it's own internally mutable inverted map to such usages.
>
>> We can fix this by having the GC treat all key entries of inverted maps as weak references. Note that this requires weak reference support in the GC (definitely a complication) but does not require that the GC implement ephemeron semantics (a bigger complication).
>
> I would appreciate a reply to the previous comment ("That's not clear to me") before going on. This may be just a clarifying point, or a tangent, with respect to the feasibility of .clear for maps with inverted representations, but I hope it will clarify things so no one (including me!) goes down a bogus path.
>
> /be

hope it's clearer now

Allen

>
>>
>> Note that nothing above requires (or allows) enumerating the entries of a WC.  Explicit enumeration operations or a 'clear' method would require some way to enumerate the objects that are used as keys of a WC.
>>
>> This is the end of my assumed inverted WC design and why I assert that a clear method is incompatible with it.
>>
>> Lets explore how we can extend this design to support iteration (of the sort need to implement 'clear').  To do iteration we need to have some way to find all of the objects that are used as keys of a WC. Since were are using an inverted representation (and assuming we don't want to examine all objects to see if the WC is a key in its inverted map). We need a way to efficiently find all objects that are used as key of a particular WC.  The obvious way to do that would be to add to the internal state of a WC a list of all active key values. This would need to be a list represented using weak references.  Otherwise, it would prevent key objected from being properly GC'ed.  So, at the very least, clear or other iteration requires an additional weak reference list for each WC (or perhaps a likely less efficient single global weak list) that would not otherwise be needed.
>>
>> Allen
>>
>>
>> _______________________________________________
>> es-discuss mailing list
>> [hidden email]
>> https://mail.mozilla.org/listinfo/es-discuss
>>
>

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
1234