JSON.parse should be simple and idiot-proof

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

JSON.parse should be simple and idiot-proof

kai zhu
wish to express skepticism for the stage-1 proposal "JSON.parse source text access" [1], from web-integration perspective.

a common javascript-painpoint is pinpointing bug-source of end-to-end client<->server communications.  thankfully, JSON.parse is rarely suspect in this process.  this proposal however, encourage developers to introduce bugs/doubts-of-reliability to JSON.parse, making integration bug-hunting more painful than it already is.

standard-operating-procedure for reviving JSON-data is a 2-step process:
1. JSON.parse with zero-config to rule-out bugs during this step
2. second-pass of plain-JSON to revive [product-specific] string-encoded non-JSON datatypes like BigInt/Date/RegExp, where bugs can be expected

you normally do not want to complicate bug-hunts by contaminating step-1 with bugs from step-2.

[1] stage-1 proposal - JSON.parse source text access



_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Slight return to BigInt. Was: JSON.parse should be simple and idiot-proof

Anders Rundgren-2
On 2018-10-21 08:01, kai zhu wrote:
> wish to express skepticism for the stage-1 proposal "JSON.parse source text access" [1], from web-integration perspective.

Be happy, now you have not less than TWO proposals for extending the ES6 JSON object! :-)

A [potentially biased] comparison between https://github.com/gibson042/ecma262-proposal-JSON-parse-with-source and https://github.com/cyberphone/es6-bigint-json-support#21-the-jsonnumber-primitive:

The introduction of a JSONNumber type in the latter proposal:
- only requires an optional extra boolean flag to JSON.parse()
- also works in the other direction, JSON.stringify(). Without changing the API
- is compatible both with "stream parsing" using the reviver function as well as the more commonly used "see what you got after parsing" method
- addresses the only real short-coming of the ES6 JSON object, namely JSON number handling

It is also worth keeping in mind that the JSON world is slowly drifting towards "schemas" making the ES6 JSON object less interesting.


Although I indeed wrote a specification for dealing with with JSON numbers outside of IEEE double precision, I still claim that the advantages of using the RFC mode is ZERO, and in fact introduces new problems like forcing stream/schema parsing or introducing "super types" like my JSONNumber.  "Fulfilling RFCs" is not something application programmers are particularly concerned with.

Expressing "big":5656565656565656565 as "big":"5656565656565656565" is simply put not a big deal.  The way this is currently implemented in for example JDK is anyway incompatible with just about everything else out there :-)

Anders

>
> a common javascript-painpoint is pinpointing bug-source of end-to-end client<->server communications.  thankfully, JSON.parse is rarely suspect in this process.  this proposal however, encourage developers to introduce bugs/doubts-of-reliability to JSON.parse, making integration bug-hunting more painful than it already is.
>
> standard-operating-procedure for reviving JSON-data is a 2-step process:
> 1. JSON.parse with zero-config to rule-out bugs during this step
> 2. second-pass of plain-JSON to revive [product-specific] string-encoded non-JSON datatypes like BigInt/Date/RegExp, where bugs can be expected
>
> you normally do not want to complicate bug-hunts by contaminating step-1 with bugs from step-2.
>
> [1] stage-1 proposal - JSON.parse source text access
> https://github.com/gibson042/ecma262-proposal-JSON-parse-with-source
>
> kai zhu
> [hidden email] <mailto:[hidden email]>
>
>
>
>
>
> _______________________________________________
> es-discuss mailing list
> [hidden email]
> https://mail.mozilla.org/listinfo/es-discuss
>

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: JSON.parse should be simple and idiot-proof

Isiah Meadows-2
In reply to this post by kai zhu
Just because something exists doesn't mean it'll be  broadly used. Plus, reviver functions are normally incredibly simple - you don't get enough context (like key paths) to do anything crazy, and this proposal doesn't give you enough context for that.

In practice, this changes literally nothing for most consumers, since the use case is incredibly limited and usually requires server agreement as well. In fact, that's where my skepticism lies: why add 3 new reviver parameters when a single "parse ints as bigints" would solve basically the entire problem? I've yet to see any other use case that couldn't be solved by manipulating the result after it's parsed.

But personally, I don't see how bugs would be a major issue here considering its limited utility.

On Sun, Oct 21, 2018 at 02:01 kai zhu <[hidden email]> wrote:
wish to express skepticism for the stage-1 proposal "JSON.parse source text access" [1], from web-integration perspective.

a common javascript-painpoint is pinpointing bug-source of end-to-end client<->server communications.  thankfully, JSON.parse is rarely suspect in this process.  this proposal however, encourage developers to introduce bugs/doubts-of-reliability to JSON.parse, making integration bug-hunting more painful than it already is.

standard-operating-procedure for reviving JSON-data is a 2-step process:
1. JSON.parse with zero-config to rule-out bugs during this step
2. second-pass of plain-JSON to revive [product-specific] string-encoded non-JSON datatypes like BigInt/Date/RegExp, where bugs can be expected

you normally do not want to complicate bug-hunts by contaminating step-1 with bugs from step-2.

[1] stage-1 proposal - JSON.parse source text access


_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: JSON.parse should be simple and idiot-proof

Richard Gibson
First, note that reviver functions do manipulate the result after it's parsed. Second, "parse ints as bigints" is too big a hammer—changing the output for all numbers would break currently working code. And third, it's not even fully sufficient for the "big numbers" purpose, which logically also includes non-integers outside the IEEE 754 64-bit range ("BigFloat").

On Sun, Oct 21, 2018 at 10:50 AM Isiah Meadows <[hidden email]> wrote:
Just because something exists doesn't mean it'll be  broadly used. Plus, reviver functions are normally incredibly simple - you don't get enough context (like key paths) to do anything crazy, and this proposal doesn't give you enough context for that.

In practice, this changes literally nothing for most consumers, since the use case is incredibly limited and usually requires server agreement as well. In fact, that's where my skepticism lies: why add 3 new reviver parameters when a single "parse ints as bigints" would solve basically the entire problem? I've yet to see any other use case that couldn't be solved by manipulating the result after it's parsed.

But personally, I don't see how bugs would be a major issue here considering its limited utility.

On Sun, Oct 21, 2018 at 02:01 kai zhu <[hidden email]> wrote:
wish to express skepticism for the stage-1 proposal "JSON.parse source text access" [1], from web-integration perspective.

a common javascript-painpoint is pinpointing bug-source of end-to-end client<->server communications.  thankfully, JSON.parse is rarely suspect in this process.  this proposal however, encourage developers to introduce bugs/doubts-of-reliability to JSON.parse, making integration bug-hunting more painful than it already is.

standard-operating-procedure for reviving JSON-data is a 2-step process:
1. JSON.parse with zero-config to rule-out bugs during this step
2. second-pass of plain-JSON to revive [product-specific] string-encoded non-JSON datatypes like BigInt/Date/RegExp, where bugs can be expected

you normally do not want to complicate bug-hunts by contaminating step-1 with bugs from step-2.

[1] stage-1 proposal - JSON.parse source text access


_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: JSON.parse should be simple and idiot-proof

Peter Jaszkowiak
He was recommending a single parameter for "parse ints as bigints", not changing the default behavior.

On Sun, Oct 21, 2018, 09:58 Richard Gibson <[hidden email]> wrote:
First, note that reviver functions do manipulate the result after it's parsed. Second, "parse ints as bigints" is too big a hammer—changing the output for all numbers would break currently working code. And third, it's not even fully sufficient for the "big numbers" purpose, which logically also includes non-integers outside the IEEE 754 64-bit range ("BigFloat").

On Sun, Oct 21, 2018 at 10:50 AM Isiah Meadows <[hidden email]> wrote:
Just because something exists doesn't mean it'll be  broadly used. Plus, reviver functions are normally incredibly simple - you don't get enough context (like key paths) to do anything crazy, and this proposal doesn't give you enough context for that.

In practice, this changes literally nothing for most consumers, since the use case is incredibly limited and usually requires server agreement as well. In fact, that's where my skepticism lies: why add 3 new reviver parameters when a single "parse ints as bigints" would solve basically the entire problem? I've yet to see any other use case that couldn't be solved by manipulating the result after it's parsed.

But personally, I don't see how bugs would be a major issue here considering its limited utility.

On Sun, Oct 21, 2018 at 02:01 kai zhu <[hidden email]> wrote:
wish to express skepticism for the stage-1 proposal "JSON.parse source text access" [1], from web-integration perspective.

a common javascript-painpoint is pinpointing bug-source of end-to-end client<->server communications.  thankfully, JSON.parse is rarely suspect in this process.  this proposal however, encourage developers to introduce bugs/doubts-of-reliability to JSON.parse, making integration bug-hunting more painful than it already is.

standard-operating-procedure for reviving JSON-data is a 2-step process:
1. JSON.parse with zero-config to rule-out bugs during this step
2. second-pass of plain-JSON to revive [product-specific] string-encoded non-JSON datatypes like BigInt/Date/RegExp, where bugs can be expected

you normally do not want to complicate bug-hunts by contaminating step-1 with bugs from step-2.

[1] stage-1 proposal - JSON.parse source text access


_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: JSON.parse should be simple and idiot-proof

Isiah Meadows-2
This would be the correct understanding here, a 4th parameter read as a boolean.

On Sun, Oct 21, 2018 at 12:27 Peter Jaszkowiak <[hidden email]> wrote:
He was recommending a single parameter for "parse ints as bigints", not changing the default behavior.

On Sun, Oct 21, 2018, 09:58 Richard Gibson <[hidden email]> wrote:
First, note that reviver functions do manipulate the result after it's parsed. Second, "parse ints as bigints" is too big a hammer—changing the output for all numbers would break currently working code. And third, it's not even fully sufficient for the "big numbers" purpose, which logically also includes non-integers outside the IEEE 754 64-bit range ("BigFloat").

On Sun, Oct 21, 2018 at 10:50 AM Isiah Meadows <[hidden email]> wrote:
Just because something exists doesn't mean it'll be  broadly used. Plus, reviver functions are normally incredibly simple - you don't get enough context (like key paths) to do anything crazy, and this proposal doesn't give you enough context for that.

In practice, this changes literally nothing for most consumers, since the use case is incredibly limited and usually requires server agreement as well. In fact, that's where my skepticism lies: why add 3 new reviver parameters when a single "parse ints as bigints" would solve basically the entire problem? I've yet to see any other use case that couldn't be solved by manipulating the result after it's parsed.

But personally, I don't see how bugs would be a major issue here considering its limited utility.

On Sun, Oct 21, 2018 at 02:01 kai zhu <[hidden email]> wrote:
wish to express skepticism for the stage-1 proposal "JSON.parse source text access" [1], from web-integration perspective.

a common javascript-painpoint is pinpointing bug-source of end-to-end client<->server communications.  thankfully, JSON.parse is rarely suspect in this process.  this proposal however, encourage developers to introduce bugs/doubts-of-reliability to JSON.parse, making integration bug-hunting more painful than it already is.

standard-operating-procedure for reviving JSON-data is a 2-step process:
1. JSON.parse with zero-config to rule-out bugs during this step
2. second-pass of plain-JSON to revive [product-specific] string-encoded non-JSON datatypes like BigInt/Date/RegExp, where bugs can be expected

you normally do not want to complicate bug-hunts by contaminating step-1 with bugs from step-2.

[1] stage-1 proposal - JSON.parse source text access


_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: JSON.parse should be simple and idiot-proof

Richard Gibson
Yes, understood. And setting aside the undesirable characteristics of boolean trap interfaces for the moment (which will certainly be a source of regret if ECMAScript ever gets a BigFloat), my point is that doing so would affect parsing of all numbers, as opposed to only those numbers that really should be BigInts. For example, let's look at a sample Twitter API entity:

{ "created_at":"Thu Apr 06 15:24:15 +0000 2017", "id": 850006245121695744, "id_str": "850006245121695744", "text": "1/ Today we’re sharing our vision for the future of the Twitter API platform!nhttps://t.co/XweGngmxlP", "user": { "id": 6253282,
"id_str": "6253282",
"name": "Twitter API",
"screen_name": "twitterapi",
"followers_count": 21,
"friends_count": 32 }, "entities": {} }

It's nice that e.g. JSON.parse(…, null, true) would allow us to access the ids as full-fidelity numbers (Twitter documents them as an int64), but much less nice that the behavior would affect all numbers. For instance, let's say this is one tweet in a list that we want to sort by the "follower ratio" of their creators:

let creatorPopularity = tweet.user.followers_count / tweet.user.friends_count;
// (without parse-as-BigInt) → 0.65625
// (with parse-as-BigInt)    → 0n

We just silently lost floating-point arithmetic.

On Sun, Oct 21, 2018 at 1:00 PM Isiah Meadows <[hidden email]> wrote:
This would be the correct understanding here, a 4th parameter read as a boolean.

On Sun, Oct 21, 2018 at 12:27 Peter Jaszkowiak <[hidden email]> wrote:
He was recommending a single parameter for "parse ints as bigints", not changing the default behavior.

On Sun, Oct 21, 2018, 09:58 Richard Gibson <[hidden email]> wrote:
First, note that reviver functions do manipulate the result after it's parsed. Second, "parse ints as bigints" is too big a hammer—changing the output for all numbers would break currently working code. And third, it's not even fully sufficient for the "big numbers" purpose, which logically also includes non-integers outside the IEEE 754 64-bit range ("BigFloat").

On Sun, Oct 21, 2018 at 10:50 AM Isiah Meadows <[hidden email]> wrote:
Just because something exists doesn't mean it'll be  broadly used. Plus, reviver functions are normally incredibly simple - you don't get enough context (like key paths) to do anything crazy, and this proposal doesn't give you enough context for that.

In practice, this changes literally nothing for most consumers, since the use case is incredibly limited and usually requires server agreement as well. In fact, that's where my skepticism lies: why add 3 new reviver parameters when a single "parse ints as bigints" would solve basically the entire problem? I've yet to see any other use case that couldn't be solved by manipulating the result after it's parsed.

But personally, I don't see how bugs would be a major issue here considering its limited utility.

On Sun, Oct 21, 2018 at 02:01 kai zhu <[hidden email]> wrote:
wish to express skepticism for the stage-1 proposal "JSON.parse source text access" [1], from web-integration perspective.

a common javascript-painpoint is pinpointing bug-source of end-to-end client<->server communications.  thankfully, JSON.parse is rarely suspect in this process.  this proposal however, encourage developers to introduce bugs/doubts-of-reliability to JSON.parse, making integration bug-hunting more painful than it already is.

standard-operating-procedure for reviving JSON-data is a 2-step process:
1. JSON.parse with zero-config to rule-out bugs during this step
2. second-pass of plain-JSON to revive [product-specific] string-encoded non-JSON datatypes like BigInt/Date/RegExp, where bugs can be expected

you normally do not want to complicate bug-hunts by contaminating step-1 with bugs from step-2.

[1] stage-1 proposal - JSON.parse source text access


_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: JSON.parse should be simple and idiot-proof

Peter Jaszkowiak
What if `JSON.parse` took an options object as the second parameter?
There's already been an instance of a JS API changing from booleans to an options object: `addEventListener`.

I know it wasn't tc39, but it shows that it's possible. Another option is to add a different method to the `JSON` object.

On Sun, Oct 21, 2018, 13:45 Richard Gibson <[hidden email]> wrote:
Yes, understood. And setting aside the undesirable characteristics of boolean trap interfaces for the moment (which will certainly be a source of regret if ECMAScript ever gets a BigFloat), my point is that doing so would affect parsing of all numbers, as opposed to only those numbers that really should be BigInts. For example, let's look at a sample Twitter API entity:

{ "created_at":"Thu Apr 06 15:24:15 +0000 2017", "id": 850006245121695744, "id_str": "850006245121695744", "text": "1/ Today we’re sharing our vision for the future of the Twitter API platform!nhttps://t.co/XweGngmxlP", "user": { "id": 6253282,
"id_str": "6253282",
"name": "Twitter API",
"screen_name": "twitterapi",
"followers_count": 21,
"friends_count": 32 }, "entities": {} }

It's nice that e.g. JSON.parse(…, null, true) would allow us to access the ids as full-fidelity numbers (Twitter documents them as an int64), but much less nice that the behavior would affect all numbers. For instance, let's say this is one tweet in a list that we want to sort by the "follower ratio" of their creators:

let creatorPopularity = tweet.user.followers_count / tweet.user.friends_count;
// (without parse-as-BigInt) → 0.65625
// (with parse-as-BigInt)    → 0n

We just silently lost floating-point arithmetic.

On Sun, Oct 21, 2018 at 1:00 PM Isiah Meadows <[hidden email]> wrote:
This would be the correct understanding here, a 4th parameter read as a boolean.

On Sun, Oct 21, 2018 at 12:27 Peter Jaszkowiak <[hidden email]> wrote:
He was recommending a single parameter for "parse ints as bigints", not changing the default behavior.

On Sun, Oct 21, 2018, 09:58 Richard Gibson <[hidden email]> wrote:
First, note that reviver functions do manipulate the result after it's parsed. Second, "parse ints as bigints" is too big a hammer—changing the output for all numbers would break currently working code. And third, it's not even fully sufficient for the "big numbers" purpose, which logically also includes non-integers outside the IEEE 754 64-bit range ("BigFloat").

On Sun, Oct 21, 2018 at 10:50 AM Isiah Meadows <[hidden email]> wrote:
Just because something exists doesn't mean it'll be  broadly used. Plus, reviver functions are normally incredibly simple - you don't get enough context (like key paths) to do anything crazy, and this proposal doesn't give you enough context for that.

In practice, this changes literally nothing for most consumers, since the use case is incredibly limited and usually requires server agreement as well. In fact, that's where my skepticism lies: why add 3 new reviver parameters when a single "parse ints as bigints" would solve basically the entire problem? I've yet to see any other use case that couldn't be solved by manipulating the result after it's parsed.

But personally, I don't see how bugs would be a major issue here considering its limited utility.

On Sun, Oct 21, 2018 at 02:01 kai zhu <[hidden email]> wrote:
wish to express skepticism for the stage-1 proposal "JSON.parse source text access" [1], from web-integration perspective.

a common javascript-painpoint is pinpointing bug-source of end-to-end client<->server communications.  thankfully, JSON.parse is rarely suspect in this process.  this proposal however, encourage developers to introduce bugs/doubts-of-reliability to JSON.parse, making integration bug-hunting more painful than it already is.

standard-operating-procedure for reviving JSON-data is a 2-step process:
1. JSON.parse with zero-config to rule-out bugs during this step
2. second-pass of plain-JSON to revive [product-specific] string-encoded non-JSON datatypes like BigInt/Date/RegExp, where bugs can be expected

you normally do not want to complicate bug-hunts by contaminating step-1 with bugs from step-2.

[1] stage-1 proposal - JSON.parse source text access


_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: JSON.parse should be simple and idiot-proof

Jordan Harband
I'd suggest filing an issue like this directly on the proposal repo.

On Sun, Oct 21, 2018 at 12:59 PM Peter Jaszkowiak <[hidden email]> wrote:
What if `JSON.parse` took an options object as the second parameter?
There's already been an instance of a JS API changing from booleans to an options object: `addEventListener`.

I know it wasn't tc39, but it shows that it's possible. Another option is to add a different method to the `JSON` object.

On Sun, Oct 21, 2018, 13:45 Richard Gibson <[hidden email]> wrote:
Yes, understood. And setting aside the undesirable characteristics of boolean trap interfaces for the moment (which will certainly be a source of regret if ECMAScript ever gets a BigFloat), my point is that doing so would affect parsing of all numbers, as opposed to only those numbers that really should be BigInts. For example, let's look at a sample Twitter API entity:

{ "created_at":"Thu Apr 06 15:24:15 +0000 2017", "id": 850006245121695744, "id_str": "850006245121695744", "text": "1/ Today we’re sharing our vision for the future of the Twitter API platform!nhttps://t.co/XweGngmxlP", "user": { "id": 6253282,
"id_str": "6253282",
"name": "Twitter API",
"screen_name": "twitterapi",
"followers_count": 21,
"friends_count": 32 }, "entities": {} }

It's nice that e.g. JSON.parse(…, null, true) would allow us to access the ids as full-fidelity numbers (Twitter documents them as an int64), but much less nice that the behavior would affect all numbers. For instance, let's say this is one tweet in a list that we want to sort by the "follower ratio" of their creators:

let creatorPopularity = tweet.user.followers_count / tweet.user.friends_count;
// (without parse-as-BigInt) → 0.65625
// (with parse-as-BigInt)    → 0n

We just silently lost floating-point arithmetic.

On Sun, Oct 21, 2018 at 1:00 PM Isiah Meadows <[hidden email]> wrote:
This would be the correct understanding here, a 4th parameter read as a boolean.

On Sun, Oct 21, 2018 at 12:27 Peter Jaszkowiak <[hidden email]> wrote:
He was recommending a single parameter for "parse ints as bigints", not changing the default behavior.

On Sun, Oct 21, 2018, 09:58 Richard Gibson <[hidden email]> wrote:
First, note that reviver functions do manipulate the result after it's parsed. Second, "parse ints as bigints" is too big a hammer—changing the output for all numbers would break currently working code. And third, it's not even fully sufficient for the "big numbers" purpose, which logically also includes non-integers outside the IEEE 754 64-bit range ("BigFloat").

On Sun, Oct 21, 2018 at 10:50 AM Isiah Meadows <[hidden email]> wrote:
Just because something exists doesn't mean it'll be  broadly used. Plus, reviver functions are normally incredibly simple - you don't get enough context (like key paths) to do anything crazy, and this proposal doesn't give you enough context for that.

In practice, this changes literally nothing for most consumers, since the use case is incredibly limited and usually requires server agreement as well. In fact, that's where my skepticism lies: why add 3 new reviver parameters when a single "parse ints as bigints" would solve basically the entire problem? I've yet to see any other use case that couldn't be solved by manipulating the result after it's parsed.

But personally, I don't see how bugs would be a major issue here considering its limited utility.

On Sun, Oct 21, 2018 at 02:01 kai zhu <[hidden email]> wrote:
wish to express skepticism for the stage-1 proposal "JSON.parse source text access" [1], from web-integration perspective.

a common javascript-painpoint is pinpointing bug-source of end-to-end client<->server communications.  thankfully, JSON.parse is rarely suspect in this process.  this proposal however, encourage developers to introduce bugs/doubts-of-reliability to JSON.parse, making integration bug-hunting more painful than it already is.

standard-operating-procedure for reviving JSON-data is a 2-step process:
1. JSON.parse with zero-config to rule-out bugs during this step
2. second-pass of plain-JSON to revive [product-specific] string-encoded non-JSON datatypes like BigInt/Date/RegExp, where bugs can be expected

you normally do not want to complicate bug-hunts by contaminating step-1 with bugs from step-2.

[1] stage-1 proposal - JSON.parse source text access


_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss

_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: JSON.parse should be simple and idiot-proof

kai zhu
In reply to this post by Peter Jaszkowiak
i'm against all these proposals.  JSON.parse should remain zero-config and idiot-proof.  you will *always* need a second-pass in product-development to revive non-JSON datatypes anyways, like Date, and thats where reviving BigInt should take place as well.

again, i'm ok with working on web-projects where there are bugs in the second-pass (which is expected).  i'm not ok with web-projects, where i cannot reliably trust JSON.parse to do its job (and have to add it to checklist of things to inspect when figuring out where the bug is in end-to-end communications).

fyi, the twitter example is not compelling. it already has a string-form “id_str” which is more appropriate/easier-to-handle in javascript-land.  what benefit does a pseudorandom BigInt “id” (which has no need of arithmetic) have over plain-string “id_str” in javascript-land? none.


On 22 Oct 2018, at 2:59 AM, Peter Jaszkowiak <[hidden email]> wrote:

What if `JSON.parse` took an options object as the second parameter?
There's already been an instance of a JS API changing from booleans to an options object: `addEventListener`.

I know it wasn't tc39, but it shows that it's possible. Another option is to add a different method to the `JSON` object.

On Sun, Oct 21, 2018, 13:45 Richard Gibson <[hidden email]> wrote:
Yes, understood. And setting aside the undesirable characteristics of boolean trap interfaces for the moment (which will certainly be a source of regret if ECMAScript ever gets a BigFloat), my point is that doing so would affect parsing of all numbers, as opposed to only those numbers that really should be BigInts. For example, let's look at a sample Twitter API entity:

{ "created_at":"Thu Apr 06 15:24:15 +0000 2017", "id": 850006245121695744, "id_str": "850006245121695744", "text": "1/ Today we’re sharing our vision for the future of the Twitter API platform!nhttps://t.co/XweGngmxlP", "user": { "id": 6253282,
"id_str": "6253282",
"name": "Twitter API",
"screen_name": "twitterapi",
"followers_count": 21,
"friends_count": 32 }, "entities": {} }

It's nice that e.g. JSON.parse(…, null, true) would allow us to access the ids as full-fidelity numbers (Twitter documents them as an int64), but much less nice that the behavior would affect all numbers. For instance, let's say this is one tweet in a list that we want to sort by the "follower ratio" of their creators:

let creatorPopularity = tweet.user.followers_count / tweet.user.friends_count;
// (without parse-as-BigInt) → 0.65625
// (with parse-as-BigInt)    → 0n

We just silently lost floating-point arithmetic.

On Sun, Oct 21, 2018 at 1:00 PM Isiah Meadows <[hidden email]> wrote:
This would be the correct understanding here, a 4th parameter read as a boolean.

On Sun, Oct 21, 2018 at 12:27 Peter Jaszkowiak <[hidden email]> wrote:
He was recommending a single parameter for "parse ints as bigints", not changing the default behavior.

On Sun, Oct 21, 2018, 09:58 Richard Gibson <[hidden email]> wrote:
First, note that reviver functions do manipulate the result after it's parsed. Second, "parse ints as bigints" is too big a hammer—changing the output for all numbers would break currently working code. And third, it's not even fully sufficient for the "big numbers" purpose, which logically also includes non-integers outside the IEEE 754 64-bit range ("BigFloat").

On Sun, Oct 21, 2018 at 10:50 AM Isiah Meadows <[hidden email]> wrote:
Just because something exists doesn't mean it'll be  broadly used. Plus, reviver functions are normally incredibly simple - you don't get enough context (like key paths) to do anything crazy, and this proposal doesn't give you enough context for that.

In practice, this changes literally nothing for most consumers, since the use case is incredibly limited and usually requires server agreement as well. In fact, that's where my skepticism lies: why add 3 new reviver parameters when a single "parse ints as bigints" would solve basically the entire problem? I've yet to see any other use case that couldn't be solved by manipulating the result after it's parsed.

But personally, I don't see how bugs would be a major issue here considering its limited utility.

On Sun, Oct 21, 2018 at 02:01 kai zhu <[hidden email]> wrote:
wish to express skepticism for the stage-1 proposal "JSON.parse source text access" [1], from web-integration perspective.

a common javascript-painpoint is pinpointing bug-source of end-to-end client<->server communications.  thankfully, JSON.parse is rarely suspect in this process.  this proposal however, encourage developers to introduce bugs/doubts-of-reliability to JSON.parse, making integration bug-hunting more painful than it already is.

standard-operating-procedure for reviving JSON-data is a 2-step process:
1. JSON.parse with zero-config to rule-out bugs during this step
2. second-pass of plain-JSON to revive [product-specific] string-encoded non-JSON datatypes like BigInt/Date/RegExp, where bugs can be expected

you normally do not want to complicate bug-hunts by contaminating step-1 with bugs from step-2.

[1] stage-1 proposal - JSON.parse source text access


_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss


_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss
Reply | Threaded
Open this post in threaded view
|

Re: JSON.parse should be simple and idiot-proof

Isiah Meadows-2
The use is with integers 2^53 and above in general: most non-JS JSON
clients (like Java and C++) can correctly interpret 64-bit integers
and some can even represent arbitrary-size integers (like Python), but
JS can't. This can come up sometimes in math/science contexts, where
numbers can often get that large (if not sometimes larger), but it can
also come up easily in other contexts and is in general a legitimate
concern. Third-party APIs often also represent IDs as 64-bit integers
and send them accordingly, and you can't always fix those (that's
where the integration concerns exist). Twitter fixed theirs, but
broken servers will still exist, and not everyone who uses JSON widely
is even aware of JS's floating point restrictions.

-----

Isiah Meadows
[hidden email]
www.isiahmeadows.com
On Mon, Oct 22, 2018 at 4:09 AM kai zhu <[hidden email]> wrote:

>
> i'm against all these proposals.  JSON.parse should remain zero-config and idiot-proof.  you will *always* need a second-pass in product-development to revive non-JSON datatypes anyways, like Date, and thats where reviving BigInt should take place as well.
>
> again, i'm ok with working on web-projects where there are bugs in the second-pass (which is expected).  i'm not ok with web-projects, where i cannot reliably trust JSON.parse to do its job (and have to add it to checklist of things to inspect when figuring out where the bug is in end-to-end communications).
>
> fyi, the twitter example is not compelling. it already has a string-form “id_str” which is more appropriate/easier-to-handle in javascript-land.  what benefit does a pseudorandom BigInt “id” (which has no need of arithmetic) have over plain-string “id_str” in javascript-land? none.
>
> kai zhu
> [hidden email]
>
> On 22 Oct 2018, at 2:59 AM, Peter Jaszkowiak <[hidden email]> wrote:
>
> What if `JSON.parse` took an options object as the second parameter?
> There's already been an instance of a JS API changing from booleans to an options object: `addEventListener`.
>
> I know it wasn't tc39, but it shows that it's possible. Another option is to add a different method to the `JSON` object.
>
> On Sun, Oct 21, 2018, 13:45 Richard Gibson <[hidden email]> wrote:
>>
>> Yes, understood. And setting aside the undesirable characteristics of boolean trap interfaces for the moment (which will certainly be a source of regret if ECMAScript ever gets a BigFloat), my point is that doing so would affect parsing of all numbers, as opposed to only those numbers that really should be BigInts. For example, let's look at a sample Twitter API entity:
>>
>> { "created_at":"Thu Apr 06 15:24:15 +0000 2017", "id": 850006245121695744, "id_str": "850006245121695744", "text": "1/ Today we’re sharing our vision for the future of the Twitter API platform!nhttps://t.co/XweGngmxlP", "user": { "id": 6253282,
>> "id_str": "6253282",
>> "name": "Twitter API",
>> "screen_name": "twitterapi",
>> "followers_count": 21,
>> "friends_count": 32 }, "entities": {} }
>>
>>
>> It's nice that e.g. JSON.parse(…, null, true) would allow us to access the ids as full-fidelity numbers (Twitter documents them as an int64), but much less nice that the behavior would affect all numbers. For instance, let's say this is one tweet in a list that we want to sort by the "follower ratio" of their creators:
>>
>> let creatorPopularity = tweet.user.followers_count / tweet.user.friends_count;
>>
>> // (without parse-as-BigInt) → 0.65625
>>
>> // (with parse-as-BigInt)    → 0n
>>
>>
>> We just silently lost floating-point arithmetic.
>>
>> On Sun, Oct 21, 2018 at 1:00 PM Isiah Meadows <[hidden email]> wrote:
>>>
>>> This would be the correct understanding here, a 4th parameter read as a boolean.
>>>
>>> On Sun, Oct 21, 2018 at 12:27 Peter Jaszkowiak <[hidden email]> wrote:
>>>>
>>>> He was recommending a single parameter for "parse ints as bigints", not changing the default behavior.
>>>>
>>>> On Sun, Oct 21, 2018, 09:58 Richard Gibson <[hidden email]> wrote:
>>>>>
>>>>> First, note that reviver functions do manipulate the result after it's parsed. Second, "parse ints as bigints" is too big a hammer—changing the output for all numbers would break currently working code. And third, it's not even fully sufficient for the "big numbers" purpose, which logically also includes non-integers outside the IEEE 754 64-bit range ("BigFloat").
>>>>>
>>>>> On Sun, Oct 21, 2018 at 10:50 AM Isiah Meadows <[hidden email]> wrote:
>>>>>>
>>>>>> Just because something exists doesn't mean it'll be  broadly used. Plus, reviver functions are normally incredibly simple - you don't get enough context (like key paths) to do anything crazy, and this proposal doesn't give you enough context for that.
>>>>>>
>>>>>> In practice, this changes literally nothing for most consumers, since the use case is incredibly limited and usually requires server agreement as well. In fact, that's where my skepticism lies: why add 3 new reviver parameters when a single "parse ints as bigints" would solve basically the entire problem? I've yet to see any other use case that couldn't be solved by manipulating the result after it's parsed.
>>>>>>
>>>>>> But personally, I don't see how bugs would be a major issue here considering its limited utility.
>>>>>>
>>>>>> On Sun, Oct 21, 2018 at 02:01 kai zhu <[hidden email]> wrote:
>>>>>>>
>>>>>>> wish to express skepticism for the stage-1 proposal "JSON.parse source text access" [1], from web-integration perspective.
>>>>>>>
>>>>>>> a common javascript-painpoint is pinpointing bug-source of end-to-end client<->server communications.  thankfully, JSON.parse is rarely suspect in this process.  this proposal however, encourage developers to introduce bugs/doubts-of-reliability to JSON.parse, making integration bug-hunting more painful than it already is.
>>>>>>>
>>>>>>> standard-operating-procedure for reviving JSON-data is a 2-step process:
>>>>>>> 1. JSON.parse with zero-config to rule-out bugs during this step
>>>>>>> 2. second-pass of plain-JSON to revive [product-specific] string-encoded non-JSON datatypes like BigInt/Date/RegExp, where bugs can be expected
>>>>>>>
>>>>>>> you normally do not want to complicate bug-hunts by contaminating step-1 with bugs from step-2.
>>>>>>>
>>>>>>> [1] stage-1 proposal - JSON.parse source text access
>>>>>>> https://github.com/gibson042/ecma262-proposal-JSON-parse-with-source
>>>>>>>
>>>>>>> kai zhu
>>>>>>> [hidden email]
>>>>>>>
>>>>>>>
>>>>>>>
>>>>>>> _______________________________________________
>>>>>>> es-discuss mailing list
>>>>>>> [hidden email]
>>>>>>> https://mail.mozilla.org/listinfo/es-discuss
>>>>>>
>>>>>> _______________________________________________
>>>>>> es-discuss mailing list
>>>>>> [hidden email]
>>>>>> https://mail.mozilla.org/listinfo/es-discuss
>>>>>
>>>>> _______________________________________________
>>>>> es-discuss mailing list
>>>>> [hidden email]
>>>>> https://mail.mozilla.org/listinfo/es-discuss
>>>>
>>>> _______________________________________________
>>>> es-discuss mailing list
>>>> [hidden email]
>>>> https://mail.mozilla.org/listinfo/es-discuss
>
> _______________________________________________
> es-discuss mailing list
> [hidden email]
> https://mail.mozilla.org/listinfo/es-discuss
>
>
> _______________________________________________
> es-discuss mailing list
> [hidden email]
> https://mail.mozilla.org/listinfo/es-discuss
_______________________________________________
es-discuss mailing list
[hidden email]
https://mail.mozilla.org/listinfo/es-discuss