Implement jsdICompilationUnit

classic Classic list List threaded Threaded
22 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Implement jsdICompilationUnit

John J Barton
For Firefox 3.7 I want to push to get an extension to jsd that would
dramatically improve Firebug and improve Firefox's support for dynamic
Javascript. I am looking for more feedback on the approach I am taking:

  Bug 449464 -  Implement jsdICompilationUnit to extend jsd to include
information on the compilation unit structure.

https://bugzilla.mozilla.org/show_bug.cgi?id=449464

The bug has more justification and details. Broadly the API attempts to
mimic the current jsdIDebuggerService style by adding a new callback
hook, jsdICompilationHook and a new enumerate call,
enumerateCompiledScripts().  This makes the API narrow but idiosyncratic
(a good match to the current API ;-).

The implementation is "lightly salted": a few lines need to be added in
a lot of places. Every place JS can be compiled needs a call to the hook
and the hook-setting code needs to propagate up through the jsd layer
cake to the JS interface. Algorithmically the jsengine builds a simple
list of jsdIScripts threaded through JSDScript with the head in
JSDContext. If the compilation hook is taken, the list can be copied out
via enumerateCompiledScripts(), else it is not useful.

I would appreciate input. (I tried to set the followup here to
mozilla.dev.apps.js-debugger, but I know that group is not widely
followed so I've added platform and js-engine).

jjb
_______________________________________________
dev-apps-js-debugger mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-js-debugger
Reply | Threaded
Open this post in threaded view
|

Re: Implement jsdICompilationUnit

Boris Zbarsky
On 12/1/09 6:29 PM, John J Barton wrote:
> The implementation is "lightly salted": a few lines need to be added in
> a lot of places. Every place JS can be compiled needs a call to the hook
> and the hook-setting code needs to propagate up through the jsd layer
> cake to the JS interface. Algorithmically the jsengine builds a simple
> list of jsdIScripts threaded through JSDScript with the head in
> JSDContext. If the compilation hook is taken, the list can be copied out
> via enumerateCompiledScripts(), else it is not useful.

Hmm...  So does this involve firebug js running when the compile
happens?  I believe right now there are various places (event handlers,
xbl, etc) that assume that js compilation will not trigger random
script, and in particular won't spin the event loop.

-Boris
_______________________________________________
dev-apps-js-debugger mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-js-debugger
Reply | Threaded
Open this post in threaded view
|

Re: Implement jsdICompilationUnit

John J Barton
Boris Zbarsky wrote:

> On 12/1/09 6:29 PM, John J Barton wrote:
>> The implementation is "lightly salted": a few lines need to be added in
>> a lot of places. Every place JS can be compiled needs a call to the hook
>> and the hook-setting code needs to propagate up through the jsd layer
>> cake to the JS interface. Algorithmically the jsengine builds a simple
>> list of jsdIScripts threaded through JSDScript with the head in
>> JSDContext. If the compilation hook is taken, the list can be copied out
>> via enumerateCompiledScripts(), else it is not useful.
>
> Hmm...  So does this involve firebug js running when the compile
> happens?  I believe right now there are various places (event handlers,
> xbl, etc) that assume that js compilation will not trigger random
> script, and in particular won't spin the event loop.

I think the answer is "no", but it depends on you definition of "when
the compile happens". For the current Firebug purposes the hook can be
called any time after the last function compilation in the 'unit' and
before the first call to any function in the unit. (In fact we could
stretch and say this is the definition of a 'unit'; if we work out the
naming problem we can have every function in a file be a 'unit').
Typically JS compiles are immediately followed by invocation of the
outer function (file scope initializer or what ever JS calls it), so
typically there will be a clear position for the callback.

We also want events for 'compile start'/'compile end' in Firebug. We can
use this hook call as 'compile end' if it works out that all of the hook
calls are close in time to the end of compile activity. (This bit is to
create an eventual browser activity monitor feature).

jjb
_______________________________________________
dev-apps-js-debugger mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-js-debugger
Reply | Threaded
Open this post in threaded view
|

Re: Implement jsdICompilationUnit

Gijs Kruitbosch ("Hannibal")
In reply to this post by John J Barton
On 02/12/2009 00:29 AM, John J Barton wrote:

> For Firefox 3.7 I want to push to get an extension to jsd that would
> dramatically improve Firebug and improve Firefox's support for dynamic
> Javascript. I am looking for more feedback on the approach I am taking:
>
> Bug 449464 - Implement jsdICompilationUnit to extend jsd to include
> information on the compilation unit structure.
>
> https://bugzilla.mozilla.org/show_bug.cgi?id=449464
>
> The bug has more justification and details. Broadly the API attempts to
> mimic the current jsdIDebuggerService style by adding a new callback
> hook, jsdICompilationHook and a new enumerate call,
> enumerateCompiledScripts(). This makes the API narrow but idiosyncratic
> (a good match to the current API ;-).
>
> The implementation is "lightly salted": a few lines need to be added in
> a lot of places. Every place JS can be compiled needs a call to the hook
> and the hook-setting code needs to propagate up through the jsd layer
> cake to the JS interface. Algorithmically the jsengine builds a simple
> list of jsdIScripts threaded through JSDScript with the head in
> JSDContext. If the compilation hook is taken, the list can be copied out
> via enumerateCompiledScripts(), else it is not useful.
>
> I would appreciate input. (I tried to set the followup here to
> mozilla.dev.apps.js-debugger, but I know that group is not widely
> followed so I've added platform and js-engine).
>
> jjb

You didn't link this newsgroup post from the actual bug, so I hadn't seen it
before replying extensively there. I think it's very confusing that you keep
talking about "jsdICompilationUnit", which doesn't actually exist (neither in
your patch nor currently) -- at least, that confused my initial feedback. But
anyway, feedback is on the bug, no sense repeating it all here.

~ Gijs
_______________________________________________
dev-apps-js-debugger mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-js-debugger
Reply | Threaded
Open this post in threaded view
|

Re: Implement jsdICompilationUnit

Gijs Kruitbosch ("Hannibal")
In reply to this post by Boris Zbarsky
On 02/12/2009 03:34 AM, Boris Zbarsky wrote:

> On 12/1/09 6:29 PM, John J Barton wrote:
>> The implementation is "lightly salted": a few lines need to be added in
>> a lot of places. Every place JS can be compiled needs a call to the hook
>> and the hook-setting code needs to propagate up through the jsd layer
>> cake to the JS interface. Algorithmically the jsengine builds a simple
>> list of jsdIScripts threaded through JSDScript with the head in
>> JSDContext. If the compilation hook is taken, the list can be copied out
>> via enumerateCompiledScripts(), else it is not useful.
>
> Hmm... So does this involve firebug js running when the compile happens?
> I believe right now there are various places (event handlers, xbl, etc)
> that assume that js compilation will not trigger random script, and in
> particular won't spin the event loop.
>
> -Boris

I think at least Venkman works around these kinds of issues by making JSD enter
a nested event loop. I have no idea if Firebug does the same.

http://mxr.mozilla.org/mozilla-central/source/js/jsd/idl/jsdIDebuggerService.idl#373

~ Gijs
_______________________________________________
dev-apps-js-debugger mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-js-debugger
Reply | Threaded
Open this post in threaded view
|

Re: Implement jsdICompilationUnit

John J Barton
In reply to this post by Gijs Kruitbosch ("Hannibal")
Gijs Kruitbosch wrote:
> On 02/12/2009 00:29 AM, John J Barton wrote:
>> For Firefox 3.7 I want to push to get an extension to jsd that would
>> dramatically improve Firebug and improve Firefox's support for dynamic
>> Javascript. I am looking for more feedback on the approach I am taking:
>>
>> Bug 449464 - Implement jsdICompilationUnit to extend jsd to include
>> information on the compilation unit structure.
...
>
> You didn't link this newsgroup post from the actual bug, so I hadn't
>  I think it's very confusing
> that you keep talking about "jsdICompilationUnit", which doesn't
> actually exist (neither in your patch nor currently) -- at least, that
> confused my initial feedback. But anyway, feedback is on the bug, no
> sense repeating it all here.

Oops, sorry I change it to jsdICompilationHook
jjb
_______________________________________________
dev-apps-js-debugger mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-js-debugger
Reply | Threaded
Open this post in threaded view
|

Re: Implement jsdICompilationUnit

John J Barton
In reply to this post by Gijs Kruitbosch ("Hannibal")
Gijs Kruitbosch wrote:

> On 02/12/2009 03:34 AM, Boris Zbarsky wrote:
>> On 12/1/09 6:29 PM, John J Barton wrote:
>>> The implementation is "lightly salted": a few lines need to be added in
>>> a lot of places. Every place JS can be compiled needs a call to the hook
>>> and the hook-setting code needs to propagate up through the jsd layer
>>> cake to the JS interface. Algorithmically the jsengine builds a simple
>>> list of jsdIScripts threaded through JSDScript with the head in
>>> JSDContext. If the compilation hook is taken, the list can be copied out
>>> via enumerateCompiledScripts(), else it is not useful.
>>
>> Hmm... So does this involve firebug js running when the compile happens?
>> I believe right now there are various places (event handlers, xbl, etc)
>> that assume that js compilation will not trigger random script, and in
>> particular won't spin the event loop.
>>
>> -Boris
>
> I think at least Venkman works around these kinds of issues by making
> JSD enter a nested event loop. I have no idea if Firebug does the same.
>
> http://mxr.mozilla.org/mozilla-central/source/js/jsd/idl/jsdIDebuggerService.idl#373 
>
>
> ~ Gijs

Firebug does use the nested event loop, when it hits a breakpoint or
otherwise wants to stop the current control flow and allow user input to
direct subsequent flow.  Currently Firebug runs a lot of code while js
compilation is on-going, but I don't know if it every calls for a nested
event loop.

jjb
_______________________________________________
dev-apps-js-debugger mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-js-debugger
Reply | Threaded
Open this post in threaded view
|

Re: Implement jsdICompilationUnit

Boris Zbarsky
In reply to this post by John J Barton
On 12/2/09 12:33 AM, John J. Barton wrote:
> For the current Firebug purposes the hook can be
> called any time after the last function compilation in the 'unit' and
> before the first call to any function in the unit.

OK.

> Typically JS compiles are immediately followed by invocation of the
> outer function

The cases I mentioned are some of the cases where this is not in fact
the case.  XUL prototype scripts are another such case.

> We also want events for 'compile start'/'compile end' in Firebug. We can
> use this hook call as 'compile end' if it works out that all of the hook
> calls are close in time to the end of compile activity. (This bit is to
> create an eventual browser activity monitor feature).

I'm not sure what this paragraph is saying...

-Boris

_______________________________________________
dev-apps-js-debugger mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-js-debugger
Reply | Threaded
Open this post in threaded view
|

Re: Implement jsdICompilationUnit

Boris Zbarsky
In reply to this post by Gijs Kruitbosch ("Hannibal")
On 12/2/09 4:46 AM, Gijs Kruitbosch wrote:
> I think at least Venkman works around these kinds of issues by making
> JSD enter a nested event loop.

Yes, I'm aware.  The compile callsites I mentioned are places where
entering a nested event loop is a good way to crash exploitably.

-Boris
_______________________________________________
dev-apps-js-debugger mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-js-debugger
Reply | Threaded
Open this post in threaded view
|

Re: Implement jsdICompilationUnit

John J Barton
Boris Zbarsky wrote:
> On 12/2/09 4:46 AM, Gijs Kruitbosch wrote:
>> I think at least Venkman works around these kinds of issues by making
>> JSD enter a nested event loop.
>
> Yes, I'm aware.  The compile callsites I mentioned are places where
> entering a nested event loop is a good way to crash exploitably.
>
> -Boris

However that issue is orthogonal to the jsdICompilationHook proposal.
Today I could hook onScriptCreated and call jsd.enterNestedEventLoop().
The new API does not make this better or worse.

jjb
_______________________________________________
dev-apps-js-debugger mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-js-debugger
Reply | Threaded
Open this post in threaded view
|

Re: Implement jsdICompilationUnit

John J Barton
In reply to this post by Boris Zbarsky
Boris Zbarsky wrote:
>> We also want events for 'compile start'/'compile end' in Firebug. We can
>> use this hook call as 'compile end' if it works out that all of the hook
>> calls are close in time to the end of compile activity. (This bit is to
>> create an eventual browser activity monitor feature).
>
> I'm not sure what this paragraph is saying...

Independent of the compilation hook work, Firebug also wants to show
users where all of the browser time is going: reading,  parsing,
compiling, executing, rendering, waiting, .... compile start/end would
be part of that. We can do some of this already.

Sorry for the side track.
jjb
_______________________________________________
dev-apps-js-debugger mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-js-debugger
Reply | Threaded
Open this post in threaded view
|

Re: Implement jsdICompilationUnit

Gijs Kruitbosch ("Hannibal")
In reply to this post by Boris Zbarsky
On 02/12/2009 18:32 PM, Boris Zbarsky wrote:
> On 12/2/09 4:46 AM, Gijs Kruitbosch wrote:
>> I think at least Venkman works around these kinds of issues by making
>> JSD enter a nested event loop.
>
> Yes, I'm aware. The compile callsites I mentioned are places where
> entering a nested event loop is a good way to crash exploitably.
>
> -Boris

So... is that happening already? Are there bugs filed?

~ Gijs
_______________________________________________
dev-apps-js-debugger mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-js-debugger
Reply | Threaded
Open this post in threaded view
|

Re: Implement jsdICompilationUnit

Boris Zbarsky
In reply to this post by John J Barton
On 12/2/09 1:02 PM, John J. Barton wrote:
> However that issue is orthogonal to the jsdICompilationHook proposal.
> Today I could hook onScriptCreated and call jsd.enterNestedEventLoop().

Uh... That's not good.  Not good at all.  Fundamentally, I don't trust
"you" (whoever is hooking into jsd) to not do that.

-Boris
_______________________________________________
dev-apps-js-debugger mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-js-debugger
Reply | Threaded
Open this post in threaded view
|

Re: Implement jsdICompilationUnit

Boris Zbarsky
In reply to this post by Gijs Kruitbosch ("Hannibal")
On 12/2/09 1:43 PM, Gijs Kruitbosch wrote:
> So... is that happening already?

I have no idea.  You tell me.  I've never looked at either the Firebug
or venkman code in depth.

> Are there bugs filed?

No.  (I think I'm safe on that one.)

-Boris

_______________________________________________
dev-apps-js-debugger mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-js-debugger
Reply | Threaded
Open this post in threaded view
|

Re: Implement jsdICompilationUnit

John J Barton
In reply to this post by Boris Zbarsky
Boris Zbarsky wrote:
> On 12/2/09 1:02 PM, John J. Barton wrote:
>> However that issue is orthogonal to the jsdICompilationHook proposal.
>> Today I could hook onScriptCreated and call jsd.enterNestedEventLoop().
>
> Uh... That's not good.  Not good at all.  Fundamentally, I don't trust
> "you" (whoever is hooking into jsd) to not do that.
>
> -Boris

I think we must be miscommunicating.

To the best of my knowledge the Firebug code that runs in
onScriptCreated would never call jsd.enterNestedEventLoop(). Firebug
"could" call it, just like it "could" write "Happy Holidays" a billion
times on your hard drive. Hopefully you'll to trust us not to do either one.

And anyway, jsdICompilationHook just does not have this problem. It runs
after the compiles are completed, by design.

jjb
_______________________________________________
dev-apps-js-debugger mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-js-debugger
Reply | Threaded
Open this post in threaded view
|

Re: Implement jsdICompilationUnit

Boris Zbarsky
On 12/2/09 5:25 PM, John J Barton wrote:
> To the best of my knowledge the Firebug code that runs in
> onScriptCreated would never call jsd.enterNestedEventLoop(). Firebug
> "could" call it, just like it "could" write "Happy Holidays" a billion
> times on your hard drive. Hopefully you'll to trust us not to do either
> one.

I don't trust people to not accidentally call something that calls
something that spins an event loop, nested or otherwise.  This is based
on historical precedent where people do this by accident over and over.

> And anyway, jsdICompilationHook just does not have this problem. It runs
> after the compiles are completed, by design.

The point is that the compile callsite is the place that can't deal with
having the event loop spin....

-Boris
_______________________________________________
dev-apps-js-debugger mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-js-debugger
Reply | Threaded
Open this post in threaded view
|

Re: Implement jsdICompilationUnit

John J Barton
Boris Zbarsky wrote:
...
> The point is that the compile callsite is the place that can't deal with
> having the event loop spin....

Can you point me to the compile callsite?

jjb
_______________________________________________
dev-apps-js-debugger mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-js-debugger
Reply | Threaded
Open this post in threaded view
|

Re: Implement jsdICompilationUnit

Boris Zbarsky
Reply | Threaded
Open this post in threaded view
|

Re: Implement jsdICompilationUnit

John J Barton
Boris Zbarsky wrote:
So for these cases the text is compiled but no script is executed as far
as I can see. So I could bracket these calls with
beginCompile/endCompile and you'd be ok right?

>
>
> Not sure about event handlers; can point you to those too if desired.

The problem case call JS_EvaluateUCScriptForPrincipals(), because the
endCompile would need to be in that method because the outerScript is
called before the method returns. So if there are cases where your
compile call site passes through this guy its an issue.

Actually its an issue anyway, since I'd much rather have the begin/end
in the same function. Having the beginCompile in the caller and the
endCompile in some callee is asking for trouble.

jjb
_______________________________________________
dev-apps-js-debugger mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-js-debugger
Reply | Threaded
Open this post in threaded view
|

Re: Implement jsdICompilationUnit

Boris Zbarsky
On 12/8/09 6:05 PM, John J Barton wrote:
> So for these cases the text is compiled but no script is executed as far
> as I can see.

Yes...

> So I could bracket these calls with beginCompile/endCompile and you'd be ok right?

I'm not sure what you mean here.

> The problem case call JS_EvaluateUCScriptForPrincipals(), because the
> endCompile would need to be in that method because the outerScript is
> called before the method returns. So if there are cases where your
> compile call site passes through this guy its an issue.

Sure there are.  <script> elements, say.

-Boris
_______________________________________________
dev-apps-js-debugger mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-js-debugger
12