Profiler in Callgrind format

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|

Profiler in Callgrind format

"Sebastián" Marconi
Hello,
I'm a co-developer of (yet another) a web framework...Being mainly a
server-side programmer, after profiling the execution using Venkman
(and now also the great firebug1.0 feature) I miss the visualization
quality of tools like KCacheGrind or WinCacheGrind that uses the
Callgrind Format. We have a lot of classes and complex (ajax) things,
and the tabular output doesn't fit very well  (this kind of framework
may be useless, but there's where we and thousands are) you can't
follow specific threads of execution, you don't know who the callers
and calles are, even the memory consumption is missed.

Seeking this goal, I've started to develop an extension using
jsdIDebuggerService, but it's not enough because for callgrind you need
a step-by-step execution (horrible for performance) and jsdlDebugger
gives you pre-processed information. What do you think? Is it possible
to write an specific XPCOM component or improve the actual one?

I'm not blaming the current work, works very well for nearly all the
projects, but I think that a more powerfull profiler is needed in the
today's javascript. Am I crazy?

Sorry for my english. I'm trying to improve it.

Regards,
Sebastián Marconi

Links:
 - Kcachegrind: http://kcachegrind.sourceforge.net/cgi-bin/show.cgi
 - WinCachegrind: http://sourceforge.net/projects/wincachegrind/
 - Callgrind format:
http://kcachegrind.sourceforge.net/cgi-bin/show.cgi/KcacheGrindCalltreeFormat

_______________________________________________
dev-apps-js-debugger mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-js-debugger
Reply | Threaded
Open this post in threaded view
|

Re: Profiler in Callgrind format

James Ross
[hidden email] wrote:

> Hello,
> I'm a co-developer of (yet another) a web framework...Being mainly a
> server-side programmer, after profiling the execution using Venkman
> (and now also the great firebug1.0 feature) I miss the visualization
> quality of tools like KCacheGrind or WinCacheGrind that uses the
> Callgrind Format. We have a lot of classes and complex (ajax) things,
> and the tabular output doesn't fit very well  (this kind of framework
> may be useless, but there's where we and thousands are) you can't
> follow specific threads of execution, you don't know who the callers
> and calles are, even the memory consumption is missed.

Memory consumption is neigh on impossible to calculate with SpiderMonkey
because it shares all sorts of objects, not only between things in your
scripts, but often with the host application.

Following individual execution paths is indeed a limitation of the
current profiling code which is, in all fairness, exceptionally simple.
It certainly wasn't meant to care who calls who, and adding such would
be (probably) a rewrite of most of the existing profiling code.

Different layouts of the data we do have might well be very useful,
though. :)

> Seeking this goal, I've started to develop an extension using
> jsdIDebuggerService, but it's not enough because for callgrind you need
> a step-by-step execution (horrible for performance) and jsdlDebugger
> gives you pre-processed information. What do you think? Is it possible
> to write an specific XPCOM component or improve the actual one?

What do you mean "pre-processed"? Are you referring to the access to
profiling data? The debugger service is quite sufficient to single-step
through all the code, although it might be a big performance hit (hardly
surprising). You'd have to do all the work tracking profile data.

I don't know how feasible trying to do your own XPCOM components for JSD
is - you'd have to effectively replace the entire thing, I think, as the
profiling is all down in the C code and not in the XPCOM interface
(jsd_xpc.cpp) that exposes it to the world.

> I'm not blaming the current work, works very well for nearly all the
> projects, but I think that a more powerfull profiler is needed in the
> today's javascript. Am I crazy?

Only as crazy as the rest of us. ;)

If you would like to see any of your ideas in JSD in the future, you
should file bug reports at https://bugzilla.mozilla.org/ (Product: Other
Applications, Component: JavaScript Debugger). Nothing will happen
overnight, but it's always better to know what is actually wanted than
to guess and get it wrong.

--
James Ross <[hidden email]>
ChatZilla Developer
_______________________________________________
dev-apps-js-debugger mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-js-debugger
Reply | Threaded
Open this post in threaded view
|

Re: Profiler in Callgrind format

timeless-3
In reply to this post by "Sebastián" Marconi
On Dec 12, 11:19 pm, "[hidden email]"
<[hidden email]> wrote:
> I'm a co-developer of (yet another) a web framework...Being mainly a
> server-side programmer, after profiling the execution using Venkman
> (and now also the great firebug1.0 feature) I miss the visualization
> quality of tools like KCacheGrind or WinCacheGrind that uses the
> Callgrind Format. We have a lot of classes and complex (ajax) things,
> and the tabular output doesn't fit very well  (this kind of framework
> may be useless, but there's where we and thousands are) you can't
> follow specific threads of execution, you don't know who the callers
> and calles are, even the memory consumption is missed.

There's a patch floating around that allows you to watch object size
and such. It's in a bug reporter=timeless,assignee=timeless

It basically works and just needs someone to help me drive it into cvs
(i don't actively drive bugs, i write patches and move on to other
problems).

As for your core problem, well, writing something that single steps
using jsdI shouldn't be impossible, although I'd expect it not to be
much fun, and I'd recommend you configure it to ignore scripts it isn't
supposed to profile so that your system has some chance of running.

Note that James Ross essentially owns the Venkman UI. He's essentially
more of an owner than I am. I can be seen as having adopted jsdb which
is a GUI free tool, on average atm when it comes to debugging stuff
i've hacked at lower layers. (Which isn't to say that i haven't hacked
Venkman - because whenever I try to say I don't do something it turns
me into a liar.)

_______________________________________________
dev-apps-js-debugger mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-js-debugger
Reply | Threaded
Open this post in threaded view
|

Re: Profiler in Callgrind format

"Sebastián" Marconi
On Dec 15, 6:46 pm, James Ross <[hidden email]> wrote:
> [hidden email] wrote:
> What do you mean "pre-processed"? Are you referring to the access to
> profiling data? The debugger service is quite sufficient to single-step
> through all the code, although it might be a big performance hit (hardly
> surprising). You'd have to do all the work tracking profile data.
Sorry, I meant summarized times, the kind of information that
jsdIScript gives (callCount, max, min, total).
The callgrind format needs raw data (a lot of data). Let us suppose
that it's possible to observe the SpiderMonkey in every call, so the
caller/callee/time can be stored in memory during the execution. When
the profiling ends, that information can be translated to the callgrind
format. In my limited view, it doesn't seems very difficult! (too much
php here).

> I don't know how feasible trying to do your own XPCOM components for JSD
> is - you'd have to effectively replace the entire thing, I think, as the
> profiling is all down in the C code and not in the XPCOM interface
> (jsd_xpc.cpp) that exposes it to the world.a
Thanks for the advise. I'm not a C-C++ expert by a long shot, but I'm
very excited with the mozilla framework as a whole, first I'll try to
use the step-by-step feature of JSD not worrying about the performance,
then... we will see....

> If you would like to see any of your ideas in JSD in the future, you
> should file bug reports athttps://bugzilla.mozilla.org/(Product: Other
> Applications, Component: JavaScript Debugger). Nothing will happen
> overnight, but it's always better to know what is actually wanted than
> to guess and get it wrong.
Ok, I'll do my homework in the next months.

On Dec 17, 3:54 am, "timeless" <[hidden email]> wrote:
> As for your core problem, well, writing something that single steps
> using jsdI shouldn't be impossible, although I'd expect it not to be
> much fun, and I'd recommend you configure it to ignore scripts it isn't
> supposed to profile so that your system has some chance of running.
Is there any way in JSD to focus only some scripts? I mean: only track
information about the scripts in current window content. Viewing
firebug 1.0 beta, I understand that the profiler tracks all open
scripts, in the visualization all the <script> tags are searched and
they src are compared with the fileName that jsdIScript gives. How
Venkman/JSD do that? Can I give it a window content to watch?

Thanks James and timeless for the feedback.

PS: Have you ever tried KCacheGrind? We use it in PHP, with de xdebug
extension.

_______________________________________________
dev-apps-js-debugger mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-js-debugger
Reply | Threaded
Open this post in threaded view
|

Re: Profiler in Callgrind format

James Ross
[hidden email] wrote:

> On Dec 15, 6:46 pm, James Ross <[hidden email]> wrote:
>> [hidden email] wrote:
>> What do you mean "pre-processed"? Are you referring to the access to
>> profiling data? The debugger service is quite sufficient to single-step
>> through all the code, although it might be a big performance hit (hardly
>> surprising). You'd have to do all the work tracking profile data.
> Sorry, I meant summarized times, the kind of information that
> jsdIScript gives (callCount, max, min, total).
> The callgrind format needs raw data (a lot of data). Let us suppose
> that it's possible to observe the SpiderMonkey in every call, so the
> caller/callee/time can be stored in memory during the execution. When
> the profiling ends, that information can be translated to the callgrind
> format. In my limited view, it doesn't seems very difficult! (too much
> php here).

Just so we don't all confuse each other too much, the stack for
debugging in Venkman goes something like this:

   - Venkman UI (XUL, JS, etc.)
   - JSD XPC layer (jsd_xpc.cpp) - essentially just wraps JSD for JS users
   - JSD core (jsd_*.c) - the real works
   - SpiderMonkey

SpiderMonkey calls into JSD for every function call (if JSD asks it to),
which ends up (eventually) in this function:

http://bonsai.mozilla.org/cvsblame.cgi?file=mozilla/js/jsd/jsd_step.c&rev=3.16#111

That is where the profile data is collected. If I understand you right,
you just want all the data that code collects to be kept, rather than
collated into min/max/ave data.

That code does include calls up through the XPC layer and into Venkman
(or whatever front-end is attached to JSD at the time), so I would
surmise that the basis of what you want is doable even from the JS
level, but I'm not sure what the best way to get the times out without
doing your own...

> On Dec 17, 3:54 am, "timeless" <[hidden email]> wrote:
>> As for your core problem, well, writing something that single steps
>> using jsdI shouldn't be impossible, although I'd expect it not to be
>> much fun, and I'd recommend you configure it to ignore scripts it isn't
>> supposed to profile so that your system has some chance of running.
> Is there any way in JSD to focus only some scripts? I mean: only track
> information about the scripts in current window content. Viewing
> firebug 1.0 beta, I understand that the profiler tracks all open
> scripts, in the visualization all the <script> tags are searched and
> they src are compared with the fileName that jsdIScript gives. How
> Venkman/JSD do that? Can I give it a window content to watch?

Venkman can tell you what scripts are in a window, and there are some
known bits of JS (that I want to add) that'll let it go from script back
to the window it is in. In terms of profiling, though, JSD works on a
global toggle and a per-script one - I believe you could tell JSD not to
profile anything, then flag the scripts you care about. There's also a
filter you can set, but I don't believe Venkman uses this, and I don't
know how to use it either.

--
James Ross <[hidden email]>
ChatZilla Developer
_______________________________________________
dev-apps-js-debugger mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-js-debugger