Clint Talbert wrote (paraphrasing from a private email, moving the
conversation here with permission):
> I'm trying to help organize the Lightning QA effort. Do you have some ideas about a regression testing framework for mozilla? Do you have experience with what works and what doesn't?
I know of a few frameworks.
I've been playing with jssh/jssh-driver and foxunit as test frameworks
for Firefox. Both provide a mechanism to run js test code with chrome
Allan Beaufour posted here earlier about using jsUnit for XForms
testing. jsUnit tests run with content privs.
And you mentioned xpcshell - I moved the simple xpchell harness to its
own directory, and wrote one or two simple tests for it. Right now, I
think only necko and the xml serializer have tests based on this code,
but it is very similar to jsDriver stuff.
Finally, there is the js test suite, maintained by Bob Clary.
> I've been exploring the possibilities of xpcshell, and it seems very useful. However, I am having issues getting it to work consistently from the make system.
What specific problems have you been having?
I think that if you post the code of a simple test case, we could pick a
harness and integrate the test while narrating the task here . . .
Dave Liebreich wrote:
> What specific problems have you been having?
They were build integration problems, the these regression tests are
supposed to run after a build, but they weren't. I figured it out. It
turns out that it wasn't in the makefile, and that the proper targets
were not being called as you would expect.
This is really cool. I'm just now checking out a tree to run this code.
Will the trunk work, or should I use a particular branch?
This test seems like a good example, since it sits right on the edge
between external and internal interfaces. We could move this type of
test closer to the user and generate complex story or use-case tests.
Or we could cross into the component itself and generate unit-style tests.
Our tinderbox scripts support something called FileBasedTest that looks
at stdout instead of return codes. I'm going to try to adapt
mozilla/tools/test-harness/xpcshell-simple so that it can be invoked by
FileBasedTest, then plug your test into it.
Dave Liebreich wrote:
> This is really cool. I'm just now checking out a tree to run this code.
> Will the trunk work, or should I use a particular branch?
I've got a tree, and I can run this test. It passes :-)
Though now I wonder about the history of the event loop stuff in
mozilla/calendar/test/homegrown/shell.js compared to the history of the
event loop stuff in mozilla/netwerk/test/unit/head.js (which now lives
in mozilla/tools/test-harness/xpcshell-simple/head.js). I think darin
is out of town - maybe you can ask dmose?