Test harnesses, calendar testing

classic Classic list List threaded Threaded
4 messages Options
Reply | Threaded
Open this post in threaded view
|

Test harnesses, calendar testing

Dave Liebreich
Clint Talbert wrote (paraphrasing from a private email, moving the
conversation here with permission):

>  I'm trying to help organize the Lightning QA effort. Do you have some ideas about a regression testing framework for mozilla? Do you have experience with what works and what doesn't?

I know of a few frameworks.

I've been playing with jssh/jssh-driver and foxunit as test frameworks
for Firefox.  Both provide a mechanism to run js test code with chrome
privileges.

Allan Beaufour posted here earlier about using jsUnit for XForms
testing.  jsUnit tests run with content privs.

And you mentioned xpcshell - I moved the simple xpchell harness to its
own directory, and wrote one or two simple tests for it.  Right now, I
think only necko and the xml serializer have tests based on this code,
but it is very similar to jsDriver stuff.

Finally, there is the js test suite, maintained by Bob Clary.

> I've been exploring the possibilities of xpcshell, and it seems very useful. However, I am having issues getting it to work consistently from the make system.

What specific problems have you been having?

I think that if you post the code of a simple test case, we could pick a
harness and integrate the test while narrating the task here . . .

jssh: http://www.croczilla.com/jssh
jssh-driver, xpcshell-simple:
http://lxr.mozilla.org/mozilla/source/tools/test-harness/
js tests: http://lxr.mozilla.org/mozilla/source/js/tests/
--
Dave Liebreich
Test Architect, Mozilla Corporation
_______________________________________________
dev-quality mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-quality
Reply | Threaded
Open this post in threaded view
|

Re: Test harnesses, calendar testing

Clint Talbert-2
Dave Liebreich wrote:
> What specific problems have you been having?
>
They were build integration problems, the these regression tests are
supposed to run after a build, but they weren't. I figured it out.  It
turns out that it wasn't in the makefile, and that the proper targets
were not being called as you would expect.

> I think that if you post the code of a simple test case, we could pick a
> harness and integrate the test while narrating the task here . . .
>
Here is a very simple test case:
It is from
http://lxr.mozilla.org/seamonkey/source/calendar/test/homegrown/misc/simple-item-mutability.js

//
// simple test bits
//

const eventContractID = "@mozilla.org/calendar/event;1";
const eventIID = Components.interfaces.calIEvent;

dump ("Creating mutable event...\n");
var me = Components.classes[eventContractID].createInstance(eventIID);

me.title = "Test Title";
dump ("Title is: " + me.title + " (jsobj: " + me.wrappedJSObject.mTitle
+ ")\n");
if (me.title != "Test Title") {
    throw("FAILED!  Title on mutable event contained unexpected value.");
}

dump ("Setting event-to be non-mutable...\n");
me.makeImmutable();

dump ("Trying to set title (should fail)\n");
try {
    // this should cause an exception
    me.title = "Fail";

    // so we'll only get here if something is wrong, and there's no
exception
    dump("FAILED\n!");
} catch (ex) {
   }

if (me.title != "Test Title") {
     throw("FAILED!  Title on immutable event contained unexpected value: "
         + me.title);
}

Thanks for the help!

Clint
_______________________________________________
dev-quality mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-quality
Reply | Threaded
Open this post in threaded view
|

Re: Test harnesses, calendar testing

Dave Liebreich
Clint,

This is really cool.  I'm just now checking out a tree to run this code.
  Will the trunk work, or should I use a particular branch?

This test seems like a good example, since it sits right on the edge
between external and internal interfaces.  We could move this type of
test closer to the user and generate complex story or use-case tests.
Or we could cross into the component itself and generate unit-style tests.

Our tinderbox scripts support something called FileBasedTest that looks
at stdout instead of return codes.  I'm going to try to adapt
mozilla/tools/test-harness/xpcshell-simple so that it can be invoked by
FileBasedTest, then plug your test into it.

--
Dave Liebreich
Test Architect, Mozilla Corporation
_______________________________________________
dev-quality mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-quality
Reply | Threaded
Open this post in threaded view
|

Re: Test harnesses, calendar testing

Dave Liebreich
Dave Liebreich wrote:
> Clint,
>
> This is really cool.  I'm just now checking out a tree to run this code.
>  Will the trunk work, or should I use a particular branch?

I've got a tree, and I can run this test.  It passes :-)

Though now I wonder about the history of the event loop stuff in
mozilla/calendar/test/homegrown/shell.js compared to the history of the
event loop stuff in mozilla/netwerk/test/unit/head.js (which now lives
in mozilla/tools/test-harness/xpcshell-simple/head.js).  I think darin
is out of town - maybe you can ask dmose?

Regardless, I've got some ideas of what I'd like to do next.  I'll post
again when I've got some new code.
--
Dave Liebreich
Test Architect, Mozilla Corporation
_______________________________________________
dev-quality mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-quality