stylo reftest-stylo jobs are green

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

stylo reftest-stylo jobs are green

Cameron McCormack-4
Thanks to Shing's manifest updating script which he ran recently, and
some work to manually disable tests and update failure/assertion
annotations, and modulo some current leaks, the reftest-stylo jobs are
now green.  As we are doing with the crashtest job, let's try to keep
them green, by including reftest-stylo in the try pushes we do before
landing Stylo-affecting patches.

Remember that the reftest-stylo job is the one that uses the
reftest-stylo.list manifest files, and which compares a reftest test
file with Stylo enabled and disabled.  So if you need to update the
manifests as part of the patches you land, it's the reftest-stylo.list
ones you need to change.

Because the reftest-stylo.list manifests were originally generated by
Shing's scripts, which disabled crashing/intermittent tests and (I
think?) added assertion annotations automatically, we mostly don't have
comments in the manifests pointing to bugs for the underlying reason the
test is failing.  If, when you're updating annotations as part of your
changes, you notice a new crash or assertion failure pattern, please
file a bug blocking bug 1324620.  For now, I don't think we need to
bother filing bugs for reftest image comparison failures.
_______________________________________________
dev-tech-layout mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-layout
Reply | Threaded
Open this post in threaded view
|

Re: stylo reftest-stylo jobs are green

Bobby Holley-2
\o/ \o/ \o/

On Wed, Feb 1, 2017 at 3:13 AM, Cameron McCormack <[hidden email]> wrote:

> Thanks to Shing's manifest updating script which he ran recently, and
> some work to manually disable tests and update failure/assertion
> annotations, and modulo some current leaks, the reftest-stylo jobs are
> now green.  As we are doing with the crashtest job, let's try to keep
> them green, by including reftest-stylo in the try pushes we do before
> landing Stylo-affecting patches.
>
> Remember that the reftest-stylo job is the one that uses the
> reftest-stylo.list manifest files, and which compares a reftest test
> file with Stylo enabled and disabled.  So if you need to update the
> manifests as part of the patches you land, it's the reftest-stylo.list
> ones you need to change.
>
> Because the reftest-stylo.list manifests were originally generated by
> Shing's scripts, which disabled crashing/intermittent tests and (I
> think?) added assertion annotations automatically, we mostly don't have
> comments in the manifests pointing to bugs for the underlying reason the
> test is failing.  If, when you're updating annotations as part of your
> changes, you notice a new crash or assertion failure pattern, please
> file a bug blocking bug 1324620.  For now, I don't think we need to
> bother filing bugs for reftest image comparison failures.
> _______________________________________________
> dev-tech-layout mailing list
> [hidden email]
> https://lists.mozilla.org/listinfo/dev-tech-layout
>
_______________________________________________
dev-tech-layout mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-layout