RFC: Change to tree closure policy regarding SeaMonkey perf regressions

classic Classic list List threaded Threaded
2 messages Options
Reply | Threaded
Open this post in threaded view
|

RFC: Change to tree closure policy regarding SeaMonkey perf regressions

J. Paul Reed

Hey all,

Sorry for the large cross posting.

Traditionally, build breakages and performance regressions affecting
SeaMonkey have, by policy, prompted the tree to close.

Over the past months, we've been working hard to retire old, unreliable
hardware that had been used to build Seamonkey's various branches; as a
result of this, we've put a bunch of SeaMonkey machines into virtual
machines.

As we've deployed new build resources for Seamonkey, these have been
images based on our reference platforms, which are also VMs.

With the resolution of bug 294182, allowing Seamonkey to turn Cairo on,
all of SeaMonkey's functioning Win32 and Linux tinderboxen are VMs.

Additionally, it turns out that most of the performance data coming in
about Seamonkey for the past few weeks is based on data from VMs, and is
therefore of less utility to track performance regressions.

As such, we're proposing that performance regressions that are
Seamonkey-specific will no longer close the tree. I'm told that this
seldom, if ever, happens (since performance regressions much more
commonly affect Firefox and Seamonkey together).

Seamonkey build breakages will continue under the same policy.

In some sense, this already seems to be a defacto policy, since the
performance data on Seamonkey builds has been from VMs for some time
now, and we've not heard any complaints.

Please let me know if there are any issues with making this the standard
policy.

Thanks!

Later,
preed
--
J. Paul Reed
Build/Release Engineer - The Mozilla Corporation
smtp://[hidden email]
irc://irc.mozilla.org/preed
pots://650.903.0800/x256
_______________________________________________
dev-performance mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-performance
Reply | Threaded
Open this post in threaded view
|

Re: RFC: Change to tree closure policy regarding SeaMonkey perf regressions

Robert Kaiser
J. Paul Reed schrieb:
> As such, we're proposing that performance regressions that are
> Seamonkey-specific will no longer close the tree. I'm told that this
> seldom, if ever, happens (since performance regressions much more
> commonly affect Firefox and Seamonkey together).
>
> Seamonkey build breakages will continue under the same policy.

Sounds good to me - and reminds me that it would probably be nice to get
  real testing boxes some time. (But I think you're still figuring out
details for those, right?)

Ah, and while we're on that topic: I think it's still good that
tinderboxen do some ad-hoc testing of the builds (Ts/Txul tests are
often good to detect broken code by turning orange), and the bloat and
codesize tests should be independent of actual machine performance, so
it would be nice to have at least one machine on our tinderbox page that
does them (and has working graphs).

Robert Kaiser
_______________________________________________
dev-performance mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-performance