Scheduling a trunk alpha

classic Classic list List threaded Threaded
96 messages Options
12345
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha

beltzner
On 4/12/06, L. David Baron <[hidden email]> wrote:

> On Wednesday 2006-04-12 13:10 -0700, Chris Hofmann wrote:
> > Its going to be tough to get any widespread testing of trunk snapshot
> > builds named "minefield"...  I think that code name, or
> > characterization, or branding for trunk builds is the wrong direction to
> > be heading in.   We should be continuing to encourage and create a large
> > development and testing community of  10,000 to 20,000 or more to help
> > out in keeping the trunk as stable as possible at all times.  We should
> > be and keeping destabilizing changes away from the trunk until they are
> > baked and ready to land.  minefield sends the wrong message if these are
> > still our goals.  This becomes harder the more branches we are working
> > on in parallel, but I still think it is a goal.
>
> Agreed.
>
> And what's pissed me off about our release process ever since Firefox
> became the flagship app is that the Firefox leads have insisted on
> controlling the Firefox release process as if it weren't also the
> primary testing and release vehicle for Gecko.
>
> This is why, following Firefox 1.0, I sat down with Ben and many others
> to develop a plan that would allow Firefox to really become the flagship
> Gecko app:
> http://groups.google.com/group/netscape.public.mozilla.seamonkey/msg/0883b9b35d3400af
> Part of this Firefox 1.1 (later 1.5) plan involved shipping a
> Firefox-based Gecko alpha in early January of 2005.  The agreement by
> the Firefox leads to that 1.1 plan was the only reason I accepted
> dropping the suite as the flagship Gecko app.
>
> That plan was nowhere close to being met, and it now seems like the
> current Firefox leads aren't even willing to accept its rationale --
> that Gecko needs to ship reasonably frequent Web browser alphas to tens
> or hundreds of thousands of users so that regressions in handling of Web
> pages are found.  This requires more users than most other types of
> testing-by-alpha because most Web pages are used by only a tiny portion
> of Web users.

So we get tens of thousands of people testing trunk builds. And then
we get hundreds of thousands of more casual users testing product
alphas and betas. And when the product releases start building off of
trunk, the number of testers on trunk jumps up nicely to coincide with
the point in time where trunk code is ready to expand to a wider
testing audience.

Win-win, isn't it?

> If Firefox leads aren't comfortable with what being the flagship Gecko
> app means, then I think we need to restore the Mozilla Suite to that
> role -- not because I like the suite, but because we need a Web browser
> that we can release to get Gecko testing.

What do you mean when you say "release"? What's involved there in
terms of engineering, publicity and socialization?

cheers,
mike

--

[ mike beltzner / user experience lead / mozilla corporation ]
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha

beltzner
In reply to this post by Boris Zbarsky
On 4/13/06, Boris Zbarsky <[hidden email]> wrote:
> Mike Beltzner wrote:
> > I think the issue we need to worry about is splitting our testing
> > community: should they be testing the browser using trunk or branch?
>
> Some of both, unfortunately.  I thought part of the point of keeping the
> UI in sync was that trunk testing happening right now would still benefit
> Firefox 2, so we could try to move testing resources to trunk as needed.

I think it was mostly to reduce the pain of merging back to trunk from
a branch, as was experienced with aviary->gecko 1.8. That was the
rumour I heard or read off a bathroom wall or something, anyway ;)

> > In the long term, as both
> > you and dbaron mention, we need eyes on the platform related stuff.
>
> That long term is now, imo.  Again, it's been 8 months since we
> branched...  How much longer term could we really get?  ;)

You don't actually want an answer to that question, do you?

> Back to scheduling, if we _can_ do the trunk release at about the same
> time as a branch release then you're probably right that the UI should be
> fairly non-busted.  So what does the branch release schedule look like?

We're targetting Bon Echo Alpha 2 at May 9th at the moment, with code
freeze on May 5th. There's a rough schedule for B1, B2 and the RCs at
http://wiki.mozilla.org/Firefox2/Schedule

cheers,
mike
--

[ mike beltzner / user experience lead / mozilla corporation ]
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha

Boris Zbarsky
In reply to this post by Benjamin Smedberg
Mike Beltzner wrote:
> I mean that I don't know where the set of things that are going into
> trunk is being catalogued

http://wiki.mozilla.org/Global:1.9_Trunk_1.8_Branch_Plan has some info, but I
agree that this list is not being maintained.  I'll try to set up a wiki page
with the list I have compiled so far and whatever "owner", status, and date info
I can come up with...  That's just in terms of the code.

> nor do I know where I can get a sense of what the targets are for alphas, betas, and release candidates for the
> trunk. Nor, really, do I know how those releases will be expressed,
> but that seems to be precisely what we're talking about atm :)

Right.

> So I'm confused about the goal: do we want to publish an alpha
> release, where we set firm goals about what will or won't be in that
> release, and scope priorities in terms of moving the trunk from one
> milestone to the next, or do we want to simply ensure that there's a
> stable state for the trunk

I don't think we can do the latter without doing the former for trunk work.
Otherwise we'll just have big landings happening just about as we stabilize from
the previous landing.  So yes, I think we need to ensure a stable state for the
trunk and to that end we need firm goals about what will be in that stable state
(and what won't be, probably).   Getting this defined was one of the things I
wanted to come out of this thread.

> I think we all agree
> that calling it "Firefox 3 Alpha" isn't the right thing to do, since
> that's gonna get misinterpreted by the general public.

Well, it's not clear that everyone agreed, but I'm personally ok with calling it
something else (I stand by my "Firefox 3 Developer Preview" suggestion).  Unless
your problem is with the "3" in there?

-Boris
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha

L. David Baron
In reply to this post by beltzner
On Thursday 2006-04-13 01:01 -0400, Mike Beltzner wrote:
> > That plan was nowhere close to being met, and it now seems like the
> > current Firefox leads aren't even willing to accept its rationale --
> > that Gecko needs to ship reasonably frequent Web browser alphas to tens
> > or hundreds of thousands of users so that regressions in handling of Web
> > pages are found.  This requires more users than most other types of
> > testing-by-alpha because most Web pages are used by only a tiny portion
> > of Web users.
>
> So we get tens of thousands of people testing trunk builds. And then

Whatever the number we have testing trunk builds, it isn't enough.  (And
it sounds like it might be ten thousand, but not tens of thousands.)  We
got too many very-old regressions filed once Firefox 1.5 started
shipping alphas and betas, and it's probably going to happen again (and
perhaps even worse) for 3.0, at the rate we're going.

Regression bug reports are much more useful when the code is still fresh
in its authors' mind, and when there's enough time to fix them before
the release.

> we get hundreds of thousands of more casual users testing product
> alphas and betas.

Not at any reasonable interval.

> And when the product releases start building off of
> trunk, the number of testers on trunk jumps up nicely to coincide with
> the point in time where trunk code is ready to expand to a wider
> testing audience.

We need it to hit a wider testing audience more often than we do now.

And I don't see why you think the trunk Gecko code becomes more ready to
expand to a wider testing audience when Firefox has met its
product-planned goals.

> Win-win, isn't it?

No, and I think you already knew I thought not.

-David

--
L. David Baron                                <URL: http://dbaron.org/ >
           Technical Lead, Layout & CSS, Mozilla Corporation

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning

attachment0 (198 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha

beltzner
On 4/13/06, L. David Baron <[hidden email]> wrote:

> On Thursday 2006-04-13 01:01 -0400, Mike Beltzner wrote:
> > > That plan was nowhere close to being met, and it now seems like the
> > > current Firefox leads aren't even willing to accept its rationale --
> > > that Gecko needs to ship reasonably frequent Web browser alphas to tens
> > > or hundreds of thousands of users so that regressions in handling of Web
> > > pages are found.  This requires more users than most other types of
> > > testing-by-alpha because most Web pages are used by only a tiny portion
> > > of Web users.
> >
> > So we get tens of thousands of people testing trunk builds. And then
>
> Whatever the number we have testing trunk builds, it isn't enough.  (And
> it sounds like it might be ten thousand, but not tens of thousands.)  We
> got too many very-old regressions filed once Firefox 1.5 started
> shipping alphas and betas, and it's probably going to happen again (and
> perhaps even worse) for 3.0, at the rate we're going.
[...]
> Regression bug reports are much more useful when the code is still fresh
> in its authors' mind, and when there's enough time to fix them before
> the release.

You're coming up with a lot of great points for why we need more
testing on trunk, and I'm sold on the idea, but I don't see any
counter-proposals to the solutions that I'm putting forward. Are you
proposing that we do frequent releases of Firefox off the trunk? What
would constitute such a release? Why do you expect that doing so would
get more people using that version for their web browsing?

The way I see it, there are a couple of options:

1. Firefox 2 releases as planned. Simultaneously, stabilize trunk such
that a set of solid builds can be made at some regular interval (let's
say monthly?) and then call those builds "latest-stable-trunk" or
something and socialize the need for testing and how these builds are
good enough for day to day use.

2. Firefox 2 releases as planned. Quickly build out a set of milestone
dates for Firefox 3 (with several alphas) and start releasing those as
well, perhaps initially synchronized to the Firefox 2 release
schedule. Or maybe it would be better to stagger the releases such
that our testers could use one for a month, then another for a month,
etc.

3. Make Firefox 3 releases the "flagship" alphas which we ask people
to use, since any front-end bugs will be discovered from here, too.
When the front-end stuff reaches beta level, then continue publishing
alphas from trunk, but also release Firefox 2 betas based on the 1.8
branch.

> > we get hundreds of thousands of more casual users testing product
> > alphas and betas.
>
> Not at any reasonable interval.
>
> > And when the product releases start building off of
> > trunk, the number of testers on trunk jumps up nicely to coincide with
> > the point in time where trunk code is ready to expand to a wider
> > testing audience.
>
> We need it to hit a wider testing audience more often than we do now.

Just "releasing" something doesn't guarantee a wider test audience.
Were I not in the know, I'd have no idea if I should be using Firefox
2 or Firefox 3. Also, I want people paying attention and submitting
bugs on the front end changes as well; you seem to be implying that
these code changes are less risky/worth testing than trunk changes.

> And I don't see why you think the trunk Gecko code becomes more ready to
> expand to a wider testing audience when Firefox has met its
> product-planned goals.

Didn't mean to imply that, sorry, my bad. What I meant to imply was
that if we assume that the number of testers is a finite resource, and
if we have to prioritize where their attention should go, then I would
suggest that we try to focus that resource more on our n+1 product
than on our n+2 initially, and then shift it as we get close to
releasing n+1.

> > Win-win, isn't it?
>
> No, and I think you already knew I thought not.

Believe it or not, I was actually hoping that we might have been
moving towards some sort of constructive solution. Apparently I'm
still new enough to have that kind of optimism.

cheers,
mike

--

[ mike beltzner / user experience lead / mozilla corporation ]
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha

L. David Baron
On Thursday 2006-04-13 02:51 -0400, Mike Beltzner wrote:

> The way I see it, there are a couple of options:
>
> 1. Firefox 2 releases as planned. Simultaneously, stabilize trunk such
> that a set of solid builds can be made at some regular interval (let's
> say monthly?) and then call those builds "latest-stable-trunk" or
> something and socialize the need for testing and how these builds are
> good enough for day to day use.
>
> 2. Firefox 2 releases as planned. Quickly build out a set of milestone
> dates for Firefox 3 (with several alphas) and start releasing those as
> well, perhaps initially synchronized to the Firefox 2 release
> schedule. Or maybe it would be better to stagger the releases such
> that our testers could use one for a month, then another for a month,
> etc.
>
> 3. Make Firefox 3 releases the "flagship" alphas which we ask people
> to use, since any front-end bugs will be discovered from here, too.
> When the front-end stuff reaches beta level, then continue publishing
> alphas from trunk, but also release Firefox 2 betas based on the 1.8
> branch.
I'm happy with any of these, at least assuming (1) would be done in a
prominent enough way that it gets users.  I expect others would be
unhappy with (3), and I wasn't pushing for it.

> > > Win-win, isn't it?
> >
> > No, and I think you already knew I thought not.
>
> Believe it or not, I was actually hoping that we might have been
> moving towards some sort of constructive solution. Apparently I'm
> still new enough to have that kind of optimism.

Then I must have misread your message:  when you wrote "we get", I
thought you were describing the present situation, but I think you
actually were describing the result of the proposed changes (for which I
would have used "we would get").

-David

--
L. David Baron                                <URL: http://dbaron.org/ >
           Technical Lead, Layout & CSS, Mozilla Corporation

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning

attachment0 (198 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha

Robert Kaiser
In reply to this post by Benjamin Smedberg
Mike Beltzner schrieb:
> What I'm getting at is that I feel like we need to
> shift our way of thinking about things so that people developing the
> trunk aren't forced to do so *only* in service of Firefox. So the
> platform is considered to be its own product, and apps built off of it
> like Firefox will pick stable-ish points from which to cut branches
> that will become the base of their releases. This nets out to mostly a
> semantic difference, I guess.

In a world where apps are all created on top of identical XULRunner
builds (and I think we agree that this should be the future for all of
us), the model we are working towards works fine, basically as you
described:

The platform (under the name of XULRunner) gets branched for a stable
release at some point, and apps do their stabilizing on the same branch
possibly (touching only their app-specific directory, i.e. browser/,
mail/, suite/, composer/, etc.) and are able to release off a point
that's declared "stable" by XULRunner developers.
Of course, they can always release at a later point using the same
stable XULRunner state/release, e.g. by doing a minibranch (client.mk +
their app-specific dir) off the release tag/minibranch/branch.

Releasing platform/XULRunner alphas (directly off trunk, probably
without the effort of branching) in between should be very much
scheduled (monthly, date-versioned?), so that any apps that want to
release own pre-releases from such a stable-ish point can either try to
stabilize at the same point or do their own minibranch based on that
platform alpha tree. For sure, the projects should know when they can
schedule for themselves to get to such a point (give or take a few days,
of course).

Still, the platform/XULRunner alphas need some "tesing instrument" which
is a showcase for the platform, and esp. for Gecko, so releasing a
"Gecko Browser" (i.e. Firefox) xulapp along with it would be a good idea.

At least, that sounds good for the glorious future where XULRunner is a
well-defined common base for all our apps. In the mean time, where we
still have no single app that's fully XULRunner-based and ready for a
preview-release, we can work towards such a sheme though, and use all
elements of it that do already fit.
Jugding from how we are/were dealing with releases off the 1.8.0 branch
(esp. Firefox/Thunderbird and SeaMonkey), we are not quite as far from
that model as one might think - though it needs much more coodination
with affected groups as it probably will in the fully XULRunner-based
world (though coordination and communication between projects and
platform is always a good idea).

Robert Kaiser
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha

Jonas Sicking
In reply to this post by Boris Zbarsky
A thought just occurred to me (you just know this is gonna be bad, don't
you)

The primary reason we're releasing alphas and betas is that we want
people to test stuff with it. Like we want web-developers to test that
their site works, and we want extension-developers to test that their
extensions still work.

However, a lot of people are vary of installing an alpha version that
might wreck their bookmarks, eat their preferences and stomp on their
cookies.

So I wonder if it might make sense to create a distro that is more
sandboxed and is less likely to screw up their day-to-day firefox install.

In fact, there already is such a distro: Portable firefox.

If we could put portable versions of our releases right visible on the
alpha page and in the alpha announcements, people might be more willing
to give it a go.

We could even make portable firefox to optionally import the config
files of an existing install.

What do people think? Is it worth the effort? Maybe this is more
important to the later alphas?

/ Jonas
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha

Jonas Sicking
In reply to this post by Boris Zbarsky
Boris Zbarsky wrote:
> Mike Beltzner wrote:
>> I think the issue we need to worry about is splitting our testing
>> community: should they be testing the browser using trunk or branch?
>
> Some of both, unfortunately.  I thought part of the point of keeping the
> UI in sync was that trunk testing happening right now would still
> benefit Firefox 2, so we could try to move testing resources to trunk as
> needed.

I'm not sure that there is a huge advantage in shipping an alpha now, as
opposed to shipping one a bit after FF2 is out the door. Regressions are
generally not too hard to fix, it is finding them that is the problem.
So it might not be too bad to live with a regression a bit longer on the
trunk and make sure that we get a lot of testing at some point well
before we're approaching any sort of freeze.

So I think a well trumpeted, well tested alpha a bit down the road is
better then a half-trumpeted and half-tested now, and another
half-trumpeted and half-tested a bit later. In other words, I think it's
better to get more people testing, then to get the same people to test
multiple times.

I think (someone correct me if i'm overly optimistic) that we're keeping
good enough quality through the nightly releases that we don't have a
lot of test-blocking level bugs.

/ Jonas
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha

fantasai
In reply to this post by Jeff Schiller
schiller wrote:

> Slightly selfish, er, tangential question here:  Are there any
> requirements on what delta SVG functionality will make it into Fx3?  I
> know there was a big piece outstanding for Declarative Animation/SMIL
> (done by this guy:
> http://brian.sol1.net/svg/2006/01/09/smil-animation-in-mozilla-report/)
> but I'm not sure if this will make it for Fx3.
>
> Is the idea something like "whatever the SVG guys can get done and
> stable in time" or are there specific criteria, like "all of SVG 1.1"
> :) ?

I can't imagine a day when "all of SVG 1.1" will be on the requirements list.

~fantasai
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha

Boris Zbarsky
In reply to this post by Jonas Sicking
Jonas Sicking wrote:
> I'm not sure that there is a huge advantage in shipping an alpha now, as
> opposed to shipping one a bit after FF2 is out the door. Regressions are
> generally not too hard to fix, it is finding them that is the problem.

This is not my experience with changes to architecture.  Often, fixing
regressions requires further architecture changes...

> So it might not be too bad to live with a regression a bit longer on the
> trunk and make sure that we get a lot of testing at some point well
> before we're approaching any sort of freeze.

The problem is that if at that point we have a number of regressions that make
the builds not testable by the general public then we have to do multiple alpha
cycles as the regressions are discovered.  If we start having alphas later,
we'll have fewer alpha cycles, which will, imo, mean fewer regressions found.

> So I think a well trumpeted, well tested alpha a bit down the road is
> better then a half-trumpeted and half-tested now, and another
> half-trumpeted and half-tested a bit later.

I'm not sure I buy that, but in any case it's worse than a half-trumpeted one
now and a well-trumpeted one later.  I'm not proposing we skip on later alphas
just because we do one sooner.

> I think (someone correct me if i'm overly optimistic) that we're keeping
> good enough quality through the nightly releases that we don't have a
> lot of test-blocking level bugs.

I think you're being overly optimistic.  As a simple example, I'm currently
using a February nightly for most of my browsing; there have been regressions
almost constantly in various functionality areas since then, and the switch to
Cairo makes our nightlies unusably slow on my computer (3-4 seconds to repaint
the window after I do any window manager operation; 1-2 seconds to repaint when
the window loses or gains focus).

So the current trunk quality is certainly blocking testing by me, as far as that
goes.

-Boris
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha: selective summary

fantasai
In reply to this post by Boris Zbarsky
Ok, so here's what I'm understanding so far. Correct me if I'm wrong.

Issues:

   - *Gecko trunk needs more testing, especially by web-developer types*
       -> Need a browser front end, other apps not as important for this
           (Boris writes: I just want feedback on which web sites and
           intranet apps we've broken so we can fix them now, and not
           during the release crunch.)

   - *Want to catch regressions in Gecko earlier in the release cycle*

   - *There are some major Gecko changes just in / going in* within the
      next month or two that need widespread QA exposure.
       -> bz is compiling a list of these

   - Trunk firefox code is synced with FF2 branch
       -> Using its front end will be a useful QA windfall for FF team,
          even though that's not the focus here

   - Triage could use more blocking nomination flags for Gecko
       -> Should have flags for at least one alpha and one beta release
           (Axel writes: It's more important to separate a1 and b1 than a1
           and a2 at the current stage, so that one can decide "well, I'd
           ship an alpha with this, but not a beta".)

Release Planning Ideas:

   - Release a rebranded Firefox as the Gecko testing vehicle
       -> Nightlies are branded as "Minefield".
       -> Proposed names for Gecko releases include:
          - Gecko Browser Preview 2006-06-24 (or something like that)
          - restarting old Milestone system
          - Mozilla Eats Babies for Lunch Pre-Pre-Alpha Developer Preview 0.0.0.1

   - Target releases at web developers, not front-end testers.

   - Boris proposes a *Gecko 1.9* alpha release in the near future to put
     those major changes out to a wider testing audience. How this should
     sync with FF2 releases is still under debate.

   - Mike Connor proposes creating an update channel for Gecko testers,
     pushing a new known-good trunk build every fortnight. Estimated freeze
     time is one afternoon every two weeks for MozQA to run smoketests.

   - Jonas notes that potential testers would not want to install an alpha
     version that might wreck their bookmarks, eat their preferences and
     stomp on their cookies. Changing install options so it doesn't write
     to user's main FF profile might be a good idea.


My opinion:

   - Plan a Gecko alpha release as Boris proposed in the first message.

   - Call it "Gecko 1.9 Preview" or somesuch. The string should tie the
     version number to "Gecko", not to "Gecko Browser".

   - While FF development is still on the branch, don't worry too much about
     syncing the releases, just make sure the front end on the trunk is stable
     enough for the alpha release.

   - Target the Gecko 1.9 release at web developers. Say this release is from
     alpha development stages for the Firefox 3 *layout engine*, but it's still
     using the same Firefox 2 front end as the FF2 builds. If testers are
     interested in testing the latest layout engine from Mozilla, this is what
     they should get. If they're only interested in Firefox and browsing
     features, they should get the Firefox 2 branch releases.

   - If we have the resources, also pick up Mike Connor's fortnightly update
     channel idea. It'll make new layout engine developments that much more
     easy and exciting (and thereby encourage more people to get involved with
     bleeding-edge layout QA).

   Basically, that follows two principles:
     a) Don't try pushing the masses of testers around based on numbers.
        Split them strategically based on how their skills/interests meet our
        testing needs. People interested in layout engine improvements should
        be poking at Gecko trunk. Everyone else should be poking at the FF2
        branch.
     b) Communicate clearly what the different testing builds mean.
        Web developer-testers may want to have FF2 builds handy alongside
        FF1.5 for testing their website for regressions, but focus on Gecko 1.9
        when filing layout bugs. They can use builds more effectively if they
        understand what's going on.

~fantasai
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha: selective summary

Andrew Schultz-2
fantasai wrote:
>   - Target releases at web developers, not front-end testers.

This line of reasoning is appealing (and would be ideal), but ignores
the fact that most bug reports (including layout engine bugs) are not
from web developers.  They're mostly filed by users who know nothing
more than that the web page looks different than it does in IE or in the
previous version.  I've probably seen about as many comments (HTML
comments by the developers) in web pages like "<!-- workaround for
Firefox -->" as I've seen bugs filed by web developers.  Maybe they
assume that Firefox is  defined to be "standards compliant" or maybe
they're just vastly outnumber by users.

So, we have to pitch a browser because the users don't care about the
layout engine.

What pitching to web developers /will/ get is people developing
next-generation web apps who want to take advantage of the
latest/greatest features included with Gecko 1.9.  But I don't see that
as being enough.

--
Andrew Schultz
[hidden email]
http://www.sens.buffalo.edu/~ajs42/
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha

Brendan Eich
In reply to this post by beltzner
L. David Baron wrote:

> Regression bug reports are much more useful when the code is still fresh
> in its authors' mind, and when there's enough time to fix them before
> the release.

And when there are not lots of CVS revisions dogpiled on top, entangling
with the regressing change in arbitrarily complex ways.  It's bad enough
to have to do "CVS archeology" or binary-search-testing of old builds.
Having to disentangle merged code is even worse.  So time in regression
finding and fixing is of the essence.

/be

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha

Jonas Sicking
In reply to this post by Boris Zbarsky
>> So it might not be too bad to live with a regression a bit longer on
>> the trunk and make sure that we get a lot of testing at some point
>> well before we're approaching any sort of freeze.
>
> The problem is that if at that point we have a number of regressions
> that make the builds not testable by the general public then we have to
> do multiple alpha cycles as the regressions are discovered.  If we start
> having alphas later, we'll have fewer alpha cycles, which will, imo,
> mean fewer regressions found.

OTOH, if we have too many alphas I think people will just think "i'll
just wait another alpha and test then, it should be more stable". I
guess the answer is to find the right number of alphas and make sure to
announce to the public what the purpose of each release is.

>> I think (someone correct me if i'm overly optimistic) that we're
>> keeping good enough quality through the nightly releases that we don't
>> have a lot of test-blocking level bugs.
>
> I think you're being overly optimistic.  As a simple example, I'm
> currently using a February nightly for most of my browsing; there have
> been regressions almost constantly in various functionality areas since
> then, and the switch to Cairo makes our nightlies unusably slow on my
> computer (3-4 seconds to repaint the window after I do any window
> manager operation; 1-2 seconds to repaint when the window loses or gains
> focus).

Sure, but all this is stuff that the nightlies testing is finding, no?
It's just a matter of at some point deciding to focus on fixing it
before we ship an alpha.

My question remains, if we half-ass an alpha "since we havn't released
anything in 8 months", are we really going to get a lot of new testers
on it?

/ Jonas
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha

Jonas Sicking
>>> I think (someone correct me if i'm overly optimistic) that we're
>>> keeping good enough quality through the nightly releases that we
>>> don't have a lot of test-blocking level bugs.
>>
>> I think you're being overly optimistic.  As a simple example, I'm
>> currently using a February nightly for most of my browsing; there have
>> been regressions almost constantly in various functionality areas
>> since then, and the switch to Cairo makes our nightlies unusably slow
>> on my computer (3-4 seconds to repaint the window after I do any
>> window manager operation; 1-2 seconds to repaint when the window loses
>> or gains focus).
>
> Sure, but all this is stuff that the nightlies testing is finding, no?
> It's just a matter of at some point deciding to focus on fixing it
> before we ship an alpha.

Actually, what I should say is, just because we find problems, it
doesn't mean that they will go away. We still have to actually fix them :)

I still think that if we just throwing more alphas out there is not
going to give us significantly higher quality than what the nightlies do.

/ Jonas
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha

Boris Zbarsky
In reply to this post by Jonas Sicking
Jonas Sicking wrote:
> I guess the answer is to find the right number of alphas and make sure to
> announce to the public what the purpose of each release is.

Exactly.  For each one, list the new things that need testing.

> Sure, but all this is stuff that the nightlies testing is finding, no?

Could be; I have no idea.  But the point is that if the builds are not usable
(say due to perf issues), then no one's going to test them (so we won't find
correctness bugs).

> It's just a matter of at some point deciding to focus on fixing it
> before we ship an alpha.

That point should be yesterday, imo.

> My question remains, if we half-ass an alpha "since we havn't released
> anything in 8 months", are we really going to get a lot of new testers
> on it?

My point is that we should schedule an alpha (and decide what remaining large
changes we'll take for it), then focus on getting this alpha usable so that we
can get useful testing from it.  I suspect that'll take us another month or so
at least.

I'm not suggesting just shipping current trunk as-is.  That would be rather
unfortunate.

-Boris
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha

beltzner
In reply to this post by Boris Zbarsky
On 4/15/06, Brendan Eich <[hidden email]> wrote:
> Mike Beltzner wrote:
> > I think it was mostly to reduce the pain of merging back to trunk from
> > a branch, as was experienced with aviary->gecko 1.8. That was the
> > rumour I heard or read off a bathroom wall or something, anyway ;)
>
> Hey look, I wrote up a bunch of reasoning, with a FAQ, at
> http://wiki.mozilla.org/Global:1.9_Trunk_1.8_Branch_Plan.  It talks
> explicitly about testing Gecko 1.9 with the Firefox 2 front end, to
> avoid the former regressing the latter.

Precisely the bathroom wall I was referring to, actually. :) FWIW, I
didn't mean to imply that I thought this was a poor approach. Quite
the opposite, really. I'm just trying to be careful about making
assertions about relative ease and complexity when I haven't really
got the history or context to make those sorts of statements.

> So it makes no sense to release "Gecko 1.9" as some kind of library set,
> even if we had frozen APIs in place for all app front ends to use (we
> don't).  A trunk Firefox alpha, with appropriately obscure name, is
> absolutely necessary well before Firefox 2 ships.

So we're back to one of the options proposed earlier in this thread.

--
/ mike beltzner / user experience lead / mozilla corporation /
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha: selective summary

beltzner
In reply to this post by fantasai
On 4/14/06, fantasai <[hidden email]> wrote:
> Ok, so here's what I'm understanding so far. Correct me if I'm wrong.

[..snip..]

Great round up, Fantasai. That seems to be pretty right to me, in
terms of summary, and I like the recommendations you make in terms of
clearly communicating what the various releases would be for.

As Andrew mentions, focusing the release on web developers might not
get the critical mass for testing that bz and dbaron are looking for.
I think it's important to communicate the idea that the front end of
Gecko 1.9 releases won't diverge meaningfully from Firefox 2 until the
latter has been released in its final form, but more to manage
expectations of reviewers who might otherwise wonder why the two are
so identical looking at the level of the chrome.

I must once again question the idea of a named, branded release, as
opposed to a milestone (ie: dated) checkpoint for which we can make
assertions of stability of the code. This avoids the introduction of a
new name (f.e. Gecko Browser, Gecko, Firefox 3, etc) and allows us to
clearly message about how, for example, "Minefield Stable
(2006-05-15)" is a for-tester release of the next Gecko layout engine,
using the same UI as exists in Firefox 2, and so testers should feel
free to use it or the latest Bon Echo release for their testing
depending on the criteria you listed.

cheers,
mike


--
/ mike beltzner / user experience lead / mozilla corporation /
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha: selective summary

Boris Zbarsky
In reply to this post by fantasai
fantasai wrote:
>       -> bz is compiling a list of these

I've posted the current state of what I know about at
<http://wiki.mozilla.org/Gecko_1.9_Alpha_Planning>.  I crave feedback or direct
wiki edits.

My impression is that the long pole in the whole thing is the combination of
cocoa widgets, followed by cairo on Mac, followed by the nsTextFrame changes
(likely needed to get perf parity with pre-cairo).  I suspect we'll want to put
alpha 1 somewhere in the middle of that long pole; the question is where.  I'd
like it to be after we enable cairo on Mac, but we need more timing information
here.

-Boris
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
12345