Scheduling a trunk alpha

classic Classic list List threaded Threaded
96 messages Options
12345
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha: selective summary

skierpage (Bugzilla)
I found this thread trying to find the status of the trunk.  I'm one of
your bleeding-edge users that was on DeerPark 1.6a1 with the
channel-prefs.js hack to get nightly updates, and I've been waiting for
a flashing amber light to get back on the trunk ever since that stopped
updating.  (The unchanging Bon Echo 2.0a1 just isn't doing it for me
:-)

I applaud fantasai's summary.  I like mconnor's idea for a
"bruised-edge" update channel, or if that's too much work, just an
occasional "Last night's build seems OK" blog or forum post.

Naming is tricky when you've got Firefox 1.5.xx, Firefox 2 "Bon Echo",
and Gecko 1.9 as candidates for user testing.  But no matter how you
characterize a "Minefield 1.9 engine preview alpha" release, some site
will link directly to its bits with a "Firefox 3 is out!!!" headline,
so all you can do is warn users *away* from it in all its splash screen
/ Help -> About / Release Notes / Check for Updates messaging.

I strongly agree with "Communicate clearly what the different testing
builds mean"; as small help I've added clarifying text and links on
some of the Firefox3/Gecko 1.9 wiki pages.

Good luck with the alpha.

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha: selective summary

beltzner
On 17 Apr 2006 19:15:21 -0700, [hidden email] <[hidden email]> wrote:
> updating.  (The unchanging Bon Echo 2.0a1 just isn't doing it for me
> :-)

Software updates for Bon Echo were re-enabled on March 14th, fwiw. :)

> I strongly agree with "Communicate clearly what the different testing
> builds mean"; as small help I've added clarifying text and links on
> some of the Firefox3/Gecko 1.9 wiki pages.

Good idea. Thanks.

--
/ mike beltzner / user experience lead / mozilla corporation /
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha

Brendan Eich
In reply to this post by Jonas Sicking
Jonas Sicking wrote:

> Boris Zbarsky wrote:
>> Mike Beltzner wrote:
>>> I think the issue we need to worry about is splitting our testing
>>> community: should they be testing the browser using trunk or branch?
>>
>> Some of both, unfortunately.  I thought part of the point of keeping
>> the UI in sync was that trunk testing happening right now would still
>> benefit Firefox 2, so we could try to move testing resources to trunk
>> as needed.
>
> I'm not sure that there is a huge advantage in shipping an alpha now, as
> opposed to shipping one a bit after FF2 is out the door. Regressions are
> generally not too hard to fix, it is finding them that is the problem.
> So it might not be too bad to live with a regression a bit longer on the
> trunk and make sure that we get a lot of testing at some point well
> before we're approaching any sort of freeze.

This is contradicted by our experience over the years, especially during
the push to Firefox 1.0 when 1.8 alpha lingered on the trunk.  But we
have always had problems with regressions diving deep, entangling with
other changes.

What you write above "live with a regression a bit longer" implies that
regressions are known and tracked, and fixing them can be deferred.
That is not the case.  Too often, regressions are not even *found* until
an alpha-scale (100k+ downloads) release is done -- and then the hard
job of diagnosing begins.  This can take months.

/be
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha: selective summary

Ian Pottinger
In reply to this post by skierpage (Bugzilla)
[hidden email] wrote:
> But no matter how you
> characterize a "Minefield 1.9 engine preview alpha" release, some site
> will link directly to its bits with a "Firefox 3 is out!!!" headline,

I think that anytime you use the terms "alpha', "beta", or "preview",
you invite this type of confusion.

Leave "alpha', "beta", or "preview" to product releases and characterize
offerings straight from the truck as "snapshots."  I've seen that term
used previously in this thread.  It better describes what such an
offering is or, at the very least, will beg authors to investigate and
then hopefully explain the term to their audience.  Hence:

Firefox 2.0 Developer Preview
Firefox 2.0 Beta1
Minefield Snapshot 20060515
Minefield Snapshot 20060615
Firefox 3.0 Developer Preview
Firefox 3.0 Beta1

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha: selective summary

Boris Zbarsky
Ian Pottinger wrote:
> Leave "alpha', "beta", or "preview" to product releases and characterize
> offerings straight from the truck as "snapshots."

Except the point is that this IS a "developer preview" of Gecko 1.9.  That's why
we're doing it.

Now whether we should call it a "preview" of Firefox 3 is a different issue.

-Boris
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha

Boris Zbarsky
In reply to this post by Boris Zbarsky
OK, so looking at http://wiki.mozilla.org/Gecko_1.9_Alpha_Planning I'm starting
to think that one thing we should consider is shipping an alpha 1 in the next
few weeks (at the same time as the aforementioned 2.0 beta?) with cairo disabled
on either all platforms or just Mac (depending on the state of cairo at that
point and on our estimate of how much the cairo regressions would interfere with
testing).

To make that decision, we need to start actually tracking the cairo regressions,
of course, both performance and correctness (possibly under separate trackers,
so two trackers per platform or something).  If desired, we could track
correctness on the "enable cairo by default" bugs, of course.

Did we ever get bugs filed to do this tracking?  If not, would someone be
willing to do so and start adding dependencies?  It would be very helpful.

-Boris
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha: selective summary

Mike Connor-4
In reply to this post by fantasai
fantasai wrote:
>   - Triage could use more blocking nomination flags for Gecko
>       -> Should have flags for at least one alpha and one beta release
>           (Axel writes: It's more important to separate a1 and b1 than a1
>           and a2 at the current stage, so that one can decide "well, I'd
>           ship an alpha with this, but not a beta".)
At this point, I'd want to avoid targeting bugs for that far in the
future, since you can easily end up with other bugs depending on broken
behaviour, and that can chain fairly fast, making fixing the underlying
issue a game of Wac-A-Mole.
>   - Jonas notes that potential testers would not want to install an alpha
>     version that might wreck their bookmarks, eat their preferences and
>     stomp on their cookies. Changing install options so it doesn't write
>     to user's main FF profile might be a good idea.
This is fairly easy to do, we've avoided it in the past, but its
probably worth looking at for the trunk.
>     a) Don't try pushing the masses of testers around based on numbers.
>        Split them strategically based on how their skills/interests
> meet our
>        testing needs. People interested in layout engine improvements
> should
>        be poking at Gecko trunk. Everyone else should be poking at the
> FF2
>        branch.
I think the key here is "interested" users.  I don't classify this as
meaning "web developers" to the exclusion of others, but I think that
lots of people are interested in what happens on trunk, and the update
channel concept would remove a big barrier to entry for the
not-quite-bleeding-edge group, and possibly, if we do a good enough job
with those, we'll get a wide enough spread to catch more bugs faster.
>     b) Communicate clearly what the different testing builds mean.
>        Web developer-testers may want to have FF2 builds handy alongside
>        FF1.5 for testing their website for regressions, but focus on
> Gecko 1.9
>        when filing layout bugs. They can use builds more effectively
> if they
>        understand what's going on.
It'd be handy to have a nice chart somewhere on mozilla.org showing all
current development tracks, where to get latest releases/stable
builds/nightlies.  This would have 1.5.0.x/Bon Echo/Minefield for the
browser at least.

-- Mike


_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha: selective summary

Robert Kaiser
In reply to this post by skierpage (Bugzilla)
[hidden email] schrieb:
> Naming is tricky when you've got Firefox 1.5.xx, Firefox 2 "Bon Echo",
> and Gecko 1.9 as candidates for user testing.  But no matter how you
> characterize a "Minefield 1.9 engine preview alpha" release, some site
> will link directly to its bits with a "Firefox 3 is out!!!" headline,
> so all you can do is warn users *away* from it in all its splash screen
> / Help -> About / Release Notes / Check for Updates messaging.

If the UA string would show "Minefield/20060528" instead of
"Firefox/3.0a" this might be less the fact...
And that would be another test for web pages, as mayn pages currently
test for "Firefox/" in the UA string, and close out all other
Gecko-based browsers that would work.
Usually, this is something that users of Camino or SeaMonkey are seeing,
but such a UA change might shed some more light on those cases (not sure
if that's reall wanted though).

Robert Kaiser
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha

beltzner
In reply to this post by Boris Zbarsky
On 4/18/06, Boris Zbarsky <[hidden email]> wrote:
> to think that one thing we should consider is shipping an alpha 1 in
> the next few weeks (at the same time as the aforementioned 2.0 beta?)

Aforementioned 2.0 Alpha, you mean. The planned ship date for Bon Echo
Alpha 2 is May 9th. I think to get the best testing coverage and
exposure, you'd want to release a Minefield
Snapshot/Milestone/Whatever  the week after that. Gets you another
media cycle.

cheers,
mike

--
/ mike beltzner / user experience lead / mozilla corporation /
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha: selective summary

Boris Zbarsky
In reply to this post by Robert Kaiser
Robert Kaiser wrote:
> Usually, this is something that users of Camino or SeaMonkey are seeing,
> but such a UA change might shed some more light on those cases (not sure
> if that's reall wanted though).

It's wanted by me.

-Boris
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha: selective summary

Chris Hofmann
In reply to this post by Mike Connor-4
Mike Connor wrote:

> fantasai wrote:
>>     a) Don't try pushing the masses of testers around based on numbers.
>>        Split them strategically based on how their skills/interests
>> meet our
>>        testing needs. People interested in layout engine improvements
>> should
>>        be poking at Gecko trunk. Everyone else should be poking at
>> the FF2
>>        branch.
> I think the key here is "interested" users.  I don't classify this as
> meaning "web developers" to the exclusion of others, but I think that
> lots of people are interested in what happens on trunk, and the update
> channel concept would remove a big barrier to entry for the
> not-quite-bleeding-edge group, and possibly, if we do a good enough
> job with those, we'll get a wide enough spread to catch more bugs faster.
>>    

We really do need to push folks around and draw them to test gecko based
where the focus can be of the highest value.   There are back end
changes going into 2.0 and they need to be shaken out at some point as
well...  

http://wiki.mozilla.org/Firefox2/StatusMeetings/2006-04-04#Infrastructure:_Platform_Uplift

By the time these 23 approved and 110 nominations are dealt with on the
1.8.1 branch there will be need to shake those out.  More nominations
will follow during 2.0 development...

23 approved so far
https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&long_desc_type=substring&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=&resolution=DUPLICATE&resolution=---&emailassigned_to1=1&emailtype1=exact&email1=&emailassigned_to2=1&emailreporter2=1&emailqa_contact2=1&emailtype2=exact&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&cmdtype=doit&order=Reuse+same+sort+as+last+time&field0-0-0=flagtypes.name&type0-0-0=substring&value0-0-0=blocking1.8.1%2B

110 nominations
https://bugzilla.mozilla.org/buglist.cgi?query_format=advanced&short_desc_type=allwordssubstr&short_desc=&long_desc_type=substring&long_desc=&bug_file_loc_type=allwordssubstr&bug_file_loc=&status_whiteboard_type=allwordssubstr&status_whiteboard=&keywords_type=allwords&keywords=&resolution=DUPLICATE&resolution=---&emailassigned_to1=1&emailtype1=exact&email1=&emailassigned_to2=1&emailreporter2=1&emailqa_contact2=1&emailtype2=exact&email2=&bugidtype=include&bug_id=&votes=&chfieldfrom=&chfieldto=Now&chfieldvalue=&cmdtype=doit&order=Reuse+same+sort+as+last+time&field0-0-0=flagtypes.name&type0-0-0=substring&value0-0-0=blocking1.8.1%3F

The key to all this testing is to get a pretty good set of builds though
lots of focus bug triage and testing by some very active testers, then
draw 100,000+ into the mix to shake out more obscure problems.  This
will be needed on both the 1.8.1 branch and trunk.

chris h.


_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha

Boris Zbarsky
In reply to this post by Boris Zbarsky
Mike Beltzner wrote:
> Aforementioned 2.0 Alpha, you mean. The planned ship date for Bon Echo
> Alpha 2 is May 9th.

OK.  So are there major landing owners who are not reading this?  ;)  If you
know of one, poke them.  I'd like to see date estimates for all landings, to the
extent that that's possible.  That will tell us where we stand.

-Boris
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha: selective summary

Boris Zbarsky
In reply to this post by Mike Connor-4
Chris Hofmann wrote:
> There are back end
> changes going into 2.0 and they need to be shaken out at some point as
> well...

I should note that if we were testing reasonably on trunk we'd have a lot less
to worry about in terms of 2.0 at this point, imho, since all those changes are
landing on trunk first, often by weeks.

-Boris
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha: selective summary

Mike Connor-4
Boris Zbarsky wrote:
> Chris Hofmann wrote:
>> There are back end changes going into 2.0 and they need to be shaken
>> out at some point as well...
>
> I should note that if we were testing reasonably on trunk we'd have a
> lot less to worry about in terms of 2.0 at this point, imho, since all
> those changes are landing on trunk first, often by weeks.
Yes, but those patches are landing on top of other patches on trunk,
which may not be making it to the branch for various reasons, and the
interactions are thus different.  We hit that a fair bit in the endgame
for 1.5, and hopefully we can learn from that and not shortchange the
amount of testing for branch Gecko changes.

-- Mike
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha

Chris Hofmann
In reply to this post by Boris Zbarsky
Boris Zbarsky wrote:

> OK, so looking at http://wiki.mozilla.org/Gecko_1.9_Alpha_Planning I'm
> starting to think that one thing we should consider is shipping an
> alpha 1 in the next few weeks (at the same time as the aforementioned
> 2.0 beta?) with cairo disabled on either all platforms or just Mac
> (depending on the state of cairo at that point and on our estimate of
> how much the cairo regressions would interfere with testing).
>
> To make that decision, we need to start actually tracking the cairo
> regressions, of course, both performance and correctness (possibly
> under separate trackers, so two trackers per platform or something).  
> If desired, we could track correctness on the "enable cairo by
> default" bugs, of course.
>
> Did we ever get bugs filed to do this tracking?  If not, would someone
> be willing to do so and start adding dependencies?  It would be very
> helpful.
I searched though bugs with "cairo" in the title and couldn't see any
top level trackers so I filed this
https://bugzilla.mozilla.org/show_bug.cgi?id=334509

With performance and correctness under dependency bugs

performance clean up- https://bugzilla.mozilla.org/show_bug.cgi?id=334510
correctness clean up- https://bugzilla.mozilla.org/show_bug.cgi?id=334512

the pool of "cairo" titled bugs is a good place to start in finding bugs
that need to be linked in to these.

chris .
>
> -Boris
> _______________________________________________
> dev-planning mailing list
> [hidden email]
> https://lists.mozilla.org/listinfo/dev-planning

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha: selective summary

Boris Zbarsky
In reply to this post by Boris Zbarsky
Mike Connor wrote:
> Yes, but those patches are landing on top of other patches on trunk,
> which may not be making it to the branch for various reasons, and the
> interactions are thus different.  We hit that a fair bit in the endgame
> for 1.5, and hopefully we can learn from that and not shortchange the
> amount of testing for branch Gecko changes.

Of course imho we should also be limiting branch Gecko changes, generally
speaking...  Not just in terms of "this is a scary arch change" but in terms of
"this changes behavior from web authors' point of view".

-Boris
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha

Boris Zbarsky
In reply to this post by Boris Zbarsky
Chris Hofmann wrote:
> top level trackers so I filed this
> https://bugzilla.mozilla.org/show_bug.cgi?id=334509
>
> With performance and correctness under dependency bugs
>
> performance clean up- https://bugzilla.mozilla.org/show_bug.cgi?id=334510
> correctness clean up- https://bugzilla.mozilla.org/show_bug.cgi?id=334512

I assume those are for Cairo-windows?  We probably want to track each of the
three platforms separately, since so many of the regressions are
platform-specific and need platform-specific fixes....

-Boris
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha

Christian Biesinger
In reply to this post by Mike Connor-3
(I'm kind of late to this thread and still not done catching up, oh well
:-/)

Mike Connor wrote:
> We'll still have these builds on the ftp somewhere for archiving/linking
> from announcements (i.e. we would have a Gecko Team blog that we
> announce these builds on, so people know what especially needs testing).

You earlier mentioned that you wouldn't tag them/make source tarballs
available. In my opinion, everything that lands in a /release/ directory
should have a corresponding tag and source tarball - since this is an
open-source project, the source for all releases ought to be easily
available.
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha

Mike Shaver
On 4/18/06, Christian Biesinger <[hidden email]> wrote:
> You earlier mentioned that you wouldn't tag them/make source tarballs
> available. In my opinion, everything that lands in a /release/ directory
> should have a corresponding tag and source tarball - since this is an
> open-source project, the source for all releases ought to be easily
> available.

Would we want this in a releases/ directory?  Seems to me that we
wouldn't, if we could easily avoid it.  But tagging it seems virtuous,
and source tarballs are part of the nightly machinery already, so
those seem like reasonable desires both.

Mike
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Scheduling a trunk alpha: selective summary

Justin Kerk
In reply to this post by skierpage (Bugzilla)
[hidden email] wrote in news:1145326521.849409.212180
@j33g2000cwa.googlegroups.com:

> I found this thread trying to find the status of the trunk.  I'm one of
> your bleeding-edge users that was on DeerPark 1.6a1 with the
> channel-prefs.js hack to get nightly updates, and I've been waiting for
> a flashing amber light to get back on the trunk ever since that stopped
> updating.  (The unchanging Bon Echo 2.0a1 just isn't doing it for me
>:-)

I'm another bleeding-edge user, I've been using trunk on my laptop but
I've hung onto an old pre-Places February branch build on my desktop
because I don't fully trust Places not to wreck my bookmarks and so
forth; I agree with Jonas that that's an obstacle to getting people to
test unstable builds. I'm not sure what the best solution to that is
though; IMO it would be useful to have some kind of a lightweight "Gecko
in a box" kiosk-type build that doesn't use a profile at all and can be
run concurrently with a stable Firefox, for people who just want to try
out websites with the new rendering engine. Of course we do want people
to test Places....

> I strongly agree with "Communicate clearly what the different testing
> builds mean"; as small help I've added clarifying text and links on
> some of the Firefox3/Gecko 1.9 wiki pages.

I think it's also going to be important to communicate right up front in
the announcement any obvious known bugs that are going to be in these
releases, in a way similar to the "red list" in Peter(6)'s posts in the
MozillaZine Firefox Builds forum. For example, there's a big list of
Cairo bugs that probably aren't all going to be fixed in a month, and
which aren't necessarily easy to find by searching for symptoms, so we're
probably just going to get a big pile of dupes on those if we just tell
people "grab this and look for bugs", and many testers probably won't
bother to look past the obvious bugs. Bug 324706 in particular has been
getting lots of dupes even just with the nightly testers.

Acknowledging the obvious bugs up front is also more likely to get
understanding and support from users rather than a stream of "wow this is
hilariously broken, good luck pulling this together in time for Fx3" blog
comments.

--
[hidden email] is spambait
dopefish justin at gmail dot com
http://interbutt.com/
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
12345