shorter release cycles?

classic Classic list List threaded Threaded
47 messages Options
123
Reply | Threaded
Open this post in threaded view
|

shorter release cycles?

L. David Baron
We've had some discussion in various places about trying to reduce
the time it takes for code to get from landing on nightly to being
on release, so that we can get code into our users hands faster.

I wrote a blog post with one idea about how to do this that I
figure is worth sharing on this list (which is, after all, for
discussion of release planning):

  http://dbaron.org/log/20141106-release-cycles

I'm curious what others think of this idea.

(I'd note that this proposal is basically independent of Aurora
being a Developer Edition; the two things don't really interact.)

-David

--
𝄞   L. David Baron                         http://dbaron.org/   𝄂
𝄢   Mozilla                          https://www.mozilla.org/   𝄂
             Before I built a wall I'd ask to know
             What I was walling in or walling out,
             And to whom I was like to give offense.
               - Robert Frost, Mending Wall (1914)

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: shorter release cycles?

Mark Filipak
On 2014/11/14 3:08 PM, L. David Baron wrote:
> We've had some discussion in various places about trying to reduce
> the time it takes for code to get from landing on nightly to being
> on release, so that we can get code into our users hands faster.

What makes you think that the users want more and faster releases?

I've been lurking this list for about a month to ascertain whether this
is the best place to complain about the development cycles that seem to
some on the support-firefox list to be out of control. There are many on
that list who are unhappy that FF keeps changing. I'm at ESR 24.4.0 and
have stopped updating altogether.
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: shorter release cycles?

Eric Shepherd
In reply to this post by L. David Baron
> On Nov 14, 2014, at 3:08 PM, L. David Baron <[hidden email]> wrote:
>
> We've had some discussion in various places about trying to reduce
> the time it takes for code to get from landing on nightly to being
> on release, so that we can get code into our users hands faster.

This is a little bit of a shock to me. Everyone I’ve talked to where release schedules come up has been very much frustrated with how often updates come out as it is. Going even faster seems like a bad, bad idea. I don’t know how many people really want an application they use constantly to change on a frequent basis.

Has any research been done that suggests that faster would be better?

Eric Shepherd
Developer Documentation Lead
Mozilla Developer Network <https://developer.mozilla.org/>
Blog: http://www.bitstampede.com/
 <http://www.bitstampede.com/>Twitter: http://twitter.com/sheppy <http://twitter.com/sheppy>
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: shorter release cycles?

mhoye
On 2014-11-14 3:35 PM, Eric Shepherd wrote:
>> On Nov 14, 2014, at 3:08 PM, L. David Baron <[hidden email]> wrote:
>>
>> We've had some discussion in various places about trying to reduce
>> the time it takes for code to get from landing on nightly to being
>> on release, so that we can get code into our users hands faster.
> This is a little bit of a shock to me. Everyone I’ve talked to where release schedules come up has been very much frustrated with how often updates come out as it is. Going even faster seems like a bad, bad idea. I don’t know how many people really want an application they use constantly to change on a frequent basis.
For whatever it's worth I think that's because we're interrupting
people's by asking them to update, not because they're getting updated.
I've had that same conversations, but I follow it up by asking what
version of Chrome or Safari people are using. Nobody knows or cares, and
I suspect that's because they're never asked to notice.

Getting performance improvements, new features and bug fixes out to our
users faster is definitely better for the users who want a faster, more
useful browser with fewer bugs in it, which seems like it's probably a
lot of them, as long as we don't interrupt what they were doing.


- mhoye

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: shorter release cycles?

Ben Kelly
In reply to this post by Eric Shepherd
> From: "Eric Shepherd" <[hidden email]>
> > On Nov 14, 2014, at 3:08 PM, L. David Baron <[hidden email]> wrote:
> >
> > We've had some discussion in various places about trying to reduce
> > the time it takes for code to get from landing on nightly to being
> > on release, so that we can get code into our users hands faster.
>
> This is a little bit of a shock to me. Everyone I’ve talked to where release
> schedules come up has been very much frustrated with how often updates come
> out as it is. Going even faster seems like a bad, bad idea. I don’t know how
> many people really want an application they use constantly to change on a
> frequent basis.

As far as I can see, David's proposal keeps the user facing release cycle at 6 weeks.

Ben
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: shorter release cycles?

Eric Shepherd
You’re right, it does.That’s what I get for replying after only partly reading the blog post in question. My apologies.

Eric Shepherd
Developer Documentation Lead
Mozilla Developer Network <https://developer.mozilla.org/>
Blog: http://www.bitstampede.com/
 <http://www.bitstampede.com/>Twitter: http://twitter.com/sheppy <http://twitter.com/sheppy>
> On Nov 14, 2014, at 3:48 PM, Ben Kelly <[hidden email]> wrote:
>
> As far as I can see, David's proposal keeps the user facing release cycle at 6 weeks.

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: shorter release cycles?

Boris Zbarsky
In reply to this post by L. David Baron
On 11/14/14, 3:21 PM, Mark Filipak wrote:
> On 2014/11/14 3:08 PM, L. David Baron wrote:
>> We've had some discussion in various places about trying to reduce
>> the time it takes for code to get from landing on nightly to being
>> on release, so that we can get code into our users hands faster.
>
> What makes you think that the users want more and faster releases?

This proposal does not cause more releases.  What it does is reduce the
latency without changing the throughput.

-Boris
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: shorter release cycles?

Chris Peterson-12
In reply to this post by L. David Baron
On 11/14/14 12:08 PM, L. David Baron wrote:
> I wrote a blog post with one idea about how to do this that I
> figure is worth sharing on this list (which is, after all, for
> discussion of release planning):
>
>    http://dbaron.org/log/20141106-release-cycles

For comparison, Chrome uses a similar release cycle, but their
Aurora-equivalent ("Dev") is just a ~weekly known-good snapshot of their
Nightly-equivalent (Canary). Chrome's Dev users have a good balance
between stability and shiny new features and their Beta retains six
weeks of low-churn bake time.

(Pre-Deveoper Edition) Aurora was a hard sell: it was not shiny like
Nightly and it is not as stable as Beta. David's proposed Aurora would
be no shinier than Beta but would have an extra week of instability. Who
would choose to run that Aurora? With the new Developer Edition, the
answer might not be "no one". :) But if the Developer Edition tracked
Nightly like Chrome's Dev does, we would get web developer feedback much
sooner and before changes break Beta users.

Also, what is our current ratio of Aurora uplifts to Beta uplifts? With
David's proposal, our Aurora uplifts would have to be as conservative
today's Beta uplifts.


chris
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: shorter release cycles?

Anne van Kesteren
In reply to this post by mhoye
On Fri, Nov 14, 2014 at 9:48 PM, Mike Hoye <[hidden email]> wrote:
> For whatever it's worth I think that's because we're interrupting people's
> by asking them to update, not because they're getting updated.

This is probably the single most annoying thing about running Firefox
Nightly compared to Chrome dev.


--
https://annevankesteren.nl/
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: shorter release cycles?

Jeff Griffiths-4
In reply to this post by Chris Peterson-12

> On Nov 14, 2014, at 1:59 PM, Chris Peterson <[hidden email]> wrote:
>
> On 11/14/14 12:08 PM, L. David Baron wrote:
>> I wrote a blog post with one idea about how to do this that I
>> figure is worth sharing on this list (which is, after all, for
>> discussion of release planning):
>>
>>   http://dbaron.org/log/20141106-release-cycles
>
> For comparison, Chrome uses a similar release cycle, but their Aurora-equivalent ("Dev") is just a ~weekly known-good snapshot of their Nightly-equivalent (Canary). Chrome's Dev users have a good balance between stability and shiny new features and their Beta retains six weeks of low-churn bake time.
>
> (Pre-Deveoper Edition) Aurora was a hard sell: it was not shiny like Nightly and it is not as stable as Beta. David's proposed Aurora would be no shinier than Beta but would have an extra week of instability. Who would choose to run that Aurora? With the new Developer Edition, the answer might not be "no one". :) But if the Developer Edition tracked Nightly like Chrome's Dev does, we would get web developer feedback much sooner and before changes break Beta users.

We’ve pondered moving to more of a weekly cadence for dev edition, similar to Canary. We certainly weren’t going to tackle that for fx10, we were actively trying to minimize disruption. I’d like to see a metope in Portland generally about the future here. I want to ship features to my users as soon as possible, while ensuring dev edition is reasonable stable.

Jeff

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: shorter release cycles?

Jeff Griffiths-4
In reply to this post by Anne van Kesteren

> On Nov 14, 2014, at 2:38 PM, Anne van Kesteren <[hidden email]> wrote:
>
> On Fri, Nov 14, 2014 at 9:48 PM, Mike Hoye <[hidden email]> wrote:
>> For whatever it's worth I think that's because we're interrupting people's
>> by asking them to update, not because they're getting updated.
>
> This is probably the single most annoying thing about running Firefox
> Nightly compared to Chrome dev.

FYI, Dev Edition avoids this by badging the hamburger menu button instead. It’s a little less intrusive - easily ignored. Update happens on the next re-start no matter what.


_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: shorter release cycles?

Mark Filipak
In reply to this post by Mark Filipak
Well, here I am replying to my own post, so it appears that what the
folks on the support-firefox list say is true, eh? ...that the
developers are not interested in listening to users and are not
interested in what users want.

According to Chris Ilias, support-firefox is for users to help users,
not for feature requests or complaints. Well, okay. Joyless, but okay.

The only other form of feedback is bugs, but that's not appropriate for
complaints or requests.

That means there's no forum for complaints and requests.

You know. There are a lot of unhappy FF users who would jump ship in a
heartbeat if there was an alternative -- note: Chrome means getting in
bed with Google and so is not an alternative.


On 2014/11/14 3:21 PM, Mark Filipak wrote:

> On 2014/11/14 3:08 PM, L. David Baron wrote:
>> We've had some discussion in various places about trying to reduce
>> the time it takes for code to get from landing on nightly to being
>> on release, so that we can get code into our users hands faster.
>
> What makes you think that the users want more and faster releases?
>
> I've been lurking this list for about a month to ascertain whether this
> is the best place to complain about the development cycles that seem to
> some on the support-firefox list to be out of control. There are many on
> that list who are unhappy that FF keeps changing. I'm at ESR 24.4.0 and
> have stopped updating altogether.
>

--
The Insect Hall of Fame:
Thunderbird Bug 121947 - 12 years and counting.
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: shorter release cycles?

L. David Baron
On Friday 2014-11-14 19:08 -0500, Mark Filipak wrote:

> Well, here I am replying to my own post, so it appears that what the
> folks on the support-firefox list say is true, eh? ...that the
> developers are not interested in listening to users and are not
> interested in what users want.
>
> According to Chris Ilias, support-firefox is for users to help users,
> not for feature requests or complaints. Well, okay. Joyless, but okay.
>
> The only other form of feedback is bugs, but that's not appropriate for
> complaints or requests.
>
> That means there's no forum for complaints and requests.
>
> You know. There are a lot of unhappy FF users who would jump ship in a
> heartbeat if there was an alternative -- note: Chrome means getting in
> bed with Google and so is not an alternative.
This proposal wouldn't change the rate of change or the frequency of
updates at all; those are independent issues.  It would, however,
generally improve the speed with which we'd be able to respond to
feedback.

-David

--
𝄞   L. David Baron                         http://dbaron.org/   𝄂
𝄢   Mozilla                          https://www.mozilla.org/   𝄂
             Before I built a wall I'd ask to know
             What I was walling in or walling out,
             And to whom I was like to give offense.
               - Robert Frost, Mending Wall (1914)

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: shorter release cycles?

Mark Filipak
On 2014/11/14 7:44 PM, L. David Baron wrote:

> On Friday 2014-11-14 19:08 -0500, Mark Filipak wrote:
>> Well, here I am replying to my own post, so it appears that what the
>> folks on the support-firefox list say is true, eh? ...that the
>> developers are not interested in listening to users and are not
>> interested in what users want.
>>
>> According to Chris Ilias, support-firefox is for users to help users,
>> not for feature requests or complaints. Well, okay. Joyless, but okay.
>>
>> The only other form of feedback is bugs, but that's not appropriate for
>> complaints or requests.
>>
>> That means there's no forum for complaints and requests.
>>
>> You know. There are a lot of unhappy FF users who would jump ship in a
>> heartbeat if there was an alternative -- note: Chrome means getting in
>> bed with Google and so is not an alternative.
>
> This proposal wouldn't change the rate of change or the frequency of
> updates at all; those are independent issues.  It would, however,
> generally improve the speed with which we'd be able to respond to
> feedback.

What feedback? Where do you get feedback?
--
The Insect Hall of Fame:
Thunderbird Bug 121947 - 12 years and counting.
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: shorter release cycles?

Robert Kaiser
In reply to this post by L. David Baron
L. David Baron schrieb:
> I'm curious what others think of this idea.

Given that we just showed with 33 that we have cases where we are unable
to reach a good stabilization point even with 12 weeks stabilization
time, and that in the last year we had at most a single release that was
actually stable enough during the whole beta phase, I do not think we
can do with any *day* less than 12 weeks of stabilization time for every
release.

Show me the actual data that says we can safely release with less and I
will be happy to discuss it.

KaiRo

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: shorter release cycles?

Robert Kaiser
In reply to this post by Mark Filipak
L. David Baron schrieb:
> This proposal wouldn't change the rate of change or the frequency of
> updates at all; those are independent issues.

It run the large danger of releasing less qualitative and less tested
software to users, though. And we also already have a weakness in the
amount of resources we have for quality. If you have a good plan on what
to do there to still ensure high quality (and better quality than the
pretty bad experience that 33 was), I'd love to hear it.

KaiRo
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: shorter release cycles?

Francesco Lodolo [:flod]
In reply to this post by L. David Baron
Il 14/11/14 21:08, L. David Baron ha scritto:
> I'm curious what others think of this idea.
Hi David,
have you considered l10n in your proposal?

Currently most l10n teams work on Aurora for 6 weeks: it's string frozen
(most of the time), strings arriving there are supposed to be in good
shape (there are a few localizers working on central and filing bugs),
therefore a stable environment to work in. Then code+localization moves
to beta with a larger audience and teams have an additional 5 weeks to
make further changes and improve quality.

If I read your proposal correctly:

  * Time available for localization goes from 11 weeks to 6 weeks top.
  * Beta population on week 2 will probably get a half translated
    product for most locales, even some of our biggest ones. That's not
    good, especially considering that most of our users are on
    non-English builds, and we need to keep the localization quality high.

I'm also ignoring the fact that currently l10n teams have to sign-off
their beta changesets for shipping, no idea how that would fit in this
proposal.
The alternative would be to move localization to Nightly, which is not
something that I suggest, being one of the few localizers active on that
channel.

I also have one question: how does Firefox Developer Edition impact this
plan?

Francesco
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: shorter release cycles?

Jet Villegas
In reply to this post by Robert Kaiser
FWIW, it's the current release systems that produced Firefox 33.

I think the reason for the graphics issues we've seen on 33 is that our testing population doesn't adequately represent the hardware/OS combinations actually in use "out there." David's proposal won't change the actual testing population, but that's a problem we should address separately.

I don't think that we'll get users with low-end hardware and older drivers to opt in to our testing channels. Instead, I like the proposal to quietly update release channel users with early releases and have them report back. I also like the proposal from Matt Woodrow to have a first-run GFx runtime check to render a fast reftest and read back from the GPU to test for successful rendering. This would have caught the failures we saw on 33, and rolled the users back from the DirectX changes.

BTW, I just contributed to this thread going off on a tangent (sorry!) Let's come to a decision on David's proposal, which I also support.

--Jet

----- Original Message -----
From: "Robert Kaiser" <[hidden email]>
To: [hidden email]
Sent: Saturday, November 15, 2014 7:23:06 AM
Subject: Re: shorter release cycles?

L. David Baron schrieb:
> This proposal wouldn't change the rate of change or the frequency of
> updates at all; those are independent issues.

It run the large danger of releasing less qualitative and less tested
software to users, though. And we also already have a weakness in the
amount of resources we have for quality. If you have a good plan on what
to do there to still ensure high quality (and better quality than the
pretty bad experience that 33 was), I'd love to hear it.

KaiRo
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: shorter release cycles?

Sebastian Hengst
In reply to this post by L. David Baron
-------- Original-Nachricht --------
Betreff: shorter release cycles?
Von: L. David Baron <[hidden email]>
Datum: 2014-11-14 21:08
> (I'd note that this proposal is basically independent of Aurora
> being a Developer Edition; the two things don't really interact.)
>
> -David
>
With the Aurora channel rebranded as Developer Edition, we could get
(and likely hope) to get more developer users to use this long neglected
branch and file web platform and developer tools specific bugs earlier.
For normal end users, the change would mean one week less of testing due
to the stabilization week, leaving 5 instead 6 weeks (subtract the final
week on both for RCs).

Firefox front-end development has partially adopted to shorter
land-to-release times by still working on the features during aurora
(and landing strings for localization earlier), e.g. Loop, download
blocking. Eliminating aurora, development should be shifted back to
central to get stabilization in beta.

Regarding the bugs found during different development stages, people
could be interested in these stats:

Bug counts for bugs landed on mozilla-beta while it was Gecko 34:

Filed when
central was Gecko <34: 23 bugs
Gecko 34 was central: 16 bugs
Gecko 34 was aurora: 104 bugs
Gecko 34 was beta: 110 bugs

It would interesting to see the main needs of the different stakeholders
for shorter land-to-release times. B2G like wants newer features faster,
also services like the self-serve support API. What else benefits
strongly from generally shorter land-to-release times? And what are the
risks for these?

Sebastian
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: shorter release cycles?

Chris Ilias-3
In reply to this post by L. David Baron
On 2014-11-14 7:56 PM, Mark Filipak wrote:
> On 2014/11/14 7:44 PM, L. David Baron wrote:
>
>> This proposal wouldn't change the rate of change or the frequency of
>> updates at all; those are independent issues.  It would, however,
>> generally improve the speed with which we'd be able to respond to
>> feedback.
>
> What feedback? Where do you get feedback?


In the Help menu, select "Submit Feedback" That info gets collected at
<http://input.mozilla.org/>, where a team of people read it and gather
data about the most common issues. That data along with data from the
support forum at <https://support.mozilla.org/> is then presented in a
report.

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
123