Build System Changes in Firefox 21

classic Classic list List threaded Threaded
11 messages Options
Reply | Threaded
Open this post in threaded view
|

Build System Changes in Firefox 21

Gregory Szorc-3
There will be a number of build system changes in Firefox 21 that I want
to be sure everyone is aware of.

When mozilla-central switches over to Firefox 21 on Monday, we will
require Python 2.7 to build the tree. This was previously announced on
this list [1]. Effort is being tracked in bug 804865.

The initial transition to moz.build files [2] should hopefully land next
week as well. This was previously announced at [3] And is being tracked
in bug 784841.

Finally, a fully rewritten packager should land in the first week or two
of the cycle. I don't believe this has been advertised on any major
list. But Mike Hommey is doing amazing work and it's a significant
change, so it deserves to be mentioned here. This is tracked in bug 780561.

If all goes to plan, you won't know anything has changed. *You will
still build the tree like you always have.*

If you are touching Makefile.ins, there is a strong possibility for
merge conflicts with the moz.build transition. Things could be rough as
that merge propagates to all project branches. Because of this, we'll
communicate details of the moz.build landing once they are known and
will work with owners of popular project branches to facilitate the
transition.

If you are planning to land significant changes to Makefiles (especially
if you are creating new ones), please ensure :ted, :gps, and :glandium
are CC'd on the bug and please get review from one of us (generally
speaking all but trivial Makefile.in changes - like adding a new item to
an existing variable/list - should have build peer review).

If you see any build failures, don't hesitate to say something in #build
and/or file a bug against Core :: Build Config.

[1]
https://groups.google.com/d/topic/mozilla.dev.platform/jWucT3Idw_4/discussion
[2]
http://gregoryszorc.com/presentations/2012-11-29-firefox-build-system/#39
[3]
https://groups.google.com/d/topic/mozilla.dev.planning/iZOrriRMYIg/discussion
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Build System Changes in Firefox 21

Mike Hommey
On Sat, Jan 05, 2013 at 01:24:03PM -0800, Gregory Szorc wrote:

> There will be a number of build system changes in Firefox 21 that I
> want to be sure everyone is aware of.
>
> When mozilla-central switches over to Firefox 21 on Monday, we will
> require Python 2.7 to build the tree. This was previously announced
> on this list [1]. Effort is being tracked in bug 804865.
>
> The initial transition to moz.build files [2] should hopefully land
> next week as well. This was previously announced at [3] And is being
> tracked in bug 784841.
>
> Finally, a fully rewritten packager should land in the first week or
> two of the cycle. I don't believe this has been advertised on any
> major list. But Mike Hommey is doing amazing work and it's a
> significant change, so it deserves to be mentioned here. This is
> tracked in bug 780561.

It's actually not a significant change from the packaging manifest
perspective. At least not for now. There's only about one file name that
changes and needs to be changed in removed-files.in, but that's about
all that should be necessary for now.

There are, however, disruptive changes planned for later, but that will
"just" make us have to update the package manifest much less frequently.

Mike
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Build System Changes in Firefox 21

Gregory Szorc-3
In reply to this post by Gregory Szorc-3
On 1/5/13 1:24 PM, Gregory Szorc wrote:
> The initial transition to moz.build files [2] should hopefully land next
> week as well. This was previously announced at [3] And is being tracked
> in bug 784841.

It took a little longer than expected, but landing the moz.build
transition in m-c is near! build-system is staged with the changes and
we're just waiting for final verification and to finalize the procedure
and date.

The proposal:

1) Pick a day (preferably ASAP).
2) Merge in the morning PST, say 8 or 9 AM, possibly earlier.
3) Trigger Nightlies as soon as the merge occurs.
4) Fix any problems we encounter in l10n repacks, etc during the day.

Open questions:

* Which branches? We were planning to merge into both m-c and inbound.
We could also merge into fx-team, etc if desired. Let me know if you
want in.
* Do we want tree closures? Which trees? How long? I'm fine with no
closure, closure for just the merge, closure until builds start to come
in, and closure until all builds come in.
* Impact on upcoming B2G tagging. (Unless directories are added,
removed, or renamed, patches to Makefile.in may contain contextual bit
rot but otherwise should have no conflicts.)
* Which day? m-c could technically happen as early as Tuesday. But, tack
on a day for us to look at unbusting comm-central and time for people to
respond and we're likely looking at Wednesday or Thursday. Friday stinks
because it's a Friday and everybody's lookin' forward to the weekend,
weekend. Fun. Fun. Fun. Fun. Does Wednesday or Thursday work for everybody?

Gregory
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Build System Changes in Firefox 21

nick.r.cameron
In reply to this post by Gregory Szorc-3
We have a fair few changes to makefiles on the graphics branch, although mostly just for adding files (+ two directories). If you could merge this for us it would be great. If it is likely to be an easy merge, then we can deal with it, but it would be useful to know what we have to do (hopefully without knowing too much build system magic).

Thanks, Nick (nrc).
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Build System Changes in Firefox 21

Joshua Cranmer 🐧
In reply to this post by Gregory Szorc-3
On 2/25/2013 6:23 PM, Gregory Szorc wrote:
> * Which branches? We were planning to merge into both m-c and inbound.
> We could also merge into fx-team, etc if desired. Let me know if you
> want in.
> * Do we want tree closures? Which trees? How long? I'm fine with no
> closure, closure for just the merge, closure until builds start to
> come in, and closure until all builds come in.

I think the standard operating procedure for these kinds of mega-merges
is a process akin to this (going by what I recall of the PRBool change):
1. Close m-c, m-i, and c-c.
2. Merge m-i into m-c.
3. Land patches on m-c.
4. Land patches on c-c.
5. When m-c goes green, merge m-c to m-i
6. Reopen everybody for checkin.

> * Which day? m-c could technically happen as early as Tuesday. But,
> tack on a day for us to look at unbusting comm-central and time for
> people to respond and we're likely looking at Wednesday or Thursday.
> Friday stinks because it's a Friday and everybody's lookin' forward to
> the weekend, weekend. Fun. Fun. Fun. Fun. Does Wednesday or Thursday
> work for everybody?

I would like to request that you don't land on m-c until you at least
have working patches to unbreak c-c awaiting review.

--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Build System Changes in Firefox 21

Gregory Szorc-3
On 2/25/13 7:50 PM, Joshua Cranmer 🐧 wrote:

> On 2/25/2013 6:23 PM, Gregory Szorc wrote:
>> * Which branches? We were planning to merge into both m-c and inbound.
>> We could also merge into fx-team, etc if desired. Let me know if you
>> want in.
>> * Do we want tree closures? Which trees? How long? I'm fine with no
>> closure, closure for just the merge, closure until builds start to
>> come in, and closure until all builds come in.
>
> I think the standard operating procedure for these kinds of mega-merges
> is a process akin to this (going by what I recall of the PRBool change):
> 1. Close m-c, m-i, and c-c.
> 2. Merge m-i into m-c.
> 3. Land patches on m-c.
> 4. Land patches on c-c.
> 5. When m-c goes green, merge m-c to m-i
> 6. Reopen everybody for checkin.

That works for me if it works for everyone else!

>> * Which day? m-c could technically happen as early as Tuesday. But,
>> tack on a day for us to look at unbusting comm-central and time for
>> people to respond and we're likely looking at Wednesday or Thursday.
>> Friday stinks because it's a Friday and everybody's lookin' forward to
>> the weekend, weekend. Fun. Fun. Fun. Fun. Does Wednesday or Thursday
>> work for everybody?
>
> I would like to request that you don't land on m-c until you at least
> have working patches to unbreak c-c awaiting review.

I have a nearly-working c-c patch in bug 845089. I'm pretty confident
the final patch requires just a few hours of work and testing and will
be ready to land by tomorrow.
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Build System Changes in Firefox 21

Gregory Szorc-3
In reply to this post by Gregory Szorc-3
On 2/25/13 4:23 PM, Gregory Szorc wrote:

> On 1/5/13 1:24 PM, Gregory Szorc wrote:
>> The initial transition to moz.build files [2] should hopefully land next
>> week as well. This was previously announced at [3] And is being tracked
>> in bug 784841.
>
> It took a little longer than expected, but landing the moz.build
> transition in m-c is near! build-system is staged with the changes and
> we're just waiting for final verification and to finalize the procedure
> and date.
>
> The proposal:
>
> 1) Pick a day (preferably ASAP).
> 2) Merge in the morning PST, say 8 or 9 AM, possibly earlier.
> 3) Trigger Nightlies as soon as the merge occurs.
> 4) Fix any problems we encounter in l10n repacks, etc during the day.

In the Platform meeting today, I was given approval to perform the merge
as soon as the comm-central patches are ready. This could happen as
early as tomorrow (Wednesday). Given I still don't have the c-c patches
working just right, it is looking more like Thursday.

m-c should be closed for no more than a few hours during the merge.

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Build System Changes in Firefox 21

Ms2ger
In reply to this post by Joshua Cranmer 🐧
On 02/26/2013 06:54 PM, Gregory Szorc wrote:

> On 2/25/13 7:50 PM, Joshua Cranmer 🐧 wrote:
>> On 2/25/2013 6:23 PM, Gregory Szorc wrote:
>>> * Which branches? We were planning to merge into both m-c and inbound.
>>> We could also merge into fx-team, etc if desired. Let me know if you
>>> want in.
>>> * Do we want tree closures? Which trees? How long? I'm fine with no
>>> closure, closure for just the merge, closure until builds start to
>>> come in, and closure until all builds come in.
>>
>> I think the standard operating procedure for these kinds of mega-merges
>> is a process akin to this (going by what I recall of the PRBool change):
>> 1. Close m-c, m-i, and c-c.
>> 2. Merge m-i into m-c.
>> 3. Land patches on m-c.
>> 4. Land patches on c-c.
>> 5. When m-c goes green, merge m-c to m-i
>> 6. Reopen everybody for checkin.
>
> That works for me if it works for everyone else!
>
>>> * Which day? m-c could technically happen as early as Tuesday. But,
>>> tack on a day for us to look at unbusting comm-central and time for
>>> people to respond and we're likely looking at Wednesday or Thursday.
>>> Friday stinks because it's a Friday and everybody's lookin' forward to
>>> the weekend, weekend. Fun. Fun. Fun. Fun. Does Wednesday or Thursday
>>> work for everybody?
>>
>> I would like to request that you don't land on m-c until you at least
>> have working patches to unbreak c-c awaiting review.
>
> I have a nearly-working c-c patch in bug 845089. I'm pretty confident
> the final patch requires just a few hours of work and testing and will
> be ready to land by tomorrow.

FYI: I have started the merge. mozilla-central, mozilla-inbound, fx-team
and services-central are all closed and have been merged to
build-system, and we're waiting for green there. Once that has happened,
I expect we will:

1. merge build-system back to mozilla-central;
2. land bug 845089 on comm-central;
3. merge to mozilla-inbound, fx-team and services-central;
4. merge to the graphics branch;
5. reopen the trees;
6. make another announcement on this list.

HTH
Ms2ger
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Build System Changes in Firefox 21

Ryan VanderMeulen
On 2/28/2013 9:28 AM, Ms2ger wrote:

> FYI: I have started the merge. mozilla-central, mozilla-inbound, fx-team
> and services-central are all closed and have been merged to
> build-system, and we're waiting for green there. Once that has happened,
> I expect we will:
>
> 1. merge build-system back to mozilla-central;
> 2. land bug 845089 on comm-central;
> 3. merge to mozilla-inbound, fx-team and services-central;
> 4. merge to the graphics branch;
> 5. reopen the trees;
> 6. make another announcement on this list.
>
> HTH
> Ms2ger

++ \o/

Thanks for shepherding it through!
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Build System Changes in Firefox 21

Ms2ger
In reply to this post by Ms2ger
On 02/28/2013 03:28 PM, Ms2ger wrote:

> On 02/26/2013 06:54 PM, Gregory Szorc wrote:
>> On 2/25/13 7:50 PM, Joshua Cranmer 🐧 wrote:
>>> On 2/25/2013 6:23 PM, Gregory Szorc wrote:
>>>> * Which branches? We were planning to merge into both m-c and inbound.
>>>> We could also merge into fx-team, etc if desired. Let me know if you
>>>> want in.
>>>> * Do we want tree closures? Which trees? How long? I'm fine with no
>>>> closure, closure for just the merge, closure until builds start to
>>>> come in, and closure until all builds come in.
>>>
>>> I think the standard operating procedure for these kinds of mega-merges
>>> is a process akin to this (going by what I recall of the PRBool change):
>>> 1. Close m-c, m-i, and c-c.
>>> 2. Merge m-i into m-c.
>>> 3. Land patches on m-c.
>>> 4. Land patches on c-c.
>>> 5. When m-c goes green, merge m-c to m-i
>>> 6. Reopen everybody for checkin.
>>
>> That works for me if it works for everyone else!
>>
>>>> * Which day? m-c could technically happen as early as Tuesday. But,
>>>> tack on a day for us to look at unbusting comm-central and time for
>>>> people to respond and we're likely looking at Wednesday or Thursday.
>>>> Friday stinks because it's a Friday and everybody's lookin' forward to
>>>> the weekend, weekend. Fun. Fun. Fun. Fun. Does Wednesday or Thursday
>>>> work for everybody?
>>>
>>> I would like to request that you don't land on m-c until you at least
>>> have working patches to unbreak c-c awaiting review.
>>
>> I have a nearly-working c-c patch in bug 845089. I'm pretty confident
>> the final patch requires just a few hours of work and testing and will
>> be ready to land by tomorrow.
>
> FYI: I have started the merge. mozilla-central, mozilla-inbound, fx-team
> and services-central are all closed and have been merged to
> build-system, and we're waiting for green there. Once that has happened,
> I expect we will:
>
> 1. merge build-system back to mozilla-central;
> 2. land bug 845089 on comm-central;
> 3. merge to mozilla-inbound, fx-team and services-central;
> 4. merge to the graphics branch;
> 5. reopen the trees;
> 6. make another announcement on this list.

And here we are. C-C is still broken; top men are working on it. For
help on what's now defined in moz.build files, use "./mach
mozbuild-reference". Gregory will be writing more extensively on what
has happened and what is still in the pipeline.

--
HTH
Ms2ger

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Build System Changes in Firefox 21

Gregory Szorc-3
On 2/28/2013 11:45 AM, Ms2ger wrote:
> And here we are. C-C is still broken; top men are working on it. For
> help on what's now defined in moz.build files, use "./mach
> mozbuild-reference". Gregory will be writing more extensively on what
> has happened and what is still in the pipeline.
>

After Ms2ger's heroic tree shepherding, all the trees are currently open
and we are in a pretty good state!

The only known issue on m-c was a hiccup on the Android Nightly due to
an existing bug that moz.build turned into a fatal packaging error. It
has been fixed and no day's Nightly was lost as part of this landing.

The Thunderbird builds were originally broken on all platforms. But we
worked through it and they are now all green and the tree is open.

Seamonkey is currently busted and is being tracked in bug 846524 (sorry
for lack of context in bug - most context was on IRC). It should be
green again sometime Friday.

I've written a post on what the landing means for everybody. I encourage
developers to take the time to read it (especially if you touch Makefiles).

http://gregoryszorc.com/blog/2013/02/28/moz.build-files-and-the-firefox-build-system/

Gregory

P.S. Somebody should persistently pester me to crank out up-to-date
build docs on MDN.
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning