Shippable Builds Incoming...

classic Classic list List threaded Threaded
3 messages Options
Reply | Threaded
Open this post in threaded view
|

Shippable Builds Incoming...

Justin Wood
Hey Everyone,

tl;dr I landed  "shippable builds" [1] to autoland, and I'll assume it
sticks. This replaces Nightly and PGO for most job types for desktop jobs.
Expect to see tests and builds in treeherder reflecting this, and no longer
seeing most PGO platforms or Nightly platforms available for your pushes.

Longer story is that we are replacing all nightlies and current pgo builds
with 'shippable' variants. These combine what we do for PGO and Nightlies
into one job type, with the main goal of consolidating build types, and
simplifying the build configurations. We will not be changing Fennec build
types at this time

This work mirrors in part what we did with opt->pgo tests [2] to keep our
overall test load down and does move osx on push to shippable (previously
was an opt build) and linux32 testing to shippable (was previously also on
an opt variant).

This will obsolete many of the taskcluster routes you have come to expect,
and instead using shippable indexes. We have made some efforts to preserve
old indexes where possible, but If you are using taskcluster indexes in any
of your own code, I'd love to know where and for what purpose, so we can
document it going forward.

Future work this unlocks is the ability to do "Nightly Promotion" which
should reduce the cost of Nightly CI and greatly speed up the turnaround
time for Nightly builds on mozilla-central. And once we are promoting
builds from the mozilla-central push to Nightly users, then we will truely
be testing against what we ship, so a net win for our code quality as well.

There will also be some future work to cleanup some of the code in the
taskgraph code itself, to remove confusing code paths that mixed 'nightly'
build types along with the 'nightly' cadence of a released product. As well
as work to remove the temporary transitional index routes I tried to add.

Feel free to find me in IRC #ci or on slack at #firefox-ci if you have any
questions or concerns.

Thank You,
~Justin Wood (Callek)

[1]
https://groups.google.com/d/msg/mozilla.dev.planning/JomJmzGOGMY/vytPViZBDgAJ
[2]
https://groups.google.com/d/msg/mozilla.dev.platform/0dYGajwXCBc/4cpYrUMyBgAJ
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Shippable Builds Incoming...

Kartikaya Gupta-6
Will the change to taskcluster indexes affect mozregression?

On Tue, Mar 26, 2019 at 2:56 PM Justin Wood <[hidden email]> wrote:

>
> Hey Everyone,
>
> tl;dr I landed  "shippable builds" [1] to autoland, and I'll assume it
> sticks. This replaces Nightly and PGO for most job types for desktop jobs.
> Expect to see tests and builds in treeherder reflecting this, and no longer
> seeing most PGO platforms or Nightly platforms available for your pushes.
>
> Longer story is that we are replacing all nightlies and current pgo builds
> with 'shippable' variants. These combine what we do for PGO and Nightlies
> into one job type, with the main goal of consolidating build types, and
> simplifying the build configurations. We will not be changing Fennec build
> types at this time
>
> This work mirrors in part what we did with opt->pgo tests [2] to keep our
> overall test load down and does move osx on push to shippable (previously
> was an opt build) and linux32 testing to shippable (was previously also on
> an opt variant).
>
> This will obsolete many of the taskcluster routes you have come to expect,
> and instead using shippable indexes. We have made some efforts to preserve
> old indexes where possible, but If you are using taskcluster indexes in any
> of your own code, I'd love to know where and for what purpose, so we can
> document it going forward.
>
> Future work this unlocks is the ability to do "Nightly Promotion" which
> should reduce the cost of Nightly CI and greatly speed up the turnaround
> time for Nightly builds on mozilla-central. And once we are promoting
> builds from the mozilla-central push to Nightly users, then we will truely
> be testing against what we ship, so a net win for our code quality as well.
>
> There will also be some future work to cleanup some of the code in the
> taskgraph code itself, to remove confusing code paths that mixed 'nightly'
> build types along with the 'nightly' cadence of a released product. As well
> as work to remove the temporary transitional index routes I tried to add.
>
> Feel free to find me in IRC #ci or on slack at #firefox-ci if you have any
> questions or concerns.
>
> Thank You,
> ~Justin Wood (Callek)
>
> [1]
> https://groups.google.com/d/msg/mozilla.dev.planning/JomJmzGOGMY/vytPViZBDgAJ
> [2]
> https://groups.google.com/d/msg/mozilla.dev.platform/0dYGajwXCBc/4cpYrUMyBgAJ
> _______________________________________________
> dev-platform mailing list
> [hidden email]
> https://lists.mozilla.org/listinfo/dev-platform
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Shippable Builds Incoming...

Justin Wood
Yes it does, I'm currently looking at a good way to improve mozregression
ASAP, which has been partially broken since the opt->pgo work mentioned
above.

This is tracked in 1532412
<https://bugzilla.mozilla.org/show_bug.cgi?id=1532412> and whatever
solution we get there should be translatable for shippable and make this
easy.

I'm hoping to touch base with :wlach soon so we can figure out a good
solution to this code.

~Justin Wood (Callek)

On Tue, Mar 26, 2019 at 3:05 PM Kartikaya Gupta <[hidden email]> wrote:

> Will the change to taskcluster indexes affect mozregression?
>
> On Tue, Mar 26, 2019 at 2:56 PM Justin Wood <[hidden email]> wrote:
> >
> > Hey Everyone,
> >
> > tl;dr I landed  "shippable builds" [1] to autoland, and I'll assume it
> > sticks. This replaces Nightly and PGO for most job types for desktop
> jobs.
> > Expect to see tests and builds in treeherder reflecting this, and no
> longer
> > seeing most PGO platforms or Nightly platforms available for your pushes.
> >
> > Longer story is that we are replacing all nightlies and current pgo
> builds
> > with 'shippable' variants. These combine what we do for PGO and Nightlies
> > into one job type, with the main goal of consolidating build types, and
> > simplifying the build configurations. We will not be changing Fennec
> build
> > types at this time
> >
> > This work mirrors in part what we did with opt->pgo tests [2] to keep our
> > overall test load down and does move osx on push to shippable (previously
> > was an opt build) and linux32 testing to shippable (was previously also
> on
> > an opt variant).
> >
> > This will obsolete many of the taskcluster routes you have come to
> expect,
> > and instead using shippable indexes. We have made some efforts to
> preserve
> > old indexes where possible, but If you are using taskcluster indexes in
> any
> > of your own code, I'd love to know where and for what purpose, so we can
> > document it going forward.
> >
> > Future work this unlocks is the ability to do "Nightly Promotion" which
> > should reduce the cost of Nightly CI and greatly speed up the turnaround
> > time for Nightly builds on mozilla-central. And once we are promoting
> > builds from the mozilla-central push to Nightly users, then we will
> truely
> > be testing against what we ship, so a net win for our code quality as
> well.
> >
> > There will also be some future work to cleanup some of the code in the
> > taskgraph code itself, to remove confusing code paths that mixed
> 'nightly'
> > build types along with the 'nightly' cadence of a released product. As
> well
> > as work to remove the temporary transitional index routes I tried to add.
> >
> > Feel free to find me in IRC #ci or on slack at #firefox-ci if you have
> any
> > questions or concerns.
> >
> > Thank You,
> > ~Justin Wood (Callek)
> >
> > [1]
> >
> https://groups.google.com/d/msg/mozilla.dev.planning/JomJmzGOGMY/vytPViZBDgAJ
> > [2]
> >
> https://groups.google.com/d/msg/mozilla.dev.platform/0dYGajwXCBc/4cpYrUMyBgAJ
> > _______________________________________________
> > dev-platform mailing list
> > [hidden email]
> > https://lists.mozilla.org/listinfo/dev-platform
>
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning