Quantcast

Thunderbird Build System Changes

classic Classic list List threaded Threaded
5 messages Options
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Thunderbird Build System Changes

Philipp Kewisch-2
Hi Folks,

most recently I ran across another set of issues that made it impossible
to build comm-central, because of mozilla-central code assuming that the
m-c directory is TOPSRCDIR. This is the root of issues like "mach build
faster" not working, "mach eslint" being notoriously hard to run without
hacks, and an error "Cannot find project mail" when running "mach build"
that can only be remedied by a clobber build as far as I know.


I found this so annoying that I checked what it would take to run
comm-central as a subdirectory of mozilla-central, effectively flipping
around the build. It turns out this was fairly easy, I have this running
locally.

I spoke with jcranmer about his efforts to merge m-c into c-c and this
also seems to be working quite well on one of the project branches, but
he hasn't found time to complete the work. Each approach has its
downsides and missing steps.


To make some progress on this issue, I want to start some discussion.

Personally I think we should start with my approach because it is easy
to change and doesn't require any extra process around integration and
merge schedules for three or more repositories. We would just need to
change buildbot to put the directories in the right place, and clean up
and push my patches. For developers, client.py could inform about the
change and provide steps to migrate.

With some luck the patches I have locally can be modified to work
regardless of if c-c is TOPSRCDIR or m-c. This would allow us to test
without changing automation first, then work on the automation bits
separately.

With my approach in place we can continue to work towards merging m-c
and c-c at our own pace without a lot of duplicate work. Some of the
changes in the patches would even still be valid after the merge. The
comm projects are in their own subdirectory which makes it easy to
separate comm and mozilla code and not accidentally change things that
might get conflicts from upstream. The downside I know of is that we
still can't run easy bisects, and due to the extra repository it
probably wouldn't make migrating to TaskCluster easier than it is now.

I'd like to ask you all if merging m-c into c-c is still the preferred
final goal, and if you think it is reasonable to flip m-c/c-c as an
intermediate step.

Based on that we can discuss the next steps.

Thanks,
Philipp
_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Thunderbird Build System Changes

R Kent James
On 3/18/2017 11:07 AM, Philipp Kewisch wrote:
> Hi Folks,
>
> I'd like to ask you all if merging m-c into c-c is still the preferred
> final goal, and if you think it is reasonable to flip m-c/c-c as an
> intermediate step.
>

I have never been enthusiastic about merging m-c into c-c, but I was
always in the minority. It is very common for projects to be built now
with multiple libraries from multiple sources, I don't see why we are
any different. Letting m-c be the root, with c-c a separate directory
and repository, is what I've always thought made sense.

Whatever approach that we take, it is going to be important to retain
our ability to make small changes to m-c, or to freeze m-c at a certain
revision level (hopefully temporarily) when changes to m-c cause massive
problems to Thunderbird. I think that will be easier if we keep the two
repositories distinct.

Like, what will we do if m-c implements a change that requires 4 months
of effort in c-c to accommodate? Freeze a combined c-c/m-c for 4 months?
Better to freeze our m-c version, and allow parallel development on c-c
of other issues while the massive problem is being fixed on a branch. We
really need to work toward shorter periods of time that c-c is closed
while dealing with m-c issues.

This is all probably possible with complex branches on a combined
m-c/c-c I just think that will be more difficult to manage.

:rkent



_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Thunderbird Build System Changes

Philipp Kewisch-2
On 3/18/17 9:34 PM, R Kent James wrote:
> On 3/18/2017 11:07 AM, Philipp Kewisch wrote:
>
> Like, what will we do if m-c implements a change that requires 4 months
> of effort in c-c to accommodate? Freeze a combined c-c/m-c for 4 months?

I guess we would not merge m-c into c-c for 4 months, which is
effectively freezing the m-c version.

I am worried that this freezing would happen more regularly if we merge
m-c into c-c, eventually leading to forking m-c. If we fully automate
the merging then it would be hard to freeze. If we do this manually it
boils down to someone doing the integration work on a daily basis, which
is an insane amount of effort.

Then there is also the question of how to manage our repositories. Is
there an comm-central-mozinbound that allows us to check if the
changesets work? What about aurora or beta or esr? Do we need separate
integration steps for those too?


> This is all probably possible with complex branches on a combined
> m-c/c-c I just think that will be more difficult to manage.

I have the same feeling and I brought up these concerns with jcranmer.
He says that once the merge has been done it is all very easy and there
should be no difficulties.

If we can automate this to a sufficient level then I'd be open to it,
but for the meanwhile I'd like a solution that reduces issues with m-c
changes.

Philipp
_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Thunderbird Build System Changes

ISHIKAWA,chiaki
In reply to this post by Philipp Kewisch-2
On 2017年03月19日 03:07, Philipp Kewisch wrote:

> Hi Folks,
>
> most recently I ran across another set of issues that made it impossible
> to build comm-central, because of mozilla-central code assuming that the
> m-c directory is TOPSRCDIR. This is the root of issues like "mach build
> faster" not working, "mach eslint" being notoriously hard to run without
> hacks, and an error "Cannot find project mail" when running "mach build"
> that can only be remedied by a clobber build as far as I know.
>
>
> I found this so annoying that I checked what it would take to run
> comm-central as a subdirectory of mozilla-central, effectively flipping
> around the build. It turns out this was fairly easy, I have this running
> locally.
>

Interesting idea.

> I spoke with jcranmer about his efforts to merge m-c into c-c and this
> also seems to be working quite well on one of the project branches, but
> he hasn't found time to complete the work. Each approach has its
> downsides and missing steps.
>
>
> To make some progress on this issue, I want to start some discussion.
>
> Personally I think we should start with my approach because it is easy
> to change and doesn't require any extra process around integration and
> merge schedules for three or more repositories. We would just need to
> change buildbot to put the directories in the right place, and clean up
> and push my patches. For developers, client.py could inform about the
> change and provide steps to migrate.
>
> With some luck the patches I have locally can be modified to work
> regardless of if c-c is TOPSRCDIR or m-c. This would allow us to test
> without changing automation first, then work on the automation bits
> separately.
>
> With my approach in place we can continue to work towards merging m-c
> and c-c at our own pace without a lot of duplicate work. Some of the
> changes in the patches would even still be valid after the merge. The
> comm projects are in their own subdirectory which makes it easy to
> separate comm and mozilla code and not accidentally change things that
> might get conflicts from upstream. The downside I know of is that we
> still can't run easy bisects, and due to the extra repository it
> probably wouldn't make migrating to TaskCluster easier than it is now.
>
> I'd like to ask you all if merging m-c into c-c is still the preferred
> final goal, and if you think it is reasonable to flip m-c/c-c as an
> intermediate step.
>
> Based on that we can discuss the next steps.
>
> Thanks,
> Philipp
>

I found the idea very interesting.
Whatever it takes to lessen the impact of m-c changes for TB development,
we should consider seriously.

One thing I am not sure is how difficult it is to make the necessary change
on try-comm-central and other TB build infrastructure.

_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Thunderbird Build System Changes

Philipp Kewisch-2
On 3/21/17 2:26 AM, ishikawa wrote:
>
> I found the idea very interesting.
> Whatever it takes to lessen the impact of m-c changes for TB development,
> we should consider seriously.
>
> One thing I am not sure is how difficult it is to make the necessary change
> on try-comm-central and other TB build infrastructure.
>

It would require some buildbot changes for both try and nightly/release
to put the repositories into the right directory, but is certainly not
too hard to do.

It sounds to me like there are no strong objections from anyone, if this
is the case I'd get this started.

I will leave another few days before I start filing bugs and sending
patches.

Thanks for the comments!
Philipp
_______________________________________________
dev-apps-thunderbird mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-apps-thunderbird
Loading...