Proposal: Move Thunderbird and SeaMonkey to mozilla-central

classic Classic list List threaded Threaded
143 messages Options
12345 ... 8
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move Thunderbird and SeaMonkey to mozilla-central

Benjamin Smedberg
On 4/3/2014 10:01 AM, Joshua Cranmer 🐧 wrote:

>
>
> That said, to reiterate Henri's point, there is a surprising amount of
> code in mozilla-central that doesn't end up being used in Firefox.
> Having filed several bugs for mozilla-central test failures found only
> in comm-central, there are definitely cases where the failure is "oh,
> we don't use <insert toolkit component here> in Firefox" (a recent
> example is the toolkit's downloader). There are also cases where there
> is dynamically-dead code in Firefox that is not dynamically dead in
> Thunderbird: I think for a time, Thunderbird was the only one using
> certain legacy HTML parser APIs.

We should remove that code from mozilla-central/libxul. Our technical
debt is substantial and one of the easiest ways to reduce it is by
removing unneeded code.

>
>>
>> Reason #3: It's unnecessary
>>
>> It is not necessary to alter mozilla-central to put all of
>> Thunderbird in one repository. You can do the merge you are
>> proposing, but keep it as a separate Thunderbird tree and regularly
>> merge changes from mozilla-central into your combined tree.
>
> That works much less well than you'd expect. In such a repository, it
> is impossible to merge changes from comm-central back into
> mozilla-central without introducing c-c's history or rather
> complicated changeset manipulation. It also produces a comm-central
> repository that has the illusion of atomic changesets: there is no way
> to make a commit that does an across-both-trees change such as the
> atomic refcounting changes or gps's DIRS introduction
It is possible with mercurial queues to easily split patches up across
multiple trees. You can also make atomic commits to comm-central by
making the mozilla-central change and avoid merging that until you've
committed the equivalent comm-central changes.

--BDS

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move Thunderbird and SeaMonkey to mozilla-central

Neil-4
In reply to this post by Benjamin Smedberg
Henri Sivonen wrote:
> I'm still looking for a volunteer for
> https://bugzilla.mozilla.org/show_bug.cgi?id=937056
Wouldn't this be an example of something that would be much easier to
achieve with a unfied repo?

--
Warning: May contain traces of nuts.

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move Thunderbird and SeaMonkey to mozilla-central

Gregory Szorc-3
In reply to this post by Ben Hearsum-2
On 4/3/14, 8:22 AM, Ben Hearsum wrote:
> On 04/03/14 11:15 AM, Mark Banner wrote:
>> Ever since we separated the repositories, we have spent significant
>> amounts of time porting build configuration patches, fire-fighting
>> bustage when something lands in m-c and needs porting to c-c (including
>> sometimes regression range finding & bisecting). Obviously fire-fighting
>> is a larger pain as it distracts us from whatever we were working on.
>
> Perhaps we should move Firefox/B2G/Android to another repo instead, then
> we'd all be living in the same world.

Fragmenting repositories is not the answer and only creates more problems.

Unified repositories are a good thing:

1. You have a single commit identifier that identifies the state of the
entire world
2. You can make large changes across multiple projects atomically
3. The tooling doesn't suck (all solutions for subrepos or multi
repository management are significantly worse than single repo experiment).

Many companies that have encountered this problem have made the
determination that having all code live in one repository and thus
available to everyone all the time is a far greater benefit than to have
people constantly agonizing with the downsides of fragmented repositories.

These companies like the benefits so much that they use scalable version
control systems (like Perforce and Mercurial) that can handle tens or
hundreds of thousands of files and gigabytes of data. (Git doesn't scale
well into this territory. Search "Facebook Git scaling" on Google if you
doubt me - you shouldn't doubt Facebook's engineering team.)

While I concede that Mozilla's needs are slightly different from that of
a large corporation with a repository behind a firewall, the benefits of
unified repositories remain. The drawbacks (large clone times) can
largely be mitigated through extensions (with Mercurial at least).

To use a Mozilla example to support claim #2, I'll use mozbase. mozbase
is a suite of Python packages. It's effectively the "Mozilla standard
library." It used to live in GitHub and was merged to mozilla-central
periodically. People working on Python in the tree would often need or
want to make changes to mozbase. They had to commit to Git, wait for a
merge to m-c, then commit whatever changes were depending on it. The
process was brutally agonizing. Often days would go by between mozbase
merges. Stop energy was everywhere. Now, mozbase lives inside
mozilla-central. In one atomic commit I can update mozbase *and* change
all in-tree consumers in one go. It's faster for everybody and makes
everyone more productive. It keeps Mozilla fast and nimble.

If we split Firefox, Android, etc into separate repositories, I think it
would add way too much overhead and pitfalls for everyone involved and
we'd all be screaming to go back to the old world.
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move Thunderbird and SeaMonkey to mozilla-central

Gregory Szorc-3
In reply to this post by Benjamin Smedberg
On 4/3/14, 6:07 AM, Benjamin Smedberg wrote:

> On 4/3/2014 6:13 AM, Mark Banner wrote:
>>
>> Proposal
>> --------
>>
>> We'd like to move Thunderbird and SeaMonkey (i.e. most of the contents
>> of comm-central) into mozilla-central.
>
> I oppose this plan.
>
> I think it's fine to reconsider the decision we made in 2008 not to
> include tbird/seamonkey in mozilla-central, but I don't think the
> arguments here are compelling or that the benefits are worth the costs.
>
> Reason #1: confusion about what code belongs to what projects
>
> Currently when hacking on Firefox, it's roughly clear that all the code
> in mozilla-central belongs to Firefox in some way, and we have been
> effectively removing most of the code that isn't part of Firefox. When
> searching through the codebase (either locally or using the online dools
> like DXR/MXR) the addition of code which belongs to different products
> will significantly harm developer understanding.
>
> Reason #2: Pull/tree size
>
> There is a non-trivial amount of code involved in this change. In terms
> of both downloading the source repo as well as the size of a local
> checkout, I don't think we should include unnecessary code in the
> Firefox tree.

I believe this reason should be ignored on the basis that it's a
solvable and inevitable problem.

If Mozilla deployed the remotefilelog Mercurial extension on
hg.mozilla.org, people could clone mozilla-central *without* having to
pull the full repository data. A fresh clone of mozilla-central would be
several hundred MB of wire transfer less than today.

We could also write a Mercurial extension that distributed a shallow
bundle. We could probably also write an extension that did partial
bundles, pulls, and checkouts.

These are all solvable problems (with Mercurial at least - Git isn't as
flexible). They just require someone to implement. If we wait long
enough, Facebook may be our savior (they wrote remotefilelog). But they
don't care too much about Windows and non-SSH access. Furthermore, there
aren't that many entities scaling version control to the limits we are.
I fear that fixing these problems will need to come out of Mozilla's
budget sooner or later.

The tree size argument also ignores the harsh reality that the tree will
continue to grow over time. We'll be hitting scaling pains with or
without comm-central. By some accounts, we're already there.

> Reason #3: It's unnecessary
>
> It is not necessary to alter mozilla-central to put all of Thunderbird
> in one repository. You can do the merge you are proposing, but keep it
> as a separate Thunderbird tree and regularly merge changes from
> mozilla-central into your combined tree.

Merging mozilla-central into comm-central periodically does seem
somewhat reasonable to me. That would get rid of most of the special
attention needs that comm-central has inflicted upon mozilla-central's
build system. (I would love to excise those.) But the "atomic commit"
issues remain.
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move Thunderbird and SeaMonkey to mozilla-central

Justin Dolske-2
In reply to this post by Neil-4
On 4/3/14 3:15 PM, Neil wrote:
> Henri Sivonen wrote:
>> I'm still looking for a volunteer for
>> https://bugzilla.mozilla.org/show_bug.cgi?id=937056
> Wouldn't this be an example of something that would be much easier to
> achieve with a unfied repo?

Not clear to me, but I'd say it definitely is an example of the costs
carried by our core-focus products due to the quasi-supported state of
SM+TB. To stir the pot again, the simplest thing to do here would be to
remove the code, and let SM+TB be responsible for adding it back to a
place if they need to keep using it. Yes, I know that sucks for SM/TB
developers.

Justin

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move Thunderbird and SeaMonkey to mozilla-central

Joshua Cranmer 🐧
In reply to this post by Joshua Cranmer 🐧
On 4/3/2014 4:39 PM, Benjamin Smedberg wrote:

> On 4/3/2014 10:01 AM, Joshua Cranmer 🐧 wrote:
>>
>>
>> That said, to reiterate Henri's point, there is a surprising amount
>> of code in mozilla-central that doesn't end up being used in Firefox.
>> Having filed several bugs for mozilla-central test failures found
>> only in comm-central, there are definitely cases where the failure is
>> "oh, we don't use <insert toolkit component here> in Firefox" (a
>> recent example is the toolkit's downloader). There are also cases
>> where there is dynamically-dead code in Firefox that is not
>> dynamically dead in Thunderbird: I think for a time, Thunderbird was
>> the only one using certain legacy HTML parser APIs.
>
> We should remove that code from mozilla-central/libxul. Our technical
> debt is substantial and one of the easiest ways to reduce it is by
> removing unneeded code.

My point is that the current situation of separate repositories ends up
being a poor guide already to the true Firefox/Fennec/B2G versus
not-those code repositories, and no one has (to my knowledge) complained
that this is the problem. Moving comm-central into mozilla-central is
not in any way, shape, or form changing the rules that commits to
mozilla-central are not required to keep Thunderbird et al functioning;
therefore, it does not affect technical debt at all. Indeed, if
anything, the solution as proposed may help reduce technical debt by
letting people afraid to remove APIs that have use in Thunderbird remove
them more easily (one person has already suggested he might do this over
IRC).

>
>>
>>>
>>> Reason #3: It's unnecessary
>>>
>>> It is not necessary to alter mozilla-central to put all of
>>> Thunderbird in one repository. You can do the merge you are
>>> proposing, but keep it as a separate Thunderbird tree and regularly
>>> merge changes from mozilla-central into your combined tree.
>>
>> That works much less well than you'd expect. In such a repository, it
>> is impossible to merge changes from comm-central back into
>> mozilla-central without introducing c-c's history or rather
>> complicated changeset manipulation. It also produces a comm-central
>> repository that has the illusion of atomic changesets: there is no
>> way to make a commit that does an across-both-trees change such as
>> the atomic refcounting changes or gps's DIRS introduction
> It is possible with mercurial queues to easily split patches up across
> multiple trees. You can also make atomic commits to comm-central by
> making the mozilla-central change and avoid merging that until you've
> committed the equivalent comm-central changes.

That sounds like the words of someone who has never had to deal with
developing a repository in this kind of situation. A normal comm-central
developer would generally be using the comm-central side of the merge
and would need to remember to update to the mozilla-central side before
pushing. It takes just one person being forgetful to end up with the
comm-central side pushed to mozilla-central. It's a recipe for disaster
in other ways too: patches that affect both non-c-c and c-c code would
still apply equally, and you're relying on all the contributors
remembering to split them up.

It also makes the maintenance burden much more complicated: it would no
longer be possible to bisect the mozilla-central changes to find the
causes of bugs that break Thunderbird code. (Note that such bugs can
also be broken Firefox code that are masqueraded by the way tests are
run--I can think of two or three times that has happened). One of the
goals of the merge is to reduce the maintenance burden to continue the
Thunderbird support to which Mozilla is committed while requiring less
Mozilla resources to maintain that support.

--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move Thunderbird and SeaMonkey to mozilla-central

Mike Hommey
In reply to this post by Gregory Szorc-3
On Thu, Apr 03, 2014 at 03:27:18PM -0700, Gregory Szorc wrote:

> On 4/3/14, 6:07 AM, Benjamin Smedberg wrote:
> >On 4/3/2014 6:13 AM, Mark Banner wrote:
> >>
> >>Proposal
> >>--------
> >>
> >>We'd like to move Thunderbird and SeaMonkey (i.e. most of the contents
> >>of comm-central) into mozilla-central.
> >
> >I oppose this plan.
> >
> >I think it's fine to reconsider the decision we made in 2008 not to
> >include tbird/seamonkey in mozilla-central, but I don't think the
> >arguments here are compelling or that the benefits are worth the costs.
> >
> >Reason #1: confusion about what code belongs to what projects
> >
> >Currently when hacking on Firefox, it's roughly clear that all the code
> >in mozilla-central belongs to Firefox in some way, and we have been
> >effectively removing most of the code that isn't part of Firefox. When
> >searching through the codebase (either locally or using the online dools
> >like DXR/MXR) the addition of code which belongs to different products
> >will significantly harm developer understanding.
> >
> >Reason #2: Pull/tree size
> >
> >There is a non-trivial amount of code involved in this change. In terms
> >of both downloading the source repo as well as the size of a local
> >checkout, I don't think we should include unnecessary code in the
> >Firefox tree.
>
> I believe this reason should be ignored on the basis that it's a solvable
> and inevitable problem.
>
> If Mozilla deployed the remotefilelog Mercurial extension on hg.mozilla.org,
> people could clone mozilla-central *without* having to pull the full
> repository data. A fresh clone of mozilla-central would be several hundred
> MB of wire transfer less than today.

remotefilelog only works when you have low latency access to the cache
server.

> We could also write a Mercurial extension that distributed a shallow bundle.
> We could probably also write an extension that did partial bundles, pulls,
> and checkouts.
>
> These are all solvable problems (with Mercurial at least - Git isn't as
> flexible).

Git has shallow clones, so it has the problem solved already.

> They just require someone to implement. If we wait long enough,
> Facebook may be our savior (they wrote remotefilelog). But they don't care
> too much about Windows and non-SSH access. Furthermore, there aren't that
> many entities scaling version control to the limits we are. I fear that
> fixing these problems will need to come out of Mozilla's budget sooner or
> later.
>
> The tree size argument also ignores the harsh reality that the tree will
> continue to grow over time. We'll be hitting scaling pains with or without
> comm-central. By some accounts, we're already there.
>
> >Reason #3: It's unnecessary
> >
> >It is not necessary to alter mozilla-central to put all of Thunderbird
> >in one repository. You can do the merge you are proposing, but keep it
> >as a separate Thunderbird tree and regularly merge changes from
> >mozilla-central into your combined tree.
>
> Merging mozilla-central into comm-central periodically does seem somewhat
> reasonable to me. That would get rid of most of the special attention needs
> that comm-central has inflicted upon mozilla-central's build system. (I
> would love to excise those.) But the "atomic commit" issues remain.

The "atomic commit" issue will likely remain whether TB and SM files are
in mozilla-central or not. Albeit probably less often.

Mike
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move Thunderbird and SeaMonkey to mozilla-central

Mike Hommey
In reply to this post by Gregory Szorc-3
On Thu, Apr 03, 2014 at 03:19:41PM -0700, Gregory Szorc wrote:

> On 4/3/14, 8:22 AM, Ben Hearsum wrote:
> >On 04/03/14 11:15 AM, Mark Banner wrote:
> >>Ever since we separated the repositories, we have spent significant
> >>amounts of time porting build configuration patches, fire-fighting
> >>bustage when something lands in m-c and needs porting to c-c (including
> >>sometimes regression range finding & bisecting). Obviously fire-fighting
> >>is a larger pain as it distracts us from whatever we were working on.
> >
> >Perhaps we should move Firefox/B2G/Android to another repo instead, then
> >we'd all be living in the same world.
>
> Fragmenting repositories is not the answer and only creates more problems.
>
> Unified repositories are a good thing:
>
> 1. You have a single commit identifier that identifies the state of the
> entire world

Which you also have if you use submodules/subrepositories

> 2. You can make large changes across multiple projects atomically

which you also can if you use submodules/subrepositories

> 3. The tooling doesn't suck (all solutions for subrepos or multi repository
> management are significantly worse than single repo experiment).

True. But it seems to me the problem is noone actively seeked fixing
those problems and just went the "easy" way by putting everything in the
same basket. And suffering as a consequence. Which leads to what you're
writing next.

> Many companies that have encountered this problem have made the
> determination that having all code live in one repository and thus available
> to everyone all the time is a far greater benefit than to have people
> constantly agonizing with the downsides of fragmented repositories.
>
> These companies like the benefits so much that they use scalable version
> control systems (like Perforce and Mercurial) that can handle tens or
> hundreds of thousands of files and gigabytes of data. (Git doesn't scale
> well into this territory. Search "Facebook Git scaling" on Google if you
> doubt me - you shouldn't doubt Facebook's engineering team.)

Arguably, they had problems with mercurial too. They just ended up
working on fixing them, which they could equally have done in git except
it's not written in python.

The fact is, they either had to fix those performance issues, or suffer
having to break down their stuff in pieces and improve the workflow with
submodules/subrepositories. The latter is likely more work, especially
the "break it down" part.

> While I concede that Mozilla's needs are slightly different from that of a
> large corporation with a repository behind a firewall, the benefits of
> unified repositories remain. The drawbacks (large clone times) can largely
> be mitigated through extensions (with Mercurial at least).
>
> To use a Mozilla example to support claim #2, I'll use mozbase. mozbase is a
> suite of Python packages. It's effectively the "Mozilla standard library."
> It used to live in GitHub and was merged to mozilla-central periodically.
> People working on Python in the tree would often need or want to make
> changes to mozbase. They had to commit to Git, wait for a merge to m-c, then
> commit whatever changes were depending on it. The process was brutally
> agonizing. Often days would go by between mozbase merges. Stop energy was
> everywhere. Now, mozbase lives inside mozilla-central. In one atomic commit
> I can update mozbase *and* change all in-tree consumers in one go. It's
> faster for everybody and makes everyone more productive. It keeps Mozilla
> fast and nimble.

Except now, it's the other users of mozbase that aren't in m-c that suffer.

Mike
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move Thunderbird and SeaMonkey to mozilla-central

Joshua Cranmer 🐧
In reply to this post by Joshua Cranmer 🐧
On 4/3/2014 11:21 AM, Nick Alexander wrote:
> On 2014-04-03, 8:53 AM, Joshua Cranmer 🐧 wrote:
>> The factors that have compelled us to propose this merge are not
>> generally applicable to most products that use Gecko.
>
> Can you elaborate on this point?  I'd like to understand this
> difference more fully.

1. The mailnews code was originally fully integrated into the Mozilla
build system since before even the public CVS repository was created.
2. Mailnews code more or less requires being linked into libxul. As a
result, this means that Thunderbird can't extricate itself from the
mozilla-central build system.
3. Mozilla-central has historically had to made accommodations
specifically for comm-central, most notably in the build system. Moving
comm-central into mozilla-central would allow us to remove those
accommodations. I don't think such accommodations have been extended to
4. Mozilla has agreed to continue supporting Thunderbird for the
foreseeable future.
5. Mozilla's source tree has historically not been mozilla-central but
the combination of mozilla-central and comm-central, at least as far as
some things like "is this used in the tree" or "tree-wide mass rewrites"
have been concerned.

--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move Thunderbird and SeaMonkey to mozilla-central

Joshua Cranmer 🐧
In reply to this post by Benjamin Smedberg
On 4/3/2014 5:27 PM, Gregory Szorc wrote:
> The tree size argument also ignores the harsh reality that the tree
> will continue to grow over time. We'll be hitting scaling pains with
> or without comm-central. By some accounts, we're already there.

I did a little experiment to get a sense of the growth rate of
mozilla-central. I selected roughly monthly revisions from
mozilla-central, made a new repository, and pulled those revisions in
one by one, measuring the size of the .hg directory at each point. The
resulting graph is here:
<https://docs.google.com/spreadsheets/d/1ZXu1-5ILH5r87cZ7G2DOVl00UWpXbsZHePSqPcN_KNw/edit?pli=1#gid=298395604>.
Also annotated not as dates are the size of the repository with the
other heads of mozilla-central included as well as the size of the
mozilla-central as it would be were comm-central included.

It turns out I was wrong. The ~140MB of extra .hg space that
comm-central would take up is not a year's worth of checkins, but 4
months' worth.

> Merging mozilla-central into comm-central periodically does seem
> somewhat reasonable to me. That would get rid of most of the special
> attention needs that comm-central has inflicted upon mozilla-central's
> build system. (I would love to excise those.) But the "atomic commit"
> issues remain.

I think the benefits of having atomic commits are rather underrated,
having been forced to deal with non-atomic commit situations not only in
comm-central but my research group's repositories.

--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move Thunderbird and SeaMonkey to mozilla-central

Joshua Cranmer 🐧
In reply to this post by Gregory Szorc-3
On 4/3/2014 6:30 PM, Mike Hommey wrote:
> The "atomic commit" issue will likely remain whether TB and SM files
> are in mozilla-central or not. Albeit probably less often.

The "atomic commit" issue is the inability to make a change that
simultaneously touches both comm-central and mozilla-central code. That
would go away if comm-central were merged in mozilla-central. The
related issue of not all changesets being able to produce a working
Thunderbird build would not go away, although it would hopefully be
reduced in situations such as "I need to add touch all package
manifests" or "I renamed this API globally."

--
Joshua Cranmer
Thunderbird and DXR developer
Source code archæologist

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move Thunderbird and SeaMonkey to mozilla-central

ISHIKAWA,chiaki
In reply to this post by Mark Banner-4
On (2014年04月04日 00:15), Mark Banner wrote:

> On 03/04/2014 14:07, Benjamin Smedberg wrote:
>> I think it's fine to reconsider the decision we made in 2008 not to
>> include tbird/seamonkey in mozilla-central, but I don't think the
>> arguments here are compelling or that the benefits are worth the costs.
>
> I probably wasn't explicit, but I think the main benefits of this proposal
> are going to be to the Thunderbird/SeaMonkey communities. I can understand
> that this isn't so compelling to Firefox developers. So let me explain a bit
> more:
>
> Ever since we separated the repositories, we have spent significant amounts
> of time porting build configuration patches, fire-fighting bustage when
> something lands in m-c and needs porting to c-c (including sometimes
> regression range finding & bisecting). Obviously fire-fighting is a larger
> pain as it distracts us from whatever we were working on.
>
> Although with recent improvements we've now vastly reduced build
> configuration porting time, the fire-fighting aspects still take up large
> amounts of time which we could spend elsewhere - especially if we were able
> to take benefits of search & replace/repository-wide updates (where
> developers are willing to include us because we're in the same repo rather
> than a separate one). It wouldn't reduce everything, but it would likely
> mean we could be down to a bustage every now and again.
>
>> Reason #1: confusion about what code belongs to what projects
>>
>> Currently when hacking on Firefox, it's roughly clear that all the code
>> in mozilla-central belongs to Firefox in some way, and we have been
>> effectively removing most of the code that isn't part of Firefox. When
>> searching through the codebase (either locally or using the online dools
>> like DXR/MXR) the addition of code which belongs to different products
>> will significantly harm developer understanding.
>
> I don't see this being a big issue. We've dealt with this for comm-central
> ever since 2008, and its easy to see where code lies in suite/ mail/ or
> mozilla/browser and realise it is specific to that product.
>
> If we need to put everything under comm/ as Joshua suggests (or some other
> directory), that would be fine by me. Generally it feels like the
> significant cost here would be first-time users, who we could educate by
> means of adequate documentation.
>
>> Reason #3: It's unnecessary
>>
>> It is not necessary to alter mozilla-central to put all of Thunderbird
>> in one repository. You can do the merge you are proposing, but keep it
>> as a separate Thunderbird tree and regularly merge changes from
>> mozilla-central into your combined tree.
>
> One thing I think this does mean is increased storage size for hmo. We'd
> effectively be doubling up the m-* repositories. With IT/releng currently
> concerned about the size, that probably isn't good. There may be resolution
> on the way, so this might become mute.
>
> It also doubles the repository sizes for developers who do want to work on
> FF and/or TB/SM. There may not be many of them, but there are a few -
> admittedly I don't know for how many disk/download size is an issue.
>
> Mark.

I think the merge would help reduce the load of build system support
once the merge happens and underlying support scripts become stable.

That would also make the newcomer to TB development to
help community-driven support because
- today, the build support is very fragile: mach command does not
  work very well (without scary warning, for example) for C-C tree.
- already mentioned in the thread, but  for C-C developres,
  -- keeping track of TWO hg repositories is a pain when one
     uses tryserver: the scarce documentation does not make sense until
     you face a strange failure and then the complex underlying framework
dawns on you :-(
  -- meaning that simply creating a working development system on local PC
takes time.

It is a really difficult situation. On one hand, tryserver use is rather
complex until
you get the hang of creating contrived patch sets to reflect the local
change to M-C portion of
copy under C-C (residing C-C/mozilla), and tryserver itself suffered a lot
of build breakage
spring and early summer last year probably to the difficulty of
maintaining complex underlying framework (tryserver did not work very well
in the first half of 2013,
to the point, I have not used it lately.)

If merging C-C to M-C alleviates even a small portion of the support issues,
I think
it is a win to mozilla community as a whole.
Also, the reduced threshold for novice developers to set up
local development environment or the better ease of using tryserver (I hope)
may invite more homebrew develoers of a sort. (Maybe I am pipe-dreaming?)

Yes, there is demand and life behind TB.
Just the other day, a friend of mine talked of helping someone converting
from outlook (because of the platform change, I think), and he says the only
choice he could think of was TB.
Fixing outstanding issues in TB is a very important goal of user community
of TB and
anything that makes it easy is a win.
(Yes, I use TB for work and want it to be a "rock-solid mail client (tm)"
that is available on cross-platform fashion.)

I support this move.

TIA
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move Thunderbird and SeaMonkey to mozilla-central

Mike Hommey
In reply to this post by Joshua Cranmer 🐧
On Thu, Apr 03, 2014 at 09:35:10PM -0500, Joshua Cranmer ? wrote:

> On 4/3/2014 5:27 PM, Gregory Szorc wrote:
> >The tree size argument also ignores the harsh reality that the tree will
> >continue to grow over time. We'll be hitting scaling pains with or without
> >comm-central. By some accounts, we're already there.
>
> I did a little experiment to get a sense of the growth rate of
> mozilla-central. I selected roughly monthly revisions from mozilla-central,
> made a new repository, and pulled those revisions in one by one, measuring
> the size of the .hg directory at each point. The resulting graph is here: <https://docs.google.com/spreadsheets/d/1ZXu1-5ILH5r87cZ7G2DOVl00UWpXbsZHePSqPcN_KNw/edit?pli=1#gid=298395604>.
> Also annotated not as dates are the size of the repository with the other
> heads of mozilla-central included as well as the size of the mozilla-central
> as it would be were comm-central included.
>
> It turns out I was wrong. The ~140MB of extra .hg space that comm-central
> would take up is not a year's worth of checkins, but 4 months' worth.

Out of curiosity, what is the apparent size difference in .hg space? (as
in du --apparent-size difference, because on my m-c checkout, the
difference between du and du --apparent-size is 25%, which is 434M (!))

Mike
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move Thunderbird and SeaMonkey to mozilla-central

Nicholas Nethercote
In reply to this post by Mark Banner-4
On Thu, Apr 3, 2014 at 9:13 PM, Mark Banner <[hidden email]> wrote:
> [follow-ups to mozilla.dev.planning]
>
> tl;dr We're proposing to move Thunderbird and SeaMonkey into
> mozilla-central to reduce maintenance complexities for build systems,
> and for releng.

Unsurprisingly, one's reaction to this proposal mirrors one's answer
to the question "do I care at all about TB/SM?"

I can see this change would make life a *lot* easier for TB/SM
developers, and it would make life slightly more difficult for Firefox
developers. But there are a lot more Firefox developers and Firefox is
a more important product. So who wins?

I'm not expressing an opinion one way or the other about the proposal,
but just trying to clarify the contours of the discussion.

Nick
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move Thunderbird and SeaMonkey to mozilla-central

Philip Chee
In reply to this post by Benjamin Smedberg
On 03/04/2014 21:28, Henri Sivonen wrote:

> (On that topic: I'm still looking for a volunteer for
> https://bugzilla.mozilla.org/show_bug.cgi?id=937056 . At some point I
> will have whined about this so much that it would have been more
> efficient to just do the move myself even though I'm not supposed to
> use time for Thunderbird development...)

Well yes, but if c-c gets merged then this will simplify the bug as then
we could just do a hg mv followed by a hg rename. Which is why I was
waiting for a decision to merge or not to merge C-C.

Phil

--
Philip Chee <[hidden email]>, <[hidden email]>
http://flashblock.mozdev.org/ http://xsidebar.mozdev.org
Guard us from the she-wolf and the wolf, and guard us from the thief,
oh Night, and so be good for us to pass.
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move Thunderbird and SeaMonkey to mozilla-central

Philip Chee
In reply to this post by Joshua Cranmer 🐧
On 04/04/2014 00:21, Nick Alexander wrote:

> On 2014-04-03, 8:53 AM, Joshua Cranmer 🐧 wrote:
>> On 4/3/2014 10:20 AM, Nick Alexander wrote:
>>> I'd like to add that there exist other apps built on top of Gecko, and
>>> in future there will be more, and that merging every such app into m-c
>>> is not feasible.
>>>
>>> I recognize that Thunderbird and SeaMonkey are special, for technical
>>> and historical reasons, but this sets a bad precedent.
>>
>> I don't think this sets a precedent. If anything, the precedent was set
>> on April 6, 2011 when the Fennec repository was merged with
>> mozilla-central.
>
> I didn't know that Fennec was once separate, so thanks for the history
> lesson!

Also more recently MetroFx was landed on m-c in one large blob.

Phil

--
Philip Chee <[hidden email]>, <[hidden email]>
http://flashblock.mozdev.org/ http://xsidebar.mozdev.org
Guard us from the she-wolf and the wolf, and guard us from the thief,
oh Night, and so be good for us to pass.
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move Thunderbird and SeaMonkey to mozilla-central

ISHIKAWA,chiaki
In reply to this post by Mark Banner-4
On (2014年04月04日 14:44), Nicholas Nethercote wrote:

> On Thu, Apr 3, 2014 at 9:13 PM, Mark Banner <[hidden email]> wrote:
>> [follow-ups to mozilla.dev.planning]
>>
>> tl;dr We're proposing to move Thunderbird and SeaMonkey into
>> mozilla-central to reduce maintenance complexities for build systems,
>> and for releng.
>
> Unsurprisingly, one's reaction to this proposal mirrors one's answer
> to the question "do I care at all about TB/SM?"
>
> I can see this change would make life a *lot* easier for TB/SM
> developers, and it would make life slightly more difficult for Firefox
> developers. But there are a lot more Firefox developers and Firefox is
> a more important product. So who wins?
>
> I'm not expressing an opinion one way or the other about the proposal,
> but just trying to clarify the contours of the discussion.
>
> Nick
>

Well said.

I think we also explicitly think about the infrastructure support
(which may be missing when one speaks of "developers" as people writing the
source code.).
IMHO, this burden on the infrastructure support in current situation is NOT
SMALL at all, and
that is why the original proposal came about.

TIA
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move Thunderbird and SeaMonkey to mozilla-central

ISHIKAWA,chiaki
On (2014年04月04日 16:30), ishikawa wrote:
> I think we also explicitly think about the infrastructure support
I meant to say "we also NEED TO explicitly ..."

TIA
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move Thunderbird and SeaMonkey to mozilla-central

Florian Quèze
In reply to this post by Nicholas Nethercote
On Fri, Apr 4, 2014 at 7:44 AM, Nicholas Nethercote
<[hidden email]> wrote:
> On Thu, Apr 3, 2014 at 9:13 PM, Mark Banner <[hidden email]> wrote:
>> [follow-ups to mozilla.dev.planning]
>>
>> tl;dr We're proposing to move Thunderbird and SeaMonkey into
>> mozilla-central to reduce maintenance complexities for build systems,
>> and for releng.
>
> Unsurprisingly, one's reaction to this proposal mirrors one's answer
> to the question "do I care at all about TB/SM?"

This is also the way I perceived the answers here.

> I can see this change would make life a *lot* easier for TB/SM
> developers,

Indeed! As someone who's doing Thunderbird work as a volunteer, I'm
really looking forward to this merge.

> and it would make life slightly more difficult for Firefox
> developers.

I understand that people who care only about Firefox are concerned
that this change _may_ make their life slightly more difficult, but
I'm not sure I see what the actual issues are.

The concerns that have been raised so far:
- merging c-c into m-c would increase the size of m-c. Joshua has
shown data indicating that this increase is equivalent to 4 months of
m-c checkins at the current rate. I think this is small enough to make
this concern a non-issue.
- possible confusion when mxr'ing in mozilla-central. As someone who
does Firefox development work, I've sometimes been confused by results
that were for metro, because they were in browser/ and have paths
similar to those of files I was looking for. I don't think I could be
confused by files in a folder named "mail/".

Whether Mozilla should keep supporting Thunderbird with releng is IMHO
a completely different discussion. As far as I understand it, the
current state is that Mozilla Corporation has committed to keep the
product building and to provide security updates in the foreseeable
future.

Florian

--
Florian Quèze
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Proposal: Move Thunderbird and SeaMonkey to mozilla-central

Ben Hearsum-2
In reply to this post by Gregory Szorc-3
Your response is pretty much what I was trying to get at (which I did
poorly, sorry about that).

On 04/03/14 07:38 PM, Mike Hommey wrote:
>> 3. The tooling doesn't suck (all solutions for subrepos or multi repository
>> management are significantly worse than single repo experiment).
>
> True. But it seems to me the problem is noone actively seeked fixing
> those problems and just went the "easy" way by putting everything in the
> same basket. And suffering as a consequence. Which leads to what you're
> writing next.

Yep. This ship has long sailed, but if Firefox had been moved to another
repository at the same time as Thunderbird/SeaMonkey (when we moved to
Mercurial originally IIRC), we would've been heavily invested into
solving this problems.

>> While I concede that Mozilla's needs are slightly different from that of a
>> large corporation with a repository behind a firewall, the benefits of
>> unified repositories remain. The drawbacks (large clone times) can largely
>> be mitigated through extensions (with Mercurial at least).
>>
>> To use a Mozilla example to support claim #2, I'll use mozbase. mozbase is a
>> suite of Python packages. It's effectively the "Mozilla standard library."
>> It used to live in GitHub and was merged to mozilla-central periodically.
>> People working on Python in the tree would often need or want to make
>> changes to mozbase. They had to commit to Git, wait for a merge to m-c, then
>> commit whatever changes were depending on it. The process was brutally
>> agonizing. Often days would go by between mozbase merges. Stop energy was
>> everywhere. Now, mozbase lives inside mozilla-central. In one atomic commit
>> I can update mozbase *and* change all in-tree consumers in one go. It's
>> faster for everybody and makes everyone more productive. It keeps Mozilla
>> fast and nimble.
>
> Except now, it's the other users of mozbase that aren't in m-c that suffer.

+1. It's hard for people to get the tools that are embedded into our
massive repo, and it's easy to accidentally (or on purpose) depend on
completely unrelated things, making them useless to others.
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
12345 ... 8