Add-on SDK Repacks Proposal

classic Classic list List threaded Threaded
23 messages Options
12
Reply | Threaded
Open this post in threaded view
|

Add-on SDK Repacks Proposal

David Mason-6
One thing we have planned to make part of the Add-on SDK story is to
have "automatic repacks" for developers using the SDK. What this means
is that if you host your add-on, built with the SDK, on AMO
(addons.mozilla.org), we would automatically rebuild it everytime we
release a new version of the Add-on SDK to maintain compatibility with a
new version of Firefox. This would make sure that your add-on was always
up-to-date with the latest version of Firefox.

This is still the plan, but for our first attempt we noticed some things
that were wrong with the plan as we currently have it and would like to
propose a way of getting past these problems and getting to a new repack
process.

What Went Wrong

The first problem we faced is that the repacker is the first service to
use AMO's brand-new APIs for updating add-ons, and we hadn't tested the
interaction between the two sites sufficiently beforehand, so we had to
implement fixes for issues we encountered during the repack testing,
which slowed down the testing.

Second, our testing was short and not deep enough for what we were
trying to accomplish. The repack team didn't get enough testing in
themselves, nor did they make it possible for other project participants
to test repacks and suggests issues and fixes before the production run
of repacking.

Finally, what we are fundamentally trying to do is repackage a derived
format without the source code it came from as reference, like
decompiling a program, making changes, and then recompiling it. This is
something that is hard to do and as it turned out, didn't work very
well. We could fix that by including the source code in the XPI.
However, including the source code in the XPI would defeat our goal of
making the XPI size as small as possible (this is extremely important
for mobile versions of Firefox).

A Proposal

So what can we do about these issues? First of all, the repack team will
answer the testing problems by starting testing earlier and sharing the
results with everyone. In addition, the use of the AMO APIs is now
solved as the calls we are using are now verified and tested.
However, the issue about the required sources for a clean rebuild is a
bit more difficult and needs a new proposal.

Short-term

In the short-term I propose that we only repack add-ons which are built
using the Add-on Builder. This is the quickest and safest solution as we
have access to all the sources with Builder-based add-ons. The repack
process is much simpler too as we don't have to unpack a XPI first. For
those who use a local installation of the Add-on SDK to build an add-on,
we will document how to repack your add-on and the process for pushing
the new one to AMO. In addition, you can move your add-on to the Add-on
Builder and take advantage of the automatic repacks if you'd like, but
do keep in mind that the Add-on Builder is still in beta at this point.

Long-term

The long-term solutions will need to be studied and debated a bit more.
One option we have is to create a "source package" that can be pushed to
AMO for use in repacking. However, this would mean that AMO would have
to store these source packages and generate XPIs from them in addition
to taking on responsibility for repacking when a new SDK is released.
Obviously this would involve adding a great deal of functionality to
AMO, as well as tooling up the services side to store these new, larger
packages.

Another option is to land parts of the Add-on SDK into Firefox itself,
including API implementations and the module loader. If an API already
in Firefox needs updating to maintain compatibility with changes to
Firefox, we won't need to repack add-ons that use that API, as the
updates can be made to that new version of the browser rather than each
individual add-on. Doing this has other benefits as well. For example,
it may help reduce XPI sizes tremendously. However, this too would take
a bit of time to implement and is thus a long-term solution.

At this point we think that the short-term solution of just repacking
Add-on Builder-based add-ons would be the best approach. We would also
provide documentation describing the simplest approach for repacking
add-ons along with best practices and troubleshooting. In addition, we
would look into how best to notify authors when their add-ons need to be
repacked manually.

Still, we also want to know what you think about this as well as the
long-term ideas. Please feel free to leave your comments or suggestions
here or on the jetpack discussion forum:
https://groups.google.com/d/topic/mozilla-labs-jetpack/s9-f1sJ0Ne0/discussion

Thanks!

Dave

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Add-on SDK Repacks Proposal

David Illsley

On 4 Oct 2011, at 21:53, David Mason wrote:

> <snip>
> Short-term
>
> In the short-term I propose that we only repack add-ons which are built using the Add-on Builder. This is the quickest and safest solution as we have access to all the sources with Builder-based add-ons. The repack process is much simpler too as we don't have to unpack a XPI first. For those who use a local installation of the Add-on SDK to build an add-on, we will document how to repack your add-on and the process for pushing the new one to AMO.

Would this process somehow avoid the updated addon needing re-reviewed? If not, it seems like a non-starter
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Add-on SDK Repacks Proposal

David Mason-6
David,

Thanks for the comment, I do understand the frustrations going on with review processes and it is something that is being talked about mostly on the AMO side of things. This proposal and discussion doesn't cover that isn't really related to those issues per se. Still, please know that we are thinking about how to improve those issues as well.


Thanks!

Dave




----- Original Message -----

> From: "David Illsley" <[hidden email]>
> To: "David Mason" <[hidden email]>
> Cc: [hidden email]
> Sent: Tuesday, October 4, 2011 5:04:01 PM
> Subject: Re: Add-on SDK Repacks Proposal
> On 4 Oct 2011, at 21:53, David Mason wrote:
>
> > <snip>
> > Short-term
> >
> > In the short-term I propose that we only repack add-ons which are
> > built using the Add-on Builder. This is the quickest and safest
> > solution as we have access to all the sources with Builder-based
> > add-ons. The repack process is much simpler too as we don't have to
> > unpack a XPI first. For those who use a local installation of the
> > Add-on SDK to build an add-on, we will document how to repack your
> > add-on and the process for pushing the new one to AMO.
>
> Would this process somehow avoid the updated addon needing
> re-reviewed? If not, it seems like a non-starter
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Add-on SDK Repacks Proposal

David Illsley

On 4 Oct 2011, at 22:16, David Mason wrote:

> David,
>
> Thanks for the comment, I do understand the frustrations going on with review processes and it is something that is being talked about mostly on the AMO side of things. This proposal and discussion doesn't cover that isn't really related to those issues per se. Still, please know that we are thinking about how to improve those issues as well.

I wasn't complaining... I'd just be surprised if the amo team have the cycles for the additional workload (especially if sdk adoption picks up).
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Add-on SDK Repacks Proposal

Jeff Griffiths-4
On 11-10-04 2:20 PM, David Illsley wrote:
...
> I wasn't complaining... I'd just be surprised if the amo team have
> the cycles for the additional workload (especially if sdk adoption
> picks up).

AMO tracks SDK usage; for September ~50% of new add-ons were using the
SDK. August was even higher, actually. The data is here:

http://areweaddeduponyet.com/

Under 'Platform', click on 'kpi'.

Obviously the vast majority of extensions hosted on AMO are not using
the SDK, but it is encouraging to see people starting new projects *are*
choosing the SDK about half the time.

cheers, Jeff
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Add-on SDK Repacks Proposal

Jeff Griffiths-4
On 11-10-04 2:33 PM, Jeff Griffiths wrote:
...
> AMO tracks SDK usage; for September ~50% of new add-ons were using the
> SDK. August was even higher, actually. The data is here:

...it was just pointed out to me by Myk Melez that I have been operating
under a gross misunderstanding of the meaning of the graphs on that
site. We're actually sitting at around 20%, but it is also unclear how
the 20% is calculated.

cheers, Jeff
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Add-on SDK Repacks Proposal

Robert O'Callahan-3
In reply to this post by David Mason-6
On Wed, Oct 5, 2011 at 9:53 AM, David Mason <[hidden email]> wrote:

> Another option is to land parts of the Add-on SDK into Firefox itself,
> including API implementations and the module loader. If an API already in
> Firefox needs updating to maintain compatibility with changes to Firefox, we
> won't need to repack add-ons that use that API, as the updates can be made
> to that new version of the browser rather than each individual add-on. Doing
> this has other benefits as well. For example, it may help reduce XPI sizes
> tremendously. However, this too would take a bit of time to implement and is
> thus a long-term solution.
>

This is what I always expected the Add-on SDK would do. I've never
understood why we aren't doing this (except for the reasons Myk pointed out
that make this less convenient for the Add-on SDK team, which shouldn't
outweigh the value to add-on developers and users, IMHO).

Rob
--
"If we claim to be without sin, we deceive ourselves and the truth is not in
us. If we confess our sins, he is faithful and just and will forgive us our
sins and purify us from all unrighteousness. If we claim we have not sinned,
we make him out to be a liar and his word is not in us." [1 John 1:8-10]
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Add-on SDK Repacks Proposal

Henri Sivonen
In reply to this post by David Mason-6
On Tue, Oct 4, 2011 at 11:53 PM, David Mason <[hidden email]> wrote:
> Another option is to land parts of the Add-on SDK into Firefox itself,
> including API implementations and the module loader. If an API already in
> Firefox needs updating to maintain compatibility with changes to Firefox, we
> won't need to repack add-ons that use that API, as the updates can be made
> to that new version of the browser rather than each individual add-on. Doing
> this has other benefits as well. For example, it may help reduce XPI sizes
> tremendously. However, this too would take a bit of time to implement and is
> thus a long-term solution.

Frankly, I find it strange that there's any question about whether
this should be the plan. (It feels to obvious to me that this should
be the plan.)

Anything short of this would still leave us with a worse
forward-compatibility story than what Chrome, Safari and Opera have.
I've seen the blog post stating reasons why shipping the stable API as
part of Firefox itself is less convenient for the SDK team, but I
think it would make sense to prioritize the experience for add-on
developers and users so that it's as good or better than the
experience with the Chrome, Safari and Opera.

--
Henri Sivonen
[hidden email]
http://hsivonen.iki.fi/
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Add-on SDK Repacks Proposal

Jonas Sicking-2
In reply to this post by Robert O'Callahan-3
On Tue, Oct 4, 2011 at 7:02 PM, Robert O'Callahan <[hidden email]> wrote:

> On Wed, Oct 5, 2011 at 9:53 AM, David Mason <[hidden email]> wrote:
>
>> Another option is to land parts of the Add-on SDK into Firefox itself,
>> including API implementations and the module loader. If an API already in
>> Firefox needs updating to maintain compatibility with changes to Firefox, we
>> won't need to repack add-ons that use that API, as the updates can be made
>> to that new version of the browser rather than each individual add-on. Doing
>> this has other benefits as well. For example, it may help reduce XPI sizes
>> tremendously. However, this too would take a bit of time to implement and is
>> thus a long-term solution.
>>
>
> This is what I always expected the Add-on SDK would do. I've never
> understood why we aren't doing this (except for the reasons Myk pointed out
> that make this less convenient for the Add-on SDK team, which shouldn't
> outweigh the value to add-on developers and users, IMHO).

Yes!

This would also fix the problem that we can't repack addons which
aren't hosted on AMO. The recent data we have shows that a very
significant chunk of the addons aren't hosted on AMO.

In fact, those are the addons that are in most dire need of a stable
API that will ensure that they keep working, since those are the ones
that we can't automatically upgrade the compatibility information for.

/ Jonas
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Add-on SDK Repacks Proposal

David Mason-6
In reply to this post by David Mason-6
Robert, Henri, and others,

Thanks for commenting on this, it is much appreciated. I think perhaps
my message gave the impression that we aren't sure if we will ever land
our APIs in Firefox itself. Reading back over it just now, I probably
should have phrased it differently.

So let me be clear here, we have always assumed that we would land our
APIs into Firefox at some point - but it is indeed a long-term project.

What I should suggest here is that we can think of that as one of the
options that would solve this particular problem and, in so doing, we
can possibly shoot for having that be the *only* long-term solution and
skip other options. There is a lot to weigh in making that decision, and
it does mean that it will be a while before we have the solution in place.

Does that make sense?

Thanks!

Dave


On 10/04/2011 10:02 PM, Robert O'Callahan wrote:

> On Wed, Oct 5, 2011 at 9:53 AM, David Mason<[hidden email]>  wrote:
>
>> Another option is to land parts of the Add-on SDK into Firefox itself,
>> including API implementations and the module loader. If an API already in
>> Firefox needs updating to maintain compatibility with changes to Firefox, we
>> won't need to repack add-ons that use that API, as the updates can be made
>> to that new version of the browser rather than each individual add-on. Doing
>> this has other benefits as well. For example, it may help reduce XPI sizes
>> tremendously. However, this too would take a bit of time to implement and is
>> thus a long-term solution.
>>
>
> This is what I always expected the Add-on SDK would do. I've never
> understood why we aren't doing this (except for the reasons Myk pointed out
> that make this less convenient for the Add-on SDK team, which shouldn't
> outweigh the value to add-on developers and users, IMHO).
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Add-on SDK Repacks Proposal

Lawrence Mandel-2
Dave,

I'm coming in to this conversation late but..

Bundling the Add-on SDK with Firefox sounds like it will have a positive impact on the Firefox update story. If so, I think we should pursue this. From previous posts I'm pretty sure that the issues with bundling the SDK have already been documented. Can you point me to some information (blog post, wiki, etc.) so that I can better understand the issues?

Thanks,

Lawrence

----- Original Message -----
From: "David Mason" <[hidden email]>
To: [hidden email]
Sent: Wednesday, October 5, 2011 10:09:23 AM
Subject: Re: Add-on SDK Repacks Proposal

Robert, Henri, and others,

Thanks for commenting on this, it is much appreciated. I think perhaps
my message gave the impression that we aren't sure if we will ever land
our APIs in Firefox itself. Reading back over it just now, I probably
should have phrased it differently.

So let me be clear here, we have always assumed that we would land our
APIs into Firefox at some point - but it is indeed a long-term project.

What I should suggest here is that we can think of that as one of the
options that would solve this particular problem and, in so doing, we
can possibly shoot for having that be the *only* long-term solution and
skip other options. There is a lot to weigh in making that decision, and
it does mean that it will be a while before we have the solution in place.

Does that make sense?

Thanks!

Dave


On 10/04/2011 10:02 PM, Robert O'Callahan wrote:

> On Wed, Oct 5, 2011 at 9:53 AM, David Mason<[hidden email]>  wrote:
>
>> Another option is to land parts of the Add-on SDK into Firefox itself,
>> including API implementations and the module loader. If an API already in
>> Firefox needs updating to maintain compatibility with changes to Firefox, we
>> won't need to repack add-ons that use that API, as the updates can be made
>> to that new version of the browser rather than each individual add-on. Doing
>> this has other benefits as well. For example, it may help reduce XPI sizes
>> tremendously. However, this too would take a bit of time to implement and is
>> thus a long-term solution.
>>
>
> This is what I always expected the Add-on SDK would do. I've never
> understood why we aren't doing this (except for the reasons Myk pointed out
> that make this less convenient for the Add-on SDK team, which shouldn't
> outweigh the value to add-on developers and users, IMHO).
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Add-on SDK Repacks Proposal

David Mason-6
On Wed 05 Oct 2011 11:08:24 AM EDT, Lawrence Mandel wrote:
> Dave,
>
> I'm coming in to this conversation late but..
>
> Bundling the Add-on SDK with Firefox sounds like it will have a positive impact on the Firefox update story. If so, I think we should pursue this. From previous posts I'm pretty sure that the issues with bundling the SDK have already been documented. Can you point me to some information (blog post, wiki, etc.) so that I can better understand the issues?
>

Hi Lawrence,

Thanks for asking! Not too long ago, Myk wrote a blog post on why we
don't land the Add-on SDK in Moz Central:
http://mykzilla.blogspot.com/2011/08/why-add-on-sdk-doesnt-land-in-mozilla.html 


That mostly talks about code repositories more than landing the APIs in
the the Firefox product, although that is touched on at the end with
references to a feature page on doing so, as well as the roadmap
showing our other current priorities.

Of course, all this comes down to priorities in the end.

Thanks!

Dave
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Add-on SDK Repacks Proposal

Myk Melez-3
In reply to this post by Robert O'Callahan-3
On 2011-10-04 7:02 PM, Robert O'Callahan wrote:

> On Wed, Oct 5, 2011 at 9:53 AM, David Mason<[hidden email]>  wrote:
>
>> Another option is to land parts of the Add-on SDK into Firefox itself,
>> including API implementations and the module loader. If an API already in
>> Firefox needs updating to maintain compatibility with changes to Firefox, we
>> won't need to repack add-ons that use that API, as the updates can be made
>> to that new version of the browser rather than each individual add-on. Doing
>> this has other benefits as well. For example, it may help reduce XPI sizes
>> tremendously. However, this too would take a bit of time to implement and is
>> thus a long-term solution.
>>
> This is what I always expected the Add-on SDK would do. I've never
> understood why we aren't doing this (except for the reasons Myk pointed out
> that make this less convenient for the Add-on SDK team, which shouldn't
> outweigh the value to add-on developers and users, IMHO).
There are two discrete questions here, and it's important to be clear
about the distinction between them.

The first is whether to land the SDK's codebase in mozilla-central, and
continue development of the SDK in mozilla-central, so it is slightly
less inconvenient for Firefox/Gecko developers to run its tests when
they occasionally break them.

That was the question I was addressing in my blog post
<http://mykzilla.blogspot.com/2011/08/why-add-on-sdk-doesnt-land-in-mozilla.html>,
and I continue to believe the answer is no: the significant and frequent
advantages for SDK engineers of a separate Git repository for this
distinct-albeit-dependent product far outweigh the occasional slight
inconvenience to Firefox/Gecko developers.

(But I do think it's worth making that inconvenience even slighter in
ways like I outlined in the blog post.)

The question Dave asks, on the other hand, is whether to integrate parts
of the SDK's functionality into Firefox so it is no longer necessary to
bundle them into each addon package and then update those packages for
each new incompatible version of Firefox.

I mentioned this idea at the end of my blog post, and I continue to
think it's a good one, for multiple reasons, especially now that Firefox
has moved to rapid releases (the old strategy was partly designed to
enable us to ship new addon-facing APIs more rapidly than Firefox). I
think that's the avenue we should pursue to address this issue (although
source packages are worth pursuing for other reasons).

-myk

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Add-on SDK Repacks Proposal

Joe Drew-5

On 2011-10-05 1:28 PM, Myk Melez wrote:

> The question Dave asks, on the other hand, is whether to integrate parts
> of the SDK's functionality into Firefox so it is no longer necessary to
> bundle them into each addon package and then update those packages for
> each new incompatible version of Firefox.
>
> I mentioned this idea at the end of my blog post, and I continue to
> think it's a good one, for multiple reasons, especially now that Firefox
> has moved to rapid releases (the old strategy was partly designed to
> enable us to ship new addon-facing APIs more rapidly than Firefox). I
> think that's the avenue we should pursue to address this issue (although
> source packages are worth pursuing for other reasons).

I understand that you're trying to optimize for SDK developers by
developing on GitHub, and obviously the benefits must be pretty great or
you wouldn't have said that :)

However, given that your end goal seems to be to integrate with the
Firefox codebase (note: I fully support this goal and believe it to be
best for add-on developers and users alike), I think that continuing to
develop in a separate repository will cause more harm than good to the
project.

If and when a piece of the SDK (or the whole thing) gets merged into
mozilla-central, it will start diverging from the GitHub upstream.
Changes will be made to Firefox that necessitate SDK updates, which will
be made to the SDK code in mozilla-central. Meanwhile, the SDK will
continue its development in GitHub, with the need to merge these
downstream changes regularly.

The alternative is to move those parts of the SDK that are merged into
mozilla-central; in the limit, this results in all of the SDK code
living in mozilla-central. However, the intermediate steps might suck
just as much, since you'll be making changes to GitHub that necessitate
changes to mozilla-central.

This type of occasional integration (vs continuous integration, what we
have for all code that lives in mozilla-central) is a recipe for tears.
We already have a lot of pain from this (Cf. the Jetpack tests
constantly going orange due to m-c changes), and it could get a lot worse.

All of this is a long way of trying to convince you that your best
long-term plan is to move your development to mozilla-central and
integrate (and ship) with Firefox. I truly believe that anything else is
going to lead to less productivity all around, and thus a worse
experience for our users.

Thanks,

joe
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Add-on SDK Repacks Proposal

Myk Melez
On 2011-10-05 11:32 AM, Joe Drew wrote:
> If and when a piece of the SDK (or the whole thing) gets merged into
> mozilla-central, it will start diverging from the GitHub upstream.
> Changes will be made to Firefox that necessitate SDK updates, which
> will be made to the SDK code in mozilla-central. Meanwhile, the SDK
> will continue its development in GitHub, with the need to merge these
> downstream changes regularly.
Indeed, that would be problematic, so we wouldn't do that. We would
instead remove the relevant code from the SDK and then continue
development only in mozilla-central (or perhaps in a separate
branch/repository that we merge to mozilla-central regularly).

> The alternative is to move those parts of the SDK that are merged into
> mozilla-central; in the limit, this results in all of the SDK code
> living in mozilla-central. However, the intermediate steps might suck
> just as much, since you'll be making changes to GitHub that
> necessitate changes to mozilla-central.
It is already the case that changes to one of those codebases
necessitate changes to the other, and this is normal for two distinct
products with a dependency relationship (AMO/Firefox is another
example). It doesn't inherently justify merging their codebases.

And in fact it doesn't suck very much. It does mean that we have to wait
for a Gecko/Firefox feature to land before shipping an SDK feature that
depends on it, but we'd have to do so in a unified codebase as well,
since the Gecko/Firefox features tend to be developed by different teams
(or at least different engineers).

> This type of occasional integration (vs continuous integration, what
> we have for all code that lives in mozilla-central) is a recipe for tears.
That depends on the nature of the dependencies. Loosely-coupled
codebases that interact using explicit and stablish interfaces (f.e.
SDK/mozrunner) do just fine with occasional integration, as can feature
development within a unified and tightly-coupled codebase in some cases.

In fact not all code in mozilla-central is continuously integrated.
JavaScript engine (tracemonkey, jaegermonkey, type-inference), Firefox
feature (fx-team), and services (services-central) work has been or is
currently being conducted on branches that are periodically rather than
continuously integrated with mozilla-central.

Indeed, such occasional integration seems to be a growing trend in
Mozilla, although it isn't always successful (the tracemonkey merge that
introduced compartments was particularly painful, and not only for the SDK).

In any case, I too favor continuous integration, which is why SDK
development employs it in one direction: on every SDK commit, the build
infrastructure runs SDK tests against the latest tip build of Firefox.

In the other direction, on the other hand, integration is occasional:
SDK tests run on every mozilla-central commit, but the version of the
SDK whose tests get run is updated only periodically. Those updates are
irregular at the moment, but we're moving to regular weekly drops, and I
would be happy to make them as frequent as every SDK commit, provided we
can automate it.

Nevertheless, none of these integration strategies, including full
bi-directional continuous integration, requires us to merge codebases
into a single repository. Whether or not to do so, and to what degree,
is a discrete question, and one we should carefully divorce from the
question of whether to do continuous integration, given the growing
number of new projects (plus old ones, like AMO!) with Firefox
dependencies that live in separate repositories and for which we should
also provide continuous integration. The SDK is just the tip of this
iceberg.

> We already have a lot of pain from this (Cf. the Jetpack tests
> constantly going orange due to m-c changes), and it could get a lot worse.
I haven't seen the SDK tests constantly going orange. Before the recent
episode, the last time I recall that happening was a couple months ago,
in late July (if I remember correctly). Nevertheless, we can address
them at any frequency by adding SDK tests to the tools mozilla-central
hackers use to test before they land (`make test`, try, inbound, etc.).

> All of this is a long way of trying to convince you that your best
> long-term plan is to move your development to mozilla-central and
> integrate (and ship) with Firefox. I truly believe that anything else
> is going to lead to less productivity all around, and thus a worse
> experience for our users.
I'm not convinced, especially regarding productivity, given my
experience with and observations of both Mercurial and Git (particularly
on GitHub) development patterns for both core and casual contributors to
Jetpack and other projects.

Nevertheless, I'm open to further discussion, and we certainly agree in
a number of respects...

In particular, I agree that we should ship significant parts of the
SDK's functionality with Firefox, including the APIs, the module loader,
and perhaps the bootstrapper, in order to provide the best possible
experience for addon developers and users (although it's worth
considering doing so via a default addon, which a variety of other teams
are also considering for their Firefox feature work).

(I also think it's worth exploring the idea of shipping the rest of the
SDK's functionality with Firefox too, i.e. the command-line tool for
trying, testing, and packaging addons; and then building graphical
interfaces for addon development directly into Firefox.)

And I agree that moving development of those parts to mozilla-central
when we ship them with Firefox is the best overall software
configuration management strategy (although it's worth considering the
alternative of developing them in a separate Git repository that we
regularly merge with mozilla-central, which I hear Andreas is doing for
one of his projects, especially if we decide to ship SDK functionality
as a default addon).

-myk

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Add-on SDK Repacks Proposal

Ehsan Akhgari
Thanks a lot, Myk, for the detailed response.

On Fri, Oct 7, 2011 at 2:51 PM, Myk Melez <[hidden email]> wrote:

> On 2011-10-05 11:32 AM, Joe Drew wrote:
>
>> If and when a piece of the SDK (or the whole thing) gets merged into
>> mozilla-central, it will start diverging from the GitHub upstream.
>> Changes will be made to Firefox that necessitate SDK updates, which
>> will be made to the SDK code in mozilla-central. Meanwhile, the SDK
>> will continue its development in GitHub, with the need to merge these
>> downstream changes regularly.
>>
> Indeed, that would be problematic, so we wouldn't do that. We would instead
> remove the relevant code from the SDK and then continue development only in
> mozilla-central (or perhaps in a separate branch/repository that we merge to
> mozilla-central regularly).


This makes a lot of sense.


>  The alternative is to move those parts of the SDK that are merged into
>> mozilla-central; in the limit, this results in all of the SDK code
>> living in mozilla-central. However, the intermediate steps might suck
>> just as much, since you'll be making changes to GitHub that
>> necessitate changes to mozilla-central.
>>
> It is already the case that changes to one of those codebases necessitate
> changes to the other, and this is normal for two distinct products with a
> dependency relationship (AMO/Firefox is another example). It doesn't
> inherently justify merging their codebases.
>

While I agree that depending on the nature of the products, this might make
a lot of sense (such as is the case for AMO and Firefox), I'm not sure if
the same analogy applies to the SDK.  I think that parts of the SDK are in
fact very tightly coupled with Gecko/Firefox.  Let me clarify.  I think we
need to talk distinctively about the command-line tools of the SDK which are
used to generate and test add-ons built on top of the SDK, the high-level
add-on APIs living inside of the SDK available to add-on authors, and the
low-level components of the SDK making the entire magic work (such as the
module loader and the low level APIs used to build the high-level ones).  To
make it easier to talk about these subcomponents, I will refer to them as
the tools, APIs and internals of the SDK (apologies if there is a better
terminology here that I'm not aware of, please let me know if there is.)

I would argue that the internals are very tightly coupled with
Gecko/Firefox, the APIs have tight or looser couplings, depending on which
ones we're talking about, and the tools are loosely coupled with
Gecko/Firefox.  I will provide some reasons for why I think this way below.


> And in fact it doesn't suck very much. It does mean that we have to wait
> for a Gecko/Firefox feature to land before shipping an SDK feature that
> depends on it, but we'd have to do so in a unified codebase as well, since
> the Gecko/Firefox features tend to be developed by different teams (or at
> least different engineers).


This is definitely true, and I think it needs to be considered when we're
deciding what code should live where, but there are other trade-offs here as
well, as we all know.


>
>  This type of occasional integration (vs continuous integration, what
>> we have for all code that lives in mozilla-central) is a recipe for tears.
>>
> That depends on the nature of the dependencies. Loosely-coupled codebases
> that interact using explicit and stablish interfaces (f.e. SDK/mozrunner) do
> just fine with occasional integration, as can feature development within a
> unified and tightly-coupled codebase in some cases.
>
> In fact not all code in mozilla-central is continuously integrated.
> JavaScript engine (tracemonkey, jaegermonkey, type-inference), Firefox
> feature (fx-team), and services (services-central) work has been or is
> currently being conducted on branches that are periodically rather than
> continuously integrated with mozilla-central.
>

I wouldn't necessary call project branches a means to have periodical as
opposed to continuous integration.  Some of those branches were merely
created to make it easier for smaller teams to work without being negatively
infected by the strict commit rules on mozilla-central.  This was the case
for the tracemonkey branch (at least when Tracemonkey itself was integrated
into mozilla-central).  We've been working very hard to solve that problem.
The introduction of the mozilla-inbound repository has solved a large
portion of these problems (which is what lead to the js team switch to that
branch in place of tracemonkey, and the termination of the tracemonkey
branch eventually).

Some other branches in that list serve as a less noisy mozilla-inbound
today, and are used in more or less the same way (such as fx-team).  Some of
the other branches (type-inference for example) were being used to develop
specific features in an isolated environment.  But even in those cases, as a
common practice, teams are encouraged to merge from mozilla-central
frequently to detect the cases where some change on mozilla-central has
broken some of their code.  It is only in this case where periodical
integration has been beneficial, but that usually is a temporary goal, and
features developed in those branches will strive to get continuous
integration when they're good enough to be shipped as a product.


> Indeed, such occasional integration seems to be a growing trend in Mozilla,
> although it isn't always successful (the tracemonkey merge that introduced
> compartments was particularly painful, and not only for the SDK).
>
> In any case, I too favor continuous integration, which is why SDK
> development employs it in one direction: on every SDK commit, the build
> infrastructure runs SDK tests against the latest tip build of Firefox.
>

I didn't know that, and I'm happy to hear about it.  Out of curiosity, is
there a dashboard of some sort which is used by the SDK contributors to
watch over the results of those tests?


> In the other direction, on the other hand, integration is occasional: SDK
> tests run on every mozilla-central commit, but the version of the SDK whose
> tests get run is updated only periodically. Those updates are irregular at
> the moment, but we're moving to regular weekly drops, and I would be happy
> to make them as frequent as every SDK commit, provided we can automate it.
>
> Nevertheless, none of these integration strategies, including full
> bi-directional continuous integration, requires us to merge codebases into a
> single repository. Whether or not to do so, and to what degree, is a
> discrete question, and one we should carefully divorce from the question of
> whether to do continuous integration, given the growing number of new
> projects (plus old ones, like AMO!) with Firefox dependencies that live in
> separate repositories and for which we should also provide continuous
> integration. The SDK is just the tip of this iceberg.
>

Continuous integration is only one piece of the puzzle here.  We have very
important questions to address about the quality of the products that we're
trying to build, which are not being addressed today.

Here is a concrete example: does the existence of the most basic SDK-based
add-on break any of the functions of the Firefox UI?  We have an entire
suite of automated tests (the browser-chrome tests) to try to answer the
question "do all of the functions of the Firefox UI work as expected?", but
today we have no way of ensuring the same quality when the most basic
SDK-based add-on is installed.  Being able to run those tests on a build of
Firefox including the internals and API parts of the SDK will enable us
answer this question.  This is possible to answer today, with some level of
automation and most likely a significant level of effort from our RelEng
team.  When we get to the level of confidence to say "yes" to this question,
the one would be: "Which commit to the SDK code broke this functionality in
the UI?".  That's harder to answer today.  Then, we can ask more interesting
questions, like "Do the SDK components cause any memory leaks?", or "Do the
SDK components cause any assertions in Firefox/Gecko?".  And the questions
keep getting harder and harder to answer.

In other words, I think running the SDK tests on every SDK commit is indeed
valuable, but it's the most basic level of testing that we can get for the
SDK.  There are a vast array of other interesting quality measures that we
can benefit from which are harder or impossible to address with today's
development process.


>  We already have a lot of pain from this (Cf. the Jetpack tests
>> constantly going orange due to m-c changes), and it could get a lot worse.
>>
> I haven't seen the SDK tests constantly going orange. Before the recent
> episode, the last time I recall that happening was a couple months ago, in
> late July (if I remember correctly). Nevertheless, we can address them at
> any frequency by adding SDK tests to the tools mozilla-central hackers use
> to test before they land (`make test`, try, inbound, etc.).
>

I don't think that how often the tests have gone orange is the most
interesting question to ask here.  I mentioned some of the interesting ones
before.

The fact of life is that today, when people see non-green JP test runs on
any try (be in try, mozilla-central, etc.), they automatically ignore them,
and just assume that something is wrong with the SDK and somebody will
magically take care of it.  This is very different to the common response to
test failures in any other part of the codebase.  You might think that part
of this is a cultural problem, and you will definitely be right, but at
least from the perspective of many developers working on Gecko/Firefox, the
SDK is a black box.  They don't get to see its code, they often don't know
what stuff are tested in its tests, they can't use the tools that they use
for working with the SDK and its tests, etc.  I'm very worried about this
trend, and I would really like us to do something to fix it.


>  All of this is a long way of trying to convince you that your best
>> long-term plan is to move your development to mozilla-central and
>> integrate (and ship) with Firefox. I truly believe that anything else
>> is going to lead to less productivity all around, and thus a worse
>> experience for our users.
>>
> I'm not convinced, especially regarding productivity, given my experience
> with and observations of both Mercurial and Git (particularly on GitHub)
> development patterns for both core and casual contributors to Jetpack and
> other projects.
>

I don't really understand this part.  Fortunately, hg and git are very
similar in how they think of history, and there are tools that allow people
to work on github hosted clones of mozilla-central.  I really think that
which revision control system a person uses is mostly a personal choice, and
should not reflect at all on the decisions that can affect the quality of a
product that we're building.  To give you a concrete example, I personally
am really close to switching to git for all of my mozilla-central
development, and I'll just use the tools when I need to make my RCS talk to
Mozilla's.

So I think we have two things to discuss here:

* What are the benefits and disadvantages of integrating the SDK code (at
least the internals and APIs components) into mozilla-central for our users
and developers?
* What are the benefits and disadvantages of switching the development of
the SDK (no matter where its code lives) from github to hg/bugzilla?


> Nevertheless, I'm open to further discussion, and we certainly agree in a
> number of respects...
>
> In particular, I agree that we should ship significant parts of the SDK's
> functionality with Firefox, including the APIs, the module loader, and
> perhaps the bootstrapper, in order to provide the best possible experience
> for addon developers and users (although it's worth considering doing so via
> a default addon, which a variety of other teams are also considering for
> their Firefox feature work).
>
> (I also think it's worth exploring the idea of shipping the rest of the
> SDK's functionality with Firefox too, i.e. the command-line tool for trying,
> testing, and packaging addons; and then building graphical interfaces for
> addon development directly into Firefox.)
>
> And I agree that moving development of those parts to mozilla-central when
> we ship them with Firefox is the best overall software configuration
> management strategy (although it's worth considering the alternative of
> developing them in a separate Git repository that we regularly merge with
> mozilla-central, which I hear Andreas is doing for one of his projects,
> especially if we decide to ship SDK functionality as a default addon).


This is very similar to what I think as well, and I'm happy to see that we
agree about this.  I'm looking forward to hear your further thoughts on
this.

Cheers,
--
Ehsan
<http://ehsanakhgari.org/>
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Add-on SDK Repacks Proposal

Myk Melez
On 2011-10-08 1:33 PM, Ehsan Akhgari wrote:
> I wouldn't necessary call project branches a means to have periodical
> as opposed to continuous integration.  Some of those branches were
> merely created to make it easier for smaller teams to work without
> being negatively infected by the strict commit rules on mozilla-central.
Sure. I'm not saying periodic integration was the intent of any of these
branches, but it is a consequence of them, and it can be reasonable,
even for changes to components that are tightly coupled to the rest of
the app (like the JS engine).

> Out of curiosity, is there a dashboard of some sort which is used by
> the SDK contributors to watch over the results of those tests?
Yes, we use tbpl to watch the "Jetpack" tree
<https://tbpl.mozilla.org/?tree=Jetpack&usetinderbox=1>.

An "Addon-SDK" link to this tbpl page is on the Firefox tbpl page, right
next to the Mozilla-Aurora link, although it's missing the
"usetinderbox=1" URL parameter, which you need for the time being.

> Here is a concrete example: does the existence of the most basic
> SDK-based add-on break any of the functions of the Firefox UI?  We
> have an entire suite of automated tests (the browser-chrome tests) to
> try to answer the question "do all of the functions of the Firefox UI
> work as expected?", but today we have no way of ensuring the same
> quality when the most basic SDK-based add-on is installed.  Being able
> to run those tests on a build of Firefox including the internals and
> API parts of the SDK will enable us answer this question.
Yup, that's an important question to address, but it seems orthogonal to
the location of the SDK code, since we would address it by testing
Firefox with an SDK-based addon installed in either case.

> This is possible to answer today, with some level of automation and
> most likely a significant level of effort from our RelEng team.  When
> we get to the level of confidence to say "yes" to this question, the
> one would be: "Which commit to the SDK code broke this functionality
> in the UI?".  That's harder to answer today.  Then, we can ask more
> interesting questions, like "Do the SDK components cause any memory
> leaks?", or "Do the SDK components cause any assertions in
> Firefox/Gecko?".  And the questions keep getting harder and harder to
> answer.
I see the difficulty of addressing these questions given scarce release
engineering resources, but I don't see how the difficulty is associated
with maintaining separate codebases, except for the general problem that
Mozilla's test automation infrastructure has not been designed to handle
multiple codebases. But that's a problem we need to solve anyway, given
our shift to a many-project/product/codebase development model.

The AMO example is instructive here. Although AMO and the SDK are
different in many ways, they are similar in an important respect: it is
vital that changes to both products don't regress interactions with
Firefox. Just as a change to the SDK shouldn't produce addons that break
or fail to work with Firefox, a change to AMO shouldn't break Firefox's
ability to let users search and browse addons in the Add-ons manager or
check for and offer addon updates to users.

And just as we have automated tests to verify the SDK's behavior on
every SDK commit, we should have automated tests to verify AMO's
behavior on every AMO commit. Even though AMO lives in a separate
repository. Without merging AMO into mozilla-central.

> The fact of life is that today, when people see non-green JP test runs
> on any try (be in try, mozilla-central, etc.), they automatically
> ignore them, and just assume that something is wrong with the SDK and
> somebody will magically take care of it.
That's not because the SDK codebase is in a separate repository, it's
because we enabled that test automation before the tests were stable and
manually runnable without too much effort, i.e. before we were ready to
require mozilla-central developers to observe them. The boy cried wolf
too many times, and his flock was too hard to reach from the village, so
now the villagers ignore him.

> This is very different to the common response to test failures in any
> other part of the codebase.  You might think that part of this is a
> cultural problem, and you will definitely be right, but at least from
> the perspective of many developers working on Gecko/Firefox, the SDK
> is a black box.  They don't get to see its code, they often don't know
> what stuff are tested in its tests, they can't use the tools that they
> use for working with the SDK and its tests, etc.
Landing the code in mozilla-central will not make the box more
transparent, and we can make the SDK's tests easy to run and analyze
regardless of where its code lives.

> I'm very worried about this trend, and I would really like us to do
> something to fix it.
Me too. Bug 629263 <https://bugzilla.mozilla.org/show_bug.cgi?id=629263>
tracks the work we're doing to make SDK tests stable and manually
runnable without too much effort.

> I don't really understand this part.  Fortunately, hg and git are very
> similar in how they think of history, and there are tools that allow
> people to work on github hosted clones of mozilla-central.
Git branches are fundamental different than (both kinds of) Mercurial
branches ("branches" and "repositories"), and they are significantly
easier to use for both feature and stabilization/release branching
(Mercurial has since implemented Git-like branches as "bookmarks",
although I haven't tried the feature and can't vouch for its
equivalence); while GitHub's pull request feature is significantly
easier to use, especially for casual contributors, than the Bugzilla
bug/patch-based system we use with Mercurial.

> I really think that which revision control system a person uses is
> mostly a personal choice, and should not reflect at all on the
> decisions that can affect the quality of a product that we're
> building.  To give you a concrete example, I personally am really
> close to switching to git for all of my mozilla-central development,
> and I'll just use the tools when I need to make my RCS talk to Mozilla's.
My sense is that RCSes are mostly a collective choice, as a relative
minority of developers like you are willing to put in the extra effort
to work with a non-default system. And that is especially the case for
casual contributors. Great tooling to integrate multiple systems in a
way that is transparent to developers could theoretically make a
difference. We've experimented with some such tooling (f.e. Dietrich's
addon to create a patch and Bugzilla review request from a GitHub pull
request), although I haven't come across anything close to a magic bullet.

Also, while I don't agree that Git or having the SDK in a separate
repository hurts its quality, note that quality is not the only
important metric. Developer productivity is another. The high quality
product we can ship too late to make a difference to our target audience
also doesn't matter.

> So I think we have two things to discuss here:
>
> * What are the benefits and disadvantages of integrating the SDK code
> (at least the internals and APIs components) into mozilla-central for
> our users and developers?
> * What are the benefits and disadvantages of switching the development
> of the SDK (no matter where its code lives) from github to hg/bugzilla?
I would phrase the questions slightly differently:

  * Which components of the SDK should ship with core Firefox?
  * How should the various components of the SDK be developed?

I make that distinction because it's possible to ship components of the
SDK with core Firefox without merging their code into mozilla-central
(f.e. by shipping them as a default addon). Whether or not it's
preferable to do so is for the second question to address.

And the third question is: when should whatever changes we decide to
make occur? I don't think the answer to that question is simply "as soon
as we decide to make the changes." There are many factors to consider,
like the time it would take for SDK developers to get up to speed on
mozilla-central development, the SDK's short- and long-term goals and
their impact on the various components of the SDK, etc.

-myk

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Add-on SDK Repacks Proposal

Ehsan Akhgari
On Tue, Oct 11, 2011 at 4:26 PM, Myk Melez <[hidden email]> wrote:

>   Here is a concrete example: does the existence of the most basic
> SDK-based add-on break any of the functions of the Firefox UI?  We have an
> entire suite of automated tests (the browser-chrome tests) to try to answer
> the question "do all of the functions of the Firefox UI work as expected?",
> but today we have no way of ensuring the same quality when the most basic
> SDK-based add-on is installed.  Being able to run those tests on a build of
> Firefox including the internals and API parts of the SDK will enable us
> answer this question.
>
>
> Yup, that's an important question to address, but it seems orthogonal to
> the location of the SDK code, since we would address it by testing Firefox
> with an SDK-based addon installed in either case.
>

So, how would that work?  Are you thinking of pulling down the SDK code as
part of Firefox's build, packaging an add-on using the SDK and using that in
a separate run of the browser-chrome suite on every mozilla-central commit,
and also pulling down the most recent build and test package on the SDK
side, and using those to run the browser-chrome suite on every SDK commit?
Sure, we could make that work.  But I would argue that this is not the most
elegant solution.

>   This is possible to answer today, with some level of automation and most
> likely a significant level of effort from our RelEng team.  When we get to
> the level of confidence to say "yes" to this question, the one would be:
> "Which commit to the SDK code broke this functionality in the UI?".  That's
> harder to answer today.  Then, we can ask more interesting questions, like
> "Do the SDK components cause any memory leaks?", or "Do the SDK components
> cause any assertions in Firefox/Gecko?".  And the questions keep getting
> harder and harder to answer.
>
> I see the difficulty of addressing these questions given scarce release
> engineering resources, but I don't see how the difficulty is associated with
> maintaining separate codebases, except for the general problem that
> Mozilla's test automation infrastructure has not been designed to handle
> multiple codebases. But that's a problem we need to solve anyway, given our
> shift to a many-project/product/codebase development model.
>

Well, of course, this is all just software and anything is possible, but
yes, I did have the finite RelEng resources constraint in mind.  I'm not
aware of any plans in RelEng to provide support for automation for this type
of scenario where we need to test something tracked by two separate RCSs.

And aside from that, there are other technical reasons why shipping the SDK
built into Firefox provides a better experience for our users and add-on
developers, as pointed out by Jonas.


>  The AMO example is instructive here. Although AMO and the SDK are
> different in many ways, they are similar in an important respect: it is
> vital that changes to both products don't regress interactions with Firefox.
> Just as a change to the SDK shouldn't produce addons that break or fail to
> work with Firefox, a change to AMO shouldn't break Firefox's ability to let
> users search and browse addons in the Add-ons manager or check for and offer
> addon updates to users.
>
> And just as we have automated tests to verify the SDK's behavior on every
> SDK commit, we should have automated tests to verify AMO's behavior on every
> AMO commit. Even though AMO lives in a separate repository. Without merging
> AMO into mozilla-central.
>

The AMO example is still different to me in the level of coupling between
AMO and the add-on SDK (at least the internals part).  AMO and the Add-ons
Manager component talk to each other in well-defined web-based protocols.
But even considering that, we've had at least one problem in the past which
was caused by the coupling between AMO and Firefox (landing of automated
blocklist updates breaking mozilla-central tests).  The situation with AMO
is not ideal to the best of my knowledge, so I wouldn't take AMO as a 100%
successful pattern of how to develop two products in separate codebases.

On the same topic, I would love to hear what you think about the degrees of
coupling between the three SDK components and Firefox/Gecko as I mentioned
in my previous post.


>   The fact of life is that today, when people see non-green JP test runs
> on any try (be in try, mozilla-central, etc.), they automatically ignore
> them, and just assume that something is wrong with the SDK and somebody will
> magically take care of it.
>
> That's not because the SDK codebase is in a separate repository, it's
> because we enabled that test automation before the tests were stable and
> manually runnable without too much effort, i.e. before we were ready to
> require mozilla-central developers to observe them. The boy cried wolf too
> many times, and his flock was too hard to reach from the village, so now the
> villagers ignore him.
>

That's part of the reason.  Another part is that if somebody wants to
investigate the issue, they wouldn't know what to do (at least without the
help of the SDK folks).  They don't know where the code is, they don't know
how to run those tests on their local builds, etc.  (Cultural problem yes,
I'm just pointing out this as an example of why there is a cost to having
the two products live in different places here.)


>
>   This is very different to the common response to test failures in any
> other part of the codebase.  You might think that part of this is a cultural
> problem, and you will definitely be right, but at least from the perspective
> of many developers working on Gecko/Firefox, the SDK is a black box.  They
> don't get to see its code, they often don't know what stuff are tested in
> its tests, they can't use the tools that they use for working with the SDK
> and its tests, etc.
>
> Landing the code in mozilla-central will not make the box more transparent,
>

Would you mind clarifying why you think it won't help?


>  and we can make the SDK's tests easy to run and analyze regardless of
> where its code lives.
>

Yes, definitely.


>   I don't really understand this part.  Fortunately, hg and git are very
> similar in how they think of history, and there are tools that allow people
> to work on github hosted clones of mozilla-central.
>
>
> Git branches are fundamental different than (both kinds of) Mercurial
> branches ("branches" and "repositories"), and they are significantly easier
> to use for both feature and stabilization/release branching (Mercurial has
> since implemented Git-like branches as "bookmarks", although I haven't tried
> the feature and can't vouch for its equivalence); while GitHub's pull
> request feature is significantly easier to use, especially for casual
> contributors, than the Bugzilla bug/patch-based system we use with
> Mercurial.
>

I don't really want this thread to be converted into a discussion on RCS
internals, but in short, Mercurial bookmarks can now be synced with remote
repos, so they can serve as a pretty similar feature to git's branching (git
still has other interesting features, and there may be some weird problems
caused by the way we rely on the single-HEAD-on-default-branch invariant,
and github enables an interesting development workflow, these are all true).

My main point was, if the SDK source code lives in mozilla-central, the SDK
team can still use git and github for their development.  The question of
whether that should or should not be changed is one that is orthogonal to
the issue at hand, and I'm personally a lot less interested in having that
discussion for now.  :-)


>   I really think that which revision control system a person uses is
> mostly a personal choice, and should not reflect at all on the decisions
> that can affect the quality of a product that we're building.  To give you a
> concrete example, I personally am really close to switching to git for all
> of my mozilla-central development, and I'll just use the tools when I need
> to make my RCS talk to Mozilla's.
>
> My sense is that RCSes are mostly a collective choice, as a relative
> minority of developers like you are willing to put in the extra effort to
> work with a non-default system. And that is especially the case for casual
> contributors. Great tooling to integrate multiple systems in a way that is
> transparent to developers could theoretically make a difference. We've
> experimented with some such tooling (f.e. Dietrich's addon to create a patch
> and Bugzilla review request from a GitHub pull request), although I haven't
> come across anything close to a magic bullet.
>
> Also, while I don't agree that Git or having the SDK in a separate
> repository hurts its quality, note that quality is not the only important
> metric. Developer productivity is another. The high quality product we can
> ship too late to make a difference to our target audience also doesn't
> matter.
>

I never suggested that developer productivity should be sacrificed without a
good reason.

That being said, I really believe that the developer productivity/quality is
the wrong tradeoff to have in mind.  You're assuming that striving for
higher-quality products automatically diminishes developer productivity.  We
already have a code base with extremely high quality bars, and there are
tons of people contributing to it, even with the painful process around
bugzilla, review requirements, test requirements, try server runs, etc.
This shows that assumption to be false.

And this brings us back to what roc and Joe were arguing earlier in this
thread.  Why is the convenience of the SDK developers more valuable than
shipping a better and higher quality product for our users and add-on
developers?


>
>   So I think we have two things to discuss here:
>
> * What are the benefits and disadvantages of integrating the SDK code (at
> least the internals and APIs components) into mozilla-central for our users
> and developers?
> * What are the benefits and disadvantages of switching the development of
> the SDK (no matter where its code lives) from github to hg/bugzilla?
>
> I would phrase the questions slightly differently:
>
>    - Which components of the SDK should ship with core Firefox?
>    - How should the various components of the SDK be developed?
>
> I make that distinction because it's possible to ship components of the SDK
> with core Firefox without merging their code into mozilla-central (f.e. by
> shipping them as a default addon). Whether or not it's preferable to do so
> is for the second question to address.
>

OK, so what are your answers to those two questions?  (The second question
was mostly answered in your blog post I think, but I still think that we
need to look into ways to keep the SDK developers productive and get a
higher quality product both at the same time.)


> And the third question is: when should whatever changes we decide to make
> occur? I don't think the answer to that question is simply "as soon as we
> decide to make the changes." There are many factors to consider, like the
> time it would take for SDK developers to get up to speed on mozilla-central
> development, the SDK's short- and long-term goals and their impact on the
> various components of the SDK, etc.
>

Sure.  We can discuss the third question after we agree on an answer to the
first two, and I think that the SDK team is more likely to be able to answer
this question more accurately.

--
Ehsan
<http://ehsanakhgari.org/>
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Add-on SDK Repacks Proposal

Axel Hecht-2
In reply to this post by Myk Melez
On 12.10.11 00:26, Ehsan Akhgari wrote:

>> I see the difficulty of addressing these questions given scarce release
>> >  engineering resources, but I don't see how the difficulty is associated with
>> >  maintaining separate codebases, except for the general problem that
>> >  Mozilla's test automation infrastructure has not been designed to handle
>> >  multiple codebases. But that's a problem we need to solve anyway, given our
>> >  shift to a many-project/product/codebase development model.
>> >
> Well, of course, this is all just software and anything is possible, but
> yes, I did have the finite RelEng resources constraint in mind.  I'm not
> aware of any plans in RelEng to provide support for automation for this type
> of scenario where we need to test something tracked by two separate RCSs.

Just a comment on this one. This is beyond just "releng", the underlying
architecture buildbot doesn't handle multiple repositories well, and
fixing that is in the cards for quite a while. Being a "build automation
infrastructure", adding that capability to buildbot is beyond "just make
it work for jetpack at mozilla".

Build display was awkward for mobile when it was a separate repo still,
and I doubt that tbpl for tb is at par with it for fx, just for the
separate repos involved.

All to say, adding multiple repos is a bigger task than to just set
things up that way.

Axel
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: Add-on SDK Repacks Proposal

Myk Melez
In reply to this post by Ehsan Akhgari
On 2011-10-11 15:26, Ehsan Akhgari wrote:
> Are you thinking of pulling down the SDK code as part of Firefox's
> build, packaging an add-on using the SDK and using that in a separate
> run of the browser-chrome suite on every mozilla-central commit, and
> also pulling down the most recent build and test package on the SDK
> side, and using those to run the browser-chrome suite on every SDK commit?
Something like that, yes. We already pull SDK code to run SDK tests for
mozilla-central builds on mozilla-central changes. We could extend that
to also package an addon using the SDK and then run the browser-chrome
suite with that addon installed (although the SDK runs tests by
installing an addon containing its tests into Firefox, so we are
essentially doing this already).

> Well, of course, this is all just software and anything is possible,
> but yes, I did have the finite RelEng resources constraint in mind.  
> I'm not aware of any plans in RelEng to provide support for automation
> for this type of scenario where we need to test something tracked by
> two separate RCSs.
There are such plans, hatched not only because of Jetpack but also
RelEng's recognition that crucial product and feature development work
with Firefox dependencies that would benefit from continuous integration
is now happening in many (Git) repositories.

> And aside from that, there are other technical reasons why shipping
> the SDK built into Firefox provides a better experience for our users
> and add-on developers, as pointed out by Jonas.
Yes, I agree.

> The situation with AMO is not ideal to the best of my knowledge, so I
> wouldn't take AMO as a 100% successful pattern of how to develop two
> products in separate codebases.
I'm not saying it's a successful pattern, I'm pointing out that we
should continuously integrate AMO with Firefox, even though it
transparently belongs in a separate repository, and any lack of success
in keeping it compatible with Firefox is due to our failure to
continuously integrate it, not because it lives in a separate repository.

In other words, AMO demonstrates how the value of continuous
integration, and the importance of implementing it, is orthogonal to the
question of where continuously integrated code should live.

> On the same topic, I would love to hear what you think about the
> degrees of coupling between the three SDK components and Firefox/Gecko
> as I mentioned in my previous post.
I agree with your assessment about the relative degrees of coupling
(although some of the tightness may be loosenable via more careful
definition of the interfaces the SDK's module loader uses, and that's an
option to consider when making these decisions).

> That's part of the reason.  Another part is that if somebody wants to
> investigate the issue, they wouldn't know what to do (at least without
> the help of the SDK folks).  They don't know where the code is, they
> don't know how to run those tests on their local builds, etc.  
> (Cultural problem yes, I'm just pointing out this as an example of why
> there is a cost to having the two products live in different places here.)
Developers don't know how to run other test suites and investigate their
tests' problems on their local builds either until they learn how by
reading the docs
<https://developer.mozilla.org/en/Mozilla_automated_testing>, and we
don't need to merge SDK code into mozilla-central to make it that easy
for developers to learn how to run and investigate SDK tests too.

>> This is very different to the common response to test failures in any
>> other part of the codebase.  You might think that part of this is a
>> cultural problem, and you will definitely be right, but at least from
>> the perspective of many developers working on Gecko/Firefox, the SDK
>> is a black box.  They don't get to see its code, they often don't
>> know what stuff are tested in its tests, they can't use the tools
>> that they use for working with the SDK and its tests, etc.
>
>     Landing the code in mozilla-central will not make the box more
>     transparent,
>
>
> Would you mind clarifying why you think it won't help?
The opaque nature of the SDK is primarily due to it being a new,
unfamiliar codebase that does things existing code hasn't done before
and tests them using a harness they haven't used before.

It may be slightly easier for a developer to familiarize themselves with
code in mozilla-central, because doing so doesn't require pulling
another repository, but the majority of the pain remains, and to the
extent that pulling another repository is a cost, tooling can make it
trivial.

> That being said, I really believe that the developer
> productivity/quality is the wrong tradeoff to have in mind.  You're
> assuming that striving for higher-quality products automatically
> diminishes developer productivity.
I assume no such thing, neither generally nor in the case of the SDK
specifically, whose existing development model is just as capable of
achieving equal or better quality with higher productivity.

> We already have a code base with extremely high quality bars, and
> there are tons of people contributing to it, even with the painful
> process around bugzilla, review requirements, test requirements, try
> server runs, etc.  This shows that assumption to be false.
Firefox might have even more contributions without all the pain. Of
course, they might be different kinds of contributions, and perhaps
masochists write better code. ;-)

> And this brings us back to what roc and Joe were arguing earlier in
> this thread.  Why is the convenience of the SDK developers more
> valuable than shipping a better and higher quality product for our
> users and add-on developers?
It isn't. The Jetpack team is driven by our desire to create the best
possible experience for addon developers and users. I care about the
tools we use and the processes we put in place around them not because
they are a goal in themselves but because they affect the ability of the
team to realize its goals and achieve its mission.

> OK, so what are your answers to those two questions?
Regarding question one, I think the module loader, core API
implementations, and possibly the bootstrapper should ship with core
Firefox.

I'm not sure whether or not the command-line tools and any future local
graphical development tools we implement should ship with core Firefox,
although I do think they should at least ship as an addon rather than
the way they do today, to take advantage of Firefox's addon distribution
channels and enable the development of integrated local graphical
development tools.

Regarding question two, I think the code for components that ship with
core Firefox should probably merge into mozilla-central and be developed
in that repository, although it's worth investigating the option of
shipping some or all of them with core Firefox via a default add-on, in
which case I think they should remain in a distinct Git repository. And
components that do not ship with core Firefox should remain in a
distinct Git repository.

> (The second question was mostly answered in your blog post I think,
> but I still think that we need to look into ways to keep the SDK
> developers productive and get a higher quality product both at the
> same time.)
I agree, and not only for SDK developers. In my opinion, that way is to
evolve the release engineering infrastructure to support the continuous
integration of multiple products and product features developed in
distinct repositories.

-myk

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
12