The purpose of binary components

classic Classic list List threaded Threaded
84 messages Options
12345
Reply | Threaded
Open this post in threaded view
|

The purpose of binary components

Benjamin Smedberg
This post was prompted by the thread "Why do we keep revising intervace
ids?" in mozilla.dev.platform, and Asa's reply which suggests that we
reconsider our current policy of breaking (requiring a recompile) of
binary components in each 6-week cycle. I thought that before we got
into the details of our solution we should define the problem space.

== Background ==

The Mozilla platform has supported loadable components since the advent
of XPCOM. Those components can primarily be written in two different
languages: C++ compiled components and JavaScript components.

== Purpose ==

Loadable XPCOM components allow developers of both applications and
extensions access to the following basic functions:

* Adding new functionality to the platform available via our XPCOM
object framework.
* Extending existing functionality of the platform, for example adding a
new protocol handler.
* Replacing core functionality of existing components/services.

== Binary Versus Script Components ==

There are tradeoffs in the capabilities of C++ and JavaScript
components. JavaScript components have the following properties:

* They do not depend on specific interface layout: changes to XPCOM
interfaces which don't directly impact the specific calls made by the
component don't require any additional work and are automatically
compatible.
* They are automatically protected by the memory safety guarantees of
our JavaScript engine. This is not a 100% guarantee against crashes, but
it means that JS components are much less fragile.
* They have access to DOM workers which allow multithreading using
message passing which is well tested and memory-safe.
* They have access to load arbitrary binary code via ctypes. Note that
ctypes introduces potential type-unsafety and requires very careful
programming.

Binary components (written in C++) have different properties:

* They allow deep access to our platform; they can call into low-level
content APIs and other nonscriptable and private interfaces.
* They have access to some threading facilities which are not available
to JavaScript (but note ChromeWorkers above).
* The source code does not need to be made available to users.

== Current Compatibility Rules ==

Currently, JavaScript components are assumed to be compatible across
releases.

Binary XPCOM components are version-checked at startup and must match
the current version of the platform exactly. This means that binary
components must be recompiled for each 6-week release cycle.

It is possible to load binary XPCOM code which bypasses the version
check. We know of at least one case (bug 680927) where Oracle software
uses some kind of hooking to load a DLL into the Firefox process which
then uses XPCOM without any version checking (this caused a startup
crash for users of that software because the Oracle DLL did not check
for interface changes).

== Possible Alternatives ==

There are few possible alternatives for what we could do to make using
binary components easier or harder:

A. Change nothing, leave the status quo as-is. Extensions which use
binary components and are not recompiled every 6 weeks will "break", but
they at least have the opportunity to do cool stuff . Users will
continue to be vulnerable to Oracle-type crashes.
B. Make no technical changes, but disallow binary components in release
extensions on AMO. We may allow binary components in beta or
experimental addons, with the understanding that these would need to be
updated for each release.
C. (copied from Asa's post) Only change interfaces every 3rd release (or
something like that). This would mean that extensions which use C++
components would need to compile less frequently. Users would still be
exposed to Oracle-type issues, but less frequently.
D. Stop loading binary components from extensions. Continue exporting
the XPCOM functions so that application authors can continue to use
them. Users might still be exposed to Oracle-type issues.
E. Stop loading binary components completely: compile Firefox as a
static binary and stop exporting the XPCOM functions completely.

== bsmedberg's Opinion ==

I see binary components as a useful tool in very specific circumstances.
Primarily, they are a good way to prototype and experiment with new
features which require mucking about with Mozilla internals. But I tend
to think that we should discourage their use in any production
environment, including perhaps disallowing them on AMO. I tend to think
we should consider option "B".

--BDS

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: The purpose of binary components

Henri Sivonen
On Tue, Jan 17, 2012 at 5:30 PM, Benjamin Smedberg
<[hidden email]> wrote:
> B. Make no technical changes, but disallow binary components in release
> extensions on AMO. We may allow binary components in beta or experimental
> addons, with the understanding that these would need to be updated for each
> release.

If binary components were for experimentation only, wouldn't it be
simpler to do experimentation on an experimental branch so that the
whole app is compiled with the experiment baked in?

> D. Stop loading binary components from extensions. Continue exporting the
> XPCOM functions so that application authors can continue to use them. Users
> might still be exposed to Oracle-type issues.

Who are "application authors" in this case? Authors of XULRunner apps?
Don't authors of XULRunner apps pretty much end up compiling XULRunner
themselves and, therefore, could bake their code into a static binary
of the type mentioned in option E?

> E. Stop loading binary components completely: compile Firefox as a static
> binary and stop exporting the XPCOM functions completely.

It seems to me that options other than A constrain what can be done in
a pragmatic way (either on the extension side in cases B and D or on
the Firefox side in case C) so much that we might as well choose
option E if we don't stick with option A.

--
Henri Sivonen
[hidden email]
http://hsivonen.iki.fi/
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: The purpose of binary components

Leander-4
In reply to this post by Benjamin Smedberg
Hello,

Due to my recent interests in XULRunnner and XPCOM components, i'm just
wondering if option (B) forbids the use of XPCOM components in XULRunner in
general or does it only apply only to firefox?

Regards,

Leander

On Tue, Jan 17, 2012 at 3:30 PM, Benjamin Smedberg <[hidden email]>wrote:

> This post was prompted by the thread "Why do we keep revising intervace
> ids?" in mozilla.dev.platform, and Asa's reply which suggests that we
> reconsider our current policy of breaking (requiring a recompile) of binary
> components in each 6-week cycle. I thought that before we got into the
> details of our solution we should define the problem space.
>
> == Background ==
>
> The Mozilla platform has supported loadable components since the advent of
> XPCOM. Those components can primarily be written in two different
> languages: C++ compiled components and JavaScript components.
>
> == Purpose ==
>
> Loadable XPCOM components allow developers of both applications and
> extensions access to the following basic functions:
>
> * Adding new functionality to the platform available via our XPCOM object
> framework.
> * Extending existing functionality of the platform, for example adding a
> new protocol handler.
> * Replacing core functionality of existing components/services.
>
> == Binary Versus Script Components ==
>
> There are tradeoffs in the capabilities of C++ and JavaScript components.
> JavaScript components have the following properties:
>
> * They do not depend on specific interface layout: changes to XPCOM
> interfaces which don't directly impact the specific calls made by the
> component don't require any additional work and are automatically
> compatible.
> * They are automatically protected by the memory safety guarantees of our
> JavaScript engine. This is not a 100% guarantee against crashes, but it
> means that JS components are much less fragile.
> * They have access to DOM workers which allow multithreading using message
> passing which is well tested and memory-safe.
> * They have access to load arbitrary binary code via ctypes. Note that
> ctypes introduces potential type-unsafety and requires very careful
> programming.
>
> Binary components (written in C++) have different properties:
>
> * They allow deep access to our platform; they can call into low-level
> content APIs and other nonscriptable and private interfaces.
> * They have access to some threading facilities which are not available to
> JavaScript (but note ChromeWorkers above).
> * The source code does not need to be made available to users.
>
> == Current Compatibility Rules ==
>
> Currently, JavaScript components are assumed to be compatible across
> releases.
>
> Binary XPCOM components are version-checked at startup and must match the
> current version of the platform exactly. This means that binary components
> must be recompiled for each 6-week release cycle.
>
> It is possible to load binary XPCOM code which bypasses the version check.
> We know of at least one case (bug 680927) where Oracle software uses some
> kind of hooking to load a DLL into the Firefox process which then uses
> XPCOM without any version checking (this caused a startup crash for users
> of that software because the Oracle DLL did not check for interface
> changes).
>
> == Possible Alternatives ==
>
> There are few possible alternatives for what we could do to make using
> binary components easier or harder:
>
> A. Change nothing, leave the status quo as-is. Extensions which use binary
> components and are not recompiled every 6 weeks will "break", but they at
> least have the opportunity to do cool stuff . Users will continue to be
> vulnerable to Oracle-type crashes.
> B. Make no technical changes, but disallow binary components in release
> extensions on AMO. We may allow binary components in beta or experimental
> addons, with the understanding that these would need to be updated for each
> release.
> C. (copied from Asa's post) Only change interfaces every 3rd release (or
> something like that). This would mean that extensions which use C++
> components would need to compile less frequently. Users would still be
> exposed to Oracle-type issues, but less frequently.
> D. Stop loading binary components from extensions. Continue exporting the
> XPCOM functions so that application authors can continue to use them. Users
> might still be exposed to Oracle-type issues.
> E. Stop loading binary components completely: compile Firefox as a static
> binary and stop exporting the XPCOM functions completely.
>
> == bsmedberg's Opinion ==
>
> I see binary components as a useful tool in very specific circumstances.
> Primarily, they are a good way to prototype and experiment with new
> features which require mucking about with Mozilla internals. But I tend to
> think that we should discourage their use in any production environment,
> including perhaps disallowing them on AMO. I tend to think we should
> consider option "B".
>
> --BDS
>
> ______________________________**_________________
> dev-platform mailing list
> [hidden email]
> https://lists.mozilla.org/**listinfo/dev-platform<https://lists.mozilla.org/listinfo/dev-platform>
>
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: The purpose of binary components

Justin Wood (Callek)-2
In reply to this post by Benjamin Smedberg
Benjamin Smedberg wrote:

> This post was prompted by the thread "Why do we keep revising intervace
> ids?" in mozilla.dev.platform, and Asa's reply which suggests that we
> reconsider our current policy of breaking (requiring a recompile) of
> binary components in each 6-week cycle. I thought that before we got
> into the details of our solution we should define the problem space.
>
> == Background ==
>
> The Mozilla platform has supported loadable components since the advent
> of XPCOM. Those components can primarily be written in two different
> languages: C++ compiled components and JavaScript components.
>
> == Purpose ==
>
> Loadable XPCOM components allow developers of both applications and
> extensions access to the following basic functions:
>
> * Adding new functionality to the platform available via our XPCOM
> object framework.
> * Extending existing functionality of the platform, for example adding a
> new protocol handler.
> * Replacing core functionality of existing components/services.
>
> == Binary Versus Script Components ==
>
> There are tradeoffs in the capabilities of C++ and JavaScript
> components. JavaScript components have the following properties:
>
> * They do not depend on specific interface layout: changes to XPCOM
> interfaces which don't directly impact the specific calls made by the
> component don't require any additional work and are automatically
> compatible.
> * They are automatically protected by the memory safety guarantees of
> our JavaScript engine. This is not a 100% guarantee against crashes, but
> it means that JS components are much less fragile.
> * They have access to DOM workers which allow multithreading using
> message passing which is well tested and memory-safe.
> * They have access to load arbitrary binary code via ctypes. Note that
> ctypes introduces potential type-unsafety and requires very careful
> programming.
>
> Binary components (written in C++) have different properties:
>
> * They allow deep access to our platform; they can call into low-level
> content APIs and other nonscriptable and private interfaces.
> * They have access to some threading facilities which are not available
> to JavaScript (but note ChromeWorkers above).
> * The source code does not need to be made available to users.
>
> == Current Compatibility Rules ==
>
> Currently, JavaScript components are assumed to be compatible across
> releases.
>
> Binary XPCOM components are version-checked at startup and must match
> the current version of the platform exactly. This means that binary
> components must be recompiled for each 6-week release cycle.
>
> It is possible to load binary XPCOM code which bypasses the version
> check. We know of at least one case (bug 680927) where Oracle software
> uses some kind of hooking to load a DLL into the Firefox process which
> then uses XPCOM without any version checking (this caused a startup
> crash for users of that software because the Oracle DLL did not check
> for interface changes).
>
> == Possible Alternatives ==
>
> There are few possible alternatives for what we could do to make using
> binary components easier or harder:
>
> A. Change nothing, leave the status quo as-is. Extensions which use
> binary components and are not recompiled every 6 weeks will "break", but
> they at least have the opportunity to do cool stuff . Users will
> continue to be vulnerable to Oracle-type crashes.
> B. Make no technical changes, but disallow binary components in release
> extensions on AMO. We may allow binary components in beta or
> experimental addons, with the understanding that these would need to be
> updated for each release.
> C. (copied from Asa's post) Only change interfaces every 3rd release (or
> something like that). This would mean that extensions which use C++
> components would need to compile less frequently. Users would still be
> exposed to Oracle-type issues, but less frequently.
> D. Stop loading binary components from extensions. Continue exporting
> the XPCOM functions so that application authors can continue to use
> them. Users might still be exposed to Oracle-type issues.
> E. Stop loading binary components completely: compile Firefox as a
> static binary and stop exporting the XPCOM functions completely.
>
> == bsmedberg's Opinion ==
>
> I see binary components as a useful tool in very specific circumstances.
> Primarily, they are a good way to prototype and experiment with new
> features which require mucking about with Mozilla internals. But I tend
> to think that we should discourage their use in any production
> environment, including perhaps disallowing them on AMO. I tend to think
> we should consider option "B".

Re: Option B.

It would relegate Lightning to a non-release addon.
It would relegate Enigmail to a non-release addon.

These are two addons that users of Thunderbird for example, consider
necessary. Yes doing Lightning/Enigmail with simply binary-parts but
using js-ctypes to access those parts should be theoretically possible,
but I make no guess as to the amount of work.

So *IFF* we do B we should have a way for "special review" to allow
binary-addons to still be a release product.

I am sure there are more. And we should explicitly allow binary pieces
on AMO, just not binary-linked-with-Firefox components in the general
sense, since people can create binary dll's and load it via js-ctypes!

--
~Justin Wood (Callek)

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: The purpose of binary components

Wes Garland
In reply to this post by Benjamin Smedberg
On 17 January 2012 10:30, Benjamin Smedberg <[hidden email]> wrote:

> * They have access to load arbitrary binary code via ctypes. Note that
> ctypes introduces potential type-unsafety and requires very careful
> programming.
>

The hazards for js-ctypes are far more widespread than simple type
un-safety.

It is extremely difficult for even a seasoned C programmer to write
js-ctypes which is portable across a multitide of architectures and
operating systems, unless the APIs which are used from js-ctypes are very
narrowly defined.  I cannot think of an API list other than "Mozilla XPCOM,
JSAPI, etc" which is sufficiently controlled by Mozilla that we can make
any kind of promises or guarantees about extensions which use js-ctypes.

In some regards, components that are written with js-ctypes are in fact
*less* portable than binary components, because the JS programmer has to do
some of the work of the C compiler.

In order to completely portably use js-ctypes in an extension, the JS
developer must know at least the following:
1 - Location on disk of any libraries used that are not on the library path
2 - Calling convention of any functions used
3 - Implementation details of any of the C APIs which are implemented as
macros
4 - Struct layout of any structs used
5 - Struct packing rules of any structs used

All five of these can vary based on operating system, operating system
version, third party lib versions, and even the compiler used to build said
libraries.  None of these examples are theoretical, they are challenges
faced by any developer in any scripting language trying to write portable
FFI code.  Every one of them has bit me personally at some point in my
career.s not need to be made available to users.

Impact?  If we are allowing the use of js-ctypes on AMO, we cannot
reasonably treat extensions which use them the same as plain JS
extensions.  We can probably treat them like binary extensions, if we
"sniff" the code and make sure they are only using Mozilla's libraries.  If
they use any system or third party libraries, we cannot actually make any
guarantees for the users at all.  Maybe extensions like that should come
through vendor channels (e.g. Debian).

== Possible Alternatives ==
> C. (copied from Asa's post) Only change interfaces every 3rd release (or
> something like that). This would mean that extensions which use C++
> components would need to compile less frequently. Users would still be
> exposed to Oracle-type issues, but less frequently.
>

Observation:  This gives us, effectively, as Major.Minor release scheme,
where the numbering scheme is in trinary instead of the usual decimal.

If you're trying to remove Oracle-type issues: what kind of an impact would
it make to Firefox to have ALL XPCOM symbols hidden -- except for one.
That one returns a struct full of function pointers (or object full of
methods, you get the idea) -- but only if the caller supplies the correct
'magic cookie'.

Question: how does the binary extension issue interact with virus scanners
et al on Windows?

Wes

--
Wesley W. Garland
Director, Product Development
PageMail, Inc.
+1 613 542 2787 x 102
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: The purpose of binary components

Jorge Villalobos-2
In reply to this post by Benjamin Smedberg
On 1/17/12 9:30 AM, Benjamin Smedberg wrote:

> This post was prompted by the thread "Why do we keep revising intervace
> ids?" in mozilla.dev.platform, and Asa's reply which suggests that we
> reconsider our current policy of breaking (requiring a recompile) of
> binary components in each 6-week cycle. I thought that before we got
> into the details of our solution we should define the problem space.
>
> == Background ==
>
> The Mozilla platform has supported loadable components since the advent
> of XPCOM. Those components can primarily be written in two different
> languages: C++ compiled components and JavaScript components.
>
> == Purpose ==
>
> Loadable XPCOM components allow developers of both applications and
> extensions access to the following basic functions:
>
> * Adding new functionality to the platform available via our XPCOM
> object framework.
> * Extending existing functionality of the platform, for example adding a
> new protocol handler.
> * Replacing core functionality of existing components/services.
>
> == Binary Versus Script Components ==
>
> There are tradeoffs in the capabilities of C++ and JavaScript
> components. JavaScript components have the following properties:
>
> * They do not depend on specific interface layout: changes to XPCOM
> interfaces which don't directly impact the specific calls made by the
> component don't require any additional work and are automatically
> compatible.
> * They are automatically protected by the memory safety guarantees of
> our JavaScript engine. This is not a 100% guarantee against crashes, but
> it means that JS components are much less fragile.
> * They have access to DOM workers which allow multithreading using
> message passing which is well tested and memory-safe.
> * They have access to load arbitrary binary code via ctypes. Note that
> ctypes introduces potential type-unsafety and requires very careful
> programming.
>
> Binary components (written in C++) have different properties:
>
> * They allow deep access to our platform; they can call into low-level
> content APIs and other nonscriptable and private interfaces.
> * They have access to some threading facilities which are not available
> to JavaScript (but note ChromeWorkers above).
> * The source code does not need to be made available to users.
>
> == Current Compatibility Rules ==
>
> Currently, JavaScript components are assumed to be compatible across
> releases.
>
> Binary XPCOM components are version-checked at startup and must match
> the current version of the platform exactly. This means that binary
> components must be recompiled for each 6-week release cycle.
>
> It is possible to load binary XPCOM code which bypasses the version
> check. We know of at least one case (bug 680927) where Oracle software
> uses some kind of hooking to load a DLL into the Firefox process which
> then uses XPCOM without any version checking (this caused a startup
> crash for users of that software because the Oracle DLL did not check
> for interface changes).
>
> == Possible Alternatives ==
>
> There are few possible alternatives for what we could do to make using
> binary components easier or harder:
>
> A. Change nothing, leave the status quo as-is. Extensions which use
> binary components and are not recompiled every 6 weeks will "break", but
> they at least have the opportunity to do cool stuff . Users will
> continue to be vulnerable to Oracle-type crashes.
> B. Make no technical changes, but disallow binary components in release
> extensions on AMO. We may allow binary components in beta or
> experimental addons, with the understanding that these would need to be
> updated for each release.
> C. (copied from Asa's post) Only change interfaces every 3rd release (or
> something like that). This would mean that extensions which use C++
> components would need to compile less frequently. Users would still be
> exposed to Oracle-type issues, but less frequently.
> D. Stop loading binary components from extensions. Continue exporting
> the XPCOM functions so that application authors can continue to use
> them. Users might still be exposed to Oracle-type issues.
> E. Stop loading binary components completely: compile Firefox as a
> static binary and stop exporting the XPCOM functions completely.
>
> == bsmedberg's Opinion ==
>
> I see binary components as a useful tool in very specific circumstances.
> Primarily, they are a good way to prototype and experiment with new
> features which require mucking about with Mozilla internals. But I tend
> to think that we should discourage their use in any production
> environment, including perhaps disallowing them on AMO. I tend to think
> we should consider option "B".
>
> --BDS
>

I don't think option B is something that can be taken lightly. Add-ons
with binary components on AMO are in no way experiments, and most are
maintained by development groups, not individuals. They have very
significant usage on AMO (and much more outside of AMO):

https://addons.mozilla.org/en-US/firefox/compatibility/11.0?appver=1-11.0&type=binary

Closing the binary XPCOM way for extensions seems like a very
destructive approach, and it only worsens the problem that Asa was
trying to address.

- Jorge
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: The purpose of binary components

Asa Dotzler
In reply to this post by Benjamin Smedberg
On 1/17/2012 7:30 AM, Benjamin Smedberg wrote:

> I see binary components as a useful tool in very specific circumstances.
> Primarily, they are a good way to prototype and experiment with new
> features which require mucking about with Mozilla internals. But I tend
> to think that we should discourage their use in any production
> environment, including perhaps disallowing them on AMO. I tend to think
> we should consider option "B".

Benjamin, thank you for the best description of binary and js components
I've seen to date. It's a big help to folks like me who don't live so
deeply in the code.

There are a class of extension authors which, for what ever their
reasons, have expressed an unwillingness to move to JS ctypes. Perhaps
it's that they technically cannot or they're simply unwilling to (it
would be useful to get to the bottom of that). It's been nearly a year
now since we let them know we'd be requiring a re-compile every six
weeks and began encouraging them to move to JS solutions and they've not
done it.

Preliminary numbers say that binary add-on installs outnumber JS add-on
installs by about two to one. There are more than 30 add-ons with binary
components that each have more than a million Firefox users today. There
are only 13 JS add-ons that have more than a million Firefox users today.

I think we've lost this battle and we need to step back and revisit the
value we get from breaking that compatibility every 6 weeks and compare
that against regressing millions of users every 6 weeks.

- A
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: The purpose of binary components

Benjamin Smedberg
In reply to this post by Henri Sivonen
On 1/17/2012 10:51 AM, Henri Sivonen wrote:
> If binary components were for experimentation only, wouldn't it be
> simpler to do experimentation on an experimental branch so that the
> whole app is compiled with the experiment baked in?
I tend to think it's better to say to people "here is an experimental
addon, install it" than to say "here is an experimental build, install
it". This is because these experiments are often done not by Mozilla
itself but by third parties. We risk having users with non-updating
builds. But having custom builds is basically what we'd have to
recommend if we went with option E.

>
>> D. Stop loading binary components from extensions. Continue exporting the
>> XPCOM functions so that application authors can continue to use them. Users
>> might still be exposed to Oracle-type issues.
> Who are "application authors" in this case? Authors of XULRunner apps?
> Don't authors of XULRunner apps pretty much end up compiling XULRunner
> themselves and, therefore, could bake their code into a static binary
> of the type mentioned in option E?
Do you have data on this? I don't think that most app authors compile
their own XULRunner. In some cases they just try to use firefox -app.
> It seems to me that options other than A constrain what can be done in
> a pragmatic way (either on the extension side in cases B and D or on
> the Firefox side in case C) so much that we might as well choose
> option E if we don't stick with option A.
We're trying to balance the constraints here. Option E is really the
nuclear option, given our current history.

Leaner Bessa wrote:

> Due to my recent interests in XULRunnner and XPCOM components, i'm just
> wondering if option (B) forbids the use of XPCOM components in XULRunner in
> general or does it only apply only to firefox?
That is a question to be answered. If we go with option E) we probably
want XULRunner apps to use a similar build config for pragmatic/testing
reasons, so you'd have to compile your components into libxul itself.

Justin Wood (Callek) wrote:

> Re: Option B.
>
> It would relegate Lightning to a non-release addon.
> It would relegate Enigmail to a non-release addon.
>
> These are two addons that users of Thunderbird for example, consider
> necessary. Yes doing Lightning/Enigmail with simply binary-parts but
> using js-ctypes to access those parts should be theoretically possible,
> but I make no guess as to the amount of work.
If Firefox chooses option B, that doesn't necessarily mean that
Thunderbird must also choose it. I really don't know lightning at all,
but I believe that all enigmail needs is ipccode, which we should just
consider adding as part of core.

--BDS
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: The purpose of binary components

Benjamin Smedberg
In reply to this post by Wes Garland
On 1/17/2012 11:18 AM, Wes Garland wrote:
> It is extremely difficult for even a seasoned C programmer to write
> js-ctypes which is portable across a multitide of architectures and
> operating systems, unless the APIs which are used from js-ctypes are very
> narrowly defined.
In any non-trivial case, the recommended way to use ctypes is to compile
a DLL which exposes your well-known API, and have the C compiler do all
the dirty work for you. In this case none of the objections you have
mentioned really apply.

--BDS

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: The purpose of binary components

Wes Garland
On 17 January 2012 11:39, Benjamin Smedberg <[hidden email]> wrote:

> In any non-trivial case, the recommended way to use ctypes is to compile a
> DLL which exposes your well-known API, and have the C compiler do all the
> dirty work for you. In this case none of the objections you have mentioned
> really apply.
>

This is absolutely true, however, by following this approach, we now have a
binary component.

Asa's comment seems to indicate that he believes that js-ctypes are an
effective way to rid us of the need for binary components.   I do not
believe that is the case.

Wes

--
Wesley W. Garland
Director, Product Development
PageMail, Inc.
+1 613 542 2787 x 102
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: The purpose of binary components

Boris Zbarsky
In reply to this post by Asa Dotzler
On 1/17/12 11:28 AM, Asa Dotzler wrote:
> I think we've lost this battle and we need to step back and revisit the
> value we get from breaking that compatibility every 6 weeks

The main value is being able to sanely develop Gecko.

The alternative is basically to only do Gecko work 6 weeks out of every 18.

-Boris
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: The purpose of binary components

Asa Dotzler
In reply to this post by Benjamin Smedberg
On 1/17/2012 8:44 AM, Wes Garland wrote:

> On 17 January 2012 11:39, Benjamin Smedberg<[hidden email]>  wrote:
>
>> In any non-trivial case, the recommended way to use ctypes is to compile a
>> DLL which exposes your well-known API, and have the C compiler do all the
>> dirty work for you. In this case none of the objections you have mentioned
>> really apply.
>>
>
> This is absolutely true, however, by following this approach, we now have a
> binary component.
>
> Asa's comment seems to indicate that he believes that js-ctypes are an
> effective way to rid us of the need for binary components.   I do not
> believe that is the case.
>
> Wes
>

I don't know the tech well enough to reply in detail here, but my goal
is not to rid us the need for binary components. My goal is that our
users don't lose (and often regain a few days later) functionality every
6 weeks because add-ons weren't compatible with our update.

- A
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: The purpose of binary components

Benjamin Smedberg
In reply to this post by Wes Garland
On 1/17/2012 11:44 AM, Wes Garland wrote:
> On 17 January 2012 11:39, Benjamin Smedberg<[hidden email]>  wrote:
>
>> In any non-trivial case, the recommended way to use ctypes is to compile a
>> DLL which exposes your well-known API, and have the C compiler do all the
>> dirty work for you. In this case none of the objections you have mentioned
>> really apply.
>>
> This is absolutely true, however, by following this approach, we now have a
> binary component.
No, you have a *DLL*. I am in no way proposing that we get rid of
extensions shipping DLLs. I in fact strongly encourage it for some use
cases. IETab, for example, should almost certainly be implemented as an
NPAPI plugin, not an XPCOM component.

I am merely talking about binary *XPCOM* components, which are the root
problem when we're talking about either freezing XPCOM interface
definitions for some of the rapid release cycle or disallowing those
components based on the user experience.

--BDS

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: The purpose of binary components

Wes Garland
> No, you have a *DLL*. I am in no way proposing that we get rid of
> extensions shipping DLLs.

AH - Thank you for the clarification. That is definitely a saner approach
than trying to go js-ctypes-only.

Wes

--
Wesley W. Garland
Director, Product Development
PageMail, Inc.
+1 613 542 2787 x 102
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: The purpose of binary components

Benjamin Smedberg
In reply to this post by Asa Dotzler
On 1/17/2012 11:28 AM, Asa Dotzler wrote:
>
> There are a class of extension authors which, for what ever their
> reasons, have expressed an unwillingness to move to JS ctypes. Perhaps
> it's that they technically cannot or they're simply unwilling to (it
> would be useful to get to the bottom of that). It's been nearly a year
> now since we let them know we'd be requiring a re-compile every six
> weeks and began encouraging them to move to JS solutions and they've
> not done it.
I'm looking at
https://addons.mozilla.org/en-US/firefox/compatibility/11.0?appver=1-11.0&type=binary

What portion of these addons are not compatible at release time?

I can make educated guesses about the *possibility* of these addons
using something other than XPCOM components to achieve their
functionality, or could if we added a few APIs.

Garmin communicator: probably just code to communicate with the device
over USB: could certainly be accomplished with an intermediate ctypes
DLL, or perhaps even just from script using WebUSB (bug 674718/bug 711613).

IE Tab: the core embedding-trident functionality should be easily
accomplishable and would be far better using an NPAPI plugin. I don't
know if they also try to hook into our docshell hierarchy for the IE
tabs, which would in fact be very difficult to do from script. We could
perhaps expose an API to make that possible, with cooperation from them.

FoxTab: I really don't know why this can't just use webgl or an NPAPI
plugin.

Cooliris: also appears like it could use webgl or an NPAPI plugin.

Ant Video Downloader: does not appear to contain any binaries

Colorzilla: appears that this could easily use ctypes

> I think we've lost this battle and we need to step back and revisit
> the value we get from breaking that compatibility every 6 weeks and
> compare that against regressing millions of users every 6 weeks.
If this is true, I think we should revisit rapid release in general, and
not just this aspect of it. As Boris has mentioned, this will
significantly affect our ability to make changes to the platform.

--BDS

_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: The purpose of binary components

Henri Sivonen
On Tue, Jan 17, 2012 at 7:22 PM, Benjamin Smedberg
<[hidden email]> wrote:
> Cooliris: also appears like it could use webgl or an NPAPI plugin.

Cooliris is available for Chrome, so, clearly, the functionality can
be accomplished with NPAPI plug-in plus JS-based extension. IIRC, the
Firefox extension already contains an NPAPI plug-in.

--
Henri Sivonen
[hidden email]
http://hsivonen.iki.fi/
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: The purpose of binary components

Asa Dotzler
In reply to this post by Asa Dotzler
On 1/17/2012 9:22 AM, Benjamin Smedberg wrote:

> On 1/17/2012 11:28 AM, Asa Dotzler wrote:
>>
>> There are a class of extension authors which, for what ever their
>> reasons, have expressed an unwillingness to move to JS ctypes. Perhaps
>> it's that they technically cannot or they're simply unwilling to (it
>> would be useful to get to the bottom of that). It's been nearly a year
>> now since we let them know we'd be requiring a re-compile every six
>> weeks and began encouraging them to move to JS solutions and they've
>> not done it.
> I'm looking at
> https://addons.mozilla.org/en-US/firefox/compatibility/11.0?appver=1-11.0&type=binary
>
>
> What portion of these addons are not compatible at release time?

I believe those are only the ones hosted at AMO. That's a minority of
add-ons.

> I can make educated guesses about the *possibility* of these addons
> using something other than XPCOM components to achieve their
> functionality, or could if we added a few APIs.

How do we convince them to change, even if we have a destination that's
better?

>> I think we've lost this battle and we need to step back and revisit
>> the value we get from breaking that compatibility every 6 weeks and
>> compare that against regressing millions of users every 6 weeks.
> If this is true, I think we should revisit rapid release in general, and
> not just this aspect of it. As Boris has mentioned, this will
> significantly affect our ability to make changes to the platform.

When we moved to rapid releases, I got the impression that we needed
time to clean up but that we wouldn't be in that state forever. I think
we should discuss ways to amend rapid releases to make it work better
for platform developers and users.

- A
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: The purpose of binary components

Mark Banner-4
In reply to this post by Justin Wood (Callek)-2
On 17/01/2012 16:05, Justin Wood (Callek) wrote:
> Re: Option B.
>
> It would relegate Lightning to a non-release addon.
> It would relegate Enigmail to a non-release addon.
>
> These are two addons that users of Thunderbird for example, consider
> necessary. Yes doing Lightning/Enigmail with simply binary-parts but
> using js-ctypes to access those parts should be theoretically possible,
> but I make no guess as to the amount of work.

Work has already started on Lightning to make it non-binary. I believe
most of this is just translating the existing code to javascript.

I seem to remember from previous discussions I've had that Engmail would
be difficult without a significant amount of work, and possibly some
items landing in core.

Mark.
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: The purpose of binary components

Steve Wendt
In reply to this post by Justin Wood (Callek)-2
On 1/17/2012 8:05 AM, Justin Wood (Callek) wrote:

> It would relegate Lightning to a non-release addon.
> It would relegate Enigmail to a non-release addon.

It would be nice if those two were just included with Thunderbird and
SeaMonkey (like ChatZilla, DOM Inspector, etc.).
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
Reply | Threaded
Open this post in threaded view
|

Re: The purpose of binary components

Boris Zbarsky
In reply to this post by Asa Dotzler
On 1/17/12 12:32 PM, Asa Dotzler wrote:
> When we moved to rapid releases, I got the impression that we needed
> time to clean up but that we wouldn't be in that state forever.

I should not that we're not done with the cleanup yet...

"cleanup" includes implementing the dom4 core spec, note.

> I think we should discuss ways to amend rapid releases to make it work better
> for platform developers and users.

OK, let's define "better"?  It sounds like you would prefer to move to
an 18-week release cycle for platform (including web) features, yes?

-Boris
_______________________________________________
dev-planning mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-planning
12345