Avoiding the hard build dependency on NodeJS

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Avoiding the hard build dependency on NodeJS

Martin Husemann
I understand the desire for developers do use the best tool for each job, and integrating NodeJS in the regular work flow during development might be a good idea.

However, requiring NodeJS for each build (including the official releases) causes great problems: NodeJS itself is way less portable than Firefox, and I am building one of the strangler platforms - on sparc64. NodeJS seems to have no support at all for sparc CPUs, and I am not yet sure how hard it would be to add. I will investigate (as time permits).

So I have a concrete question: are the outputs of the NodeJS runs during a build actually target machine / OS dependent? At first I would hope not, and in this case: would it be possible to pre-generate the NodeJS output files for each official release, and then allow building with those pre-generated files instead of a local NodeJS installation?

Thanks,

Martin
_______________________________________________
dev-builds mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-builds
Reply | Threaded
Open this post in threaded view
|

Re: Avoiding the hard build dependency on NodeJS

Boris Zbarsky
On 11/7/18 8:32 AM, Martin Husemann wrote:
> NodeJS seems to have no support at all for sparc CPUs, and I am not yet sure how hard it would be to add.

You'd need to port V8 to work on sparc, right?  That sounds fairly
painful...

-Boris
_______________________________________________
dev-builds mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-builds
Reply | Threaded
Open this post in threaded view
|

Re: Avoiding the hard build dependency on NodeJS

Martin Husemann
On Wednesday, November 7, 2018 at 3:23:02 PM UTC+1, Boris Zbarsky wrote:

> You'd need to port V8 to work on sparc, right?  That sounds fairly
> painful...

Or port node to Spidermonkey, might be easier :-)
_______________________________________________
dev-builds mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-builds
Reply | Threaded
Open this post in threaded view
|

Re: Avoiding the hard build dependency on NodeJS

Boris Zbarsky
On 11/7/18 9:30 AM, Martin Husemann wrote:
> Or port node to Spidermonkey, might be easier :-)

True.  https://github.com/mozilla/spidernode/ is not really active, but
might be revivable...

-Boris
_______________________________________________
dev-builds mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-builds
Reply | Threaded
Open this post in threaded view
|

Re: Avoiding the hard build dependency on NodeJS

Gregory Szorc-3
In reply to this post by Martin Husemann
On Wed, Nov 7, 2018 at 5:35 AM Martin Husemann <[hidden email]> wrote:
I understand the desire for developers do use the best tool for each job, and integrating NodeJS in the regular work flow during development might be a good idea.

However, requiring NodeJS for each build (including the official releases) causes great problems: NodeJS itself is way less portable than Firefox, and I am building one of the strangler platforms - on sparc64. NodeJS seems to have no support at all for sparc CPUs, and I am not yet sure how hard it would be to add. I will investigate (as time permits).

So I have a concrete question: are the outputs of the NodeJS runs during a build actually target machine / OS dependent? At first I would hope not, and in this case: would it be possible to pre-generate the NodeJS output files for each official release, and then allow building with those pre-generated files instead of a local NodeJS installation?

I understand your concerns and empathize with your position. One of the reasons we held off on integrating NodeJS in Firefox's build system for so many years were concerns like this. We recognize that Firefox is the only viable modern web browser that can still be built on and for some platforms. Losing that could be devastating.

You asked a concrete question about build outputs. At this time, I don't foresee the outputs of NodeJS being target machine / OS dependent. i.e. I highly doubt we'll be producing executables, shared/static libraries, etc with NodeJS. I imagine anything using NodeJS will be related to JavaScript processing. And that should be machine / OS agnostic. Our go to language for build support tooling will likely continue to be Python for the foreseeable future. And we may start using more Rust in this space.

Would it be possible to pre-generate files requiring Node and provide a Firefox source distribution that doesn't require Node to build? In theory, yes. In fact, this is what some Firefox components have been doing! Before there was Node support, people would run Node out-of-band with the Firefox build system and check in the generated files into source control, where they were processed by the Node-less Firefox build system! (This workflow was not very efficient, hence one of the reasons for wanting to incorporate Node into the build system.)

While such a source distribution would be possible, maintaining it constitutes work. When we decided to require Node, we had to balance the loss of productivity that the lack of Node was costing us against the cost of supporting Node and other factors like the loss of some platforms being able to build Firefox natively. In the end, we decided that lack of Node was causing too much of a burden on Firefox developers and we needed native support for Node. While there was grumbling about this decision from some, overall the negative reaction seems relatively muted. At this time, I don't think an officially supported source distribution that supports building without Node can be justified. Maintaining such a distribution will constitute a non-trivial amount of work and will put a burden on our limited number of build system maintainers.

That being said, once the initial dust from requiring Node has settled and the build system Node integration bits are a bit more well-defined, it's quite possible that maintaining such a source distribution would be relatively easy. If someone were to contribute patches and offer to fix things when they break, we could try to support such a distribution. But even this could have future costs on build system maintenance. I'm hesitant to give blanket commitment to accepting patches if someone does this work. It's much easier to say "any new code not immediately beneficial to Mozilla/Firefox creates a maintenance burden and therefore should be rejected." I think a good first step would be for someone to produce a proof-of-concept demonstrating what things would like look. Take the existing Firefox source code, modify it to do what you want, and let's see what that looks like. In the worst case, you've re-enabled building Firefox on platforms like SPARC with a few patches. In the best case, the patches are accepted upstream and we commit to supporting a Firefox source distribution that doesn't require Node.

Gregory

_______________________________________________
dev-builds mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-builds
Reply | Threaded
Open this post in threaded view
|

Re: Avoiding the hard build dependency on NodeJS

Ehsan Akhgari
In reply to this post by Martin Husemann
I wonder if ChakraCore (the JS engine used inside Edge) has a JavaScript interpreter that would make it viable to run node-chakracore (https://github.com/nodejs/node-chakracore) to run on sparc64.  Have you tried that by any chance?  (Of course even if that works, how compatible the node_modules used in mozilla-central would be with node-chakracore remains to be seen...)

Cheers,
Ehsan

On Wed, Nov 7, 2018 at 8:35 AM Martin Husemann <[hidden email]> wrote:
I understand the desire for developers do use the best tool for each job, and integrating NodeJS in the regular work flow during development might be a good idea.

However, requiring NodeJS for each build (including the official releases) causes great problems: NodeJS itself is way less portable than Firefox, and I am building one of the strangler platforms - on sparc64. NodeJS seems to have no support at all for sparc CPUs, and I am not yet sure how hard it would be to add. I will investigate (as time permits).

So I have a concrete question: are the outputs of the NodeJS runs during a build actually target machine / OS dependent? At first I would hope not, and in this case: would it be possible to pre-generate the NodeJS output files for each official release, and then allow building with those pre-generated files instead of a local NodeJS installation?

Thanks,

Martin
_______________________________________________
dev-builds mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-builds


--
Ehsan

_______________________________________________
dev-builds mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-builds
Reply | Threaded
Open this post in threaded view
|

Re: Avoiding the hard build dependency on NodeJS

Mike Hommey
In reply to this post by Martin Husemann
On Wed, Nov 07, 2018 at 05:32:40AM -0800, Martin Husemann wrote:
> I understand the desire for developers do use the best tool for each job, and integrating NodeJS in the regular work flow during development might be a good idea.
>
> However, requiring NodeJS for each build (including the official releases) causes great problems: NodeJS itself is way less portable than Firefox, and I am building one of the strangler platforms - on sparc64. NodeJS seems to have no support at all for sparc CPUs, and I am not yet sure how hard it would be to add. I will investigate (as time permits).
>
> So I have a concrete question: are the outputs of the NodeJS runs during a build actually target machine / OS dependent? At first I would hope not, and in this case: would it be possible to pre-generate the NodeJS output files for each official release, and then allow building with those pre-generated files instead of a local NodeJS installation?
>

Note that for your _immediate_ needs, you can just --disable-nodejs,
because node is currently *not* actually used for anything. That won't
be the case for very long, though.

Mike
_______________________________________________
dev-builds mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-builds
Reply | Threaded
Open this post in threaded view
|

Re: Avoiding the hard build dependency on NodeJS

Dan Mosedale-4
For better and for worse, I believe that's no longer true: devtools is
now using it to transpile JSX files to JS files as part of the normal
build.

Dan
Am Do., 8. Nov. 2018 um 13:55 Uhr schrieb Mike Hommey <[hidden email]>:

>
> On Wed, Nov 07, 2018 at 05:32:40AM -0800, Martin Husemann wrote:
> > I understand the desire for developers do use the best tool for each job, and integrating NodeJS in the regular work flow during development might be a good idea.
> >
> > However, requiring NodeJS for each build (including the official releases) causes great problems: NodeJS itself is way less portable than Firefox, and I am building one of the strangler platforms - on sparc64. NodeJS seems to have no support at all for sparc CPUs, and I am not yet sure how hard it would be to add. I will investigate (as time permits).
> >
> > So I have a concrete question: are the outputs of the NodeJS runs during a build actually target machine / OS dependent? At first I would hope not, and in this case: would it be possible to pre-generate the NodeJS output files for each official release, and then allow building with those pre-generated files instead of a local NodeJS installation?
> >
>
> Note that for your _immediate_ needs, you can just --disable-nodejs,
> because node is currently *not* actually used for anything. That won't
> be the case for very long, though.
>
> Mike
> _______________________________________________
> dev-builds mailing list
> [hidden email]
> https://lists.mozilla.org/listinfo/dev-builds
_______________________________________________
dev-builds mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-builds