BUMP: Why would JS_NewGlobalObject fail

classic Classic list List threaded Threaded
6 messages Options
Reply | Threaded
Open this post in threaded view
|

BUMP: Why would JS_NewGlobalObject fail

Kent Williams
Simpler question.

My program creates one global JSRuntime object.

To evaluate a script, a C++ ScriptData object is constructed which in
turn creates a new context with JS_NewContext, and a global object
JS_NewGlobalObject, declared thusly:

JSContext *context;
JS::PersistentRooted<JSObject *> global;

These are member variables of the C++ ScriptData object.

ScriptData::~ScriptData() calls JS_DestroyContext to free the context,
but does nothing to global because the JS::PersistentRooted is
documented to have "automatic unrooting on destruction."

But apparently, something is NOT getting freed, because after many calls
to JS_NewGlobalObject, the call fails.  So memory is being leaked.

No memory problems reported by running the program with valgrind, so I
have to assume that memory is still pointed-to in the JSRuntime, or
something else has gone wrong.

What's the right way to contain a global object in a C++ object.

Note that my ScriptData object isn't owned by a JS Object -- it
encapsulates the data needed to evaluate JS Scripts.

_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: BUMP: Why would JS_NewGlobalObject fail

Terrence Cole-3
It's impossible to tell without more details.

On Mon, Nov 16, 2015 at 1:30 PM, Kent Williams <[hidden email]>
wrote:

> Simpler question.
>
> My program creates one global JSRuntime object.
>
> To evaluate a script, a C++ ScriptData object is constructed which in turn
> creates a new context with JS_NewContext, and a global object
> JS_NewGlobalObject, declared thusly:
>
> JSContext *context;
> JS::PersistentRooted<JSObject *> global;
>
> These are member variables of the C++ ScriptData object.
>
> ScriptData::~ScriptData() calls JS_DestroyContext to free the context, but
> does nothing to global because the JS::PersistentRooted is documented to
> have "automatic unrooting on destruction."
>
>
It will also get unrooted by JS_DestroyContext. Assuming, of course, that
you rooted it on the context you are destroying. JS_DestroyContext also
runs an all-zones GC, so this is unlikely to be the problem.


> But apparently, something is NOT getting freed, because after many calls
> to JS_NewGlobalObject, the call fails.  So memory is being leaked.
>

How do you know that? Is it RSS that is growing or VSS?


> No memory problems reported by running the program with valgrind, so I
> have to assume that memory is still pointed-to in the JSRuntime, or
> something else has gone wrong.
>

With a system as complicated as a GC, assuming is generally a pretty bad
way to get to the bottom of problems, at least in my experience. You need
to find the edge that is causing the global to be marked and work
backwards. Note that the global is just a normal GC thing and it is
explicitly reachable via *any* live object via the prototype chain. Thus,
*all* objects belonging to the global must be totally unreachable for the
global to be collected.


>
> What's the right way to contain a global object in a C++ object.
>

> Note that my ScriptData object isn't owned by a JS Object -- it
> encapsulates the data needed to evaluate JS Scripts.
>

> _______________________________________________
> dev-tech-js-engine mailing list
> [hidden email]
> https://lists.mozilla.org/listinfo/dev-tech-js-engine
>
_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: BUMP: Why would JS_NewGlobalObject fail

Kent Williams
Here's my cut-down test. It gets to 226 iterations before
JS_NewGlobalObject fails.

#include <iostream>
#include <js/CharacterEncoding.h>
#include <jswrapper.h>
#include <js/Conversions.h>
#include <jsapi.h>

JSClass global_class = {
     "global", JSCLASS_GLOBAL_FLAGS,
     JS_PropertyStub,
};

int main(int argc, char **argv) {
     JS_Init();
     JSRuntime *runtime = JS_NewRuntime((unsigned int)(32L * 1024L *
1024L));

     for(unsigned long i=0;true;++i) {
         JSContext *context = JS_NewContext(runtime, 8192);
         JS::PersistentRooted<JSObject *> global;
         global = JS_NewGlobalObject(context,
                                              &global_class,
                                              nullptr,
  JS::DontFireOnNewGlobalHook);
         if(!global) {
             std::cerr << "Can't create global:" << i << " iterations"
<< std::endl;
             break;
         }
         JS_EnterCompartment( context, global );
         JS_InitStandardClasses(context, global );

         JS_ClearPendingException(context);
         JS::CompileOptions jsCompileOptions(context);
         jsCompileOptions.setUTF8(true);
         JS::Rooted< JSScript * > code(context);

         char *script = "'hello' + 'world, it is '+new Date()";
         if(JS_CompileScript(context, global, script, strlen(script),
                                   jsCompileOptions, &code)) {

             JS::RootedValue result(context);
             if(JS_ExecuteScript(context, global, code, &result) &&
!result.isUndefined()) {
                 JS::RootedString resultS(context, result.toString());
                 std::cerr << "result: " <<
JS_EncodeStringToUTF8(context, resultS) << std::endl;
             } else
                 std::cerr << "execute script failed" << std::endl;
         }
         JS_DestroyContext(context);
     }
     exit(0);
}


On 11/16/2015 04:59 PM, Terrence Cole wrote:

> It's impossible to tell without more details.
>
> On Mon, Nov 16, 2015 at 1:30 PM, Kent Williams <[hidden email]>
> wrote:
>
>> Simpler question.
>>
>> My program creates one global JSRuntime object.
>>
>> To evaluate a script, a C++ ScriptData object is constructed which in turn
>> creates a new context with JS_NewContext, and a global object
>> JS_NewGlobalObject, declared thusly:
>>
>> JSContext *context;
>> JS::PersistentRooted<JSObject *> global;
>>
>> These are member variables of the C++ ScriptData object.
>>
>> ScriptData::~ScriptData() calls JS_DestroyContext to free the context, but
>> does nothing to global because the JS::PersistentRooted is documented to
>> have "automatic unrooting on destruction."
>>
>>
> It will also get unrooted by JS_DestroyContext. Assuming, of course, that
> you rooted it on the context you are destroying. JS_DestroyContext also
> runs an all-zones GC, so this is unlikely to be the problem.
>
>
>> But apparently, something is NOT getting freed, because after many calls
>> to JS_NewGlobalObject, the call fails.  So memory is being leaked.
>>
> How do you know that? Is it RSS that is growing or VSS?
>
>
>> No memory problems reported by running the program with valgrind, so I
>> have to assume that memory is still pointed-to in the JSRuntime, or
>> something else has gone wrong.
>>
> With a system as complicated as a GC, assuming is generally a pretty bad
> way to get to the bottom of problems, at least in my experience. You need
> to find the edge that is causing the global to be marked and work
> backwards. Note that the global is just a normal GC thing and it is
> explicitly reachable via *any* live object via the prototype chain. Thus,
> *all* objects belonging to the global must be totally unreachable for the
> global to be collected.
>
>
>> What's the right way to contain a global object in a C++ object.
>>
>> Note that my ScriptData object isn't owned by a JS Object -- it
>> encapsulates the data needed to evaluate JS Scripts.
>>
>> _______________________________________________
>> dev-tech-js-engine mailing list
>> [hidden email]
>> https://lists.mozilla.org/listinfo/dev-tech-js-engine
>>
> _______________________________________________
> dev-tech-js-engine mailing list
> [hidden email]
> https://lists.mozilla.org/listinfo/dev-tech-js-engine

--

*Kent Williams*| C++ Developer
*CourseLeaf from Leepfrog Technologies *

/“The Process of Academic Change”
/319-337-3877 | courseleaf.com <http://www.courseleaf.com/>

/This message contains confidential information and is intended only for
the individual named. If you are not the intended recipient of this
transmission or a person responsible for delivering it to the intended
recipient, any disclosure, copying, distribution, or other use of any of
the information in this transmission is strictly prohibited. Please
notify the sender immediately by e-mail if you have received this e-mail
by mistake and delete this e-mail from your system./

_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: BUMP: Why would JS_NewGlobalObject fail

Terrence Cole-3
The persistent rooted does not appear to be initialized, so I don't think
it's even holding global live. So I'd guess that's not your problem at all.
Besides, I have literally thousands of globals alive in this session and am
not using more than a GiB of RAM.

Ms2ger suggested in IRC that maybe it's because you are using the default
32MiB heap size limit, but that seems low. Maybe worth checking. If you
insert a call to JS_GC(runtime) at the bottom of the loop, does it get
further?

If you are not doing so already, please be sure you are running against an
--enable-debug build of SpiderMonkey; that will find tons of problems for
you.

That said, there's only so much someone can do over email with
guess-and-check -- at some point you're just going to have to just hook a
debugger up it and start working backwards from the proximate cause.

On Mon, Nov 16, 2015 at 3:40 PM, Kent Williams <[hidden email]>
wrote:

> Here's my cut-down test. It gets to 226 iterations before
> JS_NewGlobalObject fails.
>
> #include <iostream>
> #include <js/CharacterEncoding.h>
> #include <jswrapper.h>
> #include <js/Conversions.h>
> #include <jsapi.h>
>
> JSClass global_class = {
>     "global", JSCLASS_GLOBAL_FLAGS,
>     JS_PropertyStub,
> };
>
> int main(int argc, char **argv) {
>     JS_Init();
>     JSRuntime *runtime = JS_NewRuntime((unsigned int)(32L * 1024L *
> 1024L));
>
>     for(unsigned long i=0;true;++i) {
>         JSContext *context = JS_NewContext(runtime, 8192);
>         JS::PersistentRooted<JSObject *> global;
>         global = JS_NewGlobalObject(context,
>                                              &global_class,
>                                              nullptr,
>  JS::DontFireOnNewGlobalHook);
>         if(!global) {
>             std::cerr << "Can't create global:" << i << " iterations" <<
> std::endl;
>             break;
>         }
>         JS_EnterCompartment( context, global );
>         JS_InitStandardClasses(context, global );
>
>         JS_ClearPendingException(context);
>         JS::CompileOptions jsCompileOptions(context);
>         jsCompileOptions.setUTF8(true);
>         JS::Rooted< JSScript * > code(context);
>
>         char *script = "'hello' + 'world, it is '+new Date()";
>         if(JS_CompileScript(context, global, script, strlen(script),
>                                   jsCompileOptions, &code)) {
>
>             JS::RootedValue result(context);
>             if(JS_ExecuteScript(context, global, code, &result) &&
> !result.isUndefined()) {
>                 JS::RootedString resultS(context, result.toString());
>                 std::cerr << "result: " << JS_EncodeStringToUTF8(context,
> resultS) << std::endl;
>             } else
>                 std::cerr << "execute script failed" << std::endl;
>         }
>         JS_DestroyContext(context);
>     }
>     exit(0);
>
> }
>
>
> On 11/16/2015 04:59 PM, Terrence Cole wrote:
>
>> It's impossible to tell without more details.
>>
>> On Mon, Nov 16, 2015 at 1:30 PM, Kent Williams <[hidden email]>
>> wrote:
>>
>> Simpler question.
>>>
>>> My program creates one global JSRuntime object.
>>>
>>> To evaluate a script, a C++ ScriptData object is constructed which in
>>> turn
>>> creates a new context with JS_NewContext, and a global object
>>> JS_NewGlobalObject, declared thusly:
>>>
>>> JSContext *context;
>>> JS::PersistentRooted<JSObject *> global;
>>>
>>> These are member variables of the C++ ScriptData object.
>>>
>>> ScriptData::~ScriptData() calls JS_DestroyContext to free the context,
>>> but
>>> does nothing to global because the JS::PersistentRooted is documented to
>>> have "automatic unrooting on destruction."
>>>
>>>
>>> It will also get unrooted by JS_DestroyContext. Assuming, of course, that
>> you rooted it on the context you are destroying. JS_DestroyContext also
>> runs an all-zones GC, so this is unlikely to be the problem.
>>
>>
>> But apparently, something is NOT getting freed, because after many calls
>>> to JS_NewGlobalObject, the call fails.  So memory is being leaked.
>>>
>>> How do you know that? Is it RSS that is growing or VSS?
>>
>>
>> No memory problems reported by running the program with valgrind, so I
>>> have to assume that memory is still pointed-to in the JSRuntime, or
>>> something else has gone wrong.
>>>
>>> With a system as complicated as a GC, assuming is generally a pretty bad
>> way to get to the bottom of problems, at least in my experience. You need
>> to find the edge that is causing the global to be marked and work
>> backwards. Note that the global is just a normal GC thing and it is
>> explicitly reachable via *any* live object via the prototype chain. Thus,
>> *all* objects belonging to the global must be totally unreachable for the
>> global to be collected.
>>
>>
>> What's the right way to contain a global object in a C++ object.
>>>
>>> Note that my ScriptData object isn't owned by a JS Object -- it
>>> encapsulates the data needed to evaluate JS Scripts.
>>>
>>> _______________________________________________
>>> dev-tech-js-engine mailing list
>>> [hidden email]
>>> https://lists.mozilla.org/listinfo/dev-tech-js-engine
>>>
>>> _______________________________________________
>> dev-tech-js-engine mailing list
>> [hidden email]
>> https://lists.mozilla.org/listinfo/dev-tech-js-engine
>>
>
> --
>
> *Kent Williams*| C++ Developer
> *CourseLeaf from Leepfrog Technologies *
>
> /“The Process of Academic Change”
> /319-337-3877 | courseleaf.com <http://www.courseleaf.com/>
>
> /This message contains confidential information and is intended only for
> the individual named. If you are not the intended recipient of this
> transmission or a person responsible for delivering it to the intended
> recipient, any disclosure, copying, distribution, or other use of any of
> the information in this transmission is strictly prohibited. Please notify
> the sender immediately by e-mail if you have received this e-mail by
> mistake and delete this e-mail from your system./
>
>
> _______________________________________________
> dev-tech-js-engine mailing list
> [hidden email]
> https://lists.mozilla.org/listinfo/dev-tech-js-engine
>
_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: BUMP: Why would JS_NewGlobalObject fail

Boris Zbarsky
In reply to this post by Kent Williams
On 11/17/15 12:21 PM, Terrence Cole wrote:
> That said, there's only so much someone can do over email with
> guess-and-check -- at some point you're just going to have to just hook a
> debugger up it and start working backwards from the proximate cause.

A good intermediate step, by the way, is to examine the actual exception
on "context" when JS_NewGlobalObject returns false (e.g. by setting up
an error reporter on the runtime, doing JS_ReportPendingException, and
seeing what comes through in the JSErrorReport).  That might tell you
something useful about why JS_NewGlobalObject failed.

-Boris

_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: BUMP: Why would JS_NewGlobalObject fail

Kent Williams
In reply to this post by Terrence Cole-3
I found a solution, which was a blind guess with perhaps some knowledge
behind it. Adding

         JS_LeaveCompartment(context,compartment);

before the JS_DestroyContext at the bottom of the loop makes it so
JS_NewGlobalObject never fails.  Watching this program run with the
'top' command the virtual + Resident memory footprint never changes.

Thanks for giving my problem your attention!

On 11/17/2015 11:21 AM, Terrence Cole wrote:

> The persistent rooted does not appear to be initialized, so I don't
> think it's even holding global live. So I'd guess that's not your
> problem at all. Besides, I have literally thousands of globals alive
> in this session and am not using more than a GiB of RAM.
>
> Ms2ger suggested in IRC that maybe it's because you are using the
> default 32MiB heap size limit, but that seems low. Maybe worth
> checking. If you insert a call to JS_GC(runtime) at the bottom of the
> loop, does it get further?
>
> If you are not doing so already, please be sure you are running
> against an --enable-debug build of SpiderMonkey; that will find tons
> of problems for you.
>
> That said, there's only so much someone can do over email with
> guess-and-check -- at some point you're just going to have to just
> hook a debugger up it and start working backwards from the proximate
> cause.
>
> On Mon, Nov 16, 2015 at 3:40 PM, Kent Williams <[hidden email]
> <mailto:[hidden email]>> wrote:
>
>     Here's my cut-down test. It gets to 226 iterations before
>     JS_NewGlobalObject fails.
>
>     #include <iostream>
>     #include <js/CharacterEncoding.h>
>     #include <jswrapper.h>
>     #include <js/Conversions.h>
>     #include <jsapi.h>
>
>     JSClass global_class = {
>         "global", JSCLASS_GLOBAL_FLAGS,
>         JS_PropertyStub,
>     };
>
>     int main(int argc, char **argv) {
>         JS_Init();
>         JSRuntime *runtime = JS_NewRuntime((unsigned int)(32L * 1024L
>     * 1024L));
>
>         for(unsigned long i=0;true;++i) {
>             JSContext *context = JS_NewContext(runtime, 8192);
>             JS::PersistentRooted<JSObject *> global;
>             global = JS_NewGlobalObject(context,
>      &global_class,
>                                                  nullptr,
>      JS::DontFireOnNewGlobalHook);
>             if(!global) {
>                 std::cerr << "Can't create global:" << i << "
>     iterations" << std::endl;
>                 break;
>             }
>             JS_EnterCompartment( context, global );
>             JS_InitStandardClasses(context, global );
>
>             JS_ClearPendingException(context);
>             JS::CompileOptions jsCompileOptions(context);
>             jsCompileOptions.setUTF8(true);
>             JS::Rooted< JSScript * > code(context);
>
>             char *script = "'hello' + 'world, it is '+new Date()";
>             if(JS_CompileScript(context, global, script, strlen(script),
>                                       jsCompileOptions, &code)) {
>
>                 JS::RootedValue result(context);
>                 if(JS_ExecuteScript(context, global, code, &result) &&
>     !result.isUndefined()) {
>                     JS::RootedString resultS(context, result.toString());
>                     std::cerr << "result: " <<
>     JS_EncodeStringToUTF8(context, resultS) << std::endl;
>                 } else
>                     std::cerr << "execute script failed" << std::endl;
>             }
>             JS_DestroyContext(context);
>         }
>         exit(0);
>
>     }
>
>
>     On 11/16/2015 04:59 PM, Terrence Cole wrote:
>
>         It's impossible to tell without more details.
>
>         On Mon, Nov 16, 2015 at 1:30 PM, Kent Williams
>         <[hidden email] <mailto:[hidden email]>>
>         wrote:
>
>             Simpler question.
>
>             My program creates one global JSRuntime object.
>
>             To evaluate a script, a C++ ScriptData object is
>             constructed which in turn
>             creates a new context with JS_NewContext, and a global object
>             JS_NewGlobalObject, declared thusly:
>
>             JSContext *context;
>             JS::PersistentRooted<JSObject *> global;
>
>             These are member variables of the C++ ScriptData object.
>
>             ScriptData::~ScriptData() calls JS_DestroyContext to free
>             the context, but
>             does nothing to global because the JS::PersistentRooted is
>             documented to
>             have "automatic unrooting on destruction."
>
>
>         It will also get unrooted by JS_DestroyContext. Assuming, of
>         course, that
>         you rooted it on the context you are destroying.
>         JS_DestroyContext also
>         runs an all-zones GC, so this is unlikely to be the problem.
>
>
>             But apparently, something is NOT getting freed, because
>             after many calls
>             to JS_NewGlobalObject, the call fails.  So memory is being
>             leaked.
>
>         How do you know that? Is it RSS that is growing or VSS?
>
>
>             No memory problems reported by running the program with
>             valgrind, so I
>             have to assume that memory is still pointed-to in the
>             JSRuntime, or
>             something else has gone wrong.
>
>         With a system as complicated as a GC, assuming is generally a
>         pretty bad
>         way to get to the bottom of problems, at least in my
>         experience. You need
>         to find the edge that is causing the global to be marked and work
>         backwards. Note that the global is just a normal GC thing and
>         it is
>         explicitly reachable via *any* live object via the prototype
>         chain. Thus,
>         *all* objects belonging to the global must be totally
>         unreachable for the
>         global to be collected.
>
>
>             What's the right way to contain a global object in a C++
>             object.
>
>             Note that my ScriptData object isn't owned by a JS Object
>             -- it
>             encapsulates the data needed to evaluate JS Scripts.
>
>             _______________________________________________
>             dev-tech-js-engine mailing list
>             [hidden email]
>             <mailto:[hidden email]>
>             https://lists.mozilla.org/listinfo/dev-tech-js-engine
>
>         _______________________________________________
>         dev-tech-js-engine mailing list
>         [hidden email]
>         <mailto:[hidden email]>
>         https://lists.mozilla.org/listinfo/dev-tech-js-engine
>
>
>     --
>
>     *Kent Williams*| C++ Developer
>     *CourseLeaf from Leepfrog Technologies *
>
>     /“The Process of Academic Change”
>     /319-337-3877 <tel:319-337-3877> | courseleaf.com
>     <http://courseleaf.com> <http://www.courseleaf.com/>
>
>     /This message contains confidential information and is intended
>     only for the individual named. If you are not the intended
>     recipient of this transmission or a person responsible for
>     delivering it to the intended recipient, any disclosure, copying,
>     distribution, or other use of any of the information in this
>     transmission is strictly prohibited. Please notify the sender
>     immediately by e-mail if you have received this e-mail by mistake
>     and delete this e-mail from your system./
>
>
>     _______________________________________________
>     dev-tech-js-engine mailing list
>     [hidden email]
>     <mailto:[hidden email]>
>     https://lists.mozilla.org/listinfo/dev-tech-js-engine
>
>

--

*Kent Williams*| C++ Developer
*CourseLeaf from Leepfrog Technologies *

/“The Process of Academic Change”
/319-337-3877 | courseleaf.com <http://www.courseleaf.com/>

/This message contains confidential information and is intended only for
the individual named. If you are not the intended recipient of this
transmission or a person responsible for delivering it to the intended
recipient, any disclosure, copying, distribution, or other use of any of
the information in this transmission is strictly prohibited. Please
notify the sender immediately by e-mail if you have received this e-mail
by mistake and delete this e-mail from your system./

_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine