How do I trace objects on the heap with JS 59

classic Classic list List threaded Threaded
13 messages Options
Reply | Threaded
Open this post in threaded view
|

How do I trace objects on the heap with JS 59

Miles Thornton
Hi,

I'm currently trying to update my embedding from a very old version (1.8.5) to 59. I have lots of objects which I used to root using JS_AddObjectRoot.
These are stored on heap structures.
I'm trying to understand how to do the rooting in 59.

In the GC rooting guide on MDN at https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/GC_Rooting_Guide#GC_things_on_the_heap it says:

GC thing pointers on the heap must be wrapped in a JS::Heap<T>. The only exception to this is if they are added as roots with the JS_Add<T>Root() functions or JS::PersistentRooted class, but don't do this unless it's really necessary.  JS::Heap<T> pointers must also continue to be traced in the normal way, which is not covered here.

OK, so I need to use JS::Heap<JSObject *> and these need to be traced but how to do it isn't covered in the guide... Please can someone point me to some documentation/reference which says how to do it. I can't find any information anywhere...

Alternatively it looks like I could use the JS::PersistentRooted class to do what I want but it specifically says not to do this unless it's really necessary. Why is that then? Is it some sort of performance issue?

Many thanks

Miles
_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: How do I trace objects on the heap with JS 59

Boris Zbarsky
On 8/9/18 4:40 AM, Miles wrote:
> Alternatively it looks like I could use the JS::PersistentRooted class to do what I want but it specifically says not to do this unless it's really necessary. Why is that then? Is it some sort of performance issue?

Well, if nothing else it makes it easy to leak.  Say you have two C/C++
data structures X and Y.  X holds on to a PersistentRooted for object A,
object A keeps Y alive, Y holds a PersistentRooted for object B, B keeps
X alive.  Now you have a leak.

As far as how to trace things goes, it depends on what owns the heap
data structures that are pointing to the JS objects.  If they are owned
(possibly indirectly) by some SpiderMonkey object, its trace class hook
should do the tracing.  If they just exist independently, then
JS_AddExtraGCRootsTracer might be the right thing?

-Boris
_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: How do I trace objects on the heap with JS 59

Steve Fink-4
In reply to this post by Miles Thornton
On 08/09/2018 01:40 AM, Miles wrote:

> Hi,
>
> I'm currently trying to update my embedding from a very old version (1.8.5) to 59. I have lots of objects which I used to root using JS_AddObjectRoot.
> These are stored on heap structures.
> I'm trying to understand how to do the rooting in 59.
>
> In the GC rooting guide on MDN at https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/GC_Rooting_Guide#GC_things_on_the_heap it says:
>
> GC thing pointers on the heap must be wrapped in a JS::Heap<T>. The only exception to this is if they are added as roots with the JS_Add<T>Root() functions or JS::PersistentRooted class, but don't do this unless it's really necessary.  JS::Heap<T> pointers must also continue to be traced in the normal way, which is not covered here.
>
> OK, so I need to use JS::Heap<JSObject *> and these need to be traced but how to do it isn't covered in the guide... Please can someone point me to some documentation/reference which says how to do it. I can't find any information anywhere...
>
> Alternatively it looks like I could use the JS::PersistentRooted class to do what I want but it specifically says not to do this unless it's really necessary. Why is that then? Is it some sort of performance issue?

The exact equivalent of JS_Add*Root is JS::PersistentRooted. It is
discouraged because it is ridiculously easy to keep things alive
forever. For example, if the referent of a PersistentRooted field
creates a cycle with its container, then nothing in or reachable from
that cycle will ever be freed up. And since global objects are reachable
from pretty much everything, that means that if your structure is
reachable from any JS object, you'll have a cycle.

You want such fields to be traced, not rooted. That generally means that
you define a trace() method on your structures that calls JS::TraceEdge
on each of your Heap<T> fields. That is enough to put your structure in
a Rooted<yourstruct> on the stack, btw, if you happen to want to.

The exact signature of trace() is

     void trace(JSTracer* trc, const char* name);

Actually, this spurred me to update the MDN documentation to include
tracing:
https://developer.mozilla.org/en-US/docs/Mozilla/Projects/SpiderMonkey/GC_Rooting_Guide#GC_things_on_the_heap
_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: How do I trace objects on the heap with JS 59

James Stortz-2
I'd like to add a question on this topic.

What if I have JSObjects that are already rooted? (Either as global
variables, or properties of rooted objects) And I have references to those
JSObject* pointers in my backend?

Do I need to change those JSObject* pointers to JS::Heap<JSObject*> in that
case? I always assumed that would be the case, but does it affect those
rooted JSObjects? Are they still the same object? If so, how do I convert
Heap<T> back to Rooted<T>?

I am a little confused. Do I need to call some kind of GC tracer API on
construction/deconstruction of my backend objects?

Thanks,
James
_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: How do I trace objects on the heap with JS 59

James Stortz-2
So, here's an actual example to clarify. Please tell me if this simple
concept is correct.

1.) On my backend objects, I set a pointer to a `new FrontEnd(cx,
rooted_obj_js)`
//----------------------------------------------
    void trace_obj(JSTracer* tracer, void* data) {
        JS::TraceEdge(tracer, (JS::Heap<JSObject*>*)data, "obj_js");
    };

    struct FrontEnd {
           FrontEnd(JSContext *cx, JS::HandleObject obj) {
               this->cx = cx;
               this->obj = obj;
               if (this->obj.get())
                   JS_AddExtraGCRootsTracer(this->cx, trace_obj,
&this->obj);
           };
          ~FrontEnd() {
               if (this->obj.get())
                   JS_RemoveExtraGCRootsTracer(this->cx, trace_obj,
&this->obj);
           };

        JSContext *cx;
        JS::Heap<JSObject*> obj;
    };
//----------------------------------------------
2.) back in the frontend (JS), I root that JSObject again like so:
//----------------------------------------------
    JS::RootedObject obj(cx, ((FrontEnd*)backend_obj->frontend)->obj);
//----------------------------------------------


3.) and in the finalizeOp of my frontend object (JS), I delete the FrontEnd
(and BackEnd) wrappers.
//----------------------------------------------
    void obj_finalize(JSFreeOp *fop, JSObject *obj) {
        auto *backend = (BackEnd*)JS_GetPrivate(obj);
        auto *frontend = (FrontEnd*)backend->obj->frontend;
        delete frontend;
        delete backend;
    };
//----------------------------------------------


Am I using the GC/tracer API correctly? I don't want to derive from
JSObject, just link the frontend and backend with wrappers. Want to know if
`JS_AddExtraGCRootsTracer`,`JS_RemmoveExtraGCRootsTracer`, and
`JS::TraceEdge` are used appropriately for this.

Hopefully this will help Miles too.

Thanks,
James
_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: How do I trace objects on the heap with JS 59

Miles Thornton
In reply to this post by Miles Thornton
Thanks for the replies and for updating the documentation to include tracing. Much appreciated. It sounds like JS_AddExtraGCRootsTracer, JS::Heap<JSObject *> and JS::TraceEdge are what I need then.

One final question if I may...
GS thing pointers that are parameters to functions now use JS::Handle<T>. i.e. JS::Handle<JSObject *> (or JS::HandleObject) for objects.
If I use JS::Heap<JSObject *> can these be automatically coerced into JS::Handle<JSObject *> for passing to functions or do I have to do something else to be able to pass them as function parameters?

Thanks

Miles
_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: How do I trace objects on the heap with JS 59

Boris Zbarsky
On 8/13/18 7:34 AM, Miles wrote:
> If I use JS::Heap<JSObject *> can these be automatically coerced into JS::Handle<JSObject *> for passing to functions or do I have to do something else to be able to pass them as function parameters?

You generally need to put them in a Rooted on the stack.

The reason for that is that Handle<JSObject*> is really a JSObject**
under the hood.  Heap<JSObject*> stores a JSObject*.  If we allowed
creation of a Handle from it, that Handle would point inside the Heap.
Then if the code you are calling triggers some JS that changes the value
of your member (updating the JSObject* value stored), the handle would
suddenly be pointing to a different object, which is pretty confusing.

-Boris
_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: How do I trace objects on the heap with JS 59

Boris Zbarsky
In reply to this post by James Stortz-2
On 8/9/18 11:44 PM, James Stortz wrote:
> What if I have JSObjects that are already rooted? (Either as global
> variables, or properties of rooted objects) And I have references to those
> JSObject* pointers in my backend?

You must not store JSObject* directly, because that doesn't play nice
with the GC moving objects.

If you're storing JSObject** pointing into a rooted thing, that might be
OK as long as you don't write to it, I think.  So a "JSObject* const*",
basically.

> Do I need to change those JSObject* pointers to JS::Heap<JSObject*> in that
> case? I always assumed that would be the case, but does it affect those
> rooted JSObjects? Are they still the same object? If so, how do I convert
> Heap<T> back to Rooted<T>?

I'm not quite sure what you're asking here... :(

-Boris
_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: How do I trace objects on the heap with JS 59

Boris Zbarsky
In reply to this post by James Stortz-2
On 8/10/18 5:34 AM, James Stortz wrote:
> So, here's an actual example to clarify. Please tell me if this simple
> concept is correct.

What you have here will keep the thing stored in the "obj" member of a
FrontEnd struct alive as long as that FrontEnd struct is alive, I
believe.  That's assuming there is no API to mutate the value of that
member other than the FrontEnd constructor.

Given that, deleting the FrontEnd in the object's finalizer doesn't
really make sense: the FrontEnd will keep the object alive as long as
the FrontEnd is alive, as far as I can tell.

What is the desired lifetime relationship here between the FrontEnd, the
BackEnd, and the JSObject that the FrontEnd points to?

-Boris
_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: How do I trace objects on the heap with JS 59

James Stortz-2
In reply to this post by Miles Thornton
Miles, well I'm not entirely sure my example is 100% correct. I was hoping
to get some answers as well. (I also made a typo with `auto *` D:)

I went digging through some Mozilla/JSAPI code on the DXR to see how it is
used, and put that together with some info from StackOverflow.

I think I can answer your question about Heap/Handle. I believe Handles are
always for Rooted Objects, which are on the stack. Heap and Stack Objects
are handled differently, so you would have to root it before use.

Again, I'm trying to learn these things myself. These are good questions.
_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: How do I trace objects on the heap with JS 59

James Stortz-2
In reply to this post by Boris Zbarsky
On Tue, Aug 14, 2018 at 2:19 PM, Boris Zbarsky <[hidden email]> wrote:

>
> Given that, deleting the FrontEnd in the object's finalizer doesn't really
> make sense: the FrontEnd will keep the object alive as long as the FrontEnd
> is alive, as far as I can tell.
>
> What is the desired lifetime relationship here between the FrontEnd, the
> BackEnd, and the JSObject that the FrontEnd points to?
>
>
Ah, thank you for catching that! Ok, so the JSObject has a C++ Object that
it mirrors, as a high-level interface. The desired relationship is that C++
or JS can work on their respective counterpart and have changes reflected
accordingly. (FrontEnd and BackEnd are simply wrappers that contain
pointers to them.)

The problem is when the JS Object goes out of scope, it never really goes
out of scope! They all get GC'd at Context Destruction. These JSObjects are
created in my JS API with `new`, so in C++ that would be normal to keep
them alive until `delete`, but in JS, we don't want this!

I can't think of any elegant solutions off the top of my head. I don't
think storing raw JSObject* pointers in the C++ is viable. Neither is
reworking away the C++ mirror entirely. It would be great if there was a
distinction between Heap/Rooted objects that I could test for during some
GC trace/hook, but there's not a better place to delete the Heap<JSObject*>
wrapper, is there? Is there a way I can check for when a JSObject
"would've" gone out of scope? (such as: upon RootedObject normally going
out of scope)

If anybody knows of a better/common methodology, I would gladly take some
advice. I don't want to impose otherwise, I'm sure I can figure something
out. (Seems like, at least in my program, I can do some kind of a check in
a GC trace somewhere, to see if the JSObject was actually used by my API.)

Again, I really appreciate the replies, everyone! Thanks for catching that,
Boris.
Thanks,
James
_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: How do I trace objects on the heap with JS 59

Miles Thornton
In reply to this post by Miles Thornton
Please can I clarify how tracing interacts with garbage collection?

Hi, thanks for all the help and patience so far but I'm struggling with this somewhat as I don't really understand how tracing works. There is very little information on it in the JSAPI reference. I would be really grateful if someone could give some more clarification so I can try to understand how to use it properly.

In my embedding that currently uses JS 1.8.5 I have some classes defined using JSClass (e.g. I have Widget, WidgetItem and Window classes). Each of these has a constructor and this stores a structure in the private data field of the object using JS_SetPrivate. I define a JSfinalizeOp for the class where I then return the structure for this memory.

In my embedding this structure stores the pointer back to the JSObject. For example the structure I store for my 'Widget' class looks something like

typedef struct db_js_widget_struct {

    JSObject         *jsobject;      /* The Widget object in javascript */
    JSContext        *context;       /* The context in javascript */
    int               id;            /* The id of the widget */
    int               type;          /* The type of widget */
    int               bg_col;        /* The background colour */
...
} DB_JS_WIDGET;

This enables me to get back to the object.

When using JS 1.8.5 normally when the object goes out of scope the finalizeOp will be called when GC is done and my structure will be returned.

However, in some special cases I wanted to ensure that the object was not garbage collected so I rooted the object using JS_AddObjectRoot.
When I had finished with the object I could then call JS_RemoveObjectRoot and then when GC was done the object would not be rooted anymore and would be freed.

So I had two different use cases. The 'normal' case where I wanted the object to be garbage collected when it went out of scope and the 'special' case where I wanted to protect the object from being garbage collected by rooting. However in both cases I wanted to store the (JSObject *) pointer.

Now moving to JS 59 from my understanding I cannot store the (JSObject *) pointer directly on the structure as GC is now done with a moving GC so (JSObject *) pointers can 'move'. Correct?
So I have two options.
1. use JS::PersistentRootedObject
2. use JS::Heap<JSObject *> and trace it.

If I use JS::PersistentRootedObject then I think that my object will *always* be rooted so will *never* go 'out of scope' and be garbage collected. i.e. I cannot use this for my 'normal' use case.

So I think I have to use JS::Heap<JSObject *> and trace it.
My structure now becomes

typedef struct db_js_widget_struct {

    JS::Heap<JSObject *> jsobject;      /* The Widget object in javascript */
    JSContext        *context;       /* The context in javascript */
    int               id;            /* The id of the widget */
    int               type;          /* The type of widget */
    int               bg_col;        /* The background colour */
...
} DB_JS_WIDGET;

and on my JSClass I now define a traceOp which looks something like

   static void dj_widget_trace(JSTracer *trc, JSObject *obj)
/* ===========================
 *
 * Tracing for JS::Heap<T> GC things for Widget
 */
{
    DB_JS_WIDGET     *js_widget;

/* Get the private data for the widget and return it. */
    js_widget = (DB_JS_WIDGET *) JS_GetPrivate(obj);

    JS::TraceEdge(trc, &js_widget->jsobject, "widget");
}

This is where I don't understand the process.
What does calling JS::TraceEdge do? Does it
1. Ensure that jsobject is kept up to date to refer to the correct (JSObject *) pointer
or
2. Root the object to stop GC on it.

Or perhaps it is both?

Basically, how can I do my two 'normal' and 'special' use cases using tracing? Can I somehow store the JSObject pointer in a structure but still allow it to go out of scope as normal for my 'normal' case and also prevent GC in some circumstances for my 'special' case. Is it possible?

I *think* this is the same question that James is asking...

I hope this makes sense. Apologies if I haven't explained it clearly.
Many thanks in advance for any information someone might be able to give.

Miles
_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine
Reply | Threaded
Open this post in threaded view
|

Re: How do I trace objects on the heap with JS 59

James Stortz-2
On Thu, Aug 16, 2018 at 10:33 AM, Miles <[hidden email]> wrote:

> What does calling JS::TraceEdge do? Does it
> 1. Ensure that jsobject is kept up to date to refer to the correct
> (JSObject *) pointer
> or
> 2. Root the object to stop GC on it.
>
> Or perhaps it is both?
>
> Basically, how can I do my two 'normal' and 'special' use cases using
> tracing? Can I somehow store the JSObject pointer in a structure but still
> allow it to go out of scope as normal for my 'normal' case and also prevent
> GC in some circumstances for my 'special' case. Is it possible?
>
> I *think* this is the same question that James is asking...
>


Miles, did anybody help answer this? (Sometimes I get these emails delayed,
but I thought I'd try to help in some small way, since I was able to get
mine working, as far as I can tell.)

Yes, I have the same questions, and I believe our understanding is correct.
My advice is this: "You have to know when it is ok to keep the [heap]
object alive." For example, say if you had an event/request object, and
wanted to keep it alive so the backend can do work, and then return control
to the JS. Upon returning control to JS, you would be ok to free it from
the heap/destroy private, as once the object is rooted again [on the stack]
it would be treated as per usual, and able to GC.

If your special case is as straight-forward as that, then that'll work. If
not, you should try to work it out so you can know exactly when it needs to
be alive. My case was more complicated, but I was able to do that, and I
tested by calling `JS_GC(cx)` in the middle of my program, and it did
indeed GC those objects.

I like how you are using `TraceOp` on your `JSClass`, but unfortunately I
don't know how that works. I think it just adds/removes the tracer
automatically, whereas I used
`JS_AddExtraGCRootsTracer`/`JS_RemoveExtraRootsTracer`, as well as a
`FinalizeOp`. That means, elsewhere in my program, I would NULL the
`JS::Heap<>` and remove it's tracer, and that allowed FinalizeOp to delete
the private, and complete GC as per usual.

I think your approach would be cleaner, but similar using `TraceOp` ...?

Good luck and Regards,
James
_______________________________________________
dev-tech-js-engine mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-js-engine