Quantcast

Include out of viewport frames in display list

classic Classic list List threaded Threaded
27 messages Options
12
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Include out of viewport frames in display list

jip.de.beer
Hi,

I'm interested in the display-list dumps. From this dump I can accurately visualize the page structure because the frames are sorted by z-order and it contains data about frame position and dimension.

However, the display-list only contains the frames within the browser viewport. I'm trying to force Firefox to also include out of viewport frames. I'm essentially trying to force Firefox to render the whole page, all the time. Not just the parts visible in the viewport.

It's a bit like what CanvasRenderingContext2D::DrawWindow() is capable of under the hood. https://dxr.mozilla.org/mozilla-central/source/dom/canvas/CanvasRenderingContext2D.cpp#4883 This function can render also parts of the page beyond the viewport. I'm trying to make this the default behavior.

I've tried increasing the dimensions of aDirtyRegion when calling nsLayoutUtils::PaintFrame() from nsPresShell.cpp. https://dxr.mozilla.org/mozilla-central/source/layout/base/nsPresShell.cpp#6351 That didn't seem to make a difference.

I also think nsIFrame::BuildDisplayListForStackingContext() could be related somehow: https://dxr.mozilla.org/mozilla-central/source/layout/generic/nsFrame.cpp#2143

I'm sure it's possible and probably very easy if I'd know where to look.

Thanks,
Jip
_______________________________________________
dev-tech-layout mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-layout
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Include out of viewport frames in display list

Botond Ballo
On Wednesday, April 20, 2016 at 5:44:14 AM UTC-4, [hidden email] wrote:
> I've tried increasing the dimensions of aDirtyRegion when calling nsLayoutUtils::PaintFrame() from nsPresShell.cpp. https://dxr.mozilla.org/mozilla-central/source/layout/base/nsPresShell.cpp#6351 That didn't seem to make a difference.

I haven't tested this, but I'll try to point you in the right direction.

The place to start looking is probably this line in nsGfxScrollFrame::BuildDisplayList() [1] where we restrict the dirty rect (which is essentially the rect that needs to be repainted) to the scroll frame's "scroll port". For the page's root scroll frame, that's the viewport.

You may want to condition any modification you make to this line, to the scroll frame being the page's root scroll frame. The condition to test this is something like |mIsRoot && mOuter->PresContext()->IsRootContentDocument()|. (|mIsRoot| is true for the root scroll frame of any document, including an iframe. |IsRootContentDocument()| is only true for the top-level page you're viewing).

If you're using APZ (async panning/zooming; it's the default in e10s mode on 46 and newer), you probably also want to adjust this line [2] which overwrites the dirty rect to be the "display port", which is an area larger than the scroll port that we pre-render so async panning can bring it into view without repainting, but still not (necessarily) as large as the entire document.

There may also be other places that need to be adjusted that I'm not aware of or not thinking of right now.

This probably goes without saying, but rendering the entire document will kill performance for large documents, so it's suitable for debugging only (I assume that's what you have in mind).

Hope that helps!

Cheers,
Botond

[1] https://dxr.mozilla.org/mozilla-central/rev/ae7413abfa4d3954a6a4ce7c1613a7100f367f9a/layout/generic/nsGfxScrollFrame.cpp#3145
[2] https://dxr.mozilla.org/mozilla-central/rev/ae7413abfa4d3954a6a4ce7c1613a7100f367f9a/layout/generic/nsGfxScrollFrame.cpp#3512
_______________________________________________
dev-tech-layout mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-layout
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Include out of viewport frames in display list

jip.de.beer
Hi Botond,

Thanks a lot for your reply. After a bit of testing I agree with you that rendering the entire document is way too expensive. I didn't realize how expensive. But taking a full page screenshot from the Developer Tools can take several seconds (per screenshot).

I've started implementing my own function, which I based on elementsFromPoint(), what I've learned from CanvasRenderingContext2D.drawWindow() and all the help in the Google Groups.

I noticed that parsing the display-lists is not trivial: https://dxr.mozilla.org/mozilla-central/source/gfx/layers/layerviewer/layerTreeView.js and https://github.com/mozilla/layerscope. Especially for the data I'm looking for (absolute position of DOM nodes in the document and z-order).

My function currently works like document.elementsFromPoint() but returns all DOM nodes in the document (regardless of scroll position) ordered by z-order. It also dumps the absolute position for all DOM nodes to stderr so I can capture and parse this.

I've pasted the code here: http://pastebin.com/FKEC7Ejg

I'm still stuck with a few things. Just like elementsFromPoint() my function currently ignores nodes with pointer-events:none; or visibility:hidden; (and maybe more). I only want to ignore nodes with display:none; Where can I override this?

One thing that comes to mind is looping over all nodes first, then unset pointer-events:none; backup the original setting, then querying nsLayoutUtils::GetFramesForArea() (now it includes also the nodes which had pointer-events:none;) and then re-setting the backed up settings. But this sounds like a bad idea and will probably cause other problems.

While working on this I noticed a bug with elementsFromPoint(). I reported it here: https://bugzilla.mozilla.org/show_bug.cgi?id=1266932 Since my function is based on elementsFromPoint(), this currently also effects my function. I tried to fix it but did it the wrong way (and in the wrong place).

Would be great if you have some more tips. It's the first time I look at C++ so I'm already quite happy I've been able to implement a function in Firefox for my personal convenience.

On Wednesday, 20 April 2016 19:29:03 UTC+2, [hidden email]  wrote:

> On Wednesday, April 20, 2016 at 5:44:14 AM UTC-4, [hidden email] wrote:
> > I've tried increasing the dimensions of aDirtyRegion when calling nsLayoutUtils::PaintFrame() from nsPresShell.cpp. https://dxr.mozilla.org/mozilla-central/source/layout/base/nsPresShell.cpp#6351 That didn't seem to make a difference.
>
> I haven't tested this, but I'll try to point you in the right direction.
>
> The place to start looking is probably this line in nsGfxScrollFrame::BuildDisplayList() [1] where we restrict the dirty rect (which is essentially the rect that needs to be repainted) to the scroll frame's "scroll port". For the page's root scroll frame, that's the viewport.
>
> You may want to condition any modification you make to this line, to the scroll frame being the page's root scroll frame. The condition to test this is something like |mIsRoot && mOuter->PresContext()->IsRootContentDocument()|. (|mIsRoot| is true for the root scroll frame of any document, including an iframe. |IsRootContentDocument()| is only true for the top-level page you're viewing).
>
> If you're using APZ (async panning/zooming; it's the default in e10s mode on 46 and newer), you probably also want to adjust this line [2] which overwrites the dirty rect to be the "display port", which is an area larger than the scroll port that we pre-render so async panning can bring it into view without repainting, but still not (necessarily) as large as the entire document.
>
> There may also be other places that need to be adjusted that I'm not aware of or not thinking of right now.
>
> This probably goes without saying, but rendering the entire document will kill performance for large documents, so it's suitable for debugging only (I assume that's what you have in mind).
>
> Hope that helps!
>
> Cheers,
> Botond
>
> [1] https://dxr.mozilla.org/mozilla-central/rev/ae7413abfa4d3954a6a4ce7c1613a7100f367f9a/layout/generic/nsGfxScrollFrame.cpp#3145
> [2] https://dxr.mozilla.org/mozilla-central/rev/ae7413abfa4d3954a6a4ce7c1613a7100f367f9a/layout/generic/nsGfxScrollFrame.cpp#3512
_______________________________________________
dev-tech-layout mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-layout
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Include out of viewport frames in display list

Botond Ballo
Sorry for the delayed response! I'm not actually subscribed to this list, and I thought replies would be automatically cc'd to me.

On Sunday, April 24, 2016 at 6:12:31 AM UTC-4, [hidden email] wrote:
> I'm still stuck with a few things. Just like elementsFromPoint() my function currently ignores nodes with pointer-events:none; or visibility:hidden; (and maybe more). I only want to ignore nodes with display:none; Where can I override this?

It looks like visibility:none is checked here [1] and pointer-events:none is checked here [2].

Both of those functions have an nsDisplayListBuilder* argument, so you could do something like this:

  - add a flag to nsDisplayListBuilder, whether it should ignore
    visibility:none and pointer-events:none
  - condition the above checks on the flag not being set
  - add a corresponding option to GetFramesForArea()
  - when GetFramesForArea() constructs the nsDisplayListBuilder,
    set the flag is the option is set
  - for your call to GetFramesForArea(), set the option

Hope that made sense.

If you give some more details about your end goal, we may be able to suggest solutions that are easier / involve writing less custom code.

Cheers,
Botond

[1] https://dxr.mozilla.org/mozilla-central/rev/1461a4071341c282afcf7b72e33036412d2251d4/layout/generic/nsFrame.cpp#6295
[2] https://dxr.mozilla.org/mozilla-central/rev/1461a4071341c282afcf7b72e33036412d2251d4/layout/base/nsDisplayList.cpp#2061
_______________________________________________
dev-tech-layout mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-layout
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Include out of viewport frames in display list

jip.de.beer
Hi,

Thanks a lot for replying again. I'll look into it more but it already makes sense!

Does it means I also have to modify nsDisplayList.h in order to add the flag to nsDisplayListBuilder? I'm still a bit unsure how to add a flag to nsDisplayListBuilder. I think I need to implement my own function to get and set a custom parameter, right? I can then check for this parameter from the two parts you referenced. And I can set the parameter from GetFramesForArea() which constructs the nsDisplayListBuilder.

In my brief testing I noticed that overriding the pointer-events check in nsDisplayList::HitTest() allows me to also retrieve elements with pointer-events:none. Though strangely no iframes with pointer-events:none. Those are still left out.

I made this screen recording to give an idea of my end goal: https://youtu.be/EtH1MMDorKA

I basically want to visualize the position of all visible elements on a webpage, with all CSS rules applied and in the right visual order (z-order). In real time. The only elements I want to leave out are elements with display:none.

As you can see in the video I still have some issues with elements within scrolling elements...

On Monday, 2 May 2016 18:50:28 UTC+2, [hidden email]  wrote:

> Sorry for the delayed response! I'm not actually subscribed to this list, and I thought replies would be automatically cc'd to me.
>
> On Sunday, April 24, 2016 at 6:12:31 AM UTC-4, [hidden email] wrote:
> > I'm still stuck with a few things. Just like elementsFromPoint() my function currently ignores nodes with pointer-events:none; or visibility:hidden; (and maybe more). I only want to ignore nodes with display:none; Where can I override this?
>
> It looks like visibility:none is checked here [1] and pointer-events:none is checked here [2].
>
> Both of those functions have an nsDisplayListBuilder* argument, so you could do something like this:
>
>   - add a flag to nsDisplayListBuilder, whether it should ignore
>     visibility:none and pointer-events:none
>   - condition the above checks on the flag not being set
>   - add a corresponding option to GetFramesForArea()
>   - when GetFramesForArea() constructs the nsDisplayListBuilder,
>     set the flag is the option is set
>   - for your call to GetFramesForArea(), set the option
>
> Hope that made sense.
>
> If you give some more details about your end goal, we may be able to suggest solutions that are easier / involve writing less custom code.
>
> Cheers,
> Botond
>
> [1] https://dxr.mozilla.org/mozilla-central/rev/1461a4071341c282afcf7b72e33036412d2251d4/layout/generic/nsFrame.cpp#6295
> [2] https://dxr.mozilla.org/mozilla-central/rev/1461a4071341c282afcf7b72e33036412d2251d4/layout/base/nsDisplayList.cpp#2061
_______________________________________________
dev-tech-layout mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-layout
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Include out of viewport frames in display list

Botond Ballo
On Monday, May 2, 2016 at 4:55:13 PM UTC-4, [hidden email] wrote:
> Does it means I also have to modify nsDisplayList.h in order to add the flag to nsDisplayListBuilder? I'm still a bit unsure how to add a flag to nsDisplayListBuilder.

Just add a new member variable, like mIgnoreVisibility (and a getter/setter for it, if it's not public).

> In my brief testing I noticed that overriding the pointer-events check in nsDisplayList::HitTest() allows me to also retrieve elements with pointer-events:none. Though strangely no iframes with pointer-events:none. Those are still left out.

Ah, there is a separate check for iframes here [1]!

> I made this screen recording to give an idea of my end goal: https://youtu.be/EtH1MMDorKA

The video is private, I can't access it.

Cheers,
Botond

[1] https://dxr.mozilla.org/mozilla-central/rev/369a5ee3a2880a4a98df3a00bf3db8d8f36b181b/layout/generic/nsSubDocumentFrame.cpp#379
_______________________________________________
dev-tech-layout mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-layout
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Include out of viewport frames in display list

jip.de.beer
Thanks again for replying!

I was able to get everything working with your instructions :)
I also applied the same method to ignore backface-visibility:hidden;
This check is done here:
https://dxr.mozilla.org/mozilla-central/rev/e5a10bc7dac4ee2453d8319165c1f6578203eac7/layout/base/nsDisplayList.cpp#6018

Hopefully I'm now only excluding elements with display:none and elements which aren't visible because their outside the visible part of a scrollable HTML element.

Another thing I realized I should do was calling GetPrimaryFrame() on the dom::Element* I got from GetFramesForArea(). This way I could filter only the primary frames, which correspond to the HTML elements. Now I no longer have duplicate HTML tags in my output. This also filters out the list decoration of ul and ol elements and the scrollbars.

And sorry about the video! I forgot to turn that on. But the video wasn't very good anyway.

Thanks for the help and I'll post my final result when I've got something to show.

On Thursday, 5 May 2016 01:59:34 UTC+2, [hidden email]  wrote:

> On Monday, May 2, 2016 at 4:55:13 PM UTC-4, [hidden email] wrote:
> > Does it means I also have to modify nsDisplayList.h in order to add the flag to nsDisplayListBuilder? I'm still a bit unsure how to add a flag to nsDisplayListBuilder.
>
> Just add a new member variable, like mIgnoreVisibility (and a getter/setter for it, if it's not public).
>
> > In my brief testing I noticed that overriding the pointer-events check in nsDisplayList::HitTest() allows me to also retrieve elements with pointer-events:none. Though strangely no iframes with pointer-events:none. Those are still left out.
>
> Ah, there is a separate check for iframes here [1]!
>
> > I made this screen recording to give an idea of my end goal: https://youtu.be/EtH1MMDorKA
>
> The video is private, I can't access it.
>
> Cheers,
> Botond
>
> [1] https://dxr.mozilla.org/mozilla-central/rev/369a5ee3a2880a4a98df3a00bf3db8d8f36b181b/layout/generic/nsSubDocumentFrame.cpp#379
_______________________________________________
dev-tech-layout mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-layout
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Include out of viewport frames in display list

jip.de.beer
I noticed I'm still missing some elements.
Children of button HTML elements aren't returned by elementsFromPoint.
You can see it in this fiddle: https://jsfiddle.net/ztg9gppt/2/

Google Chrome does return the children within button elements, even divs.
Twitter is nesting divs within buttons on their feed.

Has it been a deliberate choice to leave out the children of button elements? Can I make another workaround or should this be reported as a bug?

On Friday, 6 May 2016 23:15:35 UTC+2, [hidden email]  wrote:

> Thanks again for replying!
>
> I was able to get everything working with your instructions :)
> I also applied the same method to ignore backface-visibility:hidden;
> This check is done here:
> https://dxr.mozilla.org/mozilla-central/rev/e5a10bc7dac4ee2453d8319165c1f6578203eac7/layout/base/nsDisplayList.cpp#6018
>
> Hopefully I'm now only excluding elements with display:none and elements which aren't visible because their outside the visible part of a scrollable HTML element.
>
> Another thing I realized I should do was calling GetPrimaryFrame() on the dom::Element* I got from GetFramesForArea(). This way I could filter only the primary frames, which correspond to the HTML elements. Now I no longer have duplicate HTML tags in my output. This also filters out the list decoration of ul and ol elements and the scrollbars.
>
> And sorry about the video! I forgot to turn that on. But the video wasn't very good anyway.
>
> Thanks for the help and I'll post my final result when I've got something to show.
>
> On Thursday, 5 May 2016 01:59:34 UTC+2, [hidden email]  wrote:
> > On Monday, May 2, 2016 at 4:55:13 PM UTC-4, [hidden email] wrote:
> > > Does it means I also have to modify nsDisplayList.h in order to add the flag to nsDisplayListBuilder? I'm still a bit unsure how to add a flag to nsDisplayListBuilder.
> >
> > Just add a new member variable, like mIgnoreVisibility (and a getter/setter for it, if it's not public).
> >
> > > In my brief testing I noticed that overriding the pointer-events check in nsDisplayList::HitTest() allows me to also retrieve elements with pointer-events:none. Though strangely no iframes with pointer-events:none. Those are still left out.
> >
> > Ah, there is a separate check for iframes here [1]!
> >
> > > I made this screen recording to give an idea of my end goal: https://youtu.be/EtH1MMDorKA
> >
> > The video is private, I can't access it.
> >
> > Cheers,
> > Botond
> >
> > [1] https://dxr.mozilla.org/mozilla-central/rev/369a5ee3a2880a4a98df3a00bf3db8d8f36b181b/layout/generic/nsSubDocumentFrame.cpp#379
_______________________________________________
dev-tech-layout mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-layout
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Include out of viewport frames in display list

Robert O'Callahan-3
On Sat, May 7, 2016 at 10:21 AM, <[hidden email]> wrote:

> I noticed I'm still missing some elements.
> Children of button HTML elements aren't returned by elementsFromPoint.
> You can see it in this fiddle: https://jsfiddle.net/ztg9gppt/2/
>
> Google Chrome does return the children within button elements, even divs.
> Twitter is nesting divs within buttons on their feed.
>
> Has it been a deliberate choice to leave out the children of button
> elements?
>

Yes. elementFromPoint uses the same algorithm as event dispatch and in
Gecko, mouse events over elements in a button target the button.

Rob
--
lbir ye,ea yer.tnietoehr  rdn rdsme,anea lurpr  edna e hnysnenh hhe uresyf
toD
selthor  stor  edna  siewaoeodm  or v sstvr  esBa  kbvted,t
rdsme,aoreseoouoto
o l euetiuruewFa  kbn e hnystoivateweh uresyf tulsa rehr  rdm  or rnea
lurpr
.a war hsrer holsa rodvted,t  nenh hneireseoouot.tniesiewaoeivatewt sstvr
esn
_______________________________________________
dev-tech-layout mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-layout
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Include out of viewport frames in display list

jip.de.beer
In reply to this post by jip.de.beer
Thanks for replying. You're right, that's why the children of button elements aren't in the display list with mode EVENT_DELIVERY. But those elements are in a display list with mode PAINTING.

I don't care if those elements receive mouse events. I have already been able to ignore pointer-events:none, and (backface-)visibility:hidden. I just want to include all HTML elements within a document in a single display-list, except display:none and elements outside of the visible part of a  scrollable element. I only care about the absolute position (x,y,width,height) and z-order of each HTML element. So I filter out everything in the resulting display-list which isn't a HTML element, like scrollbars or list decoration.

I thought I had everything until I noticed the missing button children. How can I still include those in my display list? Where in the source code are they excluded form a display list with mode EVENT_DELIVERY? Or where are they included in a display list with mode PAINTING?

I thought this might be relevant https://dxr.mozilla.org/mozilla-central/rev/e5a10bc7dac4ee2453d8319165c1f6578203eac7/layout/forms/nsButtonFrameRenderer.cpp#186 but overriding this didn't change anything for me.
_______________________________________________
dev-tech-layout mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-layout
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Include out of viewport frames in display list

Boris Zbarsky
On 5/7/16 2:26 PM, [hidden email] wrote:
> I thought I had everything until I noticed the missing button children. How can I still include those in my display list? Where in the source code are they excluded form a display list with mode EVENT_DELIVERY? Or where are they included in a display list with mode PAINTING?

http://hg.mozilla.org/mozilla-central/file/bae525a694e2/layout/forms/nsHTMLButtonControlFrame.cpp#l121

-Boris
_______________________________________________
dev-tech-layout mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-layout
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Include out of viewport frames in display list

jip.de.beer
Hi,

I'm still working on this and ran into a new problem. I'd like to return the elements in a list, but with additional metadata. Essentially an array with objects.

My function now works like elementsFromPoint() and returns sequence<Element> which is defined in source/release/dom/webidl/Document.webidl.

So instead of this:
[referenceToHtmlNode,referenceToHtmlNode,referenceToHtmlNode,referenceToHtmlNode]

I'd like my function to return something like this:
[{node:referenceToHtmlNode,number1:aNumber,number2:aNumber,number3:aNumber},
{node:referenceToHtmlNode,number1:aNumber,number2:aNumber,number3:aNumber},
{node:referenceToHtmlNode,number1:aNumber,number2:aNumber,number3:aNumber},
{node:referenceToHtmlNode,number1:aNumber,number2:aNumber,number3:aNumber}]

Actually even better would be an array of these arrays...
[[{node:referenceToHtmlNode,number1:aNumber,number2:aNumber,number3:aNumber},
{node:referenceToHtmlNode,number1:aNumber,number2:aNumber,number3:aNumber},
{node:referenceToHtmlNode,number1:aNumber,number2:aNumber,number3:aNumber},
{node:referenceToHtmlNode,number1:aNumber,number2:aNumber,number3:aNumber}],

[{node:referenceToHtmlNode,number1:aNumber,number2:aNumber,number3:aNumber},
{node:referenceToHtmlNode,number1:aNumber,number2:aNumber,number3:aNumber},
{node:referenceToHtmlNode,number1:aNumber,number2:aNumber,number3:aNumber},
{node:referenceToHtmlNode,number1:aNumber,number2:aNumber,number3:aNumber}],

[{node:referenceToHtmlNode,number1:aNumber,number2:aNumber,number3:aNumber},
{node:referenceToHtmlNode,number1:aNumber,number2:aNumber,number3:aNumber},
{node:referenceToHtmlNode,number1:aNumber,number2:aNumber,number3:aNumber},
{node:referenceToHtmlNode,number1:aNumber,number2:aNumber,number3:aNumber}]]

What kind of type would I use for this? I'd also have to change to the corresponding type in nsDocument.cpp, currently it looks like this:

nsIDocument::myFunction(nsTArray<RefPtr<Element>>& aElements){
...
}

Hopefully you don't think it's a silly question!
_______________________________________________
dev-tech-layout mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-layout
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Include out of viewport frames in display list

Cameron McCormack-4
[hidden email]:

> I'm still working on this and ran into a new problem. I'd like
> to return the elements in a list, but with additional metadata.
> Essentially an array with objects.
>
> My function now works like elementsFromPoint()
> and returns sequence<Element> which is defined in
> source/release/dom/webidl/Document.webidl.
>
> So instead of this:
> [referenceToHtmlNode,referenceToHtmlNode,referenceToHtmlNode,referenceToHtmlNode]
>
> I'd like my function to return something like this:
> [{node:referenceToHtmlNode,number1:aNumber,number2:aNumber,number3:aNumber},
> {node:referenceToHtmlNode,number1:aNumber,number2:aNumber,number3:aNumber},
> {node:referenceToHtmlNode,number1:aNumber,number2:aNumber,number3:aNumber},
> {node:referenceToHtmlNode,number1:aNumber,number2:aNumber,number3:aNumber}]

You can define a dictionary to describe the object, and then return a
sequence of those:

  dictionary MyNodeInfo {
    Node node;
    float number1;
    float number2;
    float number3;
  };

  partial interface Document {
    sequence<MyNodeInfo> myFunction();
  };

Then:

  void nsIDocument::MyFunction(nsTArray<MyNodeInfo>& aResult) {
    MyNodeInfo* e = aResult.AppendElement();
    e->mNode = …;
    e->mNumber1 = …;
    …
  }

--
Cameron McCormack ≝ http://mcc.id.au/
_______________________________________________
dev-tech-layout mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-layout
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Include out of viewport frames in display list

Boris Zbarsky
In reply to this post by jip.de.beer
On 7/19/16 10:00 AM, [hidden email] wrote:
> What kind of type would I use for this?

A sequence of dictionaries, or a sequence of sequences of dictionaries,
respectively.

> I'd also have to change to the corresponding type in nsDocument.cpp

Indeed.  See
https://developer.mozilla.org/en-US/docs/Mozilla/WebIDL_bindings#typemapping 
for how IDL types map to C++ types.

-Boris
_______________________________________________
dev-tech-layout mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-layout
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Include out of viewport frames in display list

jip.de.beer
In reply to this post by jip.de.beer
Thanks for the quick reply, both of you!

Sounds like it shouldn’t be too hard :)

But where do I define this dictionary? Now I’ve added the code to source/release/dom/webidl/Document.webidl, but it may be out of place here.

When I added the code as you suggested, the first issue I got was this:

/source/release/dom/base/nsDocument.h:1042:41: error: unknown type name 'myFunctionElementInfo'; did you mean 'mozilla::dom::myFunctionElementInfo'?

So then I changed myFunctionElementInfo to mozilla::dom::myFunctionElementInfo in nsDocument.h and nsIDocument.h.
Then next, I got this issue:

 8:43.20 In file included from /build/dom/base/Unified_cpp_dom_base5.cpp:92:
 8:43.20 /source/release/dom/base/nsDocument.cpp:3580:12: error: no member named 'node' in 'mozilla::dom::myFunctionElementInfo'
 8:43.20         e->node = node;
 8:43.20         ~  ^
 8:43.20 /source/release/dom/base/nsDocument.cpp:3581:12: error: no member named 'number1' in 'mozilla::dom::myFunctionElementInfo'
 8:43.20         e->number1 = number1;
 8:43.20         ~  ^

Perhaps I didn’t declare the dictionary in the right place?
_______________________________________________
dev-tech-layout mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-layout
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Include out of viewport frames in display list

Boris Zbarsky
On 7/19/16 11:08 AM, [hidden email] wrote:
> But where do I define this dictionary?

Any .webidl file you feel like.  But putting it in Document.webidl makes
sense if that's where you use it.

> /source/release/dom/base/nsDocument.h:1042:41: error: unknown type name 'myFunctionElementInfo'; did you mean 'mozilla::dom::myFunctionElementInfo'?

Yes.  See
https://developer.mozilla.org/en-US/docs/Mozilla/WebIDL_bindings#Dictionary_types 
the part about "whose name is the dictionary name in the mozilla::dom
namespace".

>  8:43.20 In file included from /build/dom/base/Unified_cpp_dom_base5.cpp:92:
>  8:43.20 /source/release/dom/base/nsDocument.cpp:3580:12: error: no member named 'node' in 'mozilla::dom::myFunctionElementInfo'
>  8:43.20         e->node = node;
>  8:43.20         ~  ^
>  8:43.20 /source/release/dom/base/nsDocument.cpp:3581:12: error: no member named 'number1' in 'mozilla::dom::myFunctionElementInfo'
>  8:43.20         e->number1 = number1;
>  8:43.20         ~  ^

Please see
https://developer.mozilla.org/en-US/docs/Mozilla/WebIDL_bindings#Dictionary_types 
where it says:

   The struct has one member for each of the dictionary's members with
the same name except the first letter uppercased and prefixed with "m"

Seriously, the documentation should cover most of this stuff.  ;)

Also note that the struct's member will be an Optional unless the member
is either required or has a default value, so your code above would not
work even with the correct member name.  In your case, I expect making
all the members required is fine; I assume you plan to initialize all of
them for every dictionary.

-Boris
_______________________________________________
dev-tech-layout mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-layout
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Include out of viewport frames in display list

jip.de.beer
Thanks again, you're right the documentation is a good reference. I've read it now. I see I wasn't thinking clearly yesterday, Camerons example showed I should have used "m" and uppercase.

My function now returns a sequence of sequences of dictionaries, great progress haha :).
At the moment the dict only contains key-value mappings with numbers as the value.
This is because I still can't get the Element to be returned.

This is what I have at the moment:

dictionary myFunctionElementInfo {
  Node node;
};


Then when I try to assign it, I get this error:

 8:10.55 In file included from /build/dom/base/Unified_cpp_dom_base5.cpp:92:
 8:10.55 /source/release/dom/base/nsDocument.cpp:3580:18: error: no viable overloaded '='
 8:10.55         e->mNode = node;
 8:10.55         ~~~~~~~~ ^ ~~~~
 8:10.55 /build/dist/include/mozilla/dom/BindingDeclarations.h:297:7: note: candidate function (the implicit copy assignment operator) not viable: no known conversion from 'nsIContent *' to 'const mozilla::dom::Optional<mozilla::OwningNonNull<nsINode> >' for 1st argument
 8:10.55 class Optional<OwningNonNull<T> > : public Optional_base<T, OwningNonNull<T> >
 8:10.55       ^
 8:11.94 1 error generated.

I understood from your reply that I should make the Node variable required. But the documentation doesn't show how this works. I've tried some other things, like Nullable<Node> but that didn't seem to solve it.

Also, before I said node but I actually want to return an Element, just like ElementsFromPoint(). But for some reason I'm not allowed to have a dictionary item with type Element... compiling works with Node but not with Element...

This compiles:
dictionary myFunctionElementInfo {
  Node node;
};

This doesn't:
dictionary myFunctionElementInfo {
  Element element;
};

It gives many errors similar to this:
 0:12.03 /build/dist/include/mozilla/dom/Element.h:810:12: error: incomplete type 'nsPresContext' named in nested name specifier
 0:12.03            nsPresContext::AppUnitsToIntCSSPixels(sf->GetScrollRange().XMost()) :
 0:12.03            ^~~~~~~~~~~~~~~

I'm not sure if I need Node or Element in my case, I just want to return the HTML elements so I can do more with them on the JavaScript side. The return value of ElementsFromPoint() is what I'm after, so that's why I think I need to use Element instead of Node.
_______________________________________________
dev-tech-layout mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-layout
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Include out of viewport frames in display list

Boris Zbarsky
On 7/20/16 9:16 AM, [hidden email] wrote:

> Then when I try to assign it, I get this error:
>
>  8:10.55 In file included from /build/dom/base/Unified_cpp_dom_base5.cpp:92:
>  8:10.55 /source/release/dom/base/nsDocument.cpp:3580:18: error: no viable overloaded '='
>  8:10.55         e->mNode = node;
>  8:10.55         ~~~~~~~~ ^ ~~~~
>  8:10.55 /build/dist/include/mozilla/dom/BindingDeclarations.h:297:7: note: candidate function (the implicit copy assignment operator) not viable: no known conversion from 'nsIContent *' to 'const mozilla::dom::Optional<mozilla::OwningNonNull<nsINode> >' for 1st argument
>  8:10.55 class Optional<OwningNonNull<T> > : public Optional_base<T, OwningNonNull<T> >
>  8:10.55       ^
>  8:11.94 1 error generated.
>
> I understood from your reply that I should make the Node variable required. But the documentation doesn't show how this works.

That's because "required" is a WebIDL spec concept, not a Mozilla
extenson.  How it works is covered by the spec.

The way you do it is like this:

   dictionary myFunctionElementInfo {
     required Node node;
   };

That will make the dictionary member be an OwningNonNull<nsINode>
instead of an Optional<OwningNonNull<nsINode>>.

> This doesn't:
> dictionary myFunctionElementInfo {
>   Element element;
> };

I'm guessing that the problem is that doing that causes
DocumentBinding.h to include Element.h.  Presumably Element.h
(indirectly) includes nsIDocument.h; not sure what the exact path is
here.  nsIDocument.h includes DocumentBinding.h.  So you end up with an
#include loop, and some things never end up included correctly.

> It gives many errors similar to this:
>  0:12.03 /build/dist/include/mozilla/dom/Element.h:810:12: error: incomplete type 'nsPresContext' named in nested name specifier
>  0:12.03            nsPresContext::AppUnitsToIntCSSPixels(sf->GetScrollRange().XMost()) :
>  0:12.03            ^~~~~~~~~~~~~~~

The interesting thing here is the very _first_ error it gives.

> I'm not sure if I need Node or Element in my case, I just want to return the HTML elements so I can do more with them on the JavaScript side. The return value of ElementsFromPoint() is what I'm after, so that's why I think I need to use Element instead of Node.

It doesn't matter, really.  You can use Node; it'll still be the same
thing on the JS side.

You could also put your dictionary in a separate webidl file, so it
doesn't change the includes for DocumentBinding.h.  Then as long as you
just forward-declare the dictionary type in nsIDocument.h and include
your new binding header in nsDocument.cpp things will be fine.

-Boris
_______________________________________________
dev-tech-layout mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-layout
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Include out of viewport frames in display list

jip.de.beer
Thanks a lot, again. You are terrific! I've got it working :)
_______________________________________________
dev-tech-layout mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-layout
Reply | Threaded
Open this post in threaded view
|  
Report Content as Inappropriate

Re: Include out of viewport frames in display list

jip.de.beer
I decided to improve the output of my function further.
The output I'm aiming for is like this:

{
  "documentHeight": 539,
  "documentWidth": 1436,
  "levels": [
    [
      {
        "nodeIndex": 0,
        "x": 0,
        "y": 0,
        "w": 100,
        "h": 200
      },
      {
        "nodeIndex": 1,
        "x": 0,
        "y": 0,
        "w": 300,
        "h": 400
      }
    ],
    [
      {
        "nodeIndex": 2,
        "x": 0,
        "y": 0,
        "w": 100,
        "h": 200
      },
      {
        "nodeIndex": 3,
        "x": 0,
        "y": 0,
        "w": 300,
        "h": 400
      }
    ]
  ],
  "nodes": [
    "node1",
    "node2",
    "node3",
    "node4"
  ]
}

After your previous comments I thought it would be straightforward, but I've been trying all day without success.
Somehow I have declared the dictionaries as protected?

This is what I have with the errors raised:

// in Document.webidl
dictionary myFunctionElementInfo {
  required unsigned long nodeIndex;
  required long x;
  required long y;
  required unsigned long w;
  required unsigned long h;
};

dictionary myFunctionResultContainer {
  required sequence<Node> nodes;
  required sequence<sequence<myFunctionElementInfo>> levels;
  required unsigned long documentWidth;
  required unsigned long documentHeight;
};

// http://dev.w3.org/csswg/cssom-view/#extensions-to-the-document-interface
partial interface Document {
    myFunctionResultContainer myFunction ();
};


 0:15.57 /source/release/dom/base/nsDocument.cpp:3585:24: error: 'AppendElement' is a protected member of 'nsTArray_Impl<mozilla::OwningNonNull<nsINode>, nsTArrayFallibleAllocator>'
 0:15.57         aResult.mNodes.AppendElement(node);
 0:15.57         ~~~~~~~~~~~~~~~^~~~~~~~~~~~~
 0:15.57 /build/dist/include/nsTArray.h:1589:14: note: declared protected here
 0:15.57   elem_type* AppendElement(Item&& aItem)
 0:15.57              ^
 0:15.57 In file included from /build/dom/base/Unified_cpp_dom_base5.cpp:92:
 0:15.57 /source/release/dom/base/nsDocument.cpp:3586:25: error: 'EnsureLengthAtLeast' is a protected member of 'nsTArray_Impl<mozilla::dom::Sequence<mozilla::dom::myFunctionElementInfo>, nsTArrayFallibleAllocator>'
 0:15.57         aResult.mLevels.EnsureLengthAtLeast(level+1);
 0:15.57         ~~~~~~~~~~~~~~~~^~~~~~~~~~~~~~~~~~~
 0:15.57 /build/dist/include/nsTArray.h:1809:36: note: declared protected here
 0:15.57   typename ActualAlloc::ResultType EnsureLengthAtLeast(size_type aMinLen)
 0:15.57                                    ^
 0:15.57 In file included from /build/dom/base/Unified_cpp_dom_base5.cpp:92:
 0:15.57 /source/release/dom/base/nsDocument.cpp:3587:84: error: 'AppendElement' is a protected member of 'nsTArray_Impl<mozilla::dom::myFunctionElementInfo, nsTArrayFallibleAllocator>'
 0:15.57         mozilla::dom::myFunctionElementInfo* e = aResult.mLevels.ElementAt(level).AppendElement();
 0:15.57                                                   ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~^~~~~~~~~~~~~
 0:15.57 /build/dist/include/nsTArray.h:1642:14: note: declared protected here
 0:15.57   elem_type* AppendElement()
 0:15.57              ^
 0:16.49 /build/dist/include/nsTArray.h:527:34: error: no matching constructor for initialization of 'mozilla::OwningNonNull<nsINode>'
 0:16.50     new (static_cast<void*>(aE)) E(mozilla::Forward<A>(aArg));
 0:16.50                                  ^ ~~~~~~~~~~~~~~~~~~~~~~~~~
 0:16.50 /build/dist/include/nsTArray.h:1596:18: note: in instantiation of function template specialization 'nsTArrayElementTraits<mozilla::OwningNonNull<nsINode> >::Construct<nsIContent *&>' requested here
 0:16.50     elem_traits::Construct(elem, mozilla::Forward<Item>(aItem));
 0:16.50                  ^
 0:16.50 /source/release/dom/base/nsDocument.cpp:3585:24: note: in instantiation of function template specialization 'nsTArray_Impl<mozilla::OwningNonNull<nsINode>, nsTArrayFallibleAllocator>::AppendElement<nsIContent *&, nsTArrayFallibleAllocator>' requested here
 0:16.50         aResult.mNodes.AppendElement(node);
 0:16.50                        ^
 0:16.50 /build/dist/include/mozilla/OwningNonNull.h:23:16: note: candidate constructor not viable: no known conversion from 'nsIContent *' to 'nsINode &' for 1st argument; dereference the argument with *
 0:16.50   MOZ_IMPLICIT OwningNonNull(T& aValue)
 0:16.50                ^
 0:16.50 /build/dist/include/mozilla/OwningNonNull.h:18:7: note: candidate constructor (the implicit move constructor) not viable: no known conversion from 'nsIContent *' to 'mozilla::OwningNonNull<nsINode>' for 1st argument
 0:16.50 class OwningNonNull
 0:16.50       ^
 0:16.50 /build/dist/include/mozilla/OwningNonNull.h:18:7: note: candidate constructor (the implicit copy constructor) not viable: no known conversion from 'nsIContent *' to 'const mozilla::OwningNonNull<nsINode>' for 1st argument
 0:16.50 class OwningNonNull
 0:16.50       ^
 0:16.50 /build/dist/include/mozilla/OwningNonNull.h:29:16: note: candidate template ignored: could not match 'already_AddRefed<type-parameter-0-0>' against 'nsIContent *'
 0:16.50   MOZ_IMPLICIT OwningNonNull(already_AddRefed<U>&& aValue)
 0:16.50                ^
 0:16.50 /build/dist/include/mozilla/OwningNonNull.h:21:3: note: candidate constructor not viable: requires 0 arguments, but 1 was provided
 0:16.50   OwningNonNull() {}
 0:16.50   ^
 0:16.62 4 errors generated.
_______________________________________________
dev-tech-layout mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-tech-layout
12
Loading...