IPC plans

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

IPC plans

Trevor Saunders
Hi,

 This mail is going to attempt to describe how accessibility will work
 in e10ns Gecko. follow ups to dev-accessibility is probably best.

tldr:
- the operating system level accessibility API will only talk to the
  main process
- accessibility information will be computed in the same process as the
  content it is for.
- Where it is advantagious we will cache information in the parent
  process instead of blocking on IPC.

details:
The main process will have a tree caching data for each sub tree in a
content process.  The tree of documents in in each of these subtrees
will be updated with PContent::PDocAccessibleConstructor() and
PDocAccessible::__delete__(), and the tree of accessibles in each
document will be kept up to date with show and hide events.  This means
caching the arrangement of accessibles in the tree is easy, and probably
makes things simpler in addition to faster.  At least in theory we can
cache everything that always fires an event when it changes, but I
expect at first we'll only cache the tree and then add more caching
based on performance data.

I plan on having a set of ipdl actors per document and then using
integer ids to refer to accessibles within that document, we can
repurpose the mAccessiblecache which maps pointers to accessibles to
themselves to map ids to accessibles in a manor that is safe.  This
saves the over head we'd have if we had an actor per accessible which
would result in thousands of actors for sessions with many tabs.

unresolved issues:
- xpcom API should it be one tree or tree per process? afaik AccessFoo
  assumes tree per process and our test suite assumes one tree for
everything, so we need to reconsile this somehow, but I'm not sure it
needs to happen immediately.

- we need to use the ipc infrastructure in the platform layer to handle
  accessible objects for content in child processes I'm not exactly sure
how this should look some ideas would include sub classing accessible /
making it more generic and having a proxy and local implementation, or
maybe it would be better for platform API methods to explicitly handle
remote accessibles with something like if accWrap accWrap->FOobar() else
remoteDoc->Foobar(accWrapId) but I figure we can figure this out once we
have the plumbing either case will need to call sorted out.

question / comments / whatever welcome!

Trev

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility

signature.asc (836 bytes) Download Attachment
Reply | Threaded
Open this post in threaded view
|

Re: IPC plans

Alexander Surkov
Hi.

1) If you plan to make AT to talk to the main process only then it means
the AT should see a whole MSAA tree as unique tree, the same trick can be
done for XPCOM. I.e. the way XPCOM tree is implemented shouldn't be
different from MSAA tree implementation.
2) I assume that you can have sync access to content process DOM tree from
the main process, so if the main process doesn't dupe DOM trees of content
processes then it's good to explain why accessibility tree is different.
3) Didn't you consider a scenario when AT communicates with each process?

Thanks.
Alexander.


On Tue, Mar 25, 2014 at 5:33 PM, Trevor Saunders <[hidden email]>wrote:

> Hi,
>
>  This mail is going to attempt to describe how accessibility will work
>  in e10ns Gecko. follow ups to dev-accessibility is probably best.
>
> tldr:
> - the operating system level accessibility API will only talk to the
>   main process
> - accessibility information will be computed in the same process as the
>   content it is for.
> - Where it is advantagious we will cache information in the parent
>   process instead of blocking on IPC.
>
> details:
> The main process will have a tree caching data for each sub tree in a
> content process.  The tree of documents in in each of these subtrees
> will be updated with PContent::PDocAccessibleConstructor() and
> PDocAccessible::__delete__(), and the tree of accessibles in each
> document will be kept up to date with show and hide events.  This means
> caching the arrangement of accessibles in the tree is easy, and probably
> makes things simpler in addition to faster.  At least in theory we can
> cache everything that always fires an event when it changes, but I
> expect at first we'll only cache the tree and then add more caching
> based on performance data.
>
> I plan on having a set of ipdl actors per document and then using
> integer ids to refer to accessibles within that document, we can
> repurpose the mAccessiblecache which maps pointers to accessibles to
> themselves to map ids to accessibles in a manor that is safe.  This
> saves the over head we'd have if we had an actor per accessible which
> would result in thousands of actors for sessions with many tabs.
>
> unresolved issues:
> - xpcom API should it be one tree or tree per process? afaik AccessFoo
>   assumes tree per process and our test suite assumes one tree for
> everything, so we need to reconsile this somehow, but I'm not sure it
> needs to happen immediately.
>
> - we need to use the ipc infrastructure in the platform layer to handle
>   accessible objects for content in child processes I'm not exactly sure
> how this should look some ideas would include sub classing accessible /
> making it more generic and having a proxy and local implementation, or
> maybe it would be better for platform API methods to explicitly handle
> remote accessibles with something like if accWrap accWrap->FOobar() else
> remoteDoc->Foobar(accWrapId) but I figure we can figure this out once we
> have the plumbing either case will need to call sorted out.
>
> question / comments / whatever welcome!
>
> Trev
>
> _______________________________________________
> accessibility mailing list
> [hidden email]
> https://lists.mozilla.org/listinfo/accessibility
>
>
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: IPC plans

Trevor Saunders
On Wed, Mar 26, 2014 at 09:04:36AM -0400, Alexander Surkov wrote:
> Hi.
>
> 1) If you plan to make AT to talk to the main process only then it means the AT
> should see a whole MSAA tree as unique tree, the same trick can be done for
> XPCOM. I.e. the way XPCOM tree is implemented shouldn't be different from MSAA
> tree implementation.

I agree the xpcom tree *can* all be in one process, but I'm not sure I
agree its either desirable or easier than the alternative.

> 2) I assume that you can have sync access to content process DOM tree from the

you can't unless we add it, and we'd need to add sync access to the
frame tree too...

> main process, so if the main process doesn't dupe DOM trees of content
> processes then it's good to explain why accessibility tree is different.

there's just no reason to have general access to the dom in the child
process from the parent.

> 3) Didn't you consider a scenario when AT communicates with each process?

not really, it would be hard to sand box, and it would require a bunch
of tricky platform specific code for each platform, and its not even
clear to me its possible on mac.

Trev

>     Hi,
>
>      This mail is going to attempt to describe how accessibility will work
>      in e10ns Gecko. follow ups to dev-accessibility is probably best.
>
>     tldr:
>     - the operating system level accessibility API will only talk to the
>       main process
>     - accessibility information will be computed in the same process as the
>       content it is for.
>     - Where it is advantagious we will cache information in the parent
>       process instead of blocking on IPC.
>
>     details:
>     The main process will have a tree caching data for each sub tree in a
>     content process.  The tree of documents in in each of these subtrees
>     will be updated with PContent::PDocAccessibleConstructor() and
>     PDocAccessible::__delete__(), and the tree of accessibles in each
>     document will be kept up to date with show and hide events.  This means
>     caching the arrangement of accessibles in the tree is easy, and probably
>     makes things simpler in addition to faster.  At least in theory we can
>     cache everything that always fires an event when it changes, but I
>     expect at first we'll only cache the tree and then add more caching
>     based on performance data.
>
>     I plan on having a set of ipdl actors per document and then using
>     integer ids to refer to accessibles within that document, we can
>     repurpose the mAccessiblecache which maps pointers to accessibles to
>     themselves to map ids to accessibles in a manor that is safe.  This
>     saves the over head we'd have if we had an actor per accessible which
>     would result in thousands of actors for sessions with many tabs.
>
>     unresolved issues:
>     - xpcom API should it be one tree or tree per process? afaik AccessFoo
>       assumes tree per process and our test suite assumes one tree for
>     everything, so we need to reconsile this somehow, but I'm not sure it
>     needs to happen immediately.
>
>     - we need to use the ipc infrastructure in the platform layer to handle
>       accessible objects for content in child processes I'm not exactly sure
>     how this should look some ideas would include sub classing accessible /
>     making it more generic and having a proxy and local implementation, or
>     maybe it would be better for platform API methods to explicitly handle
>     remote accessibles with something like if accWrap accWrap->FOobar() else
>     remoteDoc->Foobar(accWrapId) but I figure we can figure this out once we
>     have the plumbing either case will need to call sorted out.
>
>     question / comments / whatever welcome!
>
>     Trev
>
>     _______________________________________________
>     accessibility mailing list
>     [hidden email]
>     https://lists.mozilla.org/listinfo/accessibility
>
>
>
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: IPC plans

Alexander Surkov
On Wed, Mar 26, 2014 at 2:49 PM, Trevor Saunders <[hidden email]>wrote:

> On Wed, Mar 26, 2014 at 09:04:36AM -0400, Alexander Surkov wrote:
> > Hi.
> >
> > 1) If you plan to make AT to talk to the main process only then it means
> the AT
> > should see a whole MSAA tree as unique tree, the same trick can be done
> for
> > XPCOM. I.e. the way XPCOM tree is implemented shouldn't be different
> from MSAA
> > tree implementation.
>
> I agree the xpcom tree *can* all be in one process, but I'm not sure I
> agree its either desirable or easier than the alternative.
>

AccessFu is not quite different from platform screen readers by its nature.
I'm pretty sure they will need a mechanism to cross the boundaries between
processes. So either it'll be one tree or multiple trees with
cross-platform bridges.


>
> > 2) I assume that you can have sync access to content process DOM tree
> from the
>
> you can't unless we add it, and we'd need to add sync access to the
> frame tree too...
>

> > main process, so if the main process doesn't dupe DOM trees of content
> > processes then it's good to explain why accessibility tree is different.
>
> there's just no reason to have general access to the dom in the child
> process from the parent.
>

I'm curious though how content plugins are implemented. Their head should
be in chrome process but they operate on content.


>
> > 3) Didn't you consider a scenario when AT communicates with each process?
>
> not really, it would be hard to sand box, and it would require a bunch
> of tricky platform specific code for each platform, and its not even
> clear to me its possible on mac.
>

The stake is to win in memory and in performance. Even if it requires
platfrom specific code then it's worth to consider the option (it seems Hub
agrees on it).


>
> Trev
>
> >     Hi,
> >
> >      This mail is going to attempt to describe how accessibility will
> work
> >      in e10ns Gecko. follow ups to dev-accessibility is probably best.
> >
> >     tldr:
> >     - the operating system level accessibility API will only talk to the
> >       main process
> >     - accessibility information will be computed in the same process as
> the
> >       content it is for.
> >     - Where it is advantagious we will cache information in the parent
> >       process instead of blocking on IPC.
> >
> >     details:
> >     The main process will have a tree caching data for each sub tree in a
> >     content process.  The tree of documents in in each of these subtrees
> >     will be updated with PContent::PDocAccessibleConstructor() and
> >     PDocAccessible::__delete__(), and the tree of accessibles in each
> >     document will be kept up to date with show and hide events.  This
> means
> >     caching the arrangement of accessibles in the tree is easy, and
> probably
> >     makes things simpler in addition to faster.  At least in theory we
> can
> >     cache everything that always fires an event when it changes, but I
> >     expect at first we'll only cache the tree and then add more caching
> >     based on performance data.
> >
> >     I plan on having a set of ipdl actors per document and then using
> >     integer ids to refer to accessibles within that document, we can
> >     repurpose the mAccessiblecache which maps pointers to accessibles to
> >     themselves to map ids to accessibles in a manor that is safe.  This
> >     saves the over head we'd have if we had an actor per accessible which
> >     would result in thousands of actors for sessions with many tabs.
> >
> >     unresolved issues:
> >     - xpcom API should it be one tree or tree per process? afaik
> AccessFoo
> >       assumes tree per process and our test suite assumes one tree for
> >     everything, so we need to reconsile this somehow, but I'm not sure it
> >     needs to happen immediately.
> >
> >     - we need to use the ipc infrastructure in the platform layer to
> handle
> >       accessible objects for content in child processes I'm not exactly
> sure
> >     how this should look some ideas would include sub classing
> accessible /
> >     making it more generic and having a proxy and local implementation,
> or
> >     maybe it would be better for platform API methods to explicitly
> handle
> >     remote accessibles with something like if accWrap accWrap->FOobar()
> else
> >     remoteDoc->Foobar(accWrapId) but I figure we can figure this out
> once we
> >     have the plumbing either case will need to call sorted out.
> >
> >     question / comments / whatever welcome!
> >
> >     Trev
> >
> >     _______________________________________________
> >     accessibility mailing list
> >     [hidden email]
> >     https://lists.mozilla.org/listinfo/accessibility
> >
> >
> >
>
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: IPC plans

Trevor Saunders
On Wed, Mar 26, 2014 at 03:13:58PM -0400, Alexander Surkov wrote:

>
>
>
> On Wed, Mar 26, 2014 at 2:49 PM, Trevor Saunders <[hidden email]>
> wrote:
>
>     On Wed, Mar 26, 2014 at 09:04:36AM -0400, Alexander Surkov wrote:
>     > Hi.
>     >
>     > 1) If you plan to make AT to talk to the main process only then it means
>     the AT
>     > should see a whole MSAA tree as unique tree, the same trick can be done
>     for
>     > XPCOM. I.e. the way XPCOM tree is implemented shouldn't be different from
>     MSAA
>     > tree implementation.
>
>     I agree the xpcom tree *can* all be in one process, but I'm not sure I
>     agree its either desirable or easier than the alternative.
>
>
> AccessFu is not quite different from platform screen readers by its nature. I'm
> pretty sure they will need a mechanism to cross the boundaries between
> processes. So either it'll be one tree or multiple trees with cross-platform
> bridges.

aiui they already deal with multiple processes because they work on b2g
and they deal with it with content scripts and stuff in JS.  So since
AcessFoo seems to already work in a multiprocess world I just assume
leave it as is unless we have a reason to change it.

>     > 2) I assume that you can have sync access to content process DOM tree
>     from the
>
>     you can't unless we add it, and we'd need to add sync access to the
>     frame tree too...
>
>
>     > main process, so if the main process doesn't dupe DOM trees of content
>     > processes then it's good to explain why accessibility tree is different.
>
>     there's just no reason to have general access to the dom in the child
>     process from the parent.
>
>
> I'm curious though how content plugins are implemented. Their head should be in
> chrome process but they operate on content.

I'm not sure what you mean "content plugins"

>     > 3) Didn't you consider a scenario when AT communicates with each process?
>
>     not really, it would be hard to sand box, and it would require a bunch
>     of tricky platform specific code for each platform, and its not even
>     clear to me its possible on mac.
>
>
> The stake is to win in memory and in performance. Even if it requires platfrom

it looses on security and it would be a lot of pretty evil platform
specific code assuming it even works on mac and I'm pretty sure the
android stuff would need its own special IPC stuff anyway.

> specific code then it's worth to consider the option (it seems Hub agrees on
> it).

I haven't seen him say anything...

Trev

>  
>
>
>     Trev
>
>     >     Hi,
>     >
>     >      This mail is going to attempt to describe how accessibility will
>     work
>     >      in e10ns Gecko. follow ups to dev-accessibility is probably best.
>     >
>     >     tldr:
>     >     - the operating system level accessibility API will only talk to the
>     >       main process
>     >     - accessibility information will be computed in the same process as
>     the
>     >       content it is for.
>     >     - Where it is advantagious we will cache information in the parent
>     >       process instead of blocking on IPC.
>     >
>     >     details:
>     >     The main process will have a tree caching data for each sub tree in a
>     >     content process.  The tree of documents in in each of these subtrees
>     >     will be updated with PContent::PDocAccessibleConstructor() and
>     >     PDocAccessible::__delete__(), and the tree of accessibles in each
>     >     document will be kept up to date with show and hide events.  This
>     means
>     >     caching the arrangement of accessibles in the tree is easy, and
>     probably
>     >     makes things simpler in addition to faster.  At least in theory we
>     can
>     >     cache everything that always fires an event when it changes, but I
>     >     expect at first we'll only cache the tree and then add more caching
>     >     based on performance data.
>     >
>     >     I plan on having a set of ipdl actors per document and then using
>     >     integer ids to refer to accessibles within that document, we can
>     >     repurpose the mAccessiblecache which maps pointers to accessibles to
>     >     themselves to map ids to accessibles in a manor that is safe.  This
>     >     saves the over head we'd have if we had an actor per accessible which
>     >     would result in thousands of actors for sessions with many tabs.
>     >
>     >     unresolved issues:
>     >     - xpcom API should it be one tree or tree per process? afaik
>     AccessFoo
>     >       assumes tree per process and our test suite assumes one tree for
>     >     everything, so we need to reconsile this somehow, but I'm not sure it
>     >     needs to happen immediately.
>     >
>     >     - we need to use the ipc infrastructure in the platform layer to
>     handle
>     >       accessible objects for content in child processes I'm not exactly
>     sure
>     >     how this should look some ideas would include sub classing accessible
>     /
>     >     making it more generic and having a proxy and local implementation,
>     or
>     >     maybe it would be better for platform API methods to explicitly
>     handle
>     >     remote accessibles with something like if accWrap accWrap->FOobar()
>     else
>     >     remoteDoc->Foobar(accWrapId) but I figure we can figure this out once
>     we
>     >     have the plumbing either case will need to call sorted out.
>     >
>     >     question / comments / whatever welcome!
>     >
>     >     Trev
>     >
>     >     _______________________________________________
>     >     accessibility mailing list
>     >     [hidden email]
>     >     https://lists.mozilla.org/listinfo/accessibility
>     >
>     >
>     >
>
>
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: IPC plans

Alexander Surkov
On Wed, Mar 26, 2014 at 3:34 PM, Trevor Saunders <[hidden email]>wrote:

> On Wed, Mar 26, 2014 at 03:13:58PM -0400, Alexander Surkov wrote:
> >
> >
> >
> > On Wed, Mar 26, 2014 at 2:49 PM, Trevor Saunders <
> [hidden email]>
> > wrote:
> >
> >     On Wed, Mar 26, 2014 at 09:04:36AM -0400, Alexander Surkov wrote:
> >     > Hi.
> >     >
> >     > 1) If you plan to make AT to talk to the main process only then it
> means
> >     the AT
> >     > should see a whole MSAA tree as unique tree, the same trick can be
> done
> >     for
> >     > XPCOM. I.e. the way XPCOM tree is implemented shouldn't be
> different from
> >     MSAA
> >     > tree implementation.
> >
> >     I agree the xpcom tree *can* all be in one process, but I'm not sure
> I
> >     agree its either desirable or easier than the alternative.
> >
> >
> > AccessFu is not quite different from platform screen readers by its
> nature. I'm
> > pretty sure they will need a mechanism to cross the boundaries between
> > processes. So either it'll be one tree or multiple trees with
> cross-platform
> > bridges.
>

> aiui they already deal with multiple processes because they work on b2g
> and they deal with it with content scripts and stuff in JS.  So since
> AcessFoo seems to already work in a multiprocess world I just assume
> leave it as is unless we have a reason to change it.
>

If it works then fine (I'd love to hear Eitan or Yura on this). One tree
might be useful for tools like DOMi though. But it can be an emulation of
the unique tree I guess on DOMi side.

>     > 2) I assume that you can have sync access to content process DOM
> tree
> >     from the
> >
> >     you can't unless we add it, and we'd need to add sync access to the
> >     frame tree too...
> >
> >
> >     > main process, so if the main process doesn't dupe DOM trees of
> content
> >     > processes then it's good to explain why accessibility tree is
> different.
> >
> >     there's just no reason to have general access to the dom in the child
> >     process from the parent.
> >
> >
> > I'm curious though how content plugins are implemented. Their head
> should be in
> > chrome process but they operate on content.
>
> I'm not sure what you mean "content plugins"
>

Firefox add-ons that deal with content. I can see they took a different way
on their implementation so it doesn't look like it requires sync
communications (
https://developer.mozilla.org/en-US/Add-ons/SDK/Guides/Content_Scripts)


>
> >     > 3) Didn't you consider a scenario when AT communicates with each
> process?
> >
> >     not really, it would be hard to sand box, and it would require a
> bunch
> >     of tricky platform specific code for each platform, and its not even
> >     clear to me its possible on mac.
> >
> >
> > The stake is to win in memory and in performance. Even if it requires
> platfrom
>
> it looses on security and it would be a lot of pretty evil platform
> specific code assuming it even works on mac and I'm pretty sure the
> android stuff would need its own special IPC stuff anyway.
>

ATK makes one-tree magic on their side afaik, so if we had disconnected
trees in each process then it should be ok for ATK. Windows is HWND based
so they are quite different. If we learned how to let AT to interact with
HWNDs from content processes then it'd be awesome since we could have
disconnected trees again. If that's a security treat then we need to work
with security team to figure out how that's bad because otherwise we can
end up with unperformant and memory heavy solution.


>
> > specific code then it's worth to consider the option (it seems Hub
> agrees on
> > it).
>
> I haven't seen him say anything...
>

He said:
"I'm also concerned that using more memory might be a problem on memory
constrained platform like mobile phone (we have less than half a GB of
RAM and then we have "containers" for each app). Just to keep that in mind."


>
> Trev
>
> >
> >
> >
> >     Trev
> >
> >     >     Hi,
> >     >
> >     >      This mail is going to attempt to describe how accessibility
> will
> >     work
> >     >      in e10ns Gecko. follow ups to dev-accessibility is probably
> best.
> >     >
> >     >     tldr:
> >     >     - the operating system level accessibility API will only talk
> to the
> >     >       main process
> >     >     - accessibility information will be computed in the same
> process as
> >     the
> >     >       content it is for.
> >     >     - Where it is advantagious we will cache information in the
> parent
> >     >       process instead of blocking on IPC.
> >     >
> >     >     details:
> >     >     The main process will have a tree caching data for each sub
> tree in a
> >     >     content process.  The tree of documents in in each of these
> subtrees
> >     >     will be updated with PContent::PDocAccessibleConstructor() and
> >     >     PDocAccessible::__delete__(), and the tree of accessibles in
> each
> >     >     document will be kept up to date with show and hide events.
>  This
> >     means
> >     >     caching the arrangement of accessibles in the tree is easy, and
> >     probably
> >     >     makes things simpler in addition to faster.  At least in
> theory we
> >     can
> >     >     cache everything that always fires an event when it changes,
> but I
> >     >     expect at first we'll only cache the tree and then add more
> caching
> >     >     based on performance data.
> >     >
> >     >     I plan on having a set of ipdl actors per document and then
> using
> >     >     integer ids to refer to accessibles within that document, we
> can
> >     >     repurpose the mAccessiblecache which maps pointers to
> accessibles to
> >     >     themselves to map ids to accessibles in a manor that is safe.
>  This
> >     >     saves the over head we'd have if we had an actor per
> accessible which
> >     >     would result in thousands of actors for sessions with many
> tabs.
> >     >
> >     >     unresolved issues:
> >     >     - xpcom API should it be one tree or tree per process? afaik
> >     AccessFoo
> >     >       assumes tree per process and our test suite assumes one tree
> for
> >     >     everything, so we need to reconsile this somehow, but I'm not
> sure it
> >     >     needs to happen immediately.
> >     >
> >     >     - we need to use the ipc infrastructure in the platform layer
> to
> >     handle
> >     >       accessible objects for content in child processes I'm not
> exactly
> >     sure
> >     >     how this should look some ideas would include sub classing
> accessible
> >     /
> >     >     making it more generic and having a proxy and local
> implementation,
> >     or
> >     >     maybe it would be better for platform API methods to explicitly
> >     handle
> >     >     remote accessibles with something like if accWrap
> accWrap->FOobar()
> >     else
> >     >     remoteDoc->Foobar(accWrapId) but I figure we can figure this
> out once
> >     we
> >     >     have the plumbing either case will need to call sorted out.
> >     >
> >     >     question / comments / whatever welcome!
> >     >
> >     >     Trev
> >     >
> >     >     _______________________________________________
> >     >     accessibility mailing list
> >     >     [hidden email]
> >     >     https://lists.mozilla.org/listinfo/accessibility
> >     >
> >     >
> >     >
> >
> >
>
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: IPC plans

Eitan Isaacson
In reply to this post by Trevor Saunders
This looks great! Thanks for posting. More below.

On 03/25/2014 02:33 PM, Trevor Saunders wrote:
> unresolved issues:
> - xpcom API should it be one tree or tree per process? afaik AccessFoo
>   assumes tree per process and our test suite assumes one tree for
> everything, so we need to reconsile this somehow, but I'm not sure it
> needs to happen immediately.

Ultimately, if we can get away with it performance wise, it would be
nice to use a single tree in AccessFu, this will make life easier. Also,
what are the plans for events? The current system has events only firing
in their respective process.

Cheers,
 Eitan.

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: IPC plans

Trevor Saunders
On Wed, Mar 26, 2014 at 01:04:57PM -0700, Eitan Isaacson wrote:

> This looks great! Thanks for posting. More below.
>
> On 03/25/2014 02:33 PM, Trevor Saunders wrote:
> > unresolved issues:
> > - xpcom API should it be one tree or tree per process? afaik AccessFoo
> >   assumes tree per process and our test suite assumes one tree for
> > everything, so we need to reconsile this somehow, but I'm not sure it
> > needs to happen immediately.
>
> Ultimately, if we can get away with it performance wise, it would be
> nice to use a single tree in AccessFu, this will make life easier. Also,
> what are the plans for events? The current system has events only firing
> in their respective process.

Well, we need them for platform events so if you want them I see no
problem giving them to you, we just need to coordinate changes so
AccessFoo doesn't get broken.

Trev

>
> Cheers,
>  Eitan.
>
> _______________________________________________
> dev-accessibility mailing list
> [hidden email]
> https://lists.mozilla.org/listinfo/dev-accessibility
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: IPC plans

Trevor Saunders
In reply to this post by Alexander Surkov
On Wed, Mar 26, 2014 at 03:59:35PM -0400, Alexander Surkov wrote:

>
>
>
> On Wed, Mar 26, 2014 at 3:34 PM, Trevor Saunders <[hidden email]>
> wrote:
>
>     On Wed, Mar 26, 2014 at 03:13:58PM -0400, Alexander Surkov wrote:
>     >
>     >
>     >
>     > On Wed, Mar 26, 2014 at 2:49 PM, Trevor Saunders <[hidden email]
>     >
>     > wrote:
>     >
>     >     On Wed, Mar 26, 2014 at 09:04:36AM -0400, Alexander Surkov wrote:
>     >     > Hi.
>     >     >
>     >     > 1) If you plan to make AT to talk to the main process only then it
>     means
>     >     the AT
>     >     > should see a whole MSAA tree as unique tree, the same trick can be
>     done
>     >     for
>     >     > XPCOM. I.e. the way XPCOM tree is implemented shouldn't be
>     different from
>     >     MSAA
>     >     > tree implementation.
>     >
>     >     I agree the xpcom tree *can* all be in one process, but I'm not sure
>     I
>     >     agree its either desirable or easier than the alternative.
>     >
>     >
>     > AccessFu is not quite different from platform screen readers by its
>     nature. I'm
>     > pretty sure they will need a mechanism to cross the boundaries between
>     > processes. So either it'll be one tree or multiple trees with
>     cross-platform
>     > bridges.
>
>
>     aiui they already deal with multiple processes because they work on b2g
>     and they deal with it with content scripts and stuff in JS.  So since
>     AcessFoo seems to already work in a multiprocess world I just assume
>     leave it as is unless we have a reason to change it.
>
>
> If it works then fine (I'd love to hear Eitan or Yura on this). One tree might
> be useful for tools like DOMi though. But it can be an emulation of the unique
> tree I guess on DOMi side.

Seems they want it.

>     >     > 2) I assume that you can have sync access to content process DOM
>     tree
>     >     from the
>     >
>     >     you can't unless we add it, and we'd need to add sync access to the
>     >     frame tree too...
>     >
>     >
>     >     > main process, so if the main process doesn't dupe DOM trees of
>     content
>     >     > processes then it's good to explain why accessibility tree is
>     different.
>     >
>     >     there's just no reason to have general access to the dom in the child
>     >     process from the parent.
>     >
>     >
>     > I'm curious though how content plugins are implemented. Their head should
>     be in
>     > chrome process but they operate on content.
>
>     I'm not sure what you mean "content plugins"
>
>
> Firefox add-ons that deal with content. I can see they took a different way on
> their implementation so it doesn't look like it requires sync communications (
> https://developer.mozilla.org/en-US/Add-ons/SDK/Guides/Content_Scripts)

yeah, the things for addons aren't useful for us.

>     >     > 3) Didn't you consider a scenario when AT communicates with each
>     process?
>     >
>     >     not really, it would be hard to sand box, and it would require a
>     bunch
>     >     of tricky platform specific code for each platform, and its not even
>     >     clear to me its possible on mac.
>     >
>     >
>     > The stake is to win in memory and in performance. Even if it requires
>     platfrom
>
>     it looses on security and it would be a lot of pretty evil platform
>     specific code assuming it even works on mac and I'm pretty sure the
>     android stuff would need its own special IPC stuff anyway.
>
>
> ATK makes one-tree magic on their side afaik, so if we had disconnected trees

Sure, but that assumes no sandbox, we could probably hax the dbus stuff
so that it communicated out of the sandbox with fds we sendmesg()ed in,
but I'm pretty sure it would be really gross, there was a reason I said
this is non trivial... :p

> in each process then it should be ok for ATK. Windows is HWND based so they are
> quite different. If we learned how to let AT to interact with HWNDs from
> content processes then it'd be awesome since we could have disconnected trees
> again. If that's a security treat then we need to work with security team to
> figure out how that's bad because otherwise we can end up with unperformant and
> memory heavy solution.

how is it not obvious its bad? afaik the sandboxing plan for windows
means no hwnds in content processes heck, even if we had hwnds any way
stuff in the content process has of communicating with stuff outside is
a security issue.  I'm pretty sure we've been over this a couple times
already.

>     > specific code then it's worth to consider the option (it seems Hub agrees
>     on
>     > it).
>
>     I haven't seen him say anything...
>
>
> He said:
> "I'm also concerned that using more memory might be a problem on memory
> constrained platform like mobile phone (we have less than half a GB of
> RAM and then we have "containers" for each app). Just to keep that in mind."

I'm really not that worries about memory, its just another hash table
per document, and of course the smaller the total amount of memory the
fewer the number of documents you can have.

Trev

>  
>
>
>     Trev
>
>     >  
>     >
>     >
>     >     Trev
>     >
>     >     >     Hi,
>     >     >
>     >     >      This mail is going to attempt to describe how accessibility
>     will
>     >     work
>     >     >      in e10ns Gecko. follow ups to dev-accessibility is probably
>     best.
>     >     >
>     >     >     tldr:
>     >     >     - the operating system level accessibility API will only talk
>     to the
>     >     >       main process
>     >     >     - accessibility information will be computed in the same
>     process as
>     >     the
>     >     >       content it is for.
>     >     >     - Where it is advantagious we will cache information in the
>     parent
>     >     >       process instead of blocking on IPC.
>     >     >
>     >     >     details:
>     >     >     The main process will have a tree caching data for each sub
>     tree in a
>     >     >     content process.  The tree of documents in in each of these
>     subtrees
>     >     >     will be updated with PContent::PDocAccessibleConstructor() and
>     >     >     PDocAccessible::__delete__(), and the tree of accessibles in
>     each
>     >     >     document will be kept up to date with show and hide events.
>      This
>     >     means
>     >     >     caching the arrangement of accessibles in the tree is easy, and
>     >     probably
>     >     >     makes things simpler in addition to faster.  At least in theory
>     we
>     >     can
>     >     >     cache everything that always fires an event when it changes,
>     but I
>     >     >     expect at first we'll only cache the tree and then add more
>     caching
>     >     >     based on performance data.
>     >     >
>     >     >     I plan on having a set of ipdl actors per document and then
>     using
>     >     >     integer ids to refer to accessibles within that document, we
>     can
>     >     >     repurpose the mAccessiblecache which maps pointers to
>     accessibles to
>     >     >     themselves to map ids to accessibles in a manor that is safe.
>      This
>     >     >     saves the over head we'd have if we had an actor per accessible
>     which
>     >     >     would result in thousands of actors for sessions with many
>     tabs.
>     >     >
>     >     >     unresolved issues:
>     >     >     - xpcom API should it be one tree or tree per process? afaik
>     >     AccessFoo
>     >     >       assumes tree per process and our test suite assumes one tree
>     for
>     >     >     everything, so we need to reconsile this somehow, but I'm not
>     sure it
>     >     >     needs to happen immediately.
>     >     >
>     >     >     - we need to use the ipc infrastructure in the platform layer
>     to
>     >     handle
>     >     >       accessible objects for content in child processes I'm not
>     exactly
>     >     sure
>     >     >     how this should look some ideas would include sub classing
>     accessible
>     >     /
>     >     >     making it more generic and having a proxy and local
>     implementation,
>     >     or
>     >     >     maybe it would be better for platform API methods to explicitly
>     >     handle
>     >     >     remote accessibles with something like if accWrap accWrap->
>     FOobar()
>     >     else
>     >     >     remoteDoc->Foobar(accWrapId) but I figure we can figure this
>     out once
>     >     we
>     >     >     have the plumbing either case will need to call sorted out.
>     >     >
>     >     >     question / comments / whatever welcome!
>     >     >
>     >     >     Trev
>     >     >
>     >     >     _______________________________________________
>     >     >     accessibility mailing list
>     >     >     [hidden email]
>     >     >     https://lists.mozilla.org/listinfo/accessibility
>     >     >
>     >     >
>     >     >
>     >
>     >
>
>
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: IPC plans

Alexander Surkov
On Wed, Mar 26, 2014 at 7:18 PM, Trevor Saunders <[hidden email]>wrote:

> On Wed, Mar 26, 2014 at 03:59:35PM -0400, Alexander Surkov wrote:
> >
> >
> >
> > On Wed, Mar 26, 2014 at 3:34 PM, Trevor Saunders <
> [hidden email]>
> > wrote:
> >
> >     On Wed, Mar 26, 2014 at 03:13:58PM -0400, Alexander Surkov wrote:
> >     >
> >     >
> >     >
> >     > On Wed, Mar 26, 2014 at 2:49 PM, Trevor Saunders <
> [hidden email]
> >     >
> >     > wrote:
> >     >
> >     >     On Wed, Mar 26, 2014 at 09:04:36AM -0400, Alexander Surkov
> wrote:
> >     >     > Hi.
> >     >     >
> >     >     > 1) If you plan to make AT to talk to the main process only
> then it
> >     means
> >     >     the AT
> >     >     > should see a whole MSAA tree as unique tree, the same trick
> can be
> >     done
> >     >     for
> >     >     > XPCOM. I.e. the way XPCOM tree is implemented shouldn't be
> >     different from
> >     >     MSAA
> >     >     > tree implementation.
> >     >
> >     >     I agree the xpcom tree *can* all be in one process, but I'm
> not sure
> >     I
> >     >     agree its either desirable or easier than the alternative.
> >     >
> >     >
> >     > AccessFu is not quite different from platform screen readers by its
> >     nature. I'm
> >     > pretty sure they will need a mechanism to cross the boundaries
> between
> >     > processes. So either it'll be one tree or multiple trees with
> >     cross-platform
> >     > bridges.
> >
> >
> >     aiui they already deal with multiple processes because they work on
> b2g
> >     and they deal with it with content scripts and stuff in JS.  So since
> >     AcessFoo seems to already work in a multiprocess world I just assume
> >     leave it as is unless we have a reason to change it.
> >
> >
> > If it works then fine (I'd love to hear Eitan or Yura on this). One tree
> might
> > be useful for tools like DOMi though. But it can be an emulation of the
> unique
> > tree I guess on DOMi side.
>
> Seems they want it.
>
> >     >     > 2) I assume that you can have sync access to content process
> DOM
> >     tree
> >     >     from the
> >     >
> >     >     you can't unless we add it, and we'd need to add sync access
> to the
> >     >     frame tree too...
> >     >
> >     >
> >     >     > main process, so if the main process doesn't dupe DOM trees
> of
> >     content
> >     >     > processes then it's good to explain why accessibility tree is
> >     different.
> >     >
> >     >     there's just no reason to have general access to the dom in
> the child
> >     >     process from the parent.
> >     >
> >     >
> >     > I'm curious though how content plugins are implemented. Their head
> should
> >     be in
> >     > chrome process but they operate on content.
> >
> >     I'm not sure what you mean "content plugins"
> >
> >
> > Firefox add-ons that deal with content. I can see they took a different
> way on
> > their implementation so it doesn't look like it requires sync
> communications (
> > https://developer.mozilla.org/en-US/Add-ons/SDK/Guides/Content_Scripts)
>
> yeah, the things for addons aren't useful for us.
>
> >     >     > 3) Didn't you consider a scenario when AT communicates with
> each
> >     process?
> >     >
> >     >     not really, it would be hard to sand box, and it would require
> a
> >     bunch
> >     >     of tricky platform specific code for each platform, and its
> not even
> >     >     clear to me its possible on mac.
> >     >
> >     >
> >     > The stake is to win in memory and in performance. Even if it
> requires
> >     platfrom
> >
> >     it looses on security and it would be a lot of pretty evil platform
> >     specific code assuming it even works on mac and I'm pretty sure the
> >     android stuff would need its own special IPC stuff anyway.
> >
> >
> > ATK makes one-tree magic on their side afaik, so if we had disconnected
> trees
>
> Sure, but that assumes no sandbox, we could probably hax the dbus stuff
> so that it communicated out of the sandbox with fds we sendmesg()ed in,
> but I'm pretty sure it would be really gross, there was a reason I said
> this is non trivial... :p
>

I'm not sure I follow your proposal how to implement things for ATK.


>
> > in each process then it should be ok for ATK. Windows is HWND based so
> they are
> > quite different. If we learned how to let AT to interact with HWNDs from
> > content processes then it'd be awesome since we could have disconnected
> trees
> > again. If that's a security treat then we need to work with security
> team to
> > figure out how that's bad because otherwise we can end up with
> unperformant and
> > memory heavy solution.
>
> how is it not obvious its bad? afaik the sandboxing plan for windows
> means no hwnds in content processes heck, even if we had hwnds any way
> stuff in the content process has of communicating with stuff outside is
> a security issue.  I'm pretty sure we've been over this a couple times
> already.
>

I've never heard any feedback from security team. If the solution we will
end up will be unperfromant or memory heavy then it's quire possible that
AT users will continue to run Firefox in single process. What I suggest is
to make sure we considered all alternatives.


>
> >     > specific code then it's worth to consider the option (it seems Hub
> agrees
> >     on
> >     > it).
> >
> >     I haven't seen him say anything...
> >
> >
> > He said:
> > "I'm also concerned that using more memory might be a problem on memory
> > constrained platform like mobile phone (we have less than half a GB of
> > RAM and then we have "containers" for each app). Just to keep that in
> mind."
>
> I'm really not that worries about memory, its just another hash table
> per document, and of course the smaller the total amount of memory the
> fewer the number of documents you can have.
>
> Trev
>
> >
> >
> >
> >     Trev
> >
> >     >
> >     >
> >     >
> >     >     Trev
> >     >
> >     >     >     Hi,
> >     >     >
> >     >     >      This mail is going to attempt to describe how
> accessibility
> >     will
> >     >     work
> >     >     >      in e10ns Gecko. follow ups to dev-accessibility is
> probably
> >     best.
> >     >     >
> >     >     >     tldr:
> >     >     >     - the operating system level accessibility API will only
> talk
> >     to the
> >     >     >       main process
> >     >     >     - accessibility information will be computed in the same
> >     process as
> >     >     the
> >     >     >       content it is for.
> >     >     >     - Where it is advantagious we will cache information in
> the
> >     parent
> >     >     >       process instead of blocking on IPC.
> >     >     >
> >     >     >     details:
> >     >     >     The main process will have a tree caching data for each
> sub
> >     tree in a
> >     >     >     content process.  The tree of documents in in each of
> these
> >     subtrees
> >     >     >     will be updated with
> PContent::PDocAccessibleConstructor() and
> >     >     >     PDocAccessible::__delete__(), and the tree of
> accessibles in
> >     each
> >     >     >     document will be kept up to date with show and hide
> events.
> >      This
> >     >     means
> >     >     >     caching the arrangement of accessibles in the tree is
> easy, and
> >     >     probably
> >     >     >     makes things simpler in addition to faster.  At least in
> theory
> >     we
> >     >     can
> >     >     >     cache everything that always fires an event when it
> changes,
> >     but I
> >     >     >     expect at first we'll only cache the tree and then add
> more
> >     caching
> >     >     >     based on performance data.
> >     >     >
> >     >     >     I plan on having a set of ipdl actors per document and
> then
> >     using
> >     >     >     integer ids to refer to accessibles within that
> document, we
> >     can
> >     >     >     repurpose the mAccessiblecache which maps pointers to
> >     accessibles to
> >     >     >     themselves to map ids to accessibles in a manor that is
> safe.
> >      This
> >     >     >     saves the over head we'd have if we had an actor per
> accessible
> >     which
> >     >     >     would result in thousands of actors for sessions with
> many
> >     tabs.
> >     >     >
> >     >     >     unresolved issues:
> >     >     >     - xpcom API should it be one tree or tree per process?
> afaik
> >     >     AccessFoo
> >     >     >       assumes tree per process and our test suite assumes
> one tree
> >     for
> >     >     >     everything, so we need to reconsile this somehow, but
> I'm not
> >     sure it
> >     >     >     needs to happen immediately.
> >     >     >
> >     >     >     - we need to use the ipc infrastructure in the platform
> layer
> >     to
> >     >     handle
> >     >     >       accessible objects for content in child processes I'm
> not
> >     exactly
> >     >     sure
> >     >     >     how this should look some ideas would include sub
> classing
> >     accessible
> >     >     /
> >     >     >     making it more generic and having a proxy and local
> >     implementation,
> >     >     or
> >     >     >     maybe it would be better for platform API methods to
> explicitly
> >     >     handle
> >     >     >     remote accessibles with something like if accWrap
> accWrap->
> >     FOobar()
> >     >     else
> >     >     >     remoteDoc->Foobar(accWrapId) but I figure we can figure
> this
> >     out once
> >     >     we
> >     >     >     have the plumbing either case will need to call sorted
> out.
> >     >     >
> >     >     >     question / comments / whatever welcome!
> >     >     >
> >     >     >     Trev
> >     >     >
> >     >     >     _______________________________________________
> >     >     >     accessibility mailing list
> >     >     >     [hidden email]
> >     >     >     https://lists.mozilla.org/listinfo/accessibility
> >     >     >
> >     >     >
> >     >     >
> >     >
> >     >
> >
> >
>
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: IPC plans

Trevor Saunders
On Thu, Mar 27, 2014 at 03:24:58PM -0400, Alexander Surkov wrote:

>
>
>
> On Wed, Mar 26, 2014 at 7:18 PM, Trevor Saunders <[hidden email]>
> wrote:
>
>     On Wed, Mar 26, 2014 at 03:59:35PM -0400, Alexander Surkov wrote:
>     >
>     >
>     >
>     > On Wed, Mar 26, 2014 at 3:34 PM, Trevor Saunders <[hidden email]
>     >
>     > wrote:
>     >
>     >     On Wed, Mar 26, 2014 at 03:13:58PM -0400, Alexander Surkov wrote:
>     >     >
>     >     >
>     >     >
>     >     > On Wed, Mar 26, 2014 at 2:49 PM, Trevor Saunders <
>     [hidden email]
>     >     >
>     >     > wrote:
>     >     >
>     >     >     On Wed, Mar 26, 2014 at 09:04:36AM -0400, Alexander Surkov
>     wrote:
>     >     >     > Hi.
>     >     >     >
>     >     >     > 1) If you plan to make AT to talk to the main process only
>     then it
>     >     means
>     >     >     the AT
>     >     >     > should see a whole MSAA tree as unique tree, the same trick
>     can be
>     >     done
>     >     >     for
>     >     >     > XPCOM. I.e. the way XPCOM tree is implemented shouldn't be
>     >     different from
>     >     >     MSAA
>     >     >     > tree implementation.
>     >     >
>     >     >     I agree the xpcom tree *can* all be in one process, but I'm not
>     sure
>     >     I
>     >     >     agree its either desirable or easier than the alternative.
>     >     >
>     >     >
>     >     > AccessFu is not quite different from platform screen readers by its
>     >     nature. I'm
>     >     > pretty sure they will need a mechanism to cross the boundaries
>     between
>     >     > processes. So either it'll be one tree or multiple trees with
>     >     cross-platform
>     >     > bridges.
>     >
>     >
>     >     aiui they already deal with multiple processes because they work on
>     b2g
>     >     and they deal with it with content scripts and stuff in JS.  So since
>     >     AcessFoo seems to already work in a multiprocess world I just assume
>     >     leave it as is unless we have a reason to change it.
>     >
>     >
>     > If it works then fine (I'd love to hear Eitan or Yura on this). One tree
>     might
>     > be useful for tools like DOMi though. But it can be an emulation of the
>     unique
>     > tree I guess on DOMi side.
>
>     Seems they want it.
>
>     >     >     > 2) I assume that you can have sync access to content process
>     DOM
>     >     tree
>     >     >     from the
>     >     >
>     >     >     you can't unless we add it, and we'd need to add sync access to
>     the
>     >     >     frame tree too...
>     >     >
>     >     >
>     >     >     > main process, so if the main process doesn't dupe DOM trees
>     of
>     >     content
>     >     >     > processes then it's good to explain why accessibility tree is
>     >     different.
>     >     >
>     >     >     there's just no reason to have general access to the dom in the
>     child
>     >     >     process from the parent.
>     >     >
>     >     >
>     >     > I'm curious though how content plugins are implemented. Their head
>     should
>     >     be in
>     >     > chrome process but they operate on content.
>     >
>     >     I'm not sure what you mean "content plugins"
>     >
>     >
>     > Firefox add-ons that deal with content. I can see they took a different
>     way on
>     > their implementation so it doesn't look like it requires sync
>     communications (
>     > https://developer.mozilla.org/en-US/Add-ons/SDK/Guides/Content_Scripts)
>
>     yeah, the things for addons aren't useful for us.
>
>     >     >     > 3) Didn't you consider a scenario when AT communicates with
>     each
>     >     process?
>     >     >
>     >     >     not really, it would be hard to sand box, and it would require
>     a
>     >     bunch
>     >     >     of tricky platform specific code for each platform, and its not
>     even
>     >     >     clear to me its possible on mac.
>     >     >
>     >     >
>     >     > The stake is to win in memory and in performance. Even if it
>     requires
>     >     platfrom
>     >
>     >     it looses on security and it would be a lot of pretty evil platform
>     >     specific code assuming it even works on mac and I'm pretty sure the
>     >     android stuff would need its own special IPC stuff anyway.
>     >
>     >
>     > ATK makes one-tree magic on their side afaik, so if we had disconnected
>     trees
>
>     Sure, but that assumes no sandbox, we could probably hax the dbus stuff
>     so that it communicated out of the sandbox with fds we sendmesg()ed in,
>     but I'm pretty sure it would be really gross, there was a reason I said
>     this is non trivial... :p
>
>
> I'm not sure I follow your proposal how to implement things for ATK.

then don't worry about it.

>     > in each process then it should be ok for ATK. Windows is HWND based so
>     they are
>     > quite different. If we learned how to let AT to interact with HWNDs from
>     > content processes then it'd be awesome since we could have disconnected
>     trees
>     > again. If that's a security treat then we need to work with security team
>     to
>     > figure out how that's bad because otherwise we can end up with
>     unperformant and
>     > memory heavy solution.
>
>     how is it not obvious its bad? afaik the sandboxing plan for windows
>     means no hwnds in content processes heck, even if we had hwnds any way
>     stuff in the content process has of communicating with stuff outside is
>     a security issue.  I'm pretty sure we've been over this a couple times
>     already.
>
>
> I've never heard any feedback from security team. If the solution we will end

I remember talking about this with you in the room...  besides it really
isn't hard to see its an issue.

Trev

> up will be unperfromant or memory heavy then it's quire possible that AT users
> will continue to run Firefox in single process. What I suggest is to make sure
> we considered all alternatives.
>  
>
>
>     >     > specific code then it's worth to consider the option (it seems Hub
>     agrees
>     >     on
>     >     > it).
>     >
>     >     I haven't seen him say anything...
>     >
>     >
>     > He said:
>     > "I'm also concerned that using more memory might be a problem on memory
>     > constrained platform like mobile phone (we have less than half a GB of
>     > RAM and then we have "containers" for each app). Just to keep that in
>     mind."
>
>     I'm really not that worries about memory, its just another hash table
>     per document, and of course the smaller the total amount of memory the
>     fewer the number of documents you can have.
>
>     Trev
>
>     >  
>     >
>     >
>     >     Trev
>     >
>     >     >  
>     >     >
>     >     >
>     >     >     Trev
>     >     >
>     >     >     >     Hi,
>     >     >     >
>     >     >     >      This mail is going to attempt to describe how
>     accessibility
>     >     will
>     >     >     work
>     >     >     >      in e10ns Gecko. follow ups to dev-accessibility is
>     probably
>     >     best.
>     >     >     >
>     >     >     >     tldr:
>     >     >     >     - the operating system level accessibility API will only
>     talk
>     >     to the
>     >     >     >       main process
>     >     >     >     - accessibility information will be computed in the same
>     >     process as
>     >     >     the
>     >     >     >       content it is for.
>     >     >     >     - Where it is advantagious we will cache information in
>     the
>     >     parent
>     >     >     >       process instead of blocking on IPC.
>     >     >     >
>     >     >     >     details:
>     >     >     >     The main process will have a tree caching data for each
>     sub
>     >     tree in a
>     >     >     >     content process.  The tree of documents in in each of
>     these
>     >     subtrees
>     >     >     >     will be updated with PContent::PDocAccessibleConstructor
>     () and
>     >     >     >     PDocAccessible::__delete__(), and the tree of accessibles
>     in
>     >     each
>     >     >     >     document will be kept up to date with show and hide
>     events.
>     >      This
>     >     >     means
>     >     >     >     caching the arrangement of accessibles in the tree is
>     easy, and
>     >     >     probably
>     >     >     >     makes things simpler in addition to faster.  At least in
>     theory
>     >     we
>     >     >     can
>     >     >     >     cache everything that always fires an event when it
>     changes,
>     >     but I
>     >     >     >     expect at first we'll only cache the tree and then add
>     more
>     >     caching
>     >     >     >     based on performance data.
>     >     >     >
>     >     >     >     I plan on having a set of ipdl actors per document and
>     then
>     >     using
>     >     >     >     integer ids to refer to accessibles within that document,
>     we
>     >     can
>     >     >     >     repurpose the mAccessiblecache which maps pointers to
>     >     accessibles to
>     >     >     >     themselves to map ids to accessibles in a manor that is
>     safe.
>     >      This
>     >     >     >     saves the over head we'd have if we had an actor per
>     accessible
>     >     which
>     >     >     >     would result in thousands of actors for sessions with
>     many
>     >     tabs.
>     >     >     >
>     >     >     >     unresolved issues:
>     >     >     >     - xpcom API should it be one tree or tree per process?
>     afaik
>     >     >     AccessFoo
>     >     >     >       assumes tree per process and our test suite assumes one
>     tree
>     >     for
>     >     >     >     everything, so we need to reconsile this somehow, but I'm
>     not
>     >     sure it
>     >     >     >     needs to happen immediately.
>     >     >     >
>     >     >     >     - we need to use the ipc infrastructure in the platform
>     layer
>     >     to
>     >     >     handle
>     >     >     >       accessible objects for content in child processes I'm
>     not
>     >     exactly
>     >     >     sure
>     >     >     >     how this should look some ideas would include sub
>     classing
>     >     accessible
>     >     >     /
>     >     >     >     making it more generic and having a proxy and local
>     >     implementation,
>     >     >     or
>     >     >     >     maybe it would be better for platform API methods to
>     explicitly
>     >     >     handle
>     >     >     >     remote accessibles with something like if accWrap
>     accWrap->
>     >     FOobar()
>     >     >     else
>     >     >     >     remoteDoc->Foobar(accWrapId) but I figure we can figure
>     this
>     >     out once
>     >     >     we
>     >     >     >     have the plumbing either case will need to call sorted
>     out.
>     >     >     >
>     >     >     >     question / comments / whatever welcome!
>     >     >     >
>     >     >     >     Trev
>     >     >     >
>     >     >     >     _______________________________________________
>     >     >     >     accessibility mailing list
>     >     >     >     [hidden email]
>     >     >     >     https://lists.mozilla.org/listinfo/accessibility
>     >     >     >
>     >     >     >
>     >     >     >
>     >     >
>     >     >
>     >
>     >
>
>
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: IPC plans

David Bolter-2
Yo. Please excuse the top post. One take away from this is that we
should get these discussions going on a mailing list earlier next time.
There are still questions about all the various approaches. Trevor's
foray here has been to investigate a solution he finds most hopeful.

Cheers,
David


On 2014-03-27, 3:38 PM, Trevor Saunders wrote:

> On Thu, Mar 27, 2014 at 03:24:58PM -0400, Alexander Surkov wrote:
>>
>>
>>
>> On Wed, Mar 26, 2014 at 7:18 PM, Trevor Saunders <[hidden email]>
>> wrote:
>>
>>     On Wed, Mar 26, 2014 at 03:59:35PM -0400, Alexander Surkov wrote:
>>     >
>>     >
>>     >
>>     > On Wed, Mar 26, 2014 at 3:34 PM, Trevor Saunders <[hidden email]
>>     >
>>     > wrote:
>>     >
>>     >     On Wed, Mar 26, 2014 at 03:13:58PM -0400, Alexander Surkov wrote:
>>     >     >
>>     >     >
>>     >     >
>>     >     > On Wed, Mar 26, 2014 at 2:49 PM, Trevor Saunders <
>>     [hidden email]
>>     >     >
>>     >     > wrote:
>>     >     >
>>     >     >     On Wed, Mar 26, 2014 at 09:04:36AM -0400, Alexander Surkov
>>     wrote:
>>     >     >     > Hi.
>>     >     >     >
>>     >     >     > 1) If you plan to make AT to talk to the main process only
>>     then it
>>     >     means
>>     >     >     the AT
>>     >     >     > should see a whole MSAA tree as unique tree, the same trick
>>     can be
>>     >     done
>>     >     >     for
>>     >     >     > XPCOM. I.e. the way XPCOM tree is implemented shouldn't be
>>     >     different from
>>     >     >     MSAA
>>     >     >     > tree implementation.
>>     >     >
>>     >     >     I agree the xpcom tree *can* all be in one process, but I'm not
>>     sure
>>     >     I
>>     >     >     agree its either desirable or easier than the alternative.
>>     >     >
>>     >     >
>>     >     > AccessFu is not quite different from platform screen readers by its
>>     >     nature. I'm
>>     >     > pretty sure they will need a mechanism to cross the boundaries
>>     between
>>     >     > processes. So either it'll be one tree or multiple trees with
>>     >     cross-platform
>>     >     > bridges.
>>     >
>>     >
>>     >     aiui they already deal with multiple processes because they work on
>>     b2g
>>     >     and they deal with it with content scripts and stuff in JS.  So since
>>     >     AcessFoo seems to already work in a multiprocess world I just assume
>>     >     leave it as is unless we have a reason to change it.
>>     >
>>     >
>>     > If it works then fine (I'd love to hear Eitan or Yura on this). One tree
>>     might
>>     > be useful for tools like DOMi though. But it can be an emulation of the
>>     unique
>>     > tree I guess on DOMi side.
>>
>>     Seems they want it.
>>
>>     >     >     > 2) I assume that you can have sync access to content process
>>     DOM
>>     >     tree
>>     >     >     from the
>>     >     >
>>     >     >     you can't unless we add it, and we'd need to add sync access to
>>     the
>>     >     >     frame tree too...
>>     >     >
>>     >     >
>>     >     >     > main process, so if the main process doesn't dupe DOM trees
>>     of
>>     >     content
>>     >     >     > processes then it's good to explain why accessibility tree is
>>     >     different.
>>     >     >
>>     >     >     there's just no reason to have general access to the dom in the
>>     child
>>     >     >     process from the parent.
>>     >     >
>>     >     >
>>     >     > I'm curious though how content plugins are implemented. Their head
>>     should
>>     >     be in
>>     >     > chrome process but they operate on content.
>>     >
>>     >     I'm not sure what you mean "content plugins"
>>     >
>>     >
>>     > Firefox add-ons that deal with content. I can see they took a different
>>     way on
>>     > their implementation so it doesn't look like it requires sync
>>     communications (
>>     > https://developer.mozilla.org/en-US/Add-ons/SDK/Guides/Content_Scripts)
>>
>>     yeah, the things for addons aren't useful for us.
>>
>>     >     >     > 3) Didn't you consider a scenario when AT communicates with
>>     each
>>     >     process?
>>     >     >
>>     >     >     not really, it would be hard to sand box, and it would require
>>     a
>>     >     bunch
>>     >     >     of tricky platform specific code for each platform, and its not
>>     even
>>     >     >     clear to me its possible on mac.
>>     >     >
>>     >     >
>>     >     > The stake is to win in memory and in performance. Even if it
>>     requires
>>     >     platfrom
>>     >
>>     >     it looses on security and it would be a lot of pretty evil platform
>>     >     specific code assuming it even works on mac and I'm pretty sure the
>>     >     android stuff would need its own special IPC stuff anyway.
>>     >
>>     >
>>     > ATK makes one-tree magic on their side afaik, so if we had disconnected
>>     trees
>>
>>     Sure, but that assumes no sandbox, we could probably hax the dbus stuff
>>     so that it communicated out of the sandbox with fds we sendmesg()ed in,
>>     but I'm pretty sure it would be really gross, there was a reason I said
>>     this is non trivial... :p
>>
>>
>> I'm not sure I follow your proposal how to implement things for ATK.
>
> then don't worry about it.
>
>>     > in each process then it should be ok for ATK. Windows is HWND based so
>>     they are
>>     > quite different. If we learned how to let AT to interact with HWNDs from
>>     > content processes then it'd be awesome since we could have disconnected
>>     trees
>>     > again. If that's a security treat then we need to work with security team
>>     to
>>     > figure out how that's bad because otherwise we can end up with
>>     unperformant and
>>     > memory heavy solution.
>>
>>     how is it not obvious its bad? afaik the sandboxing plan for windows
>>     means no hwnds in content processes heck, even if we had hwnds any way
>>     stuff in the content process has of communicating with stuff outside is
>>     a security issue.  I'm pretty sure we've been over this a couple times
>>     already.
>>
>>
>> I've never heard any feedback from security team. If the solution we will end
>
> I remember talking about this with you in the room...  besides it really
> isn't hard to see its an issue.
>
> Trev
>
>> up will be unperfromant or memory heavy then it's quire possible that AT users
>> will continue to run Firefox in single process. What I suggest is to make sure
>> we considered all alternatives.
>>  
>>
>>
>>     >     > specific code then it's worth to consider the option (it seems Hub
>>     agrees
>>     >     on
>>     >     > it).
>>     >
>>     >     I haven't seen him say anything...
>>     >
>>     >
>>     > He said:
>>     > "I'm also concerned that using more memory might be a problem on memory
>>     > constrained platform like mobile phone (we have less than half a GB of
>>     > RAM and then we have "containers" for each app). Just to keep that in
>>     mind."
>>
>>     I'm really not that worries about memory, its just another hash table
>>     per document, and of course the smaller the total amount of memory the
>>     fewer the number of documents you can have.
>>
>>     Trev
>>
>>     >  
>>     >
>>     >
>>     >     Trev
>>     >
>>     >     >  
>>     >     >
>>     >     >
>>     >     >     Trev
>>     >     >
>>     >     >     >     Hi,
>>     >     >     >
>>     >     >     >      This mail is going to attempt to describe how
>>     accessibility
>>     >     will
>>     >     >     work
>>     >     >     >      in e10ns Gecko. follow ups to dev-accessibility is
>>     probably
>>     >     best.
>>     >     >     >
>>     >     >     >     tldr:
>>     >     >     >     - the operating system level accessibility API will only
>>     talk
>>     >     to the
>>     >     >     >       main process
>>     >     >     >     - accessibility information will be computed in the same
>>     >     process as
>>     >     >     the
>>     >     >     >       content it is for.
>>     >     >     >     - Where it is advantagious we will cache information in
>>     the
>>     >     parent
>>     >     >     >       process instead of blocking on IPC.
>>     >     >     >
>>     >     >     >     details:
>>     >     >     >     The main process will have a tree caching data for each
>>     sub
>>     >     tree in a
>>     >     >     >     content process.  The tree of documents in in each of
>>     these
>>     >     subtrees
>>     >     >     >     will be updated with PContent::PDocAccessibleConstructor
>>     () and
>>     >     >     >     PDocAccessible::__delete__(), and the tree of accessibles
>>     in
>>     >     each
>>     >     >     >     document will be kept up to date with show and hide
>>     events.
>>     >      This
>>     >     >     means
>>     >     >     >     caching the arrangement of accessibles in the tree is
>>     easy, and
>>     >     >     probably
>>     >     >     >     makes things simpler in addition to faster.  At least in
>>     theory
>>     >     we
>>     >     >     can
>>     >     >     >     cache everything that always fires an event when it
>>     changes,
>>     >     but I
>>     >     >     >     expect at first we'll only cache the tree and then add
>>     more
>>     >     caching
>>     >     >     >     based on performance data.
>>     >     >     >
>>     >     >     >     I plan on having a set of ipdl actors per document and
>>     then
>>     >     using
>>     >     >     >     integer ids to refer to accessibles within that document,
>>     we
>>     >     can
>>     >     >     >     repurpose the mAccessiblecache which maps pointers to
>>     >     accessibles to
>>     >     >     >     themselves to map ids to accessibles in a manor that is
>>     safe.
>>     >      This
>>     >     >     >     saves the over head we'd have if we had an actor per
>>     accessible
>>     >     which
>>     >     >     >     would result in thousands of actors for sessions with
>>     many
>>     >     tabs.
>>     >     >     >
>>     >     >     >     unresolved issues:
>>     >     >     >     - xpcom API should it be one tree or tree per process?
>>     afaik
>>     >     >     AccessFoo
>>     >     >     >       assumes tree per process and our test suite assumes one
>>     tree
>>     >     for
>>     >     >     >     everything, so we need to reconsile this somehow, but I'm
>>     not
>>     >     sure it
>>     >     >     >     needs to happen immediately.
>>     >     >     >
>>     >     >     >     - we need to use the ipc infrastructure in the platform
>>     layer
>>     >     to
>>     >     >     handle
>>     >     >     >       accessible objects for content in child processes I'm
>>     not
>>     >     exactly
>>     >     >     sure
>>     >     >     >     how this should look some ideas would include sub
>>     classing
>>     >     accessible
>>     >     >     /
>>     >     >     >     making it more generic and having a proxy and local
>>     >     implementation,
>>     >     >     or
>>     >     >     >     maybe it would be better for platform API methods to
>>     explicitly
>>     >     >     handle
>>     >     >     >     remote accessibles with something like if accWrap
>>     accWrap->
>>     >     FOobar()
>>     >     >     else
>>     >     >     >     remoteDoc->Foobar(accWrapId) but I figure we can figure
>>     this
>>     >     out once
>>     >     >     we
>>     >     >     >     have the plumbing either case will need to call sorted
>>     out.
>>     >     >     >
>>     >     >     >     question / comments / whatever welcome!
>>     >     >     >
>>     >     >     >     Trev
>>     >     >     >
>>     >     >     >     _______________________________________________
>>     >     >     >     accessibility mailing list
>>     >     >     >     [hidden email]
>>     >     >     >     https://lists.mozilla.org/listinfo/accessibility
>>     >     >     >
>>     >     >     >
>>     >     >     >
>>     >     >
>>     >     >
>>     >
>>     >
>>
>>
> _______________________________________________
> accessibility mailing list
> [hidden email]
> https://lists.mozilla.org/listinfo/accessibility
>

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility