New: Firefox 3 with Screen Readers FAQ

classic Classic list List threaded Threaded
65 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

Re: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]

Bryan Garaventa-2
Thanks, makes sense.

What's confusing is that it seems like IAccessible is not accessible from a
Windows app, except that it can be accessible sometimes. Forgetting all the
DOM stuff, and where screen readers come into it, what about this...

Say I'm creating a Windows app using VB. Would it be possible for me to
access IAccessible using the VB code? If so, how would I go about doing
this?

Thanks,

Bryan


--------------------------------------------------------------------------------
Bryan Garaventa
Senior Accessibility Engineer
SSB + BART Group
[hidden email]
www.SSBBartGroup.com
Accessibility-On-Demand


----- Original Message -----
From: "Steve Lee" <[hidden email]>
To: "Bryan Garaventa" <[hidden email]>
Cc: "dev-accessibility Firefox" <[hidden email]>
Sent: Thursday, July 10, 2008 11:04 AM
Subject: Re: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]


> 2008/7/10 Bryan Garaventa <[hidden email]>:
>> Please correct me if I'm wrong here... Is IDispatch synonymous
>> with COM?
>
> Not quite  IDispatch is synonymous with access to COM servers from
> scripting language environments like VB6.
> Compiled programs like C can happily access COM with the co called
> custom interfaces and no IDispatch interface.
> This is the case with Mozilla Firefox.
>
>> It looks like, as long as Firefox does not provide a COM interface for
>> Windows apps, any form of communication between screen readers and
>> Firefox
>> will be impossible, including any successful interaction with the
>> IAccessible MSAA object model...
>
> No. Most windows screen readers work just fine with Firefox via MSAA
> and IA, both of which are COM based technologies. You just can't use
> Windows scripting facilities w/o and IDispatch. For Screen reader
> hosted scripting the screen reader could provide the equivalent to
> IDispatch for you.
>
>> Which is truly unfortunate, since the lack of COM support for Windows
>> apps
>> is the only reason that I can see for Firefox usage not succeeding IE
>> usage
>> by screen reader users.
>
> Again it only effects access from scripting languages.
>
>> As Jamal pointed out, the importance of screen reader communication with
>> the
>> browser is very important, since it allows screen reader manufacturers to
>> customize screen reader behavior at runtime to maximize accessibility for
>> screen reader users.
>
> But only Windows screen readers, and only those that assume the
> browsers COM servers has IDispatch and so don't provide their own
> scripting access. I don't know JAWS but I'm guessing it uses
> WindowsScripting to provide the script environment form what has been
> said. That assumes IDispatch. So you could say that decision has meant
> only scripting for IDispatch servers.
>
>
> --
> Steve Lee
> Open Source Assistive Technology Software and Accessibility
> fullmeasure.co.uk

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: New: Firefox 3 with Screen Readers FAQ

T.V Raman
In reply to this post by Jason White-14

Jason, For now, hold off on Firefox 3 if you plan to use it with
Emacspeak.

Fire-Vox works with Firefox 3 -- however the mozrepl extension I
used to build the Firefox <-> Emacs bridge is presently
partially  unusable under Firefox 3.

When this changes, I'll blog about it on the emacspeak blog so
that the relevant community finds out about it. Until then, I
recommend thatEmacspeak  users working wih Fire-Vox stay with
Firefox 2.

Jason White writes:
 > On Wed, Jul 09, 2008 at 05:37:13PM +0200, Aaron Leventhal wrote:
 > > I'd like to hear more about Fire Vox from the community first.
 >
 > It's important, innovative and useful.
 >
 > I plan to install it and get it running with Emacspeak, at which point I'll
 > have more comments to add.
 >
 > It ought to be mentioned in the FAQ and in the list of accessibility-related
 > extensions.
 >
 > _______________________________________________
 > dev-accessibility mailing list
 > [hidden email]
 > https://lists.mozilla.org/listinfo/dev-accessibility

--
Best Regards,
--raman

Title:  Research Scientist      
Email:  [hidden email]
WWW:    http://emacspeak.sf.net/raman/
Google: tv+raman
GTalk:  [hidden email], [hidden email]
PGP:    http://emacspeak.sf.net/raman/raman-almaden.asc

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]

James Teh-2
In reply to this post by Marco Zehe-3
Jamal Mazrui wrote:
> Since at present, the great majority of blind computer users are
> operating on the Windows platform (whether in a home or employment
> setting), Let me urge Mozilla and its developer friends to implement
> IDispatch support for Firefox on Windows, preferably with access to the
> DOM, but at least with access to all the info available via MSAA.
Even with IDispatch, you still won't be able to access what you want
easily. Remember: JAWS provides GetIECurrentDocument() or whatever it is
called. It doesn't provide any such function for Firefox, so you have no
direct way of getting the document object for the currently loaded
document. You can probably get it if you grab the current focus, get its
accessible object and work backwards from there, however. My point is
that the screen reader still needs to provide *something* in order for
this to work. You are expecting Firefox to provide the same
functionality without any help from the screen reader. Granted, none of
this can work at all without IDispatch support.

 > In my
 > opinion, the lack of such support available to scripters of Windows
 > screen readers is a serious weakness compared to Internet Explorer.
While it is true that Firefox can't be scripted in the same way, I would
argue that an excessive need for scripting on the screen reader side
indicates a lack of functionality in the screen reader core.
Many of the examples you provided actually require knowledge of the
virtual cursor within the screen reader; e.g. finding the position n
characters from the start of the document. That is very much specific to
the screen reader, as the flat "virtual" representation of the document
is generated by the screen reader from the document hierarchy. Also,
starting a continuous read from a given position again requires that you
can route the virtual cursor to a particular node, which the screen
reader must support internally. Imo, it would be far more efficient for
the screen readers in question to implement functions which allow you to
manipulate their virtual buffers. This is the more correct and elegant
solution.

It was suggested that Firefox does not support COM. Allow me to try to
clarify this:
IAccessible (MSAA) and IAccessible2 both work via COM, so Firefox most
definitely supports COM. This is how NVDA accesses Firefox. It does not
provide a COM automation object like Microsoft Office, but rather, you
get access to its COM objects via functions such as
AccessibleObjectFromEvent, which returns an accessible object given some
event parameters.

Jamie

--
James Teh
Email: [hidden email]
WWW: http://www.jantrid.net/
MSN Messenger: [hidden email]
Jabber: [hidden email]
Yahoo: jcs_teh
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: New: Firefox 3 with Screen Readers FAQ

Charles Chen-2
In reply to this post by Aaron Leventhal-3

> Where does one get support for Fire Vox?
http://firevox.clcworld.net/contact.html
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]

Bryan Garaventa-2
In reply to this post by James Teh-2
I took a look at the MSDN documentation for AccessibleObjectFromEvent at
http://msdn.microsoft.com/en-us/library/ms697277(VS.85).aspx

Can you show me a code sample that displays how AccessibleObjectFromEvent
identifies the correct instance of Firefox and returns the IAccessible
object?

I'm not clear on which parameters are relevant to Firefox, or how to
implement this within a Windows app where the correct information is
retrieved.



--------------------------------------------------------------------------------
Bryan Garaventa
Senior Accessibility Engineer
SSB + BART Group
[hidden email]
www.SSBBartGroup.com
Accessibility-On-Demand


http://msdn.microsoft.com/en-us/library/ms697277(VS.85).aspx
AccessibleObjectFromEvent



----- Original Message -----
From: "James Teh" <[hidden email]>
Newsgroups: mozilla.dev.accessibility
To: <[hidden email]>
Sent: Thursday, July 10, 2008 8:47 PM
Subject: Re: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]


> Jamal Mazrui wrote:
>> Since at present, the great majority of blind computer users are
>> operating on the Windows platform (whether in a home or employment
>> setting), Let me urge Mozilla and its developer friends to implement
>> IDispatch support for Firefox on Windows, preferably with access to the
>> DOM, but at least with access to all the info available via MSAA.
> Even with IDispatch, you still won't be able to access what you want
> easily. Remember: JAWS provides GetIECurrentDocument() or whatever it is
> called. It doesn't provide any such function for Firefox, so you have no
> direct way of getting the document object for the currently loaded
> document. You can probably get it if you grab the current focus, get its
> accessible object and work backwards from there, however. My point is
> that the screen reader still needs to provide *something* in order for
> this to work. You are expecting Firefox to provide the same
> functionality without any help from the screen reader. Granted, none of
> this can work at all without IDispatch support.
>
> > In my
> > opinion, the lack of such support available to scripters of Windows
> > screen readers is a serious weakness compared to Internet Explorer.
> While it is true that Firefox can't be scripted in the same way, I would
> argue that an excessive need for scripting on the screen reader side
> indicates a lack of functionality in the screen reader core.
> Many of the examples you provided actually require knowledge of the
> virtual cursor within the screen reader; e.g. finding the position n
> characters from the start of the document. That is very much specific to
> the screen reader, as the flat "virtual" representation of the document
> is generated by the screen reader from the document hierarchy. Also,
> starting a continuous read from a given position again requires that you
> can route the virtual cursor to a particular node, which the screen
> reader must support internally. Imo, it would be far more efficient for
> the screen readers in question to implement functions which allow you to
> manipulate their virtual buffers. This is the more correct and elegant
> solution.
>
> It was suggested that Firefox does not support COM. Allow me to try to
> clarify this:
> IAccessible (MSAA) and IAccessible2 both work via COM, so Firefox most
> definitely supports COM. This is how NVDA accesses Firefox. It does not
> provide a COM automation object like Microsoft Office, but rather, you
> get access to its COM objects via functions such as
> AccessibleObjectFromEvent, which returns an accessible object given some
> event parameters.
>
> Jamie
>
> --
> James Teh
> Email: [hidden email]
> WWW: http://www.jantrid.net/
> MSN Messenger: [hidden email]
> Jabber: [hidden email]
> Yahoo: jcs_teh
> _______________________________________________
> dev-accessibility mailing list
> [hidden email]
> https://lists.mozilla.org/listinfo/dev-accessibility 

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

RE: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]

Jamal Mazrui
In reply to this post by James Teh-2
Hi Jamie,
I think IDispatch could be implemented in a way that does not require
additional JAWS or Window-Eyes scripting functions.  It could create a
ProgID that provides access to the current document via a GetObject
function, similar to how Microsoft Word or Excel do, e.g.,

oFirefox = GetObject("Firefox.Application")

Even if that functionality is not implemented, a screen reader
manufacturer is likely to add a single method like
FirefoxGetCurrentDocument if that is all that is required to allow
scripters to access the Firefox DOM or MSAA tree.

I agree that some of my examples would best be implemented by a mix of
screen reader and Firefox methods.  

Regarding NVDA, isn't the Firefox support the result of a Mozilla grant
that has involved months of creating a C++ library that Python calls?
If so, the Firefox access achieved in that way does not seem like a fair
comparison to what should be possible with a scripting language of a
screen reader.  In any case, can you give us the low level COM calls
necessary to gain access to Firefox MSAA content?

Regards,
Jamal


-----Original Message-----
From: dev-accessibility-bounces+jamal.mazrui=[hidden email]
[mailto:dev-accessibility-bounces+jamal.mazrui=[hidden email]
] On Behalf Of James Teh
Sent: Thursday, July 10, 2008 11:47 PM
To: [hidden email]
Subject: Re: Scriptability [was Re: New: Firefox 3 with Screen Readers
FAQ]

Jamal Mazrui wrote:
> Since at present, the great majority of blind computer users are
> operating on the Windows platform (whether in a home or employment
> setting), Let me urge Mozilla and its developer friends to implement
> IDispatch support for Firefox on Windows, preferably with access to
> the DOM, but at least with access to all the info available via MSAA.
Even with IDispatch, you still won't be able to access what you want
easily. Remember: JAWS provides GetIECurrentDocument() or whatever it is
called. It doesn't provide any such function for Firefox, so you have no
direct way of getting the document object for the currently loaded
document. You can probably get it if you grab the current focus, get its
accessible object and work backwards from there, however. My point is
that the screen reader still needs to provide *something* in order for
this to work. You are expecting Firefox to provide the same
functionality without any help from the screen reader. Granted, none of
this can work at all without IDispatch support.

 > In my
 > opinion, the lack of such support available to scripters of Windows
 > screen readers is a serious weakness compared to Internet Explorer.
While it is true that Firefox can't be scripted in the same way, I would

argue that an excessive need for scripting on the screen reader side
indicates a lack of functionality in the screen reader core.
Many of the examples you provided actually require knowledge of the
virtual cursor within the screen reader; e.g. finding the position n
characters from the start of the document. That is very much specific to

the screen reader, as the flat "virtual" representation of the document
is generated by the screen reader from the document hierarchy. Also,
starting a continuous read from a given position again requires that you

can route the virtual cursor to a particular node, which the screen
reader must support internally. Imo, it would be far more efficient for
the screen readers in question to implement functions which allow you to

manipulate their virtual buffers. This is the more correct and elegant
solution.

It was suggested that Firefox does not support COM. Allow me to try to
clarify this:
IAccessible (MSAA) and IAccessible2 both work via COM, so Firefox most
definitely supports COM. This is how NVDA accesses Firefox. It does not
provide a COM automation object like Microsoft Office, but rather, you
get access to its COM objects via functions such as
AccessibleObjectFromEvent, which returns an accessible object given some

event parameters.

Jamie

--
James Teh
Email: [hidden email]
WWW: http://www.jantrid.net/
MSN Messenger: [hidden email]
Jabber: [hidden email]
Yahoo: jcs_teh
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]

Michael Curran-2
Hi,

To explain a little clearer the problems between Jaws scripting and
Firefox:
Internally Jaws interacts with Gecko applications using a mixture of
MSAA, ISimpleDom* interfaces (Gecko specific accessibility COM
interfaces) and IAccessible2 (though its not clear how much Jaws really
does use this in Gecko).

Jaws interacts with these COM interfaces (including MSAA) most likely by
including the interface definitions at compile time. What this means is
that internally, Jaws knows all MSAA's possible methods, and can use
them internally when ever it likes.

However, Jaws scripting is another case.

Even though Jaws internally uses MSAA to read the focus etc in Gecko
apps, when using scripting api functions such as getCurrentObject,
getObjectFromEvent etc, it returns an object to the script that *must*
support IDispatch. So even though Jaws internally does not need
IDispatch on the MSAA objects, as far as scripting goes, it seems to be
the case that it does need IDispatch.

I only assume this as getObjectFromEvent, getCurrentObject etc don't
return anything useful in Gecko apps, but they work everywhere else.

There could be two alternative solutions to this problem:
1. Jaws should allow scripters to use MSAA objects with out IDispatch,
as it doesn't need it for its own use anyway. This could be possible by
either assuming that getCurrentObject always returns an MSAA object. and
that getObjectFromEvent (as long as you pass it an objectID of either
OBJID_WINDOW,OBJID_SYSMENU, OBJID_TITLEBAR, OBJID_MENU, OBJID_CLIENT,
OBJID_HSCROLL, OBJID_VSCROLL, OBJID_CARET, or OBJID_CURSOR, should also
be assumed to return an MSAA object.
2. Gecko should implement IDispatch for its objects. This would allow
current Jaws scripting to use methods on these objects, from MSAA,
ISimpleDom* and IAccessible2. Let it also be noted that technically
MSAA's IAccessible interface inherits from IDispatch according to
oleacc.idl, so for completeness, Gecko would need to support that
inheritence.

However, this does bring up the issue of IAccessible2 not inheriting
from IDispatch according to its specification. Note though that IBM
Lotus product implementations of IAccessible2 support IDispatch, so if
Gecko was also willing to do this, I would suggest that the spec be
changed to reflect this.

Adding IDispatch To Gecko Accessibility is not as easy as some people
think. Apart from writing the code, There needs to be a lot of testing
to make sure it does not break compatibility with ATs, including Jaws.

In regards to using something like GetObject("firefox.application") or
something like that, this is a completely different thing. This I think
goes further than Gecko accessibility, and is a lot more work. Plus it
locks down Firefox to the system its running on i.e. it wouldn't be able
to be used portably from a USB key etc as firefox.application must be
registered with the system for it to work. Also note that IDispatch
would still need to be implemented as well. Jaws certainly does not need
this extra stuff, neither does any other AT.


It was mentioned that NVDA received a grant from Mozilla to write c++
virtualBuffers. This is true, however, forgetting about virtualBuffers
for a second, NVDA still needs to, and always has been able to, interact
with Gecko applications using MSAA and IAccessible2, directly from
Python. It is definitely possible for VB, Python, Java, and any other
scripting language that supports type libraries, to use MSAA, or
IAccessible2.

Please feel free to browse NVDA's source code, which can be found at:
http://www.nvda-project.org/
and going to the view source link on the navbar.

People wanted examples of how they could get a gecko document.

If either Jaws supported pure MSAA for scripting, or Gecko implemented
IDispatch, in a jaws script to get the current Gecko document you could do:
let o = getObjectFromEvent(window,-4,0,realChildID)
Where window is a MozillaContentWindowClass window (which can be easily
located using normal Jaws window functions), -4 means OBJID_CLIENT (look
it up on msdn), and realChildID, is the variable where Jaws will place
the resulting childID you must use with any methods on o.

I apologize if that looks a little complex or I'm not explaining myself
enough, but I knew absolutely nothing about MSAA or Windows programming
a few years ago. But, through writing JFWTechnical, and wanting to
create NVDA (and actually doing it), I have forced myself to learn it
all. Simply by reading MSDN, reading the great documentation there is on
Gecko accessibility, and just true trial and error.

*all* NVDA's source code is available for all to see. A part from it
being the right thing to do, I also did this so others can learn from
what Jamie and I, and others have learnt over the life of the project so
far.

I hope this clears a few things up for people.

The bottom line is that the Gecko accessibility team must decide whether
IDispatch support is important enough. And if it happens that they
don't, then ATs such ass Jaws still have a choice to get around this
problem.

Mick









_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]

James Teh-2
In reply to this post by Jamal Mazrui
Michael Curran wrote:
> It was mentioned that NVDA received a grant from Mozilla to write c++
> virtualBuffers. This is true, however, forgetting about virtualBuffers
> for a second, NVDA still needs to, and always has been able to, interact
> with Gecko applications using MSAA and IAccessible2, directly from
> Python.
It should also be noted that our original Gecko virtual buffer code was
implemented entirely in Python. It wasn't anywhere near as good as the
current solution, but it certainly had some limited useability.

Also, this brings me back to my point regarding scripting for virtual
buffers. The virtual buffer internals in NVDA can be accessed by any
custom appModules (our equivalent of JAWS script files). This allows one
to interact with the virtual representation presented to the user. It
might perhaps be benefitial to urge screen reader manufacturers to
provide access to their virtual buffers from their scripting languages
so that scripters can work with the virtual buffers themselves. Aside
from providing control over the virtual buffer itself and eliminating
the need for scripters to work directly with the DOM or accessibility
APIs, this would also allow scripts to be written which should work with
little to no change between different browsers.

Jamie

--
James Teh
Email: [hidden email]
WWW: http://www.jantrid.net/
MSN Messenger: [hidden email]
Jabber: [hidden email]
Yahoo: jcs_teh
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]

Jason White-14
On Sat, Jul 12, 2008 at 04:47:37PM +1000, James Teh wrote:
> It should also be noted that our original Gecko virtual buffer code was
> implemented entirely in Python. It wasn't anywhere near as good as the
> current solution, but it certainly had some limited useability.

I'm not a Windows user, but I'm curious anyway: why do screen readers for that
environment implement virtual buffers for Web content? I use Orca under Linux,
which doesn't need any virtual buffers, and this is a good thing.

Virtual buffers remind me of off-screen models and all the reasons why they
are such a bad idea, especially when client-side Javascript comes along and
changes the document via the DOM.

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Virtual buffers in Windows [was Scriptability]

James Teh-2
In reply to this post by James Teh-2
Jason White wrote:
> I'm not a Windows user, but I'm curious anyway: why do screen readers for that
> environment implement virtual buffers for Web content? I use Orca under Linux,
> which doesn't need any virtual buffers, and this is a good thing.
Three reasons that immediately come to mind:
* Although virtual buffers involve a slight rendering delay (which is
usually unnoticeable in our implementation now), once rendered, they
allow for instantaneous access to any portion of the page. One never has
to wait for a complex part of the document to be rendered into a
readable form. While I haven't personally used Orca and Firefox much, i
understand that there can occasionally be some delay while moving
throughout the page, although corrections are of course welcome.
* Cursor navigation in a virtual buffer is similar to that of any flat
document. You can move by character, word, line, block, etc. Cursor
navigation is always symmetrical; i.e. if one moves the cursor forward
in a certain way, reversing those movements will always land you back
where you started unless the document changed during the movement.
Again, I don't have any real experience with Orca and Firefox, but I
understand that the Firefox caret is not always symmetrical in this fashion.
* Virtual buffers allow for very fast "quick navigation"; i.e. moving to
headings, links and other page elements.

> Virtual buffers remind me of off-screen models and all the reasons why they
> are such a bad idea, especially when client-side Javascript comes along and
> changes the document via the DOM.
This is a fair point. We have ot be very careful to make sure we keep
the document up to date in response to Firefox events. We do this by
associating each portion of our flat buffer with the respective nodes in
the DOM.

Jamie

--
James Teh
Email: [hidden email]
WWW: http://www.jantrid.net/
MSN Messenger: [hidden email]
Jabber: [hidden email]
Yahoo: jcs_teh
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Virtual buffers in Windows [was Scriptability]

Jason White-14
On Sun, Jul 13, 2008 at 11:27:52AM +1000, James Teh wrote:
> Three reasons that immediately come to mind:
> * Although virtual buffers involve a slight rendering delay (which is
> usually unnoticeable in our implementation now), once rendered, they
> allow for instantaneous access to any portion of the page. One never has
> to wait for a complex part of the document to be rendered into a
> readable form. While I haven't personally used Orca and Firefox much, i
> understand that there can occasionally be some delay while moving
> throughout the page, although corrections are of course welcome.

With a modern, fast machine, I haven't noticed any delays peculiar to Firefox.
There are speech-related delays involving gnome-speech and eSpeak, but a patch
has been circulating to correct those. Again, I haven't been able to reproduce
them on my quad-core desktop system, although on my single-core laptop, this
issue does become slightly noticable.
> * Cursor navigation in a virtual buffer is similar to that of any flat
> document. You can move by character, word, line, block, etc. Cursor
> navigation is always symmetrical; i.e. if one moves the cursor forward
> in a certain way, reversing those movements will always land you back
> where you started unless the document changed during the movement.
> Again, I don't have any real experience with Orca and Firefox, but I
> understand that the Firefox caret is not always symmetrical in this fashion.

Correct, but Orca by default moves the focus to the beginning of the line as
this is what users mostly desire according to the Orca developers.  With this
feature switched off, my understanding is that the cursor does arrive at the
same point in the document when navigational actions are reversed - barring
bugs, of course.
> * Virtual buffers allow for very fast "quick navigation"; i.e. moving to
> headings, links and other page elements.

Orca's structural navigation is very fast indeed.

I think the Orca developers were right not to create virtual buffers, and I
applaud this decision.

this also simplifies the user interface by eliminating multiple "modes" of
interaction.

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Virtual buffers in Windows [was Scriptability]

James Teh-2
In reply to this post by James Teh-2
Jason White wrote:
>> * Cursor navigation in a virtual buffer is similar to that of any flat
>> document. You can move by character, word, line, block, etc. Cursor
>> navigation is always symmetrical; i.e. if one moves the cursor forward
>> in a certain way, reversing those movements will always land you back
>> where you started unless the document changed during the movement.
> Correct, but Orca by default moves the focus to the beginning of the line as
> this is what users mostly desire according to the Orca developers.
Actually, a correction to my own comment: Virtual buffer navigation in
NVDA is not symmetrical in the same way; i.e. it moves to the start of
the line as well. I really can't give a fair comparison between NVDA and
Orca with Firefox, as I haven't used Orca enough.

> I think the Orca developers were right not to create virtual buffers, and I
> applaud this decision.
Note that virtual buffers under ATK would be slow anyway because there
is no concept of in-process under Linux. In Windows, there is this
horrible idea of injecting one's code into another process and executing
it there. This is how we do fast virtual buffer rendering under Windows.
Out-of-process virtual buffer rendering is extremely slow; this was our
old approach. I suspect that out-of-process functionality in Windows is
also somewhat slower than in Linux, which is another reason that
on-the-fly rendering is problematic in Windows.


> this also simplifies the user interface by eliminating multiple "modes" of
> interaction.
I would argue that this is debatable; some people prefer multiple modes,
others do not. This is an argument for a different thread. :) This
aside, virtual buffers do not necessarily imply multiple modes of
interaction. I believe System Access under Windows uses virtual buffers,
but they do not employ multiple modes of interaction.

Jamie

--
James Teh
Email: [hidden email]
WWW: http://www.jantrid.net/
MSN Messenger: [hidden email]
Jabber: [hidden email]
Yahoo: jcs_teh
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Virtual buffers in Windows [was Scriptability]

Michael Curran-2
In reply to this post by Jason White-14
Hi Jason,

On Windows out-of-process access to accessibility APIs such as MSAA and
IAccessible2 is most likely some what slower than access to ATK in Unix.

Note that IAccessible2 also does not contain the collection interface
like ATK does, so jumping by particular node types such as headings
would be extreamly slow.

We have considered not using virtualBuffers for NVDA many times. But
each time we find its just too slow.

I think it is great that Orca is able to do it in Unix, and if it works
for Unix blind users then that is also great.

But, on Windows, it is just simply the case that the fastest and most
efficient way of providing access to complex documents to blind users is
via virtualBuffers.

Note that Jaws, Window Eyes, Dolphan Hal, System Access, and NVDA  all
use some form of virtualBuffers.

Even Talks (Symbian phone screen reader) now uses virtual Buffers.

Please forgive me if I'm wrong here, but isn't there some form of bridge
that can be used from Firefox to Emacspeak? if so, this to me seems very
much like virtualBuffers. No doubt the emacs buffer would  have to be
updated each time a node changed in the Gecko DOM. We both have to deal
with the same problems. However, as long as we pay attention to all
events, I can't really see a way it can get out of sync.

Many many blind users on Windows are very happy with the page model that
virtualBuffers provide. We can debate for ever whether its the most
efficient choice for a given Operating System. But at the end of the
day, blind Windows users when using a Windows screen reader come to
expect this model.

I am most certainly happy to investigate a non-virtualBuffer way for
NVDA to access web documents, if the community at large thought it was
worth it. In fact we will need to do this anyway for access to complex
editable documents in Open Office and IBM Lotus Symphony etc. Perhaps
once that work is done, then people could have the choice to use either
interface.

But, virtualBuffers were the easiest and most efficient way of giving
access to Firefox via NVDA I knew of.

Mick

Jason White wrote:

> On Sun, Jul 13, 2008 at 11:27:52AM +1000, James Teh wrote:
>> Three reasons that immediately come to mind:
>> * Although virtual buffers involve a slight rendering delay (which is
>> usually unnoticeable in our implementation now), once rendered, they
>> allow for instantaneous access to any portion of the page. One never has
>> to wait for a complex part of the document to be rendered into a
>> readable form. While I haven't personally used Orca and Firefox much, i
>> understand that there can occasionally be some delay while moving
>> throughout the page, although corrections are of course welcome.
>
> With a modern, fast machine, I haven't noticed any delays peculiar to Firefox.
> There are speech-related delays involving gnome-speech and eSpeak, but a patch
> has been circulating to correct those. Again, I haven't been able to reproduce
> them on my quad-core desktop system, although on my single-core laptop, this
> issue does become slightly noticable.
>> * Cursor navigation in a virtual buffer is similar to that of any flat
>> document. You can move by character, word, line, block, etc. Cursor
>> navigation is always symmetrical; i.e. if one moves the cursor forward
>> in a certain way, reversing those movements will always land you back
>> where you started unless the document changed during the movement.
>> Again, I don't have any real experience with Orca and Firefox, but I
>> understand that the Firefox caret is not always symmetrical in this fashion.
>
> Correct, but Orca by default moves the focus to the beginning of the line as
> this is what users mostly desire according to the Orca developers.  With this
> feature switched off, my understanding is that the cursor does arrive at the
> same point in the document when navigational actions are reversed - barring
> bugs, of course.
>> * Virtual buffers allow for very fast "quick navigation"; i.e. moving to
>> headings, links and other page elements.
>
> Orca's structural navigation is very fast indeed.
>
> I think the Orca developers were right not to create virtual buffers, and I
> applaud this decision.
>
> this also simplifies the user interface by eliminating multiple "modes" of
> interaction.
>
> _______________________________________________
> dev-accessibility mailing list
> [hidden email]
> https://lists.mozilla.org/listinfo/dev-accessibility
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Virtual buffers in Windows [was Scriptability]

Aaron Leventhal-3
In reply to this post by Jason White-14
Jason,

ATK/AT-SPI supports something called the collection interface, which
works in-process to respond to queries from the AT. That's why Orca can
navigate quickly to headings etc. in Firefox.

This interface does not exist in IA2. If it did, each app would have to
support it, whereas in ATK/AT_SPI, there is a bridge anyway, which sits
in the app's process, and can do this work for us. Apps like Firefox
don't have to do anything to support collection under ATK/AT-SPI.

On Windows this same functionality is essentially implemented by each
screen reader vendors in their own way. The inject themselves into the
process and can decide exactly what processing to do there and what to
pass "over the wall" across processes. Each has their own reasons for
whatever design they choose.

So while this doesn't mean that Windows vendors must use virtual buffers
or have two modes of interaction, they definitely need to inject
themselves into the process for performance. I'm probably not telling
you anything new, but there you have it :)

- Aaron


_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Virtual buffers in Windows [was Scriptability]

Jason White-14
On Mon, Jul 14, 2008 at 07:10:12AM +0200, Aaron Leventhal wrote:
> ATK/AT-SPI supports something called the collection interface, which
> works in-process to respond to queries from the AT. That's why Orca can
> navigate quickly to headings etc. in Firefox.

Thanks Aaron; that's the clearest explanation of the collection interface that
I've read. I will certainly take this into account if the collection interface
is discussed in my forthcoming Linux User's Group presentation focusing on
Firefox 3 and Orca.

I plan to mention Fire Vox and Google-Axsjax as well. Toward the end of the
presentation I'll note the work that is being carried out to move ATK/AT-SPI
to DBus as the inter-process communication mechanism, and efforts to make
WebKit accessible via the Gnome accessibility infrastructure.

Suggestions for topics to include or exclude are, as ever, welcome.

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Virtual buffers in Windows [was Scriptability]

Aaron Leventhal-3
In reply to this post by Aaron Leventhal-3
Jason White wrote:
> Suggestions for topics to include or exclude are, as ever, welcome.

I think it's important to at least mention WAI-ARIA and Web 2.0
accessibility and what it means to the industry. You can probably get
notes from Steve Faulkner since he's presented on that before.

If you're talking to mostly developers I'd say the big thing is we now
have an open platform with which to advance accessibility up and down
the whole stack -- create new capabilities for developers and provide
those features to end users.

- Aaron

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Virtual buffers in Windows [was Scriptability]

T.V Raman
In reply to this post by Jason White-14
well said. In addition, as  the Web transitions from pure
documents to user interfaces and interactive applications, the
model of navigating flat, static  documents breaks down --- in
general having two modes between which the user  has to
constantly switch is not something that will work well with
Web applications that implement all their widgets via DHTML. As a
case in point, try playing with any of the initial ARIA code
examples for complex widgets --- you *never* want to have to
remember which mode the screenreader you're using is in --- in
general, you want to interact with the Web Application as any
other mainstream user does, namely, by interacting with the page,
rather than interacting through an intermediary.

Jason White writes:
 > On Sun, Jul 13, 2008 at 11:27:52AM +1000, James Teh wrote:
 > > Three reasons that immediately come to mind:
 > > * Although virtual buffers involve a slight rendering delay (which is
 > > usually unnoticeable in our implementation now), once rendered, they
 > > allow for instantaneous access to any portion of the page. One never has
 > > to wait for a complex part of the document to be rendered into a
 > > readable form. While I haven't personally used Orca and Firefox much, i
 > > understand that there can occasionally be some delay while moving
 > > throughout the page, although corrections are of course welcome.
 >
 > With a modern, fast machine, I haven't noticed any delays peculiar to Firefox.
 > There are speech-related delays involving gnome-speech and eSpeak, but a patch
 > has been circulating to correct those. Again, I haven't been able to reproduce
 > them on my quad-core desktop system, although on my single-core laptop, this
 > issue does become slightly noticable.
 > > * Cursor navigation in a virtual buffer is similar to that of any flat
 > > document. You can move by character, word, line, block, etc. Cursor
 > > navigation is always symmetrical; i.e. if one moves the cursor forward
 > > in a certain way, reversing those movements will always land you back
 > > where you started unless the document changed during the movement.
 > > Again, I don't have any real experience with Orca and Firefox, but I
 > > understand that the Firefox caret is not always symmetrical in this fashion.
 >
 > Correct, but Orca by default moves the focus to the beginning of the line as
 > this is what users mostly desire according to the Orca developers.  With this
 > feature switched off, my understanding is that the cursor does arrive at the
 > same point in the document when navigational actions are reversed - barring
 > bugs, of course.
 > > * Virtual buffers allow for very fast "quick navigation"; i.e. moving to
 > > headings, links and other page elements.
 >
 > Orca's structural navigation is very fast indeed.
 >
 > I think the Orca developers were right not to create virtual buffers, and I
 > applaud this decision.
 >
 > this also simplifies the user interface by eliminating multiple "modes" of
 > interaction.
 >
 > _______________________________________________
 > dev-accessibility mailing list
 > [hidden email]
 > https://lists.mozilla.org/listinfo/dev-accessibility

--
Best Regards,
--raman

Title:  Research Scientist      
Email:  [hidden email]
WWW:    http://emacspeak.sf.net/raman/
Google: tv+raman
GTalk:  [hidden email], [hidden email]
PGP:    http://emacspeak.sf.net/raman/raman-almaden.asc

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Virtual buffers in Windows [was Scriptability]

Michael Curran-2
A major reason why NVDA does not render virtualBuffers if the page has
an ARIA role of application, rather than document.

NVDA uses virtualBuffers and nothing can change this right now, but, if
app developers write a web application that should feel like an
application, then my belief is that they must set role application.  ATs
must be able to tell the difference between documents and applications.
Its not simply just whether or not to use virtualBuffers, its actually
to understand whether this is a document or application.

Mick


T.V Raman wrote:

> well said. In addition, as  the Web transitions from pure
> documents to user interfaces and interactive applications, the
> model of navigating flat, static  documents breaks down --- in
> general having two modes between which the user  has to
> constantly switch is not something that will work well with
> Web applications that implement all their widgets via DHTML. As a
> case in point, try playing with any of the initial ARIA code
> examples for complex widgets --- you *never* want to have to
> remember which mode the screenreader you're using is in --- in
> general, you want to interact with the Web Application as any
> other mainstream user does, namely, by interacting with the page,
> rather than interacting through an intermediary.
>
> Jason White writes:
>   >  On Sun, Jul 13, 2008 at 11:27:52AM +1000, James Teh wrote:
>   >  >  Three reasons that immediately come to mind:
>   >  >  * Although virtual buffers involve a slight rendering delay (which is
>   >  >  usually unnoticeable in our implementation now), once rendered, they
>   >  >  allow for instantaneous access to any portion of the page. One never has
>   >  >  to wait for a complex part of the document to be rendered into a
>   >  >  readable form. While I haven't personally used Orca and Firefox much, i
>   >  >  understand that there can occasionally be some delay while moving
>   >  >  throughout the page, although corrections are of course welcome.
>   >
>   >  With a modern, fast machine, I haven't noticed any delays peculiar to Firefox.
>   >  There are speech-related delays involving gnome-speech and eSpeak, but a patch
>   >  has been circulating to correct those. Again, I haven't been able to reproduce
>   >  them on my quad-core desktop system, although on my single-core laptop, this
>   >  issue does become slightly noticable.
>   >  >  * Cursor navigation in a virtual buffer is similar to that of any flat
>   >  >  document. You can move by character, word, line, block, etc. Cursor
>   >  >  navigation is always symmetrical; i.e. if one moves the cursor forward
>   >  >  in a certain way, reversing those movements will always land you back
>   >  >  where you started unless the document changed during the movement.
>   >  >  Again, I don't have any real experience with Orca and Firefox, but I
>   >  >  understand that the Firefox caret is not always symmetrical in this fashion.
>   >
>   >  Correct, but Orca by default moves the focus to the beginning of the line as
>   >  this is what users mostly desire according to the Orca developers.  With this
>   >  feature switched off, my understanding is that the cursor does arrive at the
>   >  same point in the document when navigational actions are reversed - barring
>   >  bugs, of course.
>   >  >  * Virtual buffers allow for very fast "quick navigation"; i.e. moving to
>   >  >  headings, links and other page elements.
>   >
>   >  Orca's structural navigation is very fast indeed.
>   >
>   >  I think the Orca developers were right not to create virtual buffers, and I
>   >  applaud this decision.
>   >
>   >  this also simplifies the user interface by eliminating multiple "modes" of
>   >  interaction.
>   >
>   >  _______________________________________________
>   >  dev-accessibility mailing list
>   >  [hidden email]
>   >  https://lists.mozilla.org/listinfo/dev-accessibility
>
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Virtual buffers in Windows [was Scriptability]

James Teh-2
In reply to this post by Jason White-14
T.V Raman wrote:
> well said. In addition, as  the Web transitions from pure
> documents to user interfaces and interactive applications, the
> model of navigating flat, static  documents breaks down --- in
> general having two modes between which the user  has to
> constantly switch is not something that will work well with
> Web applications that implement all their widgets via DHTML.
I would argue that documents and applications are conceptually different
in some ways. While the borders are thinning, they are perhaps not as
fine as some might think. For example, a news article, blog entry, etc.
is very much a document and is easier to read as such. However, advanced
web mail clients such as GMail are more like applications which display
email messages as documents. Following this logic, the main part of the
"application" should have an ARIA role of application, while the mail
message portion should have a role of document. Screen readers (and
other ATS) which do provide other functionality in documents should then
obey the ARIA role and not treat an element with a role of application
as a document. NVDA obeys this rule.

Jamie


--
James Teh
Email: [hidden email]
WWW: http://www.jantrid.net/
MSN Messenger: [hidden email]
Jabber: [hidden email]
Yahoo: jcs_teh
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Virtual buffers in Windows [was Scriptability]

David Bolter
In reply to this post by T.V Raman
+1 for 1 mode!  Is this on the roadmap for commercial screen readers?

cheers,
David

T.V Raman wrote:

> well said. In addition, as  the Web transitions from pure
> documents to user interfaces and interactive applications, the
> model of navigating flat, static  documents breaks down --- in
> general having two modes between which the user  has to
> constantly switch is not something that will work well with
> Web applications that implement all their widgets via DHTML. As a
> case in point, try playing with any of the initial ARIA code
> examples for complex widgets --- you *never* want to have to
> remember which mode the screenreader you're using is in --- in
> general, you want to interact with the Web Application as any
> other mainstream user does, namely, by interacting with the page,
> rather than interacting through an intermediary.
>
> Jason White writes:
>  > On Sun, Jul 13, 2008 at 11:27:52AM +1000, James Teh wrote:
>  > > Three reasons that immediately come to mind:
>  > > * Although virtual buffers involve a slight rendering delay (which is
>  > > usually unnoticeable in our implementation now), once rendered, they
>  > > allow for instantaneous access to any portion of the page. One never has
>  > > to wait for a complex part of the document to be rendered into a
>  > > readable form. While I haven't personally used Orca and Firefox much, i
>  > > understand that there can occasionally be some delay while moving
>  > > throughout the page, although corrections are of course welcome.
>  >
>  > With a modern, fast machine, I haven't noticed any delays peculiar to Firefox.
>  > There are speech-related delays involving gnome-speech and eSpeak, but a patch
>  > has been circulating to correct those. Again, I haven't been able to reproduce
>  > them on my quad-core desktop system, although on my single-core laptop, this
>  > issue does become slightly noticable.
>  > > * Cursor navigation in a virtual buffer is similar to that of any flat
>  > > document. You can move by character, word, line, block, etc. Cursor
>  > > navigation is always symmetrical; i.e. if one moves the cursor forward
>  > > in a certain way, reversing those movements will always land you back
>  > > where you started unless the document changed during the movement.
>  > > Again, I don't have any real experience with Orca and Firefox, but I
>  > > understand that the Firefox caret is not always symmetrical in this fashion.
>  >
>  > Correct, but Orca by default moves the focus to the beginning of the line as
>  > this is what users mostly desire according to the Orca developers.  With this
>  > feature switched off, my understanding is that the cursor does arrive at the
>  > same point in the document when navigational actions are reversed - barring
>  > bugs, of course.
>  > > * Virtual buffers allow for very fast "quick navigation"; i.e. moving to
>  > > headings, links and other page elements.
>  >
>  > Orca's structural navigation is very fast indeed.
>  >
>  > I think the Orca developers were right not to create virtual buffers, and I
>  > applaud this decision.
>  >
>  > this also simplifies the user interface by eliminating multiple "modes" of
>  > interaction.
>  >
>  > _______________________________________________
>  > dev-accessibility mailing list
>  > [hidden email]
>  > https://lists.mozilla.org/listinfo/dev-accessibility
>
>  

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
1234