New: Firefox 3 with Screen Readers FAQ

classic Classic list List threaded Threaded
65 messages Options
1234
Reply | Threaded
Open this post in threaded view
|

Re: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]

Steve Lee-3
James: IDispatch is not supported though I recall a  discussion about
this some time back. That does exclude scripting environment access.

Bryan, the accessibility APIs (e.g MSAA) are designed to give ATs
access to applications GUIs and in the case of a web browser that
obviously includes the web document.

IMHO scripting of the DOM through COM is quite a different requirement
and as Firefox is a cross platform product would require extra
specific support for one platform that has nothing to do with
accessibility per se (CORBA should then probably be supported as well
for Linux). IE obviously has this as it is the object model used on
that platform and the only possibility in a closed source environment.

Steve

2008/7/10 Bryan Garaventa <[hidden email]>:

> Thanks, that explanation helps.
>
> Does anyone have a way of doing this from VB6? For instance, some sample
> code that shows how to access the IAccessible interface through this method?
>
> If it is possible for me to make a function within VB6 that can access the
> current running instance of Firefox, and then access its IAccessible
> interface, would it be possible for me to get the DOM from this? I know I'm
> missing a few steps here, for instance where IDL fits into the picture, but
> if this is possible, it may go a long way to solving the problem. At least
> for me that is.
>
> It is easy for me to create an ActiveX control that I can instantiate from
> within JAWS, and then pass and receive data from the object's methods and
> functions.
>
> So if I could create a function in VB6 that detects the current browser
> window, perhaps by passing the window handle to the ActiveX control, it
> could presumably access the IAccessible interface using IDispatch, then do
> what is necessary to get a copy of the DOM as mentioned before. This could
> then be stored into a public property within the ActiveX. The DOM should be
> bound at that point while the object remains instantiated within the JAWS
> script.
>
> I've done this before using IE, and successfully retrieved the document
> object.
>
> Again, I think I'm missing some pretty important points to get it working,
> such as the part that refers to the IDL lib, so some further info would be
> much appreciated. I assume I need to generate the IDL dll lib and use this
> as a reference within the VB6 ActiveX? This would make sense if so.
>
> Thanks,
>
> Bryan
>
>
> --------------------------------------------------------------------------------
> Bryan Garaventa
> Senior Accessibility Engineer
> SSB + BART Group
> [hidden email]
> www.SSBBartGroup.com
> Accessibility-On-Demand
>
> ----- Original Message -----
> From: "James Teh" <[hidden email]>
> Newsgroups: mozilla.dev.accessibility
> To: <[hidden email]>
> Sent: Wednesday, July 09, 2008 5:21 PM
> Subject: Re: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]
>
>
>> In Python, you can access any COM interface specified in idl if you can
>> generate a typelib for it. This is how NVDA accesses MSAA and
>> IAccessible2. Note that we do not use ISimpleDOM.
>>
>> As far as I know, in order to access COM objects from JAWS, they need to
>> support IDispatch. As I understand it, Mozilla does not support
>> IDispatch on its accessibility COM objects.
>>
>> Although you can access MSAA objects from JAWS (again, assuming they
>> support IDispatch), you cannot use QueryInterface because QueryInterface
>> accepts an interface, which JAWS scripting can't support. There is no
>> way to add a property to access ISimpleDOM from MSAA, because MSAA
>> (IAccessible) is a predefined interface; you can't add arbitrary
>> properties without creating a new interface (which may or may not be a
>> subclass of an existing interface).
>>
>> Even if Mozilla did support IDispatch on its COM objects, screen readers
>> really should provide a way to get access to the current Firefox
>> document object, similar to the way they do for IE> Without this, the
>> only way to access the current document would be to provide some sort of
>> COM automation object which could be retrieved with GetActiveObject
>> (known as GetObject in JAWS). This could be potentially unreliable. It
>> could perhaps be improved by accepting the root window handle of the
>> document or some such. I still think screen reader scripting languages
>> need to provide some sort of support; it cannot properly be done without
>> something from the screen readers.
>>
>> Jamie
>>
>> Aaron Leventhal wrote:
>>> Bryan,
>>>
>>> Forgive me for being done. What would the property be on? You can get to
>>> the ISimpleDOMNode tree by taking taking any IAccessible* and doing
>>> QueryInterface to ISimpleDOMNode.
>>>
>>> It's scriptable if the screen reader includes the IDL and binds it. I
>>> don't know exactly how it's done, but NVDA uses all of Firefox's a11y
>>> interfaces in script.
>>>
>>> Keep in mind, that one difference with IE is that Firefox only uses COM
>>> for the Windows a11y stuff. Com is not usable for a general solution in
>>> a cross platform product.
>>>
>>> - Aaron
>>
>> --
>> James Teh
>> Email: [hidden email]
>> WWW: http://www.jantrid.net/
>> MSN Messenger: [hidden email]
>> Jabber: [hidden email]
>> Yahoo: jcs_teh
>> _______________________________________________
>> dev-accessibility mailing list
>> [hidden email]
>> https://lists.mozilla.org/listinfo/dev-accessibility
>
> _______________________________________________
> dev-accessibility mailing list
> [hidden email]
> https://lists.mozilla.org/listinfo/dev-accessibility
>



--
Steve Lee
Open Source Assistive Technology Software and Accessibility
fullmeasure.co.uk
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]

Steve Lee-3
I feel I should expand on that a bit.

I agree that being able to access the DOM from scripting really opens
up possibilities for accessibility. My own program PowerTalk works by
scripting the PowerPoint DOM and I chose that rather than MSAA as it
gives more control. However the decision to add COM IDispatch access
to the Firefox DOM really goes beyond accessibility and is not a small
task.

Also the model exposed through IAccessible (and IA2) parallels the DOM
to a large extent, though the lack of IDispatch still limits where you
can access it  from.

Steve

2008/7/10 Steve Lee <[hidden email]>:

> James: IDispatch is not supported though I recall a  discussion about
> this some time back. That does exclude scripting environment access.
>
> Bryan, the accessibility APIs (e.g MSAA) are designed to give ATs
> access to applications GUIs and in the case of a web browser that
> obviously includes the web document.
>
> IMHO scripting of the DOM through COM is quite a different requirement
> and as Firefox is a cross platform product would require extra
> specific support for one platform that has nothing to do with
> accessibility per se (CORBA should then probably be supported as well
> for Linux). IE obviously has this as it is the object model used on
> that platform and the only possibility in a closed source environment.
>
> Steve
>
> 2008/7/10 Bryan Garaventa <[hidden email]>:
>> Thanks, that explanation helps.
>>
>> Does anyone have a way of doing this from VB6? For instance, some sample
>> code that shows how to access the IAccessible interface through this method?
>>
>> If it is possible for me to make a function within VB6 that can access the
>> current running instance of Firefox, and then access its IAccessible
>> interface, would it be possible for me to get the DOM from this? I know I'm
>> missing a few steps here, for instance where IDL fits into the picture, but
>> if this is possible, it may go a long way to solving the problem. At least
>> for me that is.
>>
>> It is easy for me to create an ActiveX control that I can instantiate from
>> within JAWS, and then pass and receive data from the object's methods and
>> functions.
>>
>> So if I could create a function in VB6 that detects the current browser
>> window, perhaps by passing the window handle to the ActiveX control, it
>> could presumably access the IAccessible interface using IDispatch, then do
>> what is necessary to get a copy of the DOM as mentioned before. This could
>> then be stored into a public property within the ActiveX. The DOM should be
>> bound at that point while the object remains instantiated within the JAWS
>> script.
>>
>> I've done this before using IE, and successfully retrieved the document
>> object.
>>
>> Again, I think I'm missing some pretty important points to get it working,
>> such as the part that refers to the IDL lib, so some further info would be
>> much appreciated. I assume I need to generate the IDL dll lib and use this
>> as a reference within the VB6 ActiveX? This would make sense if so.
>>
>> Thanks,
>>
>> Bryan
>>
>>
>> --------------------------------------------------------------------------------
>> Bryan Garaventa
>> Senior Accessibility Engineer
>> SSB + BART Group
>> [hidden email]
>> www.SSBBartGroup.com
>> Accessibility-On-Demand
>>
>> ----- Original Message -----
>> From: "James Teh" <[hidden email]>
>> Newsgroups: mozilla.dev.accessibility
>> To: <[hidden email]>
>> Sent: Wednesday, July 09, 2008 5:21 PM
>> Subject: Re: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]
>>
>>
>>> In Python, you can access any COM interface specified in idl if you can
>>> generate a typelib for it. This is how NVDA accesses MSAA and
>>> IAccessible2. Note that we do not use ISimpleDOM.
>>>
>>> As far as I know, in order to access COM objects from JAWS, they need to
>>> support IDispatch. As I understand it, Mozilla does not support
>>> IDispatch on its accessibility COM objects.
>>>
>>> Although you can access MSAA objects from JAWS (again, assuming they
>>> support IDispatch), you cannot use QueryInterface because QueryInterface
>>> accepts an interface, which JAWS scripting can't support. There is no
>>> way to add a property to access ISimpleDOM from MSAA, because MSAA
>>> (IAccessible) is a predefined interface; you can't add arbitrary
>>> properties without creating a new interface (which may or may not be a
>>> subclass of an existing interface).
>>>
>>> Even if Mozilla did support IDispatch on its COM objects, screen readers
>>> really should provide a way to get access to the current Firefox
>>> document object, similar to the way they do for IE> Without this, the
>>> only way to access the current document would be to provide some sort of
>>> COM automation object which could be retrieved with GetActiveObject
>>> (known as GetObject in JAWS). This could be potentially unreliable. It
>>> could perhaps be improved by accepting the root window handle of the
>>> document or some such. I still think screen reader scripting languages
>>> need to provide some sort of support; it cannot properly be done without
>>> something from the screen readers.
>>>
>>> Jamie
>>>
>>> Aaron Leventhal wrote:
>>>> Bryan,
>>>>
>>>> Forgive me for being done. What would the property be on? You can get to
>>>> the ISimpleDOMNode tree by taking taking any IAccessible* and doing
>>>> QueryInterface to ISimpleDOMNode.
>>>>
>>>> It's scriptable if the screen reader includes the IDL and binds it. I
>>>> don't know exactly how it's done, but NVDA uses all of Firefox's a11y
>>>> interfaces in script.
>>>>
>>>> Keep in mind, that one difference with IE is that Firefox only uses COM
>>>> for the Windows a11y stuff. Com is not usable for a general solution in
>>>> a cross platform product.
>>>>
>>>> - Aaron
>>>
>>> --
>>> James Teh
>>> Email: [hidden email]
>>> WWW: http://www.jantrid.net/
>>> MSN Messenger: [hidden email]
>>> Jabber: [hidden email]
>>> Yahoo: jcs_teh
>>> _______________________________________________
>>> dev-accessibility mailing list
>>> [hidden email]
>>> https://lists.mozilla.org/listinfo/dev-accessibility
>>
>> _______________________________________________
>> dev-accessibility mailing list
>> [hidden email]
>> https://lists.mozilla.org/listinfo/dev-accessibility
>>
>
>
>
> --
> Steve Lee
> Open Source Assistive Technology Software and Accessibility
> fullmeasure.co.uk
>



--
Steve Lee
Open Source Assistive Technology Software and Accessibility
fullmeasure.co.uk
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]

Aaron Leventhal-3
In reply to this post by Aaron Leventhal-3
Bryan, can I still play Devil's advocate for a moment and see if there
is another Mozilla-ish way to solve the problem?

In Firefox, many people write development tools that access the DOM,
using the Firefox extension mechanism. This is very powerful and the end
result can be used by anyone, regardless of whether they use a screen
reader, and it does not depend on what operating system you're on. This
is why things like the Web Developer Toolbar or Firebug have such
widespread usage. In the end you can access the DOM of any page from an
extension, using JavaScript.

You're never going to get a mechanism that allows you to use the same IE
JAWS script in Firefox. Even if we provided ISimpleDOM* interfaces to
script, it would not work exactly the same, or be as powerful, as the IE
DOM scripting with JAWS. (As mentioned before, this is because IE uses
COM all over the place so it's a totally different architecture.) In
addition, IE uses a non-standard DOM, and Firefox uses just the W3C DOM.
So, there are many differences that will bite you.

As far as using scripting in Firefox to get the results out of the UI
you want. Here is another case where an extension could help. In
addition, if it's a matter of making Firefox work right for JAWS users,
we'd really prefer to always fix the bug for everyone, right inside
Firefox. We don't want JAWS users or scripters to have to work around
Firefox bugs.

If you are trying to use scripting to get different results out of a web
page or web application, then it becomes more interesting to me. I think
Orca is working on a per-webpage scripting mechanism. However, they use
ATK/AT-SPI. In general, Firefox exposes all the semantics you need via
it's accessibility interfaces. You should not need to use the DOM at
all. MSAA + IAccessible2 should be enough, if they were scriptable.

I'm not saying we won't use IDispatch, but it will probably be a while,
and I want to make sure everyone understands what the other
possibilities are.

This is a good conversation -- we should keep it going! We really want
to do the right thing (and wish we could do everything at once sometimes).

- Aaron


Bryan Garaventa wrote:

> No problem, it's a bit confusing for me as well, since I'm only familiar
> with Windows programming methods.
>
> To back up a bit, my interest is to obtain the DOM tree equivalent
> within the JAWS scripting environment as an object reference. Since I
> work in the accessibility field, this information would be immensely
> valuable for obtaining data within html elements, such as attribute
> values, innerHTML, etc. Also, this would allow for diagnostic utilities
> to be created within the screen reader scripting language, which would
> allow users to examine the tree at runtime to debug web apps. The value
> for this capability is limitless really. This would also allow screen
> reader scriptors to supplement current web apps with customized scripts
> that would make the web apps more accessible.
>
> There is a built in function within JAWS called IEGetCurrentDocument,
> which returns the DOM that currently has focus within the screen reader.
> This object is obtained as a ByRef object within the browser, so the
> various nodes can be retrieved or changed within the browser instance at
> runtime using standard object oriented commands like
>
> let LiveDOM = IEGetCurrentDocument()
> let FrameCount = LiveDOM.frames.length
> let TargetInnerHTML = LiveDOM.getElementById("MyTarget").innerHTML
>
> In this way, JAWS scriptors can create powerful solutions within
> web-based environments to retrieve and parse data. Unfortunately though,
> since there is no similar function for the Firefox browser, all of these
> solutions are limited to IE.
>
> However, there is a way within the JAWS scripting language, to
> instantiate a ByRef COM object. This object can then be used to run
> various methods that are publically available to that object. This would
> be the only way for a JAWS scriptor to access the correct object if this
> were available within FF.
>
> Is there any way to instantiate an instance of IAccessible using COM? As
> an example, by using VB or by any other Windows scripting language? This
> is the only way that I can see it working for a screen reader. This
> would allow the JAWS scriptor to instantiate an object that was of type
> IAccessible, which, presumably, could then call the method
> QueryInterface like so...
>
> let FFDOMNode = IAccessibleObj.QueryInterface
>
> This assumes that QueryInterface is a toggle, I wasn't really sure about
> this when reading the docs. Of course the class name of the COM object
> would need to be known in advance, in order to create the COM object
> instance like so...
>
> let IAccessibleObj = CreateObject("Firefox/IAccessible")
>
> "Firefox/IAccessible" being the COM object NameSpace/Class string. I
> realize this would only be a Windows compatible solution, but it appears
> to be an unavoidable situation given the current AT limitations we have
> to work around.
>
> Hopefully this helps a bit.
>
> Best wishes,
>
> Bryan
>
>
> --------------------------------------------------------------------------------
>
> Bryan Garaventa
> Senior Accessibility Engineer
> SSB + BART Group
> [hidden email]
> www.SSBBartGroup.com
> Accessibility-On-Demand
>
>
> ----- Original Message ----- From: "Aaron Leventhal"
> <[hidden email]>
> To: "Bryan Garaventa" <[hidden email]>
> Cc: "dev-accessibility Firefox" <[hidden email]>
> Sent: Wednesday, July 09, 2008 12:59 PM
> Subject: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]
>
>
>> Bryan,
>>
>> Forgive me for being done. What would the property be on? You can get
>> to the ISimpleDOMNode tree by taking taking any IAccessible* and doing
>> QueryInterface to ISimpleDOMNode.
>>
>> It's scriptable if the screen reader includes the IDL and binds it. I
>> don't know exactly how it's done, but NVDA uses all of Firefox's a11y
>> interfaces in script.
>>
>> Keep in mind, that one difference with IE is that Firefox only uses
>> COM for the Windows a11y stuff. Com is not usable for a general
>> solution in a cross platform product.
>>
>> - Aaron
>>
>>
>> Bryan Garaventa wrote:
>>> (Woops, forgot to reply to all...)
>>>
>>> Would it be possible to expose a public property within FF that
>>> represents the live DOM? I know that the functionality is more MSAA
>>> based, but if such a property could default to the ISimpleDOMNode
>>> interface, then it could be directly accessed using any screen reader
>>> scripting language without requiring additional development from the
>>> screen reader manufacturers to add a compatible interface within the
>>> scripting language, which wouldn't likely happen any time soon given
>>> their current pace of development.
>>>
>>> If such a DOM property were exposed in FF, it would act similarly to
>>> IE's MSHTML.Document property, which could then be scripted. All that
>>> would need be done, is that the DOM property be bound to the main doc
>>> object so it would update accordingly at runtime. Well, I think, not
>>> being an FF developer.
>>>
>>>
>>>
>>> --------------------------------------------------------------------------------
>>>
>>>
>>> Bryan Garaventa
>>> Senior Accessibility Engineer
>>> SSB + BART Group
>>> [hidden email]
>>> www.SSBBartGroup.com
>>> Accessibility-On-Demand
>>>
>>> ----- Original Message ----- From: "Aaron Leventhal"
>>> <[hidden email]>
>>> Newsgroups: mozilla.dev.accessibility
>>> To: "Bryan Garaventa" <[hidden email]>
>>> Cc: "Alexander Surkov" <[hidden email]>
>>> Sent: Wednesday, July 09, 2008 11:12 AM
>>> Subject: Re: New: Firefox 3 with Screen Readers FAQ
>>>
>>>
>>>> This is good criticism.
>>>>
>>>> I would say that our DOM is scriptable but not via IDispatch. For
>>>> example, NVDA is written in Python and can access the Firefox DOM. We
>>>> have several DOM interfaces -- ISimpleDOMNode, ISimpleDOMText and
>>>> ISimpleDOMDocument that provide read only access to the DOM.
>>>>
>>>> Screen readers could use our IDL for these interfaces to make the
>>>> Firefox DOM scriptable.
>>>>
>>>> Similarly, screen readers could use the IDL for IAccessible2 to
>>>> provide that interfaces to scripters (and not just in Firefox).
>>>>
>>>> On the other hand I'd like to support IDispatch in Firefox. We do have
>>>> a bug open for that. I'm not willing to break any kind of binary
>>>> compatibility, but I'm not sure if that would be necessary.
>>>>
>>>> - Aaron
>>>>
>>>
>>
>>
>

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: New: Firefox 3 with Screen Readers FAQ

Aaron Leventhal-3
In reply to this post by Aaron Leventhal-3
Before Mozilla mentions something in a "Support" FAQ, I just want to
make sure it's possible to get support for a given recommendation.

For example, for JAWS, Window-Eyes, Orca and NVDA, I know that users
cannot get support.

Where does one get support for Fire Vox?

- Aaron



Jason White wrote:

> On Wed, Jul 09, 2008 at 05:37:13PM +0200, Aaron Leventhal wrote:
>> I'd like to hear more about Fire Vox from the community first.
>
> It's important, innovative and useful.
>
> I plan to install it and get it running with Emacspeak, at which point I'll
> have more comments to add.
>
> It ought to be mentioned in the FAQ and in the list of accessibility-related
> extensions.
>

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: New: Firefox 3 with Screen Readers FAQ

Aaron Leventhal-3
Aaron Leventhal wrote:
> For example, for JAWS, Window-Eyes, Orca and NVDA, I know that users
> cannot get support.
Correction. Of course I meant "can get support".

- Aaron
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]

James Teh-2
In reply to this post by James Teh-2
Bryan Garaventa wrote:
> If it is possible for me to make a function within VB6 that can access
> the current running instance of Firefox, and then access its IAccessible
> interface, would it be possible for me to get the DOM from this?
Even if you could get the DOM object, it doesn't support IDispatch, so
JAWS still wouldn't be able to access its properties and methods. Just
in case you are unfamiliar with it (apologies if you are), IDispatch is
an interface which allows you to test for and access properties and
methods by name without actually having knowledge of the interface
definition itself.

> It is easy for me to create an ActiveX control that I can instantiate
> from within JAWS, and then pass and receive data from the object's
> methods and functions.
As I understand it, without IDispatch on the DOM, you'd literally have
to act as a relay between JAWS and the DOM for every single call. You'd
have to know all possible methods and write functions to wrap every one
of them. This said, your idea does solve the first part of the problem:
getting the right object. However, I think IDispatch is required for the
rest, and as has been noted, that is a *huge* undertaking for Mozilla.

Disclaimer: I'm not a COM guru, not even close. :) Anyone who knows
more, please do feel free to correct me.

As Aaron noted, perhaps there are other ways you can solve your
particular issues, perhaps for the benefit of all screen readers.

Jamie

--
James Teh
Email: [hidden email]
WWW: http://www.jantrid.net/
MSN Messenger: [hidden email]
Jabber: [hidden email]
Yahoo: jcs_teh
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]

Steve Lee-3
In reply to this post by Aaron Leventhal-3
2008/7/10 Aaron Leventhal <[hidden email]>:
> If you are trying to use scripting to get different results out of a web
> page or web application, then it becomes more interesting to me. I think
> Orca is working on a per-webpage scripting mechanism. However, they use
> ATK/AT-SPI. In general, Firefox exposes all the semantics you need via
> it's accessibility interfaces. You should not need to use the DOM at
> all. MSAA + IAccessible2 should be enough, if they were scriptable.

Here's a thought, you could use the greasemonkey extension to script
behaviour changes so that JAWs sees the right thing through MSAA (the
CSS scripter Stylish might even be useful in some circumstances).
Separating the scripting from JAWS gives the opportunity to fix broken
sites so they work with several readers or other ATs (however we don't
want to stop pestering/educating authors to create good a11y sites)


--
Steve Lee
Open Source Assistive Technology Software and Accessibility
fullmeasure.co.uk
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]

Steve Lee-3
In reply to this post by James Teh-2
2008/7/10 James Teh <[hidden email]>:
> Disclaimer: I'm not a COM guru, not even close. :) Anyone who knows
> more, please do feel free to correct me.

Me neither, but I did enough hacking to know you are correct. There a
basically 2 ways to access COM server interfaces, 1) binary where you
know the memory layout of pointers to functions (works with compiled
languages like C ) and DIspatch where you work it out at run time,
optionally with the help of some info in a 'type library' (this is for
scripted/interpreted languages like JAWS, WindowsScripting and
Python). Dispatch is a polymorphic interface as you call all other
methods through a single function call. COM server's usually support
one or both styles.

--
Steve Lee
Open Source Assistive Technology Software and Accessibility
fullmeasure.co.uk
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]

Jason White-14
In reply to this post by Steve Lee-3
On Thu, Jul 10, 2008 at 10:11:35AM +0100, Steve Lee wrote:
 
> Here's a thought, you could use the greasemonkey extension to script
> behaviour changes so that JAWs sees the right thing through MSAA (the
> CSS scripter Stylish might even be useful in some circumstances).
> Separating the scripting from JAWS gives the opportunity to fix broken
> sites so they work with several readers or other ATs (however we don't
> want to stop pestering/educating authors to create good a11y sites)

You've almost described Google-Axsjax, except that it isn't primarily designed
to fix "broken" sites, but to enhance the accessibility of Web applications
that are already basically accessible. Of course, I am sure Axsjax modules can
be written to fix otherwise inaccessible content in some circumstances.

Axsjax relies on Aria live regions, and should thus work with any assistive
technology that supports this feature.

As a more general comment, I think there is an impression among some Web site
developers that if their content works with assistive technologies, never mind
how slow, cumbersome or difficult this may be for the user, it is
"accessible". The inefficiency and difficulty of interacting with the
resulting user interfaces is conveniently overlooked. This is where Axsjax is
a major innovation: it tries to improve the effectiveness of the user's
interaction, not merely to make that interaction, via an assistive technology,
possible.

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: New: Firefox 3 with Screen Readers FAQ

Steve Lee-3
In reply to this post by Jason White-14
2008/7/10 Jason White <[hidden email]>:
> It ought to be mentioned in the [snip] in the list of accessibility-related
> extensions.

And its sibling CLiCk, speak which is simpler and provides reading support.

Plus it's a javascript library for creating such applications....

Steve
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: New: Firefox 3 with Screen Readers FAQ

Steve Lee-3
In reply to this post by Aaron Leventhal-3
2008/7/9 Aaron Leventhal <[hidden email]>:
> On the other hand I'd like to support IDispatch in Firefox. We do have a
> bug open for that. I'm not willing to break any kind of binary
> compatibility, but I'm not sure if that would be necessary.

I really don't think so. AFAIK the dispinterface simply 'dispatches'
to the binary interface and marshals data with conversions.

Look and 'And now the detail' on

http://microsoft.apress.com/asptodayarchive/71780/the-idispatch-interface

Hopefully you can add the new methods without changing the binary layout.

--
Steve Lee
Open Source Assistive Technology Software and Accessibility
fullmeasure.co.uk
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]

James Teh-2
In reply to this post by James Teh-2
Steve Lee wrote:
> There [are]
> basically 2 ways to access COM server interfaces, 1) binary where you
> know the memory layout of pointers to functions (works with compiled
> languages like C ) and DIspatch where you work it out at run time,
> optionally with the help of some info in a 'type library' (this is for
> scripted/interpreted languages like JAWS, WindowsScripting and
> Python).
Just a slight correction: languages like Python, through the use of a
type library, can access COM using the binary interface method. Dispatch
can also be used. Note that we use the binary method to access
IAccessible and IAccessible2. A type library provides the necessary
information to achieve this. Dispatch does not require a type library.

Jamie

--
James Teh
Email: [hidden email]
WWW: http://www.jantrid.net/
MSN Messenger: [hidden email]
Jabber: [hidden email]
Yahoo: jcs_teh
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]

Steve Lee-3
2008/7/10 James Teh <[hidden email]>:
> Just a slight correction: languages like Python, through the use of a
> type library, can access COM using the binary interface method. Dispatch
> can also be used. Note that we use the binary method to access
> IAccessible and IAccessible2. A type library provides the necessary
> information to achieve this. Dispatch does not require a type library.

Aha Jammes, you obviously know it better than I do. I thought Python
(at least Mark Hammonds genpy stuff) used the type library to skip the
GetIDofName() step but still called Invoke() (so called early-bound
automation). Or are you talking about something you can do with ctypes
magic?

Steve
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

RE: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]

Jamal Mazrui
In reply to this post by Aaron Leventhal-3
Below are some concrete examples of what I'd like a JAWS or Window-Eyes
script to be able to do with Firefox.  Is this information provided by
the MSAA interface?
Jamal

*  Get the URL and title of the current page

*  Get the URL of a link at the current cursor position

*  Get the complete text of the page

*  Enumerate the links of a page, getting the URL and clickable text of
each

*  Move the cursor to a location on the page based on a 0 or 1 based
index of characters in the page text

*  Move the cursor to a particular line and column position of the page

*  Get the index, line, or column position of the cursor

*  Determine if the page contains a "printable version" type of link and
activate it

*  Determine if the page contains a "skip navigation" type of link,
activate it with speech suppressed, and then begin a continuous read at
the line to which the cursor has moved

* Toggle on a mode in which a "skip navigation" link is searched for
every time a new page is opened, and if found, the cursor is
automatically moved there and a continuous read commences at that point
going forward


Jamal
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]

Marco Zehe-3
In reply to this post by Aaron Leventhal-3
Hi Jamal,

Jamal Mazrui wrote:
> *  Get the URL and title of the current page

Provided by the AccName (title) and accValue (URL) of the Document
Accessible.

> *  Get the URL of a link at the current cursor position
Get the Link's object and ask for its accValue, or use IA2 methods from
the nsIAccessibleHyperLink interface if JAWS or Window-Eyes allows
access to these.

> *  Get the complete text of the page
Can be done if you want to do your own traversal of the accessible tree
down from the Document accessible.

> *  Enumerate the links of a page, getting the URL and clickable text of
> each
See above: For links, the URL is exposed through the accValue, the
clickable text is exposed via the accName. Note for graphical links,
this may be the alt text or title of the image, or the link title if
none of the two former are present.

> *  Move the cursor to a location on the page based on a 0 or 1 based
> index of characters in the page text

You mean the application's cursor (as in normal PC cursor) or the cursor
within the virtual document? If the latter, that's a screen reader
specific feature.

> *  Move the cursor to a particular line and column position of the page
See above.

> *  Get the index, line, or column position of the cursor
For the virtual cursor, this is again only possible through screen
reader specific functions which Firefox has no control over. If you mean
the line and column position within a multi-line textfield, for example,
there are IA2 functions that give you that.

> *  Determine if the page contains a "printable version" type of link and
> activate it
This would be done through searching through the links (see above) and
look for specific text in the URL (accValue) or name of the link.
> *  Determine if the page contains a "skip navigation" type of link,
> activate it with speech suppressed, and then begin a continuous read at
> the line to which the cursor has moved
Search for that link, activate its "Jump" action.

> * Toggle on a mode in which a "skip navigation" link is searched for
> every time a new page is opened, and if found, the cursor is
> automatically moved there and a continuous read commences at that point
> going forward
See previous.

Marco
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

RE: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]

Jamal Mazrui
Thanks, Marco.  

Since at present, the great majority of blind computer users are
operating on the Windows platform (whether in a home or employment
setting), Let me urge Mozilla and its developer friends to implement
IDispatch support for Firefox on Windows, preferably with access to the
DOM, but at least with access to all the info available via MSAA.  In my
opinion, the lack of such support available to scripters of Windows
screen readers is a serious weakness compared to Internet Explorer.

Regards,
Jamal

 


-----Original Message-----
From: dev-accessibility-bounces+jamal.mazrui=[hidden email]
[mailto:dev-accessibility-bounces+jamal.mazrui=[hidden email]
] On Behalf Of Marco Zehe
Sent: Thursday, July 10, 2008 11:19 AM
To: [hidden email]
Subject: Re: Scriptability [was Re: New: Firefox 3 with Screen Readers
FAQ]

Hi Jamal,

Jamal Mazrui wrote:
> *  Get the URL and title of the current page

Provided by the AccName (title) and accValue (URL) of the Document
Accessible.

> *  Get the URL of a link at the current cursor position
Get the Link's object and ask for its accValue, or use IA2 methods from
the nsIAccessibleHyperLink interface if JAWS or Window-Eyes allows
access to these.

> *  Get the complete text of the page
Can be done if you want to do your own traversal of the accessible tree
down from the Document accessible.

> *  Enumerate the links of a page, getting the URL and clickable text
> of each
See above: For links, the URL is exposed through the accValue, the
clickable text is exposed via the accName. Note for graphical links,
this may be the alt text or title of the image, or the link title if
none of the two former are present.

> *  Move the cursor to a location on the page based on a 0 or 1 based
> index of characters in the page text

You mean the application's cursor (as in normal PC cursor) or the cursor
within the virtual document? If the latter, that's a screen reader
specific feature.

> *  Move the cursor to a particular line and column position of the
> page
See above.

> *  Get the index, line, or column position of the cursor
For the virtual cursor, this is again only possible through screen
reader specific functions which Firefox has no control over. If you mean
the line and column position within a multi-line textfield, for example,
there are IA2 functions that give you that.

> *  Determine if the page contains a "printable version" type of link
> and activate it
This would be done through searching through the links (see above) and
look for specific text in the URL (accValue) or name of the link.
> *  Determine if the page contains a "skip navigation" type of link,
> activate it with speech suppressed, and then begin a continuous read
> at the line to which the cursor has moved
Search for that link, activate its "Jump" action.

> * Toggle on a mode in which a "skip navigation" link is searched for
> every time a new page is opened, and if found, the cursor is
> automatically moved there and a continuous read commences at that
> point going forward
See previous.

Marco
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]

Aaron Leventhal-3
In reply to this post by Marco Zehe-3

>> * Get the complete text of the page
> Can be done if you want to do your own traversal of the accessible tree
> down from the Document accessible.
Can be done via IAccessible or IAccessibleText. If you use
IAccessibleText you won't have to visit as many nodes, which is good
when you're out of process. If you use IAccessibleText, I suggest
reading up on how we support that:
http://developer.mozilla.org/en/docs/Accessibility:Architecture
>
>> * Enumerate the links of a page, getting the URL and clickable text of
>> each
> See above: For links, the URL is exposed through the accValue, the
> clickable text is exposed via the accName. Note for graphical links,
> this may be the alt text or title of the image, or the link title if
> none of the two former are present.
Yes, all this can be done, and is done. For example, NVDA uses
IAccessible2/IAccessibleText to get all this info. Their code is open
source -- take a look if you want, or check out our docs, or take a look
at the accessible object tree in Firefox using accProbe.

>> * Move the cursor to a location on the page based on a 0 or 1 based
>> index of characters in the page text
You can manipulate the caret or selection using IAccessibleText.


>> * Get the index, line, or column position of the cursor
IAccessibleText will help you compute all of these things. We also have
an object attribute to give you the line number that the caret is on.

For what it's worth, the caret and selection are not part of the W3C
DOM. So, if it's in Microsoft's DOM that's because they have a
nonstandard DOM anyway. No other browser DOM would have that info.

>> * Determine if the page contains a "printable version" type of link and
>> activate it
> This would be done through searching through the links (see above) and
> look for specific text in the URL (accValue) or name of the link.
Interesting. Maybe there should be a semantic in HTML 5 or ARIA for
that. Interestingly, there is @media print, I believe -- a CSS way of
specifying the printable version.

Anyway, of course you can do all of these things, otherwise we wouldn't
be able to support screen readers at all! People have been working with
the screen reader vendors and putting in what they've asked for into the
Mozilla a11y support since 2001 -- seven years.

>> * Determine if the page contains a "skip navigation" type of link,
>> activate it with speech suppressed, and then begin a continuous read at
>> the line to which the cursor has moved
> Search for that link, activate its "Jump" action.
Sure, you need to walk the tree and look at the links. Use
doDefaultAction to jump to it.

>> * Toggle on a mode in which a "skip navigation" link is searched for
>> every time a new page is opened, and if found, the cursor is
>> automatically moved there and a continuous read commences at that point
>> going forward
In general it sounds like you want to start out at the main content of a
page. This is something that webvisum could mark with the social tagging
as well. In addition, there is an attribute in ARIA that can be used to
mark the main section of content. If you put role="main" that means this
section is the main content on the page. You can check for that role
using the xml-roles object attribute (see object attributes in IA2).

If you need more help for your project let us know. We'll be glad to
help! You can also bug us on IRC.

- Aaron

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]

Bryan Garaventa-2
In reply to this post by Jamal Mazrui
Thanks for all of the feedback, it helps to isolate the main problem.

It looks like Firefox does not expose a COM interface on a Windows system
when installed.

To my knowledge, this would prevent any other Windows app such as a screen
reader from being able to communicate with the running instance of FF at
runtime.

This would also prevent any Windows app from being able to access the
running instance of IAccessible through FF, so the MSAA object could not be
successfully queried by any Windows app such as a screen reader.

I have made successful use of IDispatch before, within some VB apps that
I've written, though the actual mechanics behind the functionality are a bit
of a mystery to me. It all appears to work though when communicating with
running instances of other apps. Still though, without the COM interface
support by the third party app, I don't believe this communication would be
possible. Please correct me if I'm wrong here... Is IDispatch synonymous
with COM?

I have no familiarity with the creation of FF extensions, so I'm not sure if
it's possible to create an extension that simulates a COM interface with FF,
which may bridge the gap if possible.

It looks like, as long as Firefox does not provide a COM interface for
Windows apps, any form of communication between screen readers and Firefox
will be impossible, including any successful interaction with the
IAccessible MSAA object model...

Which is truly unfortunate, since the lack of COM support for Windows apps
is the only reason that I can see for Firefox usage not succeeding IE usage
by screen reader users.

As Jamal pointed out, the importance of screen reader communication with the
browser is very important, since it allows screen reader manufacturers to
customize screen reader behavior at runtime to maximize accessibility for
screen reader users.

It's an interesting problem, but it doesn't look like it has a simple
solution one way or the other.

Best wishes,

Bryan


--------------------------------------------------------------------------------
Bryan Garaventa
Senior Accessibility Engineer
SSB + BART Group
[hidden email]
www.SSBBartGroup.com
Accessibility-On-Demand


----- Original Message -----
From: "Jamal Mazrui" <[hidden email]>
To: "Aaron Leventhal" <[hidden email]>;
<[hidden email]>
Sent: Thursday, July 10, 2008 7:33 AM
Subject: RE: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]


> Below are some concrete examples of what I'd like a JAWS or Window-Eyes
> script to be able to do with Firefox.  Is this information provided by
> the MSAA interface?
> Jamal
>
> *  Get the URL and title of the current page
>
> *  Get the URL of a link at the current cursor position
>
> *  Get the complete text of the page
>
> *  Enumerate the links of a page, getting the URL and clickable text of
> each
>
> *  Move the cursor to a location on the page based on a 0 or 1 based
> index of characters in the page text
>
> *  Move the cursor to a particular line and column position of the page
>
> *  Get the index, line, or column position of the cursor
>
> *  Determine if the page contains a "printable version" type of link and
> activate it
>
> *  Determine if the page contains a "skip navigation" type of link,
> activate it with speech suppressed, and then begin a continuous read at
> the line to which the cursor has moved
>
> * Toggle on a mode in which a "skip navigation" link is searched for
> every time a new page is opened, and if found, the cursor is
> automatically moved there and a continuous read commences at that point
> going forward
>
>
> Jamal
> _______________________________________________
> dev-accessibility mailing list
> [hidden email]
> https://lists.mozilla.org/listinfo/dev-accessibility 

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]

Steve Lee-3
2008/7/10 Bryan Garaventa <[hidden email]>:
> Please correct me if I'm wrong here... Is IDispatch synonymous
> with COM?

Not quite  IDispatch is synonymous with access to COM servers from
scripting language environments like VB6.
Compiled programs like C can happily access COM with the co called
custom interfaces and no IDispatch interface.
This is the case with Mozilla Firefox.

> It looks like, as long as Firefox does not provide a COM interface for
> Windows apps, any form of communication between screen readers and Firefox
> will be impossible, including any successful interaction with the
> IAccessible MSAA object model...

No. Most windows screen readers work just fine with Firefox via MSAA
and IA, both of which are COM based technologies. You just can't use
Windows scripting facilities w/o and IDispatch. For Screen reader
hosted scripting the screen reader could provide the equivalent to
IDispatch for you.

> Which is truly unfortunate, since the lack of COM support for Windows apps
> is the only reason that I can see for Firefox usage not succeeding IE usage
> by screen reader users.

Again it only effects access from scripting languages.

> As Jamal pointed out, the importance of screen reader communication with the
> browser is very important, since it allows screen reader manufacturers to
> customize screen reader behavior at runtime to maximize accessibility for
> screen reader users.

But only Windows screen readers, and only those that assume the
browsers COM servers has IDispatch and so don't provide their own
scripting access. I don't know JAWS but I'm guessing it uses
WindowsScripting to provide the script environment form what has been
said. That assumes IDispatch. So you could say that decision has meant
only scripting for IDispatch servers.


--
Steve Lee
Open Source Assistive Technology Software and Accessibility
fullmeasure.co.uk
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

RE: Scriptability [was Re: New: Firefox 3 with Screen Readers FAQ]

Jamal Mazrui
In reply to this post by Jamal Mazrui
Hi Marco and Aaron,
Isn't there already custom Windows code in Firefox?  Is MSAA also
implemented on Linux and the Mac?  

I am asking for the COM interface to be dual in nature, supporting both
IDispatch as well as the lower level interface.  This is a common
practice when an application provides COM server support to clients.

I do not understand how a Firefox extension could do some of my
examples.  An extension could find the target of a SkipNav link, but how
could it cause the screen reader to commence a continuous read at that
point?  It seems to me that full synchronization with assistive
technology is best achieved by giving the AT scripting power over the
application.  It can then mix AT functions with application functions as
appropriate to maximize usability.

Jamal


-----Original Message-----
From: Marco Zehe [mailto:[hidden email]]
Sent: Thursday, July 10, 2008 1:55 PM
To: Jamal Mazrui
Cc: [hidden email]
Subject: Re: Scriptability [was Re: New: Firefox 3 with Screen Readers
FAQ]

Hi Jamal,

Jamal Mazrui wrote:
> Since at present, the great majority of blind computer users are
> operating on the Windows platform (whether in a home or employment
> setting), Let me urge Mozilla and its developer friends to implement
> IDispatch support for Firefox on Windows, preferably with access to
> the DOM, but at least with access to all the info available via MSAA.

> In my opinion, the lack of such support available to scripters of
> Windows screen readers is a serious weakness compared to Internet
Explorer.

Well, there are pros and cons to this approach. The biggest con is that
this is then something we'd solely need to do for the Windows platform,
but other platforms would not benefit from that.
As Aaron also said: You'll never be able to take a script you wrote for
IE and simply dump it onto Firefox.

On the other hand, there's the open, well-documented Mozilla platform
that allows you to develop extensions that could, for example, do such
things as find a "Skip to main content" link and activate that on every
page load. This not only has the advantage of being much faster than any
iDispatch cross-process calls could ever be (extensions are in-process);
It would also be possible to use this on any platform Firefox supports.
So if the user suddenly decided to switch to Linux, for example, they'd
just need to reinstall that extension and have the same functionality on
the new platform without any hazzle.

All the use cases you presented are things that can be done through the
assistive technology APIs or via an extension to Firefox, with the above
benefits right in place.

I am not discounting that we may get around to implementing iDispatch in
a future version of the Mozilla platform. However, the scenarios you
proposed can be done today, could even be done for Firefox 1.5 or 2.0,
and you wouldn't have to wait for us to implement a very specific
platform API.

Marco
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
1234