Help with popups and AT-SPI

classic Classic list List threaded Threaded
8 messages Options
Reply | Threaded
Open this post in threaded view
|

Help with popups and AT-SPI

Steve Lee-3
I'm having problems working with popups which contain interactive
items like pulldown menus from the menu bar and combo boxes.

I'm happy to raise bugs but I'm trying to work out what is a good
approach to take.

So the issue is when driving the UI with Action:doAction I don't see
consistently reliably events for notification of popups and accessing
their contents. I tried switching to synthetic mouse events but while
that avoids problems due to using the untested access via doAction(),
I still don't see usable events.

Orca works with its key access and focus tracking and I've started
looking at how I might do something similar but there's a lot to
understand about how it works and the user experience (for example how
do users access the menu bar without knowing a mnemnonic to use with
Alt, and combo menus don't have a focussed item when alt down is
used).

The menus are all in the main application tree but I can find no way
to tie up the popup events with the right part of the tree. When a
menu appears we get object visibility events but the source window
object only has a ROLE_REDUNDANT child (no doubt as the objects are in
the main tree).

Currently when the user moves around the interface it does not touch
the UI but simply draws a rectangle over it. Perhaps the best solution
will be to change that actually drive the UI focus as a screen reader
user does, however that may in some cases do the wrong thing.

Any ideas/thoughts?

--
Steve Lee
--
Jambu - Alternative Access to Computers
www.fullmeasure.co.uk
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Help with popups and AT-SPI

Marco Zehe-3
Hi Steve,

let me concentrate on the keyboard driving part you're asking about.

Steve Lee wrote:
> Orca works with its key access and focus tracking and I've started
> looking at how I might do something similar but there's a lot to
> understand about how it works and the user experience (for example how
> do users access the menu bar without knowing a mnemnonic to use with
> Alt, and combo menus don't have a focussed item when alt down is
> used).
On Linux, the key to focus the menu bar without activating a popup menu
is the F10 key. Pressing that will make Orca speak the first highlighted
menu item, "File" in Firefox, for example.
As for popups in comboboxes, I have found that pressing F4 instead of
Alt+DownArrow is often the more reliable way to open the popup and then
interact with it.

Hope this helps!

Marco

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Help with popups and AT-SPI

Peter Korn-2
In reply to this post by Steve Lee-3
Hi Steve,

I think David Bolter monitors this list.  His work on both GOK and
Mozilla/Firefox make him an excellent candidate to wade into this
discussion (hint hint).  Separately, you might look at the GOK source
for both its use of doAction and how it recognizes popups in Mozilla.


Regards,

Peter Korn
Accessibility Architect,
Sun Microsystems, Inc.

> I'm having problems working with popups which contain interactive
> items like pulldown menus from the menu bar and combo boxes.
>
> I'm happy to raise bugs but I'm trying to work out what is a good
> approach to take.
>
> So the issue is when driving the UI with Action:doAction I don't see
> consistently reliably events for notification of popups and accessing
> their contents. I tried switching to synthetic mouse events but while
> that avoids problems due to using the untested access via doAction(),
> I still don't see usable events.
>
> Orca works with its key access and focus tracking and I've started
> looking at how I might do something similar but there's a lot to
> understand about how it works and the user experience (for example how
> do users access the menu bar without knowing a mnemnonic to use with
> Alt, and combo menus don't have a focussed item when alt down is
> used).
>
> The menus are all in the main application tree but I can find no way
> to tie up the popup events with the right part of the tree. When a
> menu appears we get object visibility events but the source window
> object only has a ROLE_REDUNDANT child (no doubt as the objects are in
> the main tree).
>
> Currently when the user moves around the interface it does not touch
> the UI but simply draws a rectangle over it. Perhaps the best solution
> will be to change that actually drive the UI focus as a screen reader
> user does, however that may in some cases do the wrong thing.
>
> Any ideas/thoughts?
>
>  

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Help with popups and AT-SPI

Steve Lee-3
Thanks Peter, I've been looking a Orca as that is also Python and
plain forgot GOK in this case. GOK is going to be closer in useage.

Steve

On 05/12/2007, Peter Korn <[hidden email]> wrote:

> Hi Steve,
>
> I think David Bolter monitors this list.  His work on both GOK and
> Mozilla/Firefox make him an excellent candidate to wade into this
> discussion (hint hint).  Separately, you might look at the GOK source
> for both its use of doAction and how it recognizes popups in Mozilla.
>
>
> Regards,
>
> Peter Korn
> Accessibility Architect,
> Sun Microsystems, Inc.
>
> > I'm having problems working with popups which contain interactive
> > items like pulldown menus from the menu bar and combo boxes.
> >
> > I'm happy to raise bugs but I'm trying to work out what is a good
> > approach to take.
> >
> > So the issue is when driving the UI with Action:doAction I don't see
> > consistently reliably events for notification of popups and accessing
> > their contents. I tried switching to synthetic mouse events but while
> > that avoids problems due to using the untested access via doAction(),
> > I still don't see usable events.
> >
> > Orca works with its key access and focus tracking and I've started
> > looking at how I might do something similar but there's a lot to
> > understand about how it works and the user experience (for example how
> > do users access the menu bar without knowing a mnemnonic to use with
> > Alt, and combo menus don't have a focussed item when alt down is
> > used).
> >
> > The menus are all in the main application tree but I can find no way
> > to tie up the popup events with the right part of the tree. When a
> > menu appears we get object visibility events but the source window
> > object only has a ROLE_REDUNDANT child (no doubt as the objects are in
> > the main tree).
> >
> > Currently when the user moves around the interface it does not touch
> > the UI but simply draws a rectangle over it. Perhaps the best solution
> > will be to change that actually drive the UI focus as a screen reader
> > user does, however that may in some cases do the wrong thing.
> >
> > Any ideas/thoughts?
> >
> >
>
>


--
Steve Lee
--
Jambu - Alternative Access to Computers
www.fullmeasure.co.uk
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Help with popups and AT-SPI

David Bolter
Hi guys.

GOK for the most part provides its own OSK based navigation into the
a11y tree of widgets, but in the case of popups it may take a reactive
strategy: to look for the events a popup sends and react to those
(building a keyboard or whatever context shift is required).

Steve we can VOIP about it if you like :)  (let me know and I can look
over the code).

cheers,
D


Steve Lee wrote:

> Thanks Peter, I've been looking a Orca as that is also Python and
> plain forgot GOK in this case. GOK is going to be closer in useage.
>
> Steve
>
> On 05/12/2007, Peter Korn <[hidden email]> wrote:
>  
>> Hi Steve,
>>
>> I think David Bolter monitors this list.  His work on both GOK and
>> Mozilla/Firefox make him an excellent candidate to wade into this
>> discussion (hint hint).  Separately, you might look at the GOK source
>> for both its use of doAction and how it recognizes popups in Mozilla.
>>
>>
>> Regards,
>>
>> Peter Korn
>> Accessibility Architect,
>> Sun Microsystems, Inc.
>>
>>    
>>> I'm having problems working with popups which contain interactive
>>> items like pulldown menus from the menu bar and combo boxes.
>>>
>>> I'm happy to raise bugs but I'm trying to work out what is a good
>>> approach to take.
>>>
>>> So the issue is when driving the UI with Action:doAction I don't see
>>> consistently reliably events for notification of popups and accessing
>>> their contents. I tried switching to synthetic mouse events but while
>>> that avoids problems due to using the untested access via doAction(),
>>> I still don't see usable events.
>>>
>>> Orca works with its key access and focus tracking and I've started
>>> looking at how I might do something similar but there's a lot to
>>> understand about how it works and the user experience (for example how
>>> do users access the menu bar without knowing a mnemnonic to use with
>>> Alt, and combo menus don't have a focussed item when alt down is
>>> used).
>>>
>>> The menus are all in the main application tree but I can find no way
>>> to tie up the popup events with the right part of the tree. When a
>>> menu appears we get object visibility events but the source window
>>> object only has a ROLE_REDUNDANT child (no doubt as the objects are in
>>> the main tree).
>>>
>>> Currently when the user moves around the interface it does not touch
>>> the UI but simply draws a rectangle over it. Perhaps the best solution
>>> will be to change that actually drive the UI focus as a screen reader
>>> user does, however that may in some cases do the wrong thing.
>>>
>>> Any ideas/thoughts?
>>>
>>>
>>>      
>>    
>
>
>  

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Help with popups and AT-SPI

Steve Lee-3
David, does it work with Minefield?
I just went through with Eitan and GAIL does show all the menus in the
event.source acc you get in the window showing event so it looks like
a bug.
I'll investigate more in the next couple of days after I have resolved
my other problems with event callbacks.

Steve

On 05/12/2007, David Bolter <[hidden email]> wrote:

> Hi guys.
>
> GOK for the most part provides its own OSK based navigation into the
> a11y tree of widgets, but in the case of popups it may take a reactive
> strategy: to look for the events a popup sends and react to those
> (building a keyboard or whatever context shift is required).
>
> Steve we can VOIP about it if you like :)  (let me know and I can look
> over the code).
>
> cheers,
> D
>
>
> Steve Lee wrote:
> > Thanks Peter, I've been looking a Orca as that is also Python and
> > plain forgot GOK in this case. GOK is going to be closer in useage.
> >
> > Steve
> >
> > On 05/12/2007, Peter Korn <[hidden email]> wrote:
> >
> >> Hi Steve,
> >>
> >> I think David Bolter monitors this list.  His work on both GOK and
> >> Mozilla/Firefox make him an excellent candidate to wade into this
> >> discussion (hint hint).  Separately, you might look at the GOK source
> >> for both its use of doAction and how it recognizes popups in Mozilla.
> >>
> >>
> >> Regards,
> >>
> >> Peter Korn
> >> Accessibility Architect,
> >> Sun Microsystems, Inc.
> >>
> >>
> >>> I'm having problems working with popups which contain interactive
> >>> items like pulldown menus from the menu bar and combo boxes.
> >>>
> >>> I'm happy to raise bugs but I'm trying to work out what is a good
> >>> approach to take.
> >>>
> >>> So the issue is when driving the UI with Action:doAction I don't see
> >>> consistently reliably events for notification of popups and accessing
> >>> their contents. I tried switching to synthetic mouse events but while
> >>> that avoids problems due to using the untested access via doAction(),
> >>> I still don't see usable events.
> >>>
> >>> Orca works with its key access and focus tracking and I've started
> >>> looking at how I might do something similar but there's a lot to
> >>> understand about how it works and the user experience (for example how
> >>> do users access the menu bar without knowing a mnemnonic to use with
> >>> Alt, and combo menus don't have a focussed item when alt down is
> >>> used).
> >>>
> >>> The menus are all in the main application tree but I can find no way
> >>> to tie up the popup events with the right part of the tree. When a
> >>> menu appears we get object visibility events but the source window
> >>> object only has a ROLE_REDUNDANT child (no doubt as the objects are in
> >>> the main tree).
> >>>
> >>> Currently when the user moves around the interface it does not touch
> >>> the UI but simply draws a rectangle over it. Perhaps the best solution
> >>> will be to change that actually drive the UI focus as a screen reader
> >>> user does, however that may in some cases do the wrong thing.
> >>>
> >>> Any ideas/thoughts?
> >>>
> >>>
> >>>
> >>
> >
> >
> >
>
>


--
Steve Lee
--
Jambu - Alternative Access to Computers
www.fullmeasure.co.uk
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Help with popups and AT-SPI

David Bolter
Are you thinking GAIL bug?

D
Steve Lee wrote:

> David, does it work with Minefield?
> I just went through with Eitan and GAIL does show all the menus in the
> event.source acc you get in the window showing event so it looks like
> a bug.
> I'll investigate more in the next couple of days after I have resolved
> my other problems with event callbacks.
>
> Steve
>
> On 05/12/2007, David Bolter <[hidden email]> wrote:
>  
>> Hi guys.
>>
>> GOK for the most part provides its own OSK based navigation into the
>> a11y tree of widgets, but in the case of popups it may take a reactive
>> strategy: to look for the events a popup sends and react to those
>> (building a keyboard or whatever context shift is required).
>>
>> Steve we can VOIP about it if you like :)  (let me know and I can look
>> over the code).
>>
>> cheers,
>> D
>>
>>
>> Steve Lee wrote:
>>    
>>> Thanks Peter, I've been looking a Orca as that is also Python and
>>> plain forgot GOK in this case. GOK is going to be closer in useage.
>>>
>>> Steve
>>>
>>> On 05/12/2007, Peter Korn <[hidden email]> wrote:
>>>
>>>      
>>>> Hi Steve,
>>>>
>>>> I think David Bolter monitors this list.  His work on both GOK and
>>>> Mozilla/Firefox make him an excellent candidate to wade into this
>>>> discussion (hint hint).  Separately, you might look at the GOK source
>>>> for both its use of doAction and how it recognizes popups in Mozilla.
>>>>
>>>>
>>>> Regards,
>>>>
>>>> Peter Korn
>>>> Accessibility Architect,
>>>> Sun Microsystems, Inc.
>>>>
>>>>
>>>>        
>>>>> I'm having problems working with popups which contain interactive
>>>>> items like pulldown menus from the menu bar and combo boxes.
>>>>>
>>>>> I'm happy to raise bugs but I'm trying to work out what is a good
>>>>> approach to take.
>>>>>
>>>>> So the issue is when driving the UI with Action:doAction I don't see
>>>>> consistently reliably events for notification of popups and accessing
>>>>> their contents. I tried switching to synthetic mouse events but while
>>>>> that avoids problems due to using the untested access via doAction(),
>>>>> I still don't see usable events.
>>>>>
>>>>> Orca works with its key access and focus tracking and I've started
>>>>> looking at how I might do something similar but there's a lot to
>>>>> understand about how it works and the user experience (for example how
>>>>> do users access the menu bar without knowing a mnemnonic to use with
>>>>> Alt, and combo menus don't have a focussed item when alt down is
>>>>> used).
>>>>>
>>>>> The menus are all in the main application tree but I can find no way
>>>>> to tie up the popup events with the right part of the tree. When a
>>>>> menu appears we get object visibility events but the source window
>>>>> object only has a ROLE_REDUNDANT child (no doubt as the objects are in
>>>>> the main tree).
>>>>>
>>>>> Currently when the user moves around the interface it does not touch
>>>>> the UI but simply draws a rectangle over it. Perhaps the best solution
>>>>> will be to change that actually drive the UI focus as a screen reader
>>>>> user does, however that may in some cases do the wrong thing.
>>>>>
>>>>> Any ideas/thoughts?
>>>>>
>>>>>
>>>>>
>>>>>          
>>>
>>>      
>>    
>
>
>  

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Help with popups and AT-SPI

Steve Lee-3
No Minefield, Gail has all the menus a children of the pulldown menu
(at least in GEDIT)

On 05/12/2007, David Bolter <[hidden email]> wrote:

> Are you thinking GAIL bug?
>
> D
> Steve Lee wrote:
> > David, does it work with Minefield?
> > I just went through with Eitan and GAIL does show all the menus in the
> > event.source acc you get in the window showing event so it looks like
> > a bug.
> > I'll investigate more in the next couple of days after I have resolved
> > my other problems with event callbacks.
> >
> > Steve
> >
> > On 05/12/2007, David Bolter <[hidden email]> wrote:
> >
> >> Hi guys.
> >>
> >> GOK for the most part provides its own OSK based navigation into the
> >> a11y tree of widgets, but in the case of popups it may take a reactive
> >> strategy: to look for the events a popup sends and react to those
> >> (building a keyboard or whatever context shift is required).
> >>
> >> Steve we can VOIP about it if you like :)  (let me know and I can look
> >> over the code).
> >>
> >> cheers,
> >> D
> >>
> >>
> >> Steve Lee wrote:
> >>
> >>> Thanks Peter, I've been looking a Orca as that is also Python and
> >>> plain forgot GOK in this case. GOK is going to be closer in useage.
> >>>
> >>> Steve
> >>>
> >>> On 05/12/2007, Peter Korn <[hidden email]> wrote:
> >>>
> >>>
> >>>> Hi Steve,
> >>>>
> >>>> I think David Bolter monitors this list.  His work on both GOK and
> >>>> Mozilla/Firefox make him an excellent candidate to wade into this
> >>>> discussion (hint hint).  Separately, you might look at the GOK source
> >>>> for both its use of doAction and how it recognizes popups in Mozilla.
> >>>>
> >>>>
> >>>> Regards,
> >>>>
> >>>> Peter Korn
> >>>> Accessibility Architect,
> >>>> Sun Microsystems, Inc.
> >>>>
> >>>>
> >>>>
> >>>>> I'm having problems working with popups which contain interactive
> >>>>> items like pulldown menus from the menu bar and combo boxes.
> >>>>>
> >>>>> I'm happy to raise bugs but I'm trying to work out what is a good
> >>>>> approach to take.
> >>>>>
> >>>>> So the issue is when driving the UI with Action:doAction I don't see
> >>>>> consistently reliably events for notification of popups and accessing
> >>>>> their contents. I tried switching to synthetic mouse events but while
> >>>>> that avoids problems due to using the untested access via doAction(),
> >>>>> I still don't see usable events.
> >>>>>
> >>>>> Orca works with its key access and focus tracking and I've started
> >>>>> looking at how I might do something similar but there's a lot to
> >>>>> understand about how it works and the user experience (for example how
> >>>>> do users access the menu bar without knowing a mnemnonic to use with
> >>>>> Alt, and combo menus don't have a focussed item when alt down is
> >>>>> used).
> >>>>>
> >>>>> The menus are all in the main application tree but I can find no way
> >>>>> to tie up the popup events with the right part of the tree. When a
> >>>>> menu appears we get object visibility events but the source window
> >>>>> object only has a ROLE_REDUNDANT child (no doubt as the objects are in
> >>>>> the main tree).
> >>>>>
> >>>>> Currently when the user moves around the interface it does not touch
> >>>>> the UI but simply draws a rectangle over it. Perhaps the best solution
> >>>>> will be to change that actually drive the UI focus as a screen reader
> >>>>> user does, however that may in some cases do the wrong thing.
> >>>>>
> >>>>> Any ideas/thoughts?
> >>>>>
> >>>>>
> >>>>>
> >>>>>
> >>>
> >>>
> >>
> >
> >
> >
>
>


--
Steve Lee
--
Jambu - Alternative Access to Computers
www.fullmeasure.co.uk
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility