redundant object: What did user click?

classic Classic list List threaded Threaded
5 messages Options
sbq
Reply | Threaded
Open this post in threaded view
|

redundant object: What did user click?

sbq
Greetings,
 
I've got an application that uses at-spi 1.22.1 on Ubuntu 8.04 (Hardy
Heron).  My current goal is to listen for the right events so when the
user clicks the left-mouse-button on a GUI object my application can
identify/understand the object that was clicked.
 
For Gtk applications, I've had pretty good success listening for the
Gtk:GtkWidget:button-press-event.  The event->source object that is
delivered to that listener is usually the right one.  My application is
working pretty well on Gtk applications.
 
When I run Firefox (V3.0.1) on Ubuntu my app does get the
button-press-event when the mouse in Firefox.  Unfortunately, the
event->source object is "redundant object".  That Accessible *object has
no children and no parent.  Obviously button-press-event is not a good
event for which to listen when it comes to Firefox.
 
Does anyone know what event(s) my app should be listening for and how to
best retrieve the object clicked by the user?  
 
Thanks.
 
-Sam
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: redundant object: What did user click?

Aaron Leventhal-3
Hmm, if you can get the coordinates of the mouse then you can use AT-SPI
to get the accessible object at that point. Does that work?

I probably won't be able to read any replies until around January 5.

- Aaron

On 12/19/2008 8:04 PM, Quiring, Sam wrote:

> Greetings,
>
> I've got an application that uses at-spi 1.22.1 on Ubuntu 8.04 (Hardy
> Heron).  My current goal is to listen for the right events so when the
> user clicks the left-mouse-button on a GUI object my application can
> identify/understand the object that was clicked.
>
> For Gtk applications, I've had pretty good success listening for the
> Gtk:GtkWidget:button-press-event.  The event->source object that is
> delivered to that listener is usually the right one.  My application is
> working pretty well on Gtk applications.
>
> When I run Firefox (V3.0.1) on Ubuntu my app does get the
> button-press-event when the mouse in Firefox.  Unfortunately, the
> event->source object is "redundant object".  That Accessible *object has
> no children and no parent.  Obviously button-press-event is not a good
> event for which to listen when it comes to Firefox.
>
> Does anyone know what event(s) my app should be listening for and how to
> best retrieve the object clicked by the user?
>
> Thanks.
>
> -Sam

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
sbq
Reply | Threaded
Open this post in threaded view
|

RE: redundant object: What did user click?

sbq
In reply to this post by sbq
= Aaron Leventhal wrote:
= if you can get the coordinates of the mouse then you can use AT-SPI to
get the accessible object at that point. Does that work?

No.  I tried. I implemented a keyboard listener for the F8 key which
gets the current mouse coordinates and tries to figure out the object
under the mouse.

My current test web page is http://www.weather.com.  The object I'm
trying to identify is the "Site" radio button in "The Weather Channel"
search box near the top.  There are 3 radio buttons (l-r): Local
Weather, Site, and Web.

If I hover the mouse over the "Site" radio button and press F8, it finds
two objects there.  One is the radio button.  The other comes from the
Firefox address bar dropdown menu, which appears to be a history of most
recently visited sites.  The menu is not visible when I press F8, but
the at-spi interface does not deliver enough information for me to tell
whether it is visible or not.  The menu item itself shows up like this:

   app 2/2 (n:Storm watch, Tornadoes, Thunderstorms, Weather News -
weather.com
http://www.weather.com/newscenter/stormwatch/?from=hp_news,d:,r:menu
item) [actions: ] [subs: Action Component ] (children: 0)
   app 2/2   - [states: enabled, focusable, opaque, selectable,
sensitive, showing, vertical, visible]
   app 2/2   - attr 1/4 class:autocomplete-richlistitem
   app 2/2   - attr 2/4 posinset:3
   app 2/2   - attr 3/4 tag:richlistitem
   app 2/2   - attr 4/4 setsize:12

As you can see, it is marked *enabled* and *visible*.  It's parent looks
like this:

app 2/2 (n:,d:,r:menu) [actions: click] [subs: Action Component
Selection ] (children: 12)
app 2/2   - [states: enabled, sensitive, vertical]
app 2/2   - attr 1/2 class:autocomplete-richlistbox
app 2/2   - attr 2/2 tag:xul:richlistbox

If I click on the drop-down button in the address bar to show the menu
and then press f8, All the information for these two items is identical.
So either there is some other information available to at-spi to tell if
the menu is really visible that I do not know about, or Firefox's at-spi
interface is incomplete.

-Sam

PS: I originally implemented a handler for F11 instead of F8.  The
at-spi documentation says if the handler returns SPI_TRUE, then the F11
will not be passed on.  However, if Firefox has the focus, then the F11
is not delivered to my app's F11 Handler, it is delivered only to
Firefox and it causes Firefox to become full screen.  Is this an
intentional limitation of Firefox's at-spi interface?


-----Original Message-----
From:
dev-accessibility-bounces+sam.quiring=[hidden email]
[mailto:dev-accessibility-bounces+sam.quiring=[hidden email]
a.org] On Behalf Of Aaron Leventhal
Sent: Monday, December 22, 2008 3:05 PM
To: [hidden email]
Subject: Re: redundant object: What did user click?

Hmm, if you can get the coordinates of the mouse then you can use AT-SPI
to get the accessible object at that point. Does that work?

I probably won't be able to read any replies until around January 5.

- Aaron

On 12/19/2008 8:04 PM, Quiring, Sam wrote:
> Greetings,
>
> I've got an application that uses at-spi 1.22.1 on Ubuntu 8.04 (Hardy
> Heron).  My current goal is to listen for the right events so when the

> user clicks the left-mouse-button on a GUI object my application can
> identify/understand the object that was clicked.
>
> For Gtk applications, I've had pretty good success listening for the
> Gtk:GtkWidget:button-press-event.  The event->source object that is
> delivered to that listener is usually the right one.  My application
> is working pretty well on Gtk applications.
>
> When I run Firefox (V3.0.1) on Ubuntu my app does get the
> button-press-event when the mouse in Firefox.  Unfortunately, the
> event->source object is "redundant object".  That Accessible *object
> event->has
> no children and no parent.  Obviously button-press-event is not a good

> event for which to listen when it comes to Firefox.
>
> Does anyone know what event(s) my app should be listening for and how
> to best retrieve the object clicked by the user?
>
> Thanks.
>
> -Sam

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: redundant object: What did user click?

Steve Lee-3
2008/12/29 Quiring, Sam <[hidden email]>:
> If I hover the mouse over the "Site" radio button and press F8, it finds
> two objects there.  One is the radio button.  The other comes from the
> Firefox address bar dropdown menu, which appears to be a history of most
> recently visited sites.  The menu is not visible when I press F8, but
> the at-spi interface does not deliver enough information for me to tell
> whether it is visible or not.

Hi Sam, you may need to check both visible and showing and look up the
parent chain. This is an area of the spec that causes quite a few
FAQs. if it is a menu you may need to work your way up to the popup
window. Willie explained some of the issues when I thought I had an
issue here.

http://www.nabble.com/AT_SPI-states-of-documents-and-elements-td14023595.html#a14023595

Steve Lee
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
sbq
Reply | Threaded
Open this post in threaded view
|

RE: redundant object: What did user click?

sbq
Steve,

Thank you very much for this pointer.  Armed with the information
there about STATE_SHOWING and STATE_VISIBLE, I figured out how to
log the at-spi tree with a drop-down menu being displayed and one
not being displayed.  As you suspected, the menu item is VISIBLE
and SHOWING in either case (why is that?), but the parent menu is
only VISIBLE and SHOWING when the dropdown is being displayed.

-Sam

-----Original Message-----
From: Steve Lee [mailto:[hidden email]]
Sent: Monday, December 29, 2008 1:02 PM
To: Quiring, Sam
Cc: Aaron Leventhal; [hidden email]
Subject: Re: redundant object: What did user click?

2008/12/29 Quiring, Sam <[hidden email]>:
> If I hover the mouse over the "Site" radio button and press F8, it
> finds two objects there.  One is the radio button.  The other comes
> from the Firefox address bar dropdown menu, which appears to be a
> history of most recently visited sites.  The menu is not visible when
> I press F8, but the at-spi interface does not deliver enough
> information for me to tell whether it is visible or not.

Hi Sam, you may need to check both visible and showing and look up the
parent chain. This is an area of the spec that causes quite a few FAQs.
if it is a menu you may need to work your way up to the popup window.
Willie explained some of the issues when I thought I had an issue here.

http://www.nabble.com/AT_SPI-states-of-documents-and-elements-td14023595
.html#a14023595

Steve Lee
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility