Live region interruptions

classic Classic list List threaded Threaded
12 messages Options
Reply | Threaded
Open this post in threaded view
|

Live region interruptions

Peter Parente
I'm looking at the very nice report by Charles about his experience
supporting live regions in FireVox at http://developer.mozilla.org/en/docs/AJAX:WAI_ARIA_Live_Regions.
A question comes to mind about interruptions for a speech screen
reader user. Take a chat example again, where messages can be marked
with politeness levels individually. Assume everything is polite in
the following chat history:

<John> hello everyone
<Jane> hi John
<Bill> hi!

Say the user (Jill) starts typing a lengthy response (e.g., "John, i
need to talk to you ASAP about the report we have due today") to John
immediately after hearing his initial "hello everyone" message. Assume
also that the user has some echo on that causes the AT to speak either
the letters or words as Jill types them. Typically, new user input
interrupts output from the screen reader. In this case, the first
letter or word typed by Jill would stop the screen reader from
speaking the messages from Jane and Bill. Does anyone disagree?

Now pretend that while the user is typing, the following rude message
appears:

<Chat Server> You connection has closed unexpectedly.

According to the ARIA spec and Charles's report, rude message should
be announced immediately, possibly clearing all other live region
messages except those that are rude. But what happens when the user is
busy typing? Without any special handling on the part of the screen
reader, I imagine the rude speech starts, but then is immediately
interrupted as Jill continues typing her message. She may miss the
rude message entirely if she presses a key or finishes a word just as
the announcement starts. With some smarts, the screen reader might
suppress all of its announcements, including those triggered by user
actions away from the live region, while the rude message is being
spoken. But now Jill is required to press an explicit "Stop" key so
that the screen reader continues respond to her input instead of
continuing to speak the rude message.

Has FireVox handled this situation? Has this topic come up in past
conversation that I missed?

Finally, it's worth noting that this situation is not limited to chat.
Say the user starts a long running process in a Web application and
then switches to another tab to continue browsing some other page.
Imagine the Web app brings up a rude error dialog saying the process
halted and needs attention. If the user is browsing another page in
the second tab, typing in some edit field, etc. his next action might
interrupt the rude alert speech.

Comments and guidance greatly appreciated!

Pete

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Live region interruptions

Gijs Kruitbosch ("Hannibal")
Peter Parente wrote:

> I'm looking at the very nice report by Charles about his experience
> supporting live regions in FireVox at http://developer.mozilla.org/en/docs/AJAX:WAI_ARIA_Live_Regions.
> A question comes to mind about interruptions for a speech screen
> reader user. Take a chat example again, where messages can be marked
> with politeness levels individually. Assume everything is polite in
> the following chat history:
>
> <John> hello everyone
> <Jane> hi John
> <Bill> hi!
>
> Say the user (Jill) starts typing a lengthy response (e.g., "John, i
> need to talk to you ASAP about the report we have due today") to John
> immediately after hearing his initial "hello everyone" message. Assume
> also that the user has some echo on that causes the AT to speak either
> the letters or words as Jill types them. Typically, new user input
> interrupts output from the screen reader. In this case, the first
> letter or word typed by Jill would stop the screen reader from
> speaking the messages from Jane and Bill. Does anyone disagree?
>
> Now pretend that while the user is typing, the following rude message
> appears:
>
> <Chat Server> You connection has closed unexpectedly.
>
> According to the ARIA spec and Charles's report, rude message should
> be announced immediately, possibly clearing all other live region
> messages except those that are rude. But what happens when the user is
> busy typing? Without any special handling on the part of the screen
> reader, I imagine the rude speech starts, but then is immediately
> interrupted as Jill continues typing her message. She may miss the
> rude message entirely if she presses a key or finishes a word just as
> the announcement starts. With some smarts, the screen reader might
> suppress all of its announcements, including those triggered by user
> actions away from the live region, while the rude message is being
> spoken. But now Jill is required to press an explicit "Stop" key so
> that the screen reader continues respond to her input instead of
> continuing to speak the rude message.
>
> Has FireVox handled this situation? Has this topic come up in past
> conversation that I missed?
>
> Finally, it's worth noting that this situation is not limited to chat.
> Say the user starts a long running process in a Web application and
> then switches to another tab to continue browsing some other page.
> Imagine the Web app brings up a rude error dialog saying the process
> halted and needs attention. If the user is browsing another page in
> the second tab, typing in some edit field, etc. his next action might
> interrupt the rude alert speech.
>
> Comments and guidance greatly appreciated!
>
> Pete
>

Would a non-speech sound solve this issue? ie, a loud buzzer-like thing
announcing the error coming up before actually speaking it? I would
imagine that in that case, the user can decide for themselves whether
they'd like to continue typing or listen to the error message. The trick
would be for the 'beep' to be long enough to get attention yet short
enough to not drown out Jill's typing.

~ Gijs

PS: I have no idea what screenreaders/FireVox currently do, this just
comes to mind as the most straightforward way of solving the issue (in
my mind, anyway).

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Live region interruptions

Peter Parente
> Would a non-speech sound solve this issue? ie, a loud buzzer-like thing
> announcing the error coming up before actually speaking it?

Sure. If you can have more than one output stream playing at a time,
this problem is easy to solve. I'm wondering about the case where you
only have one output channel (i.e., one speech stream).

Pete

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

RE: Live region interruptions

Sina Bahram
In reply to this post by Peter Parente
This topic has definitely come up before, and I'm glad you're bringing it up
again in a different context.

Pete, have you gotten a chance to examine my priority suggestion for
"notification" which Charles, Gijs, and myself recommended?

Basically it handles this situation by allowing an interuption that is
queueable.

I firmly believe that rude's should queue, because those letters being
echoed should not be rude in the first place, they should be a form of
notification that clears.

Rudes should queue; however, in my opinion. We should then have a super rude
that essentially clears everything.

Take care,
Sina

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Peter
Parente
Sent: Tuesday, May 01, 2007 1:32 PM
To: [hidden email]
Subject: Live region interruptions

I'm looking at the very nice report by Charles about his experience
supporting live regions in FireVox at
http://developer.mozilla.org/en/docs/AJAX:WAI_ARIA_Live_Regions.
A question comes to mind about interruptions for a speech screen reader
user. Take a chat example again, where messages can be marked with
politeness levels individually. Assume everything is polite in the following
chat history:

<John> hello everyone
<Jane> hi John
<Bill> hi!

Say the user (Jill) starts typing a lengthy response (e.g., "John, i need to
talk to you ASAP about the report we have due today") to John immediately
after hearing his initial "hello everyone" message. Assume also that the
user has some echo on that causes the AT to speak either the letters or
words as Jill types them. Typically, new user input interrupts output from
the screen reader. In this case, the first letter or word typed by Jill
would stop the screen reader from speaking the messages from Jane and Bill.
Does anyone disagree?

Now pretend that while the user is typing, the following rude message
appears:

<Chat Server> You connection has closed unexpectedly.

According to the ARIA spec and Charles's report, rude message should be
announced immediately, possibly clearing all other live region messages
except those that are rude. But what happens when the user is busy typing?
Without any special handling on the part of the screen reader, I imagine the
rude speech starts, but then is immediately interrupted as Jill continues
typing her message. She may miss the rude message entirely if she presses a
key or finishes a word just as the announcement starts. With some smarts,
the screen reader might suppress all of its announcements, including those
triggered by user actions away from the live region, while the rude message
is being spoken. But now Jill is required to press an explicit "Stop" key so
that the screen reader continues respond to her input instead of continuing
to speak the rude message.

Has FireVox handled this situation? Has this topic come up in past
conversation that I missed?

Finally, it's worth noting that this situation is not limited to chat.
Say the user starts a long running process in a Web application and then
switches to another tab to continue browsing some other page.
Imagine the Web app brings up a rude error dialog saying the process halted
and needs attention. If the user is browsing another page in the second tab,
typing in some edit field, etc. his next action might interrupt the rude
alert speech.

Comments and guidance greatly appreciated!

Pete

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Live region interruptions

T.V Raman
In reply to this post by Peter Parente

Peter --
You raise some very good UI questions,  and I believe these are
broader than just WebApps or live-regions.

To date, screenreader users have been limited to  the following
interaction model:

A)  There is  one TTS engine talking.

B)  There is only one stream of speech.

C)  There is very limited use of non-speech auditory feedback.

The above assumptions  tend to sandbox the UI paradigms you can
expose to the types of interactions we see in screenreaders
today; they also naturally raise the type of questions you ask
when it comes to a multitasking, multithreaded environment.

As an example, I'm typing this message in an email buffer in
emacs with character and word echo on.
If you were to IM me this instant (and were on my set of "can
interrupt" buddies),
I'd hear an audio cue and your name. Depending on your priority
level in my buddy list, I might also hear what you said. But that
spoken message would get interrupted by word/char echo --- with
the added functionality that I can have the last  mesage
repeated. In general, in the emacspeak environment, I can
actually go back and hear all such queued up messages.

So notice that adding multiple auditory output streams, and
enabling a  layered customization mechanism allows  one to answer
a lot of these questions at the UI level.

Going back to what needs to be exposed from Firefox via the live
properties, I think the right means of arriving at the correct
answer is to enumerate the various types of user interaction
tools like LSR would like to enable, and then work backward from
there to ensuring that the DOM carries sufficiently rich
information to enable those interactions.



Peter Parente writes:
 > I'm looking at the very nice report by Charles about his experience
 > supporting live regions in FireVox at http://developer.mozilla.org/en/docs/AJAX:WAI_ARIA_Live_Regions.
 > A question comes to mind about interruptions for a speech screen
 > reader user. Take a chat example again, where messages can be marked
 > with politeness levels individually. Assume everything is polite in
 > the following chat history:
 >
 > <John> hello everyone
 > <Jane> hi John
 > <Bill> hi!
 >
 > Say the user (Jill) starts typing a lengthy response (e.g., "John, i
 > need to talk to you ASAP about the report we have due today") to John
 > immediately after hearing his initial "hello everyone" message. Assume
 > also that the user has some echo on that causes the AT to speak either
 > the letters or words as Jill types them. Typically, new user input
 > interrupts output from the screen reader. In this case, the first
 > letter or word typed by Jill would stop the screen reader from
 > speaking the messages from Jane and Bill. Does anyone disagree?
 >
 > Now pretend that while the user is typing, the following rude message
 > appears:
 >
 > <Chat Server> You connection has closed unexpectedly.
 >
 > According to the ARIA spec and Charles's report, rude message should
 > be announced immediately, possibly clearing all other live region
 > messages except those that are rude. But what happens when the user is
 > busy typing? Without any special handling on the part of the screen
 > reader, I imagine the rude speech starts, but then is immediately
 > interrupted as Jill continues typing her message. She may miss the
 > rude message entirely if she presses a key or finishes a word just as
 > the announcement starts. With some smarts, the screen reader might
 > suppress all of its announcements, including those triggered by user
 > actions away from the live region, while the rude message is being
 > spoken. But now Jill is required to press an explicit "Stop" key so
 > that the screen reader continues respond to her input instead of
 > continuing to speak the rude message.
 >
 > Has FireVox handled this situation? Has this topic come up in past
 > conversation that I missed?
 >
 > Finally, it's worth noting that this situation is not limited to chat.
 > Say the user starts a long running process in a Web application and
 > then switches to another tab to continue browsing some other page.
 > Imagine the Web app brings up a rude error dialog saying the process
 > halted and needs attention. If the user is browsing another page in
 > the second tab, typing in some edit field, etc. his next action might
 > interrupt the rude alert speech.
 >
 > Comments and guidance greatly appreciated!
 >
 > Pete
 >
 > _______________________________________________
 > dev-accessibility mailing list
 > [hidden email]
 > https://lists.mozilla.org/listinfo/dev-accessibility

--
Best Regards,
--raman

Title:  Research Scientist      
Email:  [hidden email]
WWW:    http://emacspeak.sf.net/raman/
Google: tv+raman
GTalk:  [hidden email], [hidden email]
PGP:    http://emacspeak.sf.net/raman/raman-almaden.asc

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

RE: Live region interruptions

Sina Bahram
In reply to this post by Peter Parente
I actually think there is a larger problem of semantic mapping here.

How does the user know that this sound is mapped to the semantics of
"warning message" or "error message" or "important message waiting"?

The use of a sound would offer an extra dimension of access to the user's
attention, but how would the user associate the proper semantics with this
sound? Furthermore, the point of such a sound would be to notify the user of
the existence of such a condition or of a message that explains such a
condition. This implies that there is a facility to access the information
content of this message. How would the user know how to do this, and in what
way would this actually be facilitated? Would it be via the AT offering a
keystroke or other mechanism to buffer this message for such time that the
user requests to know about it? Doesn't that, by its nature, remove the
importance of the message; thus, changing the semantics through the way it's
being conveyed? Possibly, the message would be queued up application side,
but this presents a burden, not in complexity, but just as an annoyance for
application developers to keep state information; thus, rendering the entire
point of the priority system a bit useless in the first place?

I'm not sure anyone knows the perfect answer to this, but I think it's
important to think about the whole process before immediately going to sound
icons. They are superbly powerful, and can be even more efficient than the
fastest speech engine in conveying short chunks of semantic information to
the user, but only if used properly so that the user can extract such
semantics from them, rather than simply being confused at what's going on.

Take care,
Sina

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Peter
Parente
Sent: Tuesday, May 01, 2007 2:31 PM
To: [hidden email]
Subject: Re: Live region interruptions

> Would a non-speech sound solve this issue? ie, a loud buzzer-like
> thing announcing the error coming up before actually speaking it?

Sure. If you can have more than one output stream playing at a time, this
problem is easy to solve. I'm wondering about the case where you only have
one output channel (i.e., one speech stream).

Pete

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Live region interruptions

Peter Parente
In reply to this post by Peter Parente
Just to let people know...

I'm pushing this question to get people thinking about problems
inherent in a single auditory stream. I'm all for supporting
multichannel audio. It's my thesis topic (see http://www.cs.unc.edu/~parente/clique)
and there's actually support for it in LSR (see vids #2 and #4 at
http://www.gnome.org/~parente/lsr/screencast). The problem is, I
haven't seen a good, open source audio mixer that can provide highly
responsive mixing and HRTF spatialization, two features needed to make
a concurrent stream solution usable. Since a single speech channel is
the norm today, and will certainly remain a baseline no matter what
the future brings, it has to be a solid, usable fallback when multiple
channels, multiple voices, auditory icons/earcons, etc. are not
readily available.

Pete

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Live region interruptions

Steve Lee-3
At the risk of being left field that makes me think you might benefit
from a form of 'semantic mixer' component provided at the desktop
level. By that I mean a standard component that provides services for
screen readers or applications for queuing and prioritizing events
with callbacks for detail. it would provide users a 'place' to go with
aural UI providing management of mixing/queuing and discovery e.g.
repeat alerts (icons), drill down to description, switch to
application that presented the info. Perhaps ALL TTS could go through
it and it could eventually be expanded to handle multichannel.

That effectively pulls out some of the functionality into a system
service. Perhaps a little like the current sys trays, but not as rude
and certainly not grabbing input focus when alerting the user ;-) It
could even be part of an enhanced tray as some of those services for
managing asynchronous events or sound icons should be of use to all
users.

Just an idea, and not well thought out as I've not been following this
thread in detail.

On 1 May 2007 18:41:48 -0700, Peter Parente <[hidden email]> wrote:

> Just to let people know...
>
> I'm pushing this question to get people thinking about problems
> inherent in a single auditory stream. I'm all for supporting
> multichannel audio. It's my thesis topic (see http://www.cs.unc.edu/~parente/clique)
> and there's actually support for it in LSR (see vids #2 and #4 at
> http://www.gnome.org/~parente/lsr/screencast). The problem is, I
> haven't seen a good, open source audio mixer that can provide highly
> responsive mixing and HRTF spatialization, two features needed to make
> a concurrent stream solution usable. Since a single speech channel is
> the norm today, and will certainly remain a baseline no matter what
> the future brings, it has to be a solid, usable fallback when multiple
> channels, multiple voices, auditory icons/earcons, etc. are not
> readily available.
>
> Pete
>
> _______________________________________________
> dev-accessibility mailing list
> [hidden email]
> https://lists.mozilla.org/listinfo/dev-accessibility
>


--
Steve Lee
www.fullmeasure.co.uk
www.oatsoft.org
www.schoolforge.org.uk
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Live region interruptions

Aaron Leventhal-3
In reply to this post by Peter Parente
Does your model allow consideration of the Braille display as a 2nd or 3rd channel?

- Aaron

Peter Parente wrote:

> Just to let people know...
>
> I'm pushing this question to get people thinking about problems
> inherent in a single auditory stream. I'm all for supporting
> multichannel audio. It's my thesis topic (see http://www.cs.unc.edu/~parente/clique)
> and there's actually support for it in LSR (see vids #2 and #4 at
> http://www.gnome.org/~parente/lsr/screencast). The problem is, I
> haven't seen a good, open source audio mixer that can provide highly
> responsive mixing and HRTF spatialization, two features needed to make
> a concurrent stream solution usable. Since a single speech channel is
> the norm today, and will certainly remain a baseline no matter what
> the future brings, it has to be a solid, usable fallback when multiple
> channels, multiple voices, auditory icons/earcons, etc. are not
> readily available.
>
> Pete
>
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Live region interruptions

Peter Parente
In reply to this post by Peter Parente
I'm thinking about using an exponential backoff approach to rude
messages that get interrupted by user input in LSR. If the rude
message is interrupted immediately by some user input (e.g., char
echo), it is re-queued for announcement 1 second later. If it's
interrupted again when it's re-announced, it's re-queued for 2 seconds
later. Once again, for 4 seconds, and so on. The re-queuing attempts
stop after a configurable number of retries, an explicit stop command
from the user, or when all of the message has been reported.

The requirement here is that speech index markers are supported (which
isn't always the case) by the speech engine. A less-than-elegant fix
for this shortcoming is to use time between rude announcement start
and interruption.

Thoughts?
Pete

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

Re: Live region interruptions

Aaron Leventhal-3
I was realizing there should be a limit to how much it's pushed off, because if
I'm typing quickly, the push off could be 2^10 seconds, which isn't sensible.

- Aaron

Peter Parente wrote:

> I'm thinking about using an exponential backoff approach to rude
> messages that get interrupted by user input in LSR. If the rude
> message is interrupted immediately by some user input (e.g., char
> echo), it is re-queued for announcement 1 second later. If it's
> interrupted again when it's re-announced, it's re-queued for 2 seconds
> later. Once again, for 4 seconds, and so on. The re-queuing attempts
> stop after a configurable number of retries, an explicit stop command
> from the user, or when all of the message has been reported.
>
> The requirement here is that speech index markers are supported (which
> isn't always the case) by the speech engine. A less-than-elegant fix
> for this shortcoming is to use time between rude announcement start
> and interruption.
>
> Thoughts?
> Pete
>
_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility
Reply | Threaded
Open this post in threaded view
|

RE: Live region interruptions

Sina Bahram
In reply to this post by Peter Parente
The mathematician  side of me says, use a Fibonacci back off, instead, but
honestly, does any of that matter after around five seconds or so? In other
words, couldn't you handle this case with proper notification levels, and in
your case, speak the message simultaneously, not allowing, temporarily, the
key echo to interrupt the message because of its priority?

Take care,
Sina

-----Original Message-----
From: [hidden email]
[mailto:[hidden email]] On Behalf Of Peter
Parente
Sent: Wednesday, May 02, 2007 10:37 AM
To: [hidden email]
Subject: Re: Live region interruptions

I'm thinking about using an exponential backoff approach to rude messages
that get interrupted by user input in LSR. If the rude message is
interrupted immediately by some user input (e.g., char echo), it is
re-queued for announcement 1 second later. If it's interrupted again when
it's re-announced, it's re-queued for 2 seconds later. Once again, for 4
seconds, and so on. The re-queuing attempts stop after a configurable number
of retries, an explicit stop command from the user, or when all of the
message has been reported.

The requirement here is that speech index markers are supported (which isn't
always the case) by the speech engine. A less-than-elegant fix for this
shortcoming is to use time between rude announcement start and interruption.

Thoughts?
Pete

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility

_______________________________________________
dev-accessibility mailing list
[hidden email]
https://lists.mozilla.org/listinfo/dev-accessibility