PhET Forces and Motion simulation - Keyboard / switch and non-visual interactions

[Jonathan Hung]

NVDA has a navigation mode called "focus mode" or "form mode" which is
designed to enable easy navigation of forms. While in this mode, the cursor
keys allow the user to navigate through lists and Tab / Shift+Tab through
interactive elements. This article at WebAim has some details:
http://webaim.org/articles/nvda/#forms . Here's an example of a keyboard
accessible image carousel which works with NVDA in form mode:
http://hanshillen.github.io/jqtest/#goto_carousel. I imagine something
similar can be done with the simulation.

A live region on the cart would definitely help since it will convey
changes. However, we will still need a way to describe the cart to the
user, especially for the initial stages of the game when the user is
getting oriented. Perhaps a description of the cart can be presented
through an "Overview" option or something similar? It would be similar to
those introduction screens popular with phone apps.

The text labels help the user understand what will happen if they select
that option. A speaker icon that looks like a button alone may not convey
enough information - does the button mean it will mute the audio, or is it
telling me it's already muted and that activating the button will turn on
the sound? I think we will need to come up with a different widget design
if we want to remove text labels. Perhaps a toggle switch instead? This way
the user at a glance can see if the audio is muted or not, and they would
understand what happens if they activate the switch. Accessibility wise it
would be implemented like a radio button group.

As for the restart button - it's important that it's clear what happens
when the button is pressed since it's a "destructive" operation. A text
label seems like the easiest way to do this, but perhaps there's another
way? What if we just had a button that said "Clear" instead?

Using aria-label or aria-labelledby is a good way to label something if
we're not using any visible text.

Thanks Sam!

- Jon.

On Thu, Dec 11, 2014 at 11:04 PM, Sam Reid <reids at colorado.edu> wrote:
>
>  Jon,
>
> Excellent work, your descriptions are very clear and I think you have
> addressed many of the questions we had about how things should act (such as
> focusing an invisible element).  I also like how you portrayed the rope as
> "falling" to the ground when not held up, and your ideas for the text of
> the scene.
>
> I had a question about using the arrow keys to move the pullers.  We
> noticed that NVDA intercepts the arrow keys (so that, for example, pressing
> the right arrow key says the next letter and does not get processed by the
> browser).  How should we handle this?
>
> For the text of the scene: you mentioned making the cart focusable so that
> it could read out information to the user (and also integrating the brake
> controls into the cart).  However, unless paired with a "live region", a
> user would have to navigate to the cart to hear how it had changed
> (otherwise they might not know something had happened after pressing
> "go").  What is your opinion of using a live region to read out the status
> of the net forces and motion of the cart?  If we do use a live region,
> would the cart-focusability still be necessary?
>
> Regarding the text on the "restart game" and "mute audio" buttons: would
> it be sufficient for this text to be in an offscreen aria-label so that the
> screen reader can still read it aloud, or should it be visible as text on
> the screen as well?  Sometimes in our simulations we try to reduce text on
> the screen (for a variety of reasons).  May be a good topic for discussion
> at our next meeting.
>
> Best Regards,
> Sam
>
>
> On 12/11/14 8:34 PM, Jonathan Hung wrote:
>
>  The original message was not delivered because the PDF attachment was
> too big. Here is a link to the PDF instead:
>
> http://wiki.fluidproject.org/download/attachments/42009249/Forces-and-motion-01-jh.pdf?api=v2
>
>  I apologize if you receive duplicate messages.
>
> The original message follows:
>
>
> Hi everyone,
>
> I have created some early designs for keyboard interaction, and for
> non-visual interaction for the Net Force simulation under "Forces and
> Motion Basics" (link to Forces and Motion Basics at PhET
> <http://phet.colorado.edu/sims/html/forces-and-motion-basics/latest/forces-and-motion-basics_en.html>
> ).
>
> *Keyboard Interaction*
> Attached to this email is a PDF containing a visual walk through the Net
> Force simulation using a switch or keyboard only. Note: at this time the
> PDF is completely visual and lacks appropriate accessible text descriptions.
>
> *Non-Visual Interaction*
> In addition to the keyboard interaction, I have begun sketching out what a
> non-visual interaction for the same simulation. The results can be found on
> this wiki page:
> wiki.fluidproject.org/display/fluid/PhET+Forces+and+Motion+Simulation+Design
>
> In exploring non-visual interaction, two significant issues have come up:
>
> 1. Non-visual users will require an up-front description of the scene so
> the user understands their context. Currently the simulation does not have
> a description for this purpose.
>
> 2. Additional descriptions need to accompany non-interactive objects. For
> example, in the simulation the cart is never interacted with directly, but
> the cart contains crucial information to the understanding of the
> simulation: where is it located? has it moved, and in which direction? how
> fast is it going? how many people are pulling on it? what is the net force?
>
> A sighted user can discern this information from all the visual cues, and
> somehow this information needs to be conveyed to a non-sighted user.
>
> *"Braking" the Cart*
> A possible solution is to make the cart part of the tab order, and upon
> focusing, provide relevant information to the user. However, this breaks
> the established interaction pattern of the simulation where only
> interactive items can get tab focus. The cart lacks any interactive parts -
> so it may make sense to add some interactivity to the cart itself.
>
> One possible idea (and this may not be a good idea) is to remove the
> Play/Pause button that controls the simulation currently. Instead, we add a
> brake on the cart that controls its movement. So upon focusing the cart, a
> summary of the cart's status can be given to the user, and the user can
> apply / remove the brakes on the cart to stop and start the simulation.
>
> *What now?*
>
> The designs are in an early state and we're just beginning to uncover some
> issues we have not considered before. I would love some feedback from the
> community regarding the designs and the issues outlined in this email.
>
> Thanks in advance for any feedback.
>
> - Jon.
>
>
>
>  --
>
> *JONATHAN HUNG*
>
> INCLUSIVE DESIGNER, IDRC
>
>
>
> *T:* 416 977 6000 x3951
>
> *F:* 416 977 9844
>
> *E:* jhung at ocadu.ca
>
>
>
> *OCAD UNIVERSITY*
>
> Inclusive Design Research Centre
>
> 205 Richmond Street W, Toronto, ON, M5V 1V3
>
>
>
> www.ocadu.ca
>
> www.idrc.ocad.ca
>
>
>

-- 

*JONATHAN HUNG*

INCLUSIVE DESIGNER, IDRC



*T:* 416 977 6000 x3951

*F:* 416 977 9844

*E:* jhung at ocadu.ca



*OCAD UNIVERSITY*

Inclusive Design Research Centre

205 Richmond Street W, Toronto, ON, M5V 1V3



www.ocadu.ca

www.idrc.ocad.ca
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idrc.ocad.ca/pipermail/fluid-work/attachments/20141212/6aa64c56/attachment.html>