PhET Forces and Motion simulation - Keyboard / switch and non-visual interactions

Sun Dec 14 17:41:20 EST 2014

Jon,

I don't really have much to add, 'cept the keyboard interaction for this is impressive. It's different than I originally went about thinking on it. Seems more "natural" than duplicating the mouse action, as literally as you can. So easy to get stuck in a rut. So interesting.

And in step number 6 of the pdf explaining the keyboard interaction explicitly, at first I didn't see much use in allowing a user to focus the disabled play/ pause and rewind buttons. But allowing the user to interact with all interactive elements, even if disabled, is key.

When you need this tested, please let me know.

Well done indeed,

Johnny

http://abledaccess.com <http://abledaccess.com/>
http://abledaccess.com/twitter <http://abledaccess.com/twitter>

Access must be abled

> On Dec 12, 2014, at 3:17 PM, Jonathan Hung <jhung at ocadu.ca> wrote:
> 
> NVDA has a navigation mode called "focus mode" or "form mode" which is designed to enable easy navigation of forms. While in this mode, the cursor keys allow the user to navigate through lists and Tab / Shift+Tab through interactive elements. This article at WebAim has some details: http://webaim.org/articles/nvda/#forms <http://webaim.org/articles/nvda/#forms> . Here's an example of a keyboard accessible image carousel which works with NVDA in form mode: http://hanshillen.github.io/jqtest/#goto_carousel <http://hanshillen.github.io/jqtest/#goto_carousel>. I imagine something similar can be done with the simulation.
> 
> A live region on the cart would definitely help since it will convey changes. However, we will still need a way to describe the cart to the user, especially for the initial stages of the game when the user is getting oriented. Perhaps a description of the cart can be presented through an "Overview" option or something similar? It would be similar to those introduction screens popular with phone apps.
> 
> The text labels help the user understand what will happen if they select that option. A speaker icon that looks like a button alone may not convey enough information - does the button mean it will mute the audio, or is it telling me it's already muted and that activating the button will turn on the sound? I think we will need to come up with a different widget design if we want to remove text labels. Perhaps a toggle switch instead? This way the user at a glance can see if the audio is muted or not, and they would understand what happens if they activate the switch. Accessibility wise it would be implemented like a radio button group.
> 
> As for the restart button - it's important that it's clear what happens when the button is pressed since it's a "destructive" operation. A text label seems like the easiest way to do this, but perhaps there's another way? What if we just had a button that said "Clear" instead?
> 
> Using aria-label or aria-labelledby is a good way to label something if we're not using any visible text.
> 
> Thanks Sam!
> 
> - Jon.
> 
> On Thu, Dec 11, 2014 at 11:04 PM, Sam Reid <reids at colorado.edu <mailto:reids at colorado.edu>> wrote:
> Jon,
> 
> Excellent work, your descriptions are very clear and I think you have addressed many of the questions we had about how things should act (such as focusing an invisible element).  I also like how you portrayed the rope as "falling" to the ground when not held up, and your ideas for the text of the scene.
> 
> I had a question about using the arrow keys to move the pullers.  We noticed that NVDA intercepts the arrow keys (so that, for example, pressing the right arrow key says the next letter and does not get processed by the browser).  How should we handle this?
> 
> For the text of the scene: you mentioned making the cart focusable so that it could read out information to the user (and also integrating the brake controls into the cart).  However, unless paired with a "live region", a user would have to navigate to the cart to hear how it had changed (otherwise they might not know something had happened after pressing "go").  What is your opinion of using a live region to read out the status of the net forces and     motion of the cart?  If we do use a live region, would the cart-focusability still be necessary?
> 
> Regarding the text on the "restart game" and "mute audio" buttons: would it be sufficient for this text to be in an offscreen aria-label so that the screen reader can still read it aloud, or should it be visible as text on the screen as well?  Sometimes in our simulations we try to reduce text on the screen (for a variety of reasons).  May be a good topic for discussion at our next meeting.
> 
> Best Regards,
> Sam
> 
> 
> On 12/11/14 8:34 PM, Jonathan Hung wrote:
>> The original message was not delivered because the PDF attachment was too big. Here is a link to the PDF instead: 
>> http://wiki.fluidproject.org/download/attachments/42009249/Forces-and-motion-01-jh.pdf?api=v2 <http://wiki.fluidproject.org/download/attachments/42009249/Forces-and-motion-01-jh.pdf?api=v2>
>> 
>> I apologize if you receive duplicate messages.
>> 
>> The original message follows:
>> 
>> 
>> Hi everyone,
>> 
>> I have created some early designs for keyboard interaction, and for non-visual interaction for the Net Force simulation under "Forces and Motion Basics" (link to Forces and Motion Basics at PhET <http://phet.colorado.edu/sims/html/forces-and-motion-basics/latest/forces-and-motion-basics_en.html>). 
>> 
>> Keyboard Interaction
>> Attached to this email is a PDF containing a visual walk through the Net Force simulation using a switch or keyboard only. Note: at this time the PDF is completely visual and lacks appropriate accessible text descriptions.
>> 
>> Non-Visual Interaction
>> In addition to the keyboard interaction, I have begun sketching out what a non-visual interaction for the same simulation. The results can be found on this wiki page: wiki.fluidproject.org/display/fluid/PhET+Forces+and+Motion+Simulation+Design <http://wiki.fluidproject.org/display/fluid/PhET+Forces+and+Motion+Simulation+Design>
>> 
>> In exploring non-visual interaction, two significant issues have come up: 
>> 
>> 1. Non-visual users will require an up-front description of the scene so the user understands their context. Currently the simulation does not have a description for this purpose.
>> 
>> 2. Additional descriptions need to accompany non-interactive objects. For example, in the simulation the cart is never interacted with directly, but the cart contains crucial information to the understanding of the simulation: where is it located? has it moved, and in which direction? how fast is it going? how many people are pulling on it? what is the net force?
>> 
>> A sighted user can discern this information from all the visual cues, and somehow this information needs to be conveyed to a non-sighted user. 
>> 
>> "Braking" the Cart
>> A possible solution is to make the cart part of the tab order, and upon focusing, provide relevant information to the user. However, this breaks the established interaction pattern of the simulation where only interactive items can get tab focus. The cart lacks any interactive parts - so it may make sense to add some interactivity to the cart itself.
>> 
>> One possible idea (and this may not be a good idea) is to remove the Play/Pause button that controls the simulation currently. Instead, we add a brake on the cart that controls its movement. So upon focusing the cart, a summary of the cart's status can be given to the user, and the user can apply / remove the brakes on the cart to stop and start the simulation.
>> 
>> What now?
>> 
>> The designs are in an early state and we're just beginning to uncover some issues we have not considered before. I would love some feedback from the community regarding the designs and the issues outlined in this email. 
>> 
>> Thanks in advance for any feedback.
>> 
>> - Jon.
>> 
>> 
>> 
>> -- 
>> JONATHAN HUNG
>> 
>> INCLUSIVE DESIGNER, IDRC
>> 
>>  
>> T: 416 977 6000 x3951 <>
>> F: 416 977 9844 <>
>> E: jhung at ocadu.ca <mailto:jhung at ocadu.ca>
>>  
>> OCAD UNIVERSITY
>> 
>> Inclusive Design Research Centre
>> 
>> 205 Richmond Street W, Toronto, ON, M5V 1V3
>> 
>>  
>> www.ocadu.ca <http://www.ocadu.ca/>
>> www.idrc.ocad.ca <http://www.idrc.ocad.ca/>
> 
> 
> -- 
> JONATHAN HUNG
> 
> INCLUSIVE DESIGNER, IDRC
>  
> T: 416 977 6000 x3951 <>
> F: 416 977 9844 <>
> E: jhung at ocadu.ca <mailto:jhung at ocadu.ca>
>  
> OCAD UNIVERSITY
> Inclusive Design Research Centre
> 205 Richmond Street W, Toronto, ON, M5V 1V3
>  
> www.ocadu.ca <http://www.ocadu.ca/>
> www.idrc.ocad.ca <http://www.idrc.ocad.ca/>_______________________________________________________
> fluid-work mailing list - fluid-work at fluidproject.org
> To unsubscribe, change settings or access archives,
> see http://lists.idrc.ocad.ca/mailman/listinfo/fluid-work

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.idrc.ocad.ca/pipermail/fluid-work/attachments/20141214/05ff3098/attachment.html>