Articles on Mobile Application User Experience Evaluation

[James William Yoon]

In fact, I think you met Muriel once, Hugues--she was around for the Fluid
all-hands meeting last summer! Thanks for the papers--I'll also be reading
through these.

Also, we set up a page to work on our 0.3 user testing notes. It's a copy of
our previous protocol, with some modifications and additions (list of all
possible tasks, for instance). It's still very rough and early right now,
and we have a lot we need to take out that's irrelevant and more to pump in
(esp. wrt to guiding the semi-structured exit interview):

http://wiki.fluidproject.org/display/fluid/Mobile+user+testing+protocol+-+Engage+0.3

Cheers,
James

On Fri, Feb 12, 2010 at 1:48 PM, tona monjo <amonjop at uoc.edu> wrote:

> Thanks, Hugues, I will read them all. They seem really interesting, because
> testing for mobile implies working in a quite different paradigm from what
> we are used to desktop apps.
>
> ... and yes, Eva and Muriel are working a lot in the field of emotional
> heuristics! Eva is  a member of the board of Directors of FE, so she's also
> implied in this project.
>
> Thanks for these references! Have a nice weekend,
>
> Tona
>
>
> On Fri, Feb 12, 2010 at 6:00 PM, Hugues Boily <
> hugues.boily at mccord.mcgill.ca> wrote:
>
>>  Hi guys,
>>
>> I thought you might be interested in some of the mobile evaluation
>> literature i have been reviewing. (Tona, the last paper, about emotion
>> heuristics, is by researchers from the Universitat Oberta de Catalunya!)
>>
>> Cheers,
>> Hugues
>>
>>
>> Jambon, F. (2006). Reality Testing of Mobile Devices: How to Ensure
>> Analysis Validity? CHI 2006 Workshop "Reality Testing: HCI Challenges in
>> Non-Traditional Environments", Montreal, Quebec, Canada, April 22-27
>>
>> http://www.msh-alpes.prd.fr/ADAMOS/public_html/material/chi2006RealityTesting-jambon.pdf
>>
>> ABSTRACT : This article describes, in the first part, the motivations of
>> the MultiCom group at CLIPS-IMAG laboratory in reality testing of mobile
>> devices. The second part of the article is dedicated to the methodological
>> aspects of data collection and analysis when evaluating the usability of a
>> mobile device in a quasi-realistic context.*
>> *
>> --
>> Jurgen Kawalek, Annegret Stark, and Marcel Riebeck 2008. A New Approach to
>> Analyze Human- Mobile Computer Interaction, in Journal of Usability Studies,
>> Volume 3, Issue 2<http://www.upassoc.org/upa_publications/jus/2008february/index.html>,
>> February 2008, pp. 90-98
>>
>> http://www.upassoc.org/upa_publications/jus/2008february/JUS_kawalek_feb2008.pdf
>>
>> ABSTRACT : This paper describes a tool for log file recording and a method
>> for quickly and easily analyzing human-computer interaction with mobile
>> devices. The tool logs screen shots and quantitative interaction data, such
>> as number of clicks and timestamps. The analyzing tool provides the ability
>> to evaluate the interaction sequences and to export an MS Excel®-sheet for
>> statistical analysis. To evaluate the tool, a usability study was conducted
>> comparing the effectiveness of this tool in the laboratory and in the mobile
>> context. Findings show that the tool is the first step toward a very
>> effective, unobtrusive analysing method for user interaction in the mobile
>> context. Combined with debriefing methods, it would be an optimized way for
>> usability testing with mobile devices.
>>
>> --
>> Roto, V., Oulasvirta, A., Haikarainene, T., Kuorelahti, J., Lehmuskallio,
>> H., and Nyyssönen, T. (2004). Examining mobile phone use in the wild with
>> quasi-experimentation. HIIT Technical Report 2004-1, Helsinki Institute for
>> Information Technology, Helsinky, Finland.
>>
>> http://www.hiit.fi/files/admin/publications/Technical_Reports/hiit2004-1.pdf
>>
>> ABSTRACT : This paper describes a methodology for conducting controlled
>> experiments in the wild. The authors describe the limitation of many
>> methodologies for conducting research with mobile technology. Their initial
>> argument is that currently it is not possible to conduct controlled
>> experiments in the wild because the difficulty of controlling nuisance
>> variables and because it is difficult to record comprensively user’s data
>> while on the move. They propose a technique to overcome many of these
>> limitations with micro-cameras mounted on the device and on the body of the
>> subject which record continuously what is happening around the user.
>>
>> This methodology was proven to be effective to conduct reseach on how
>> users assign attentional resources between the different elements of the
>> environment while on the move (Oulasvirta et al., 2005). This paper discuss
>> the pros and cons of this *quasi-experimental methodology*.
>>
>> --
>> de Lera, E. and Garreta-Domingo, M. 2007. Ten emotion heuristics:
>> guidelines for assessing the user's affective dimension easily and
>> cost-effectively. In *Proceedings of the 21st British HCI Group Annual
>> Conference on HCI 2008: People and Computers Xxi: Hci..But Not As We Know It
>> - Volume 2* (University of Lancaster, United Kingdom, September 03 - 07,
>> 2007). British Computer Society Conference on Human-Computer Interaction.
>> British Computer Society, Swinton, UK, 163-166.
>> www.bcs.org/upload/pdf/ewic_hc07_popaper2.pdf
>>
>> ABSTRACT : Emotional appeal is a key dimension in user experience that
>> often goes unmeasured in most user-centered design projects. This paper
>> presents preliminary work for developing a set of guidelines for
>> efficiently, easily and cost-effectively assessing the users' affective
>> state by evaluating their expressive reactions during an interface
>> evaluation process. The evaluation of this dimension complements the
>> analysis of the objective and quantitative data gathered through usability
>> tests and the subjective feedback provided through post-test questionnaires.
>>
>>
>>
>>
>>
>>
>
>
> --
> Tona Monjo
> Disseny d'interficies | Diseño de interfaces | Interface design
> http://www.tonamonjo.com
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://fluidproject.org/pipermail/fluid-work/attachments/20100212/2ddd7ce8/attachment.html>