Lightbox User Testing Protocol

Michael S Elledge elledge at msu.edu
Thu Oct 18 15:24:49 UTC 2007 

I have a 2:00 - 3:00 PM meeting on Sakai Accessibility Testing; other 
than that I'm available all day until 4:30 EST/1:30 PST. Nothing like 
being in the office. :-)

Just let me know what works for you!

Mike

Barbara Glover wrote:
> Hi Daphne/Mike
> I wanted to thank you for your comments.  I agree it will be good to 
> get a more comprehensive section of faculty than just Eileens.  
> Actually I think it may be difficult to get faculty in general.  In 
> light of us wanting a more comprehensive sample than just our primary 
> persona, Allison's matrices will be helpful to foster us thinking when 
> recruiting faculty.
>
> My thought is that we start recruiting and see who we can get, see how 
> these early faculty fit into Allison's matrices and then, later on we 
> can target other faculty that we might be missing.
>
> I'm starting to talk to some faculty here to see what might be 
> possible.  My understanding is that we would test the demo of the 
> light box that is on the Sakai site.  I'll put a link on the wiki.
>
> Mike and Daphne are you available for a breeze meeting later today?  
> Anyone else interested let me know.
>
> cheers
> Barbara
>
> On 18-Oct-07, at 11:00 AM, Michael S Elledge wrote:
>
>> Hi Everyone--
>>
>> Barbara, you've done a very nice job putting together the user study 
>> info and protocol. The categories are quite comprehensive and will be 
>> very helpful when we move into more contextual testing.
>>
>> I want to second Daphne's comment about being flexible when testing 
>> the tool. Faculty (at least on this campus) are notoriously difficult 
>> to recruit, especially on short notice. I also imagine we'll find out 
>> what we need to know after a fairly small number, say ten, keyboard 
>> and non-keyboard users try it out (that is 5 each).
>>
>> In the meantime, can someone post a link on the site to the instance 
>> of Lightbox we'll be testing?
>>
>> Also, I've made some edits to the page. Feel free to comment, etc.
>>
>> Thanks!
>>
>> Mike
>>
>> Daphne Ogle wrote:
>>>
>>> On Oct 17, 2007, at 8:17 AM, Barbara Glover wrote:
>>>
>>>> Hi Allison
>>>> Thanks for sending the link to the chart.  That will help out.  
>>>> Originally I wanted to do a more comprehensive study of users 
>>>> representing both our faculty persona as well as student personas.  
>>>> Daphne and I had a conversation about that this week and with the 
>>>> shortened time-line we thought to focus this round on faculty with 
>>>> the addition of a few key-board only users.
>>> Right now, students are not able to use the reorderer -- no 
>>> permissions in the Image Gallery tool to reorder images.  This is 
>>> likely to come but not for a couple versions.  Our focus right now 
>>> is to make the experience of posting images for students to view as 
>>> easy and pleasant as possible for instructors.  That said, testing 
>>> some students on the lightbox will give us some good user data on 
>>> the reorder generally -- how do the controls and interesting moments 
>>> work; do they know what they can do when and where they can do it 
>>> (related to my final comment).
>>>
>>> IMO focusing on our primary persona is not critical at this point.  
>>> The current functionality is pretty generic around reordering 
>>> images.  When the personas get interesting is in future phases of 
>>> the Image Gallery (and perhaps some of this will end up in the 
>>> lightbox component) when we support image heavy course instructors 
>>> in building their course content.  We of course want to test some 
>>> users that are used to working with images like Eileen is but I also 
>>> think we want to test a diverse mix of people to get a sense of 
>>> where and when the lightbox will support users best.
>>>>
>>>> Since our planning meeting yesterday though it appears we may have 
>>>> more time to complete further testing.  In light of this, I would 
>>>> like to propose we start the testing with faculty matching our 
>>>> primary persona for Sakai and then, branch out to include the 
>>>> students.
>>>>
>>>> As far as extrapolating to the Re-orderer in general, that was not 
>>>> the intent of these studies.  However it is a good suggestion and 
>>>> one as we do more planning around user testing in general that we 
>>>> should consider.
>>> We should get some good user data about the reorderer in relation to 
>>> images by testing the lightbox.  We could also think about testing 
>>> the uPortal portlet layout implementation to round out the results a 
>>> bit.  Perhaps we could outline what we want to know about the 
>>> reorderer and make sure we include tasks that get at that information.
>>>
>>> -Daphne
>>>>
>>>> cheers
>>>> Barbara
>>>> On 16-Oct-07, at 8:46 PM, Allison Bloodworth wrote:
>>>>
>>>>> Hi there,
>>>>> It seems that one addition we should make to the testing protocol 
>>>>> is identifying the attributes of the users we want to test, and 
>>>>> then keeping track of which user types we've been able to find. 
>>>>> See http://wiki.fluidproject.org/display/fluid/Users+to+Research 
>>>>> for an example of an (admittedly complicated--we probably don't 
>>>>> need nearly this much detail) matrix Daphne, Judy and I came up 
>>>>> with for the Content Management Research. Are we only planning to 
>>>>> target folks who are similar to the two Lightbox personas 
>>>>> (http://wiki.fluidproject.org/display/fluid/Lightbox+Personas), 
>>>>> both of which are faculty members? It sounds like we at a minimum 
>>>>> need to add "keyboard-only user" to the list of target 
>>>>> characteristics. Are there others?
>>>>>
>>>>> A question that may help answer that is to ask whether we trying 
>>>>> to extrapolate testing for the Re-orderer from testing of the 
>>>>> Lightbox? If so, I am wondering if we should consider also testing 
>>>>> students and people who are using Sakai for research or project 
>>>>> management as well (e.g. 
>>>>> http://wiki.fluidproject.org/display/fluid/Sakai+Personas), since 
>>>>> they may be using a version of the re-orderer in another Sakai 
>>>>> tool (e.g. Page Order Helper). It wouldn't be testing in context, 
>>>>> but at least it would be testing user types who may encounter the 
>>>>> Re-orderer. There would also be the users of other Fluid 
>>>>> applications to consider, but I'm guessing they may be covered by 
>>>>> some of the Lightbox or Sakai personas, and am not sure it would 
>>>>> make sense to define our target testing subjects in an incredibly 
>>>>> granular fashion (because we are going to be limited as to who we 
>>>>> can access).
>>>>>
>>>>> Allison
>>>>>
>>>>> On Oct 16, 2007, at 2:54 PM, Colin Clark wrote:
>>>>>
>>>>>> Hi Barbara and all,
>>>>>>
>>>>>> This is a really good testing protocol for the Lightbox. Nice work!
>>>>>>
>>>>>> The UX team talked about user testing briefly on our planning 
>>>>>> call today. For those of you who weren't there, here's a quick 
>>>>>> summary.
>>>>>>
>>>>>> The 0.1 release is unusual in that we haven't had the opportunity 
>>>>>> to do a lot of user testing on the Lightbox yet. Now that we've 
>>>>>> got a solid test plan, it will serve as the basis for identifying 
>>>>>> and fixing usability issues with the Lightbox and Reorderer over 
>>>>>> the next few releases. User testing will become an ongoing 
>>>>>> activity for us from here on.
>>>>>>
>>>>>> We'll fix as many issues as we can before 0.1, and then continue 
>>>>>> with user testing and refactoring into 0.2, refining the test 
>>>>>> plan as we go.
>>>>>>
>>>>>> Barbara mentioned during the meeting that she's planning to run 
>>>>>> this plan with Justin Obara early next week here in Toronto. Is 
>>>>>> anyone else able to do some testing with users at their institution?
>>>>>>
>>>>>> Colin
>>>>>>
>>>>>> Barbara Glover wrote:
>>>>>>> Hi Daphne/Mike/Allison/Shaw-Han
>>>>>>> We're trying to get some quick user testing done for the release 
>>>>>>> of  the Lightbox at the end of October.  Several of you have 
>>>>>>> offered to  help with this testing especially the accessibility 
>>>>>>> sections at your  universities.
>>>>>>> I've put together a protocol that we can use with the 
>>>>>>> instructor  user.  Please review and make changes at the 
>>>>>>> folllowing wiki page.
>>>>>>> http://wiki.fluidproject.org/display/fluid/Lightbox+User+Study+Protocol 
>>>>>>>
>>>>>>> Let's try and determine if we can get any testing completed in 
>>>>>>> the  next week before the release is done.
>>>>>>> cheers
>>>>>>> Barbara
>>>>>>> _______________________________________________
>>>>>>> fluid-work mailing list
>>>>>>> fluid-work at fluidproject.org <mailto:fluid-work at fluidproject.org>
>>>>>>> http://fluidproject.org/mailman/listinfo/fluid-work
>>>>>>
>>>>>> --Colin Clark
>>>>>> Technical Lead, Fluid Project
>>>>>> Adaptive Technology Resource Centre, University of Toronto
>>>>>> http://fluidproject.org
>>>>>
>>>>> Allison Bloodworth
>>>>> Senior User Interaction Designer
>>>>> Educational Technology Services
>>>>> University of California, Berkeley
>>>>> (415) 377-8243
>>>>> abloodworth at berkeley.edu <mailto:abloodworth at berkeley.edu>
>>>>>
>>>>>
>>>>>
>>>>>
>>>>
>>>
>>> Daphne Ogle
>>> Senior Interaction Designer
>>> University of California, Berkeley
>>> Educational Technology Services
>>> daphne at media.berkeley.edu <mailto:daphne at media.berkeley.edu>
>>> cell (510)847-0308
>>>
>>>
>>>
>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: elledge.vcf
Type: text/x-vcard
Size: 313 bytes
Desc: not available
URL: <http://fluidproject.org/pipermail/fluid-work/attachments/20071018/581921cc/attachment.vcf>

More information about the fluid-work mailing list