February 03, 2014

Building A User Testing Lab, Part 3

I wrote a couple of posts about preparing our discount usability testing and user research lab at Fullscreen with software, hardware and a physical space. In this third and final post, we get to put it through the paces for the first time.

Faking It (since we don’t want to make it yet)

The team used Keynote to prototype two different takes on a feature we’re building into the Creator Platform. Keynote is basically awesome for early-stage interactive prototypes because not only is it super-quick to learn and use (especially if you start with something like Keynotopia’s UI stencils) but also it’s relatively simplistic design features force a lower-fidelity approach than using Photoshop to create screens for a prototype. Bonus: it’s free!

It has limitations, though: for example, you can’t scroll within a screen. Since one of the concepts we wanted to test was heavily reliant on vertical scrolling, we had to (really inelegantly) fake it. This particular concept tested unanimously worse than the other one, which didn’t happen to require such overt trickery. And while I don’t attribute that to the scrolling fakery alone, I’m sure they didn’t help make the experience any easier to understand.


Using big blue arrows to “scroll” up and down a page: not even minimally real.

More faking: we used a static screenshot of browser chrome to frame the content of each prototype and make them seem minimally real. Which worked so well that it only took a couple minutes for someone to try the static browser “back” button when clicking through prototypes. It just took a quick adjustment to build a fake back button by placing a transparent hyperlink over it leading to the “previous slide” and it was a reminder that anything we present users with during testing is fair game and the more we can anticipate, the more realistic interactions we’ll be able to observe.

Capturing It

Since we were testing prototypes of a web app, there was no need for much of the laundry list of equipment that we’ll need to properly capture mobile user testing. Just a Macbook running Silverback to record a screencast and the user’s reactions. It worked pretty well to capture the 45-minute sessions, although sometimes we had a tough time making out what the subjects were saying when we played them back. Next time I’ll definitely try using an external mic which should take care of that.

Once the sessions were wrapped up, we got the team together to watch them and talk through the results. This almost didn’t happen because I didn’t realize that you actually need to export recordings from Silverback, which can take a good amount of time for these 45-minute sessions. Watching low-res previews within Silverback worked in a pinch, but next time I’ll make sure to allow for a little bit of time to get those exports done before getting everyone in a room.

All in all, testing web app prototypes was pretty seamless. I’m looking forward to getting some users in to do some mobile testing next time.