All Posts in Startups

February 03, 2014

Building A User Testing Lab, Part 3

I wrote a couple of posts about preparing our discount usability testing and user research lab at Fullscreen with software, hardware and a physical space. In this third and final post, we get to put it through the paces for the first time.

Faking It (since we don’t want to make it yet)

The team used Keynote to prototype two different takes on a feature we’re building into the Creator Platform. Keynote is basically awesome for early-stage interactive prototypes because not only is it super-quick to learn and use (especially if you start with something like Keynotopia’s UI stencils) but also it’s relatively simplistic design features force a lower-fidelity approach than using Photoshop to create screens for a prototype. Bonus: it’s free!

It has limitations, though: for example, you can’t scroll within a screen. Since one of the concepts we wanted to test was heavily reliant on vertical scrolling, we had to (really inelegantly) fake it. This particular concept tested unanimously worse than the other one, which didn’t happen to require such overt trickery. And while I don’t attribute that to the scrolling fakery alone, I’m sure they didn’t help make the experience any easier to understand.


Using big blue arrows to “scroll” up and down a page: not even minimally real.

More faking: we used a static screenshot of browser chrome to frame the content of each prototype and make them seem minimally real. Which worked so well that it only took a couple minutes for someone to try the static browser “back” button when clicking through prototypes. It just took a quick adjustment to build a fake back button by placing a transparent hyperlink over it leading to the “previous slide” and it was a reminder that anything we present users with during testing is fair game and the more we can anticipate, the more realistic interactions we’ll be able to observe.

Capturing It

Since we were testing prototypes of a web app, there was no need for much of the laundry list of equipment that we’ll need to properly capture mobile user testing. Just a Macbook running Silverback to record a screencast and the user’s reactions. It worked pretty well to capture the 45-minute sessions, although sometimes we had a tough time making out what the subjects were saying when we played them back. Next time I’ll definitely try using an external mic which should take care of that.

Once the sessions were wrapped up, we got the team together to watch them and talk through the results. This almost didn’t happen because I didn’t realize that you actually need to export recordings from Silverback, which can take a good amount of time for these 45-minute sessions. Watching low-res previews within Silverback worked in a pinch, but next time I’ll make sure to allow for a little bit of time to get those exports done before getting everyone in a room.

All in all, testing web app prototypes was pretty seamless. I’m looking forward to getting some users in to do some mobile testing next time.

January 17, 2014

Building A User Testing Lab, Part 2

In this follow-up to my first post about building a discount usability testing and user research lab at Fullscreen, I’ll talk about setting up a space for testing and getting the right equipment on a budget.

The Space

Though it’d be nice, a permanent room for our user lab just isn’t in the cards so I needed to find a space to set up shop in and conduct interviews and sessions for a few hours at a time. It was easy to narrow down the candidate pool since there are really only two private and distraction-free rooms in our building that aren’t also occupied by people who, you know, need to work in them.

We’ve used our conference room for user testing sessions in the past. It’s worked out fine, but it also feels a bit overwhelmingly large when you’ve only got two or three people in it. And we also have a little loungey game room with a couch and two comfy chairs—which is what we’re going to use. Since the room itself is more like a place where people would actually use the products we’ll be testing, on some level it’ll be easier for them to engage in authentic behaviors under admittedly artificial circumstances.


Even if you don’t have an Arduino-controlled keggerator at your house, it’s a pretty easy guess which room you’d feel more at home in

The Equipment

OK, that was easy. I’ve basically got a living room to work with the users in, and the Silverback and Reflector apps installed on my laptop. Now it’s time to go shopping, keeping in mind a budget of $1000 max including devices.

Webcam: Silverback uses the built-in iSight by default, but to test mobile prototypes I’ll need a webcam to capture the user’s reactions. We had a 1080p Logitech C615 ($49) sitting around, but if I had to buy one I’d just go for this Microsoft LifeCam ($19) which is still 720p and still more than adequate.

Mouse: Good to have one handy for users who aren’t used to trackpads. Since non-Mac users won’t be familiar with the $70 Magic Mouse anyway, a standard-issue Logitech USB mouse ($17) should do it.

Microphone: I’ve had a Blue Snowball ($59) for a couple of years. Could probably get away with using the laptop’s built-in microphone, but I don’t want to miss a thing.

Mobile devices: Here’s where things get a little expensive. I went for an entry-level iPod Touch ($229) to test iOS apps and demo mobile prototypes (using Invision, an app that makes it simple to string together screens into passably-real prototypes and get them onto devices). Way cheaper than a $600 unlocked iPhone, and no need to pay another $10 for a blank SIM card just so you can open mobile Safari.

The total cost for all this equipment, as well as the software to record sessions and stream from devices, would be just $410. Way under budget, I also took a protip to heart from Pocket’s experience setting up a mobile testing lab. They note that the iPad mini is a perfect size for demoing iPhone prototypes built in Keynote, which is another amazingly quick and effective way to get early validation of rough concepts. So I got hold of one of those as well. All in, the total cost for this multi-device testing setup including software, camera, microphone, mouse, iPod Touch and iPad Mini is $659.

I’m excited to put it all to use (tomorrow!) and report back on what went wrong and hopefully a lot of what went right.

January 10, 2014

Building A User Testing Lab, Part 1

This is the first of three posts about my experience building a discount usability testing and user research lab with the team at Fullscreen. I’ll talk about why we’re doing this and touch on the software that we’ll use to capture sessions.

Google Ventures-style product design sprints seem to be popping up all over the place these days. Design sprints have become an important piece of our process toolkit at Fullscreen, not just because I love me a good trendy methodology but really because they’ve helped us us immensely with quick parallel concept development and rapid validation.

In a product design sprint, you bring ideas from concept to low-fi prototype in less than a week and then get them in front of people—preferably, real users of your product—to see how they hold up. So you need a place to work with the users during these sessions as well as a way to capture the sessions so the whole team can review and “score” them.

Now our relatively new office, despite its many lovely features including a human-size terrarium, doesn’t have one of those fancy interview chambers with a one-way mirror and HD simulcast capabilities to the adjacent observation room’s three flatscreen displays. What we do have, though, is our first design sprint of 2014 starting next week. And we need a way to conduct and capture those user sessions by Friday.

So we’re building a user testing lab that makes use of our flexible space and can be set up and taken down with extremely minimal effort, that can handle both desktop and mobile prototypes, and that costs less than $1000 including devices. I’m going to document this exercise for posterity starting with the software we’ve chosen to capture the sessions, then the hardware and the space itself and finally what will hopefully be a triumphant recounting of the first user sessions we run in it.

The Software

Our discount usability lab needs to record two things: the interaction with the prototypes onscreen and the user’s face while they’re interacting with it. We want this captured in a single video file so it’s easy to review and discuss later.

A few years back I used an app called Silverback to record some user testing done while building a group messaging app called Volly. Silverback doesn’t seem to have changed altogether too much, but that’s a good thing. It’s designed for doing exactly what we need, which is recording two streams of video—a screencast and the user’s face so we can get the full brunt of their reactions and thought process. It’ll use your laptop’s built-in iSight by default but getting it to recognize a USB webcam is as easy as plugging it in. It even records hotspots whenever the user clicks so you can tell exactly what’s going on.


Note the awesome little hotspot where I clicked. Also the depth of that v-neck

I’ve also heard that ScreenFlow gets the job done despite having an old-timey website, and it also has built-in editing features which could be handy if you also wanted to use it for something else, like recording product demos or tutorials. ScreenFlow’s extra features have a $99 price tag, though; since Silverback is only $70, that’s what I went with.

We’re sorted for capturing desktop user sessions. But what about testing mobile apps and prototypes? My first thought was to try awkwardly clamping a USB document camera to the mobile device. But this post by one of the creators of Silverback points out how ridiculously simple it is to mirror any iOS device using the $13 Reflector app.


Silverback x Reflector. Since you’re wondering, the app is called Tuxedo Kittie

With Reflector, you can mirror whatever’s happening on any iOS device that supports AirPlay mirroring (devices with just plain old AirPlay need not apply; my old iPhone 4 was a no-go). Open Reflector, and your computer will appear as an AirPlay receiver to your phone. From there, it literally just works.

That is, for iOS. Looming large is the day when we find ourselves wanting to capture user testing using Android devices. There’s no doubt in my mind that this will in no way be as painless as Reflector makes it for iOS devices. But as someone wiser than myself once said, any testing is better than no testing at all.

October 29, 2011

Lean And Mean

So, like, sometimes it’s hard for me to get things done at work, and most days it’s because people are always coming up to me asking about how I manage to keep such an incredibly chiseled, trim and well-manicured physique while working 14-hour days in environments filled with steady streams of pizza and Del Taco. It’s hard to refactor Javascript when people have their hands all over your biceps all day.

Okay FINE so maybe that’s not true but I do alright for myself considering the hours I’ve kept first running my own company and now at a pre-funding startup. Inspired in part by this Quora thread  and my man Andrew Skotzko’s series of posts on fitness I felt like throwing out a couple of mostly common-sense, simple tips on how to keep it super lean and super mean, or at least to ward off the startup version of the Freshman 15 for as long as possible.

1. Convenience Over Everything

I’ve been working out on and off for like six years and at this point I feel like it’s not just a privilege but an inalienable right for me to enjoy a beautiful, clean gym filled with well-oiled equipment and mirrors and mood lighting designed to make me look like a Men’s Fitness cover model and mahogany-paneled locker rooms replete with sauna, jacuzzi and cigar lounge.

For now, though, I’m making do with the following. First, there’s the gym that’s literally directly next door to the office I work out of. The people who work there are well-meaning good folks, so I’ll try to be nice, but their one rusty flat bench was probably Jack LaLanne’s first piece of equipment ever and they played a Coldplay album front-to-back the other day. And secondly, there’s the 24-Hour Fitness three minutes from my house that’s good enough, I guess, if you like the smell of mold and you only want to work out one of your two arms since there’s only been one of each dumbbell around since probably the mid-nineties.

The thing that these two gyms have in common besides harsh flourescent lighting are that they’re so convenient to the rest of my life that I simply can’t ignore their presence. Having a gym next door to the office means I can work out literally whenever I have 45 minutes. And you can always find 45 minutes. I could work out at Equinox but trading luxury for convenience means I get there once a day, not once a week. Bonus: works great with early-stage salaries!

2. Prepare So You Don’t Have To Eat Prepared (Food)

The most effective diet I’ve ever followed is presented below in all of the detail it was presented to me when I first started it. You might need a pen and paper to capture all this:

Breakfast: 6 egg whites and oatmeal
4 daily meals: 6-8 oz. of lean meat and 12 oz. of vegetables
Eat snacks, but don’t eat snacks stupidly

It might not be the best diet ever, but it was the best diet ever for me because I didn’t have to think about it. If you don’t have the luxury of time to think and plan every meal, this is really key. Within those loose constraints was a lot of freedom, and there just isn’t a whole lot to remember. As a formerly-single guy who still sucks at cooking, I could make most of the meals with just my man George Foreman and a microwave.

The one important thing I needed to do when I was following this diet was to take a little bit of extra time to prepare myself and to have healthy meals with me all the time. Eating pre-packaged, pre-prepared food is pretty much always worse than eating something you made yourself. Even the stuff I think is good for me like deli-sliced turkey always manages to have 8 million grams of sodium or something.

Being prepared was as easy as throwing 4 chicken breasts into the George Foreman at night instead of just 1. Even though George has since taken the long walk of shame down our driveway, I still try to make more than I could possibly eat for dinner and bring it the next day. And I constantly anger co-workers by filling up the fridge with raw vegetables from Trader Joe’s. If I have the meals around, I don’t need to put in on the Chipotle order in desperation or reach for a deliciously-marketed though lacking-in-nutrition Clif Bar.

3. Be Predictable. Be Flexible.

Routine keeps me in line. If you haven’t noticed a pattern with the two things above, it’s that the less I have to think about having to work out or eat right, the better. There are enough things vying for our attention that if we don’t have to go out of our way to think about being healthy it’s just more likely to happen.

For me, that means picking a time to go to the gym and sticking to it. Picking a class (for me it’s spin class) or a sport or, if you’re the type of sick person who likes that kind of thing, a long-distance run to go on and doing it regularly.

And it also means having a little latitude with the routine at the same time. If my trainer, who I promise is at least 3 times your size and has like .01% body fat, can go to Sweet Lady Jane and eat a half a red velvet cake on Sundays then it’s totally fine for me to order a pizza and buy a bag of Flipz from 7-11 once in a while.

4. Treadmill Desk

HAHAHAHA no one really has one of these. But if you’re considering it, promise to at least try the simple, common-sense things first?