The MacGyvers of mobile usability testing - Belén Barros Pena and Bernard Tyers at EuroIA 2010
As I tweeted at the time, any presentation that featured a slide of The Muppets' Swedish chef, then immediately followed it up with some close-ups of specified Meccano parts, is alright with me.
At EuroIA this year, Belén Barros Pena and Bernard Tyers were presenting their approach to DIY usability testing on mobile devices. Having listed various expensive ways of doing it, and their drawbacks, the duo then MacGyver'd up a device that clamped to a phone, and filmed the screen as a user, in this case my friend Theba Islam, carried out the test.
Theba Islam being 'user tested' live on stage at EuroIA by Belén Barros Pena and Bernard Tyers
Their device had several distinct advantages over other testing approaches:
- It worked on any model of phone, and you didn't have to install software on the device.
- It was light enough so that the user could still operate their phone as normal.
- It was portable, so mobile testing could be done in the field.
- The fixed camera nature of the gadget meant the user didn't have to be conscious of holding the screen at a certain angle, or within a certain field of recording.
You can view Belén and Bernard's brilliant slides here "Do it yourself mobile usability testing at EuroIA 2010":
There was something great about watching live Meccano building on stage at an IA conference. When I was a kid I always wanted to build really complex models with Meccano and Lego. Lego I was very good with, but I struggled with Meccano. I've often reflected that getting into computer programming at a young age allowed me to explore that engineering creativity in a virtual space, where I wasn't penalised for being a bit clumsy.
As I say, I thought the talk was brilliant, and a perfect opening session on the day for a crowd mostly feeling a little bit hungover. Belén demonstrated that their DIY gadget fulfilled the required criteria - that it was cheap, didn't inconvenience users when testing, and was built from commodities that could easily be replaced.
I've done several sessions of lo-fidelity mobile testing myself, and I've found it to be ok to use a simpler set-up as well.
I use my laptop webcam to get a facing static shot of the whole session. I use, as always, Silverback to record the video. I know other video capture software on the Mac could do the job just as well, but I like the way that it helps me organise the clips into projects and sessions.
I then additionally have a second person in the room using a Flip camcorder. They film the screen of the phone when it is being used. It isn't perfect - screen glare and close-up focus can be an issue - but I find the footage is "good enough", which is as good as it needs to be.
The main drawback of this approach is that testing has to be done in an office situation, and so it is harder to observe your software being used in the real context of mobile use.
And so maybe, just maybe, if I can get over my childhood fear of Maccano engineering failure, I'll jerry-rig myself up a bit of usability testing kit for the future...
Hi Martin,
Thanks very much for the very kind write-up of our talk. We're glad you liked it.
About the Meccano fear - honestly, don't be. If you can handle Lego, then you can handle Meccano.
If you want to see a pretty simple build video of the full test rig, you can on Belén's blogpost here. Overall it took about 8 minutes to build it.
We'd love to see people take our design and improve it, and feed back changes. For me (the Meccano-head) it was great to put a "toy" to actual industrial usage. Liberating and democritising user testing.
You are more capable as an adult than a kid. Don't be afraid of meccano. The thing that really scared me when I was a teenager was automatically typing a program that went way beyond what I wanted, that is write a program to read some bytes from a file when in reality I was writing a database. It was as if I had another part of my brain that worked on its own. Hey, dont even be afraid of failure: people who succeed fail a lot too, but they go beyond their fear and failures.