After 8 months of research and interviews, writing more than 20 drafts and receiving tons support from my wife and my family, I presented my point-of-view on one of the toughest challenges that experience designers face today.
The experience and the dialogs that I started with the audience made all the hard work worth while.
A recent discussion in the client project touched upon an often debated topic – What is the value of user testing?
On the surface, the value of user testing is obvious. It invites representatives of people who will use a design to provide unbiased feedback to a proposed solution. It reduces the possibility of unrealistic expectations on users (e.g. User would put this around his neck every morning) and weeds out usability problems that designer may overlook. On the flip side, it’s often argued that companies over-rely on usability testing and defer design decisions to a limited number of randomly chosen people. Lab settings and paper prototypes do not provide enough context, or even the right context, for the user to realistically judge the experience of use. A lot of designers operate under the mentality that ‘some testing is better than no testing’, while others believe in this age of constant drive for disruptive innovation and the common practice of having a digital service in a prolonged beta phase, user testing is a waste of time. Users simply don’t know what they want until you show the design to them in its full glory as a working solution. All these arguments lead me to believe that user research, which user testing is part of, is ill-presentated in companies. Companies trapped in such kind of debate miss the value of usability testing because they mis-understand the broader role of user research in the design. This needs to be fixed.
The typical design process is divided into 4 phases – Discover, Define, Design and Deliver. These step repeat multiple times during a project, sometimes in this logical sequence, sometimes not. In a user-centered design process, understanding of the user’s needs is included in every step. User testing is most appropriate during the Design phases.
During the Design phase, user testing provides grounded evidence to improve details of the design. However, there is merit to the claim that users don’t know what they want but they know very well how to judge what is in front of them. For that reasons, designer should provide test materials that are well thought out, so the users can judge the quality of each design in their details. Sometimes it is OK to test an incomplete design, but request users to provide over-arching feedback on an incomplete design is not (e.g. Will you find this solution pleasurable to use on a daily basis?). Incomplete design in, incomplete user input out. With that said, it is important to recognize that results that stem disruptive innovations rarely come put of user testing. That is because by the time the process reaches the Design phase, designers and stakeholders have already defined the problem space of the project. Expecting users to react to a prototype and articulate an unmet need that will lead to a brand new service is just wishful thinking. In the rare cases where profound user input is captured, it will most likely documented as a footnote in the report, under ‘Further Considerations’. Most user-supported evidence for disruptive innovation result from design research during the Discover phase, when the problem space is still fuzzy, or after the Delivery phase when unexpected patterns are found in user’s operations with the solution through analytics, support calls and review comments.
Contrary to common believe, designers by themselves don’t hold the solution to most complex design problems that we are tasked to solve today. Understanding the design problem with the help of user’s input offer a much higher rate of success. User testing is not the only tool, nor it is always the right tool, to incorporate user input. To realize the value of user research, designers, especially design leaders, must education companies to use the right user research tool at the right time.
I finally budged and bought a Kindle.
Being an iPad owner, I couldn’t see the appeal of paying for another tablet-like device for the longest time. I also find it wrong on both business and service design levels for Amazon to charge people for a device in a highly commoditized category. I didn’t want to support a bad idea. However, as my work travels became more frequent and longer, I found myself needing a screen that offers better readability than the iPad. After few months of on and off research, I ate my humble pie and placing an order for a Kindle. To my surprise, that led to a revelation about Touch UIs.
Amazon announced the new Kindle Fire just a few days before I made the purchase decision. Along with Kindle Fire came a low-end Kindle and the Kindle Touch. The last-gen Kindle with Keyboard was still available at a reduced price. Because I was buying readability, I automatically weeded out the Kindle Fire. I picked the Kindle with Keyboard simply because it was cheaper. After just one week into read on my daily news and e-books new toy, I felt very happy about my purchase. In additional the display being miles better to read from than my iPad, I felt more confident and relaxed while navigating through the contents. With just about every type of consumer devices moving toward Touch displays, that observation begs for a deeper understanding.
I used the Kindle for a few more weeks. I deliberately slowed down my activities to self-observe and reflecting on my experience with the device. I can explain my joy with navigating contents with the Kindle down to one reason – physical hard keys are better, by far, for controlling single-dimension, linear contents.
We read text forward and sometimes jump backward to reference. Dedicated hardware controls arranged linearly provides better detectability, higher active-feedback compatibility and ultimately simpler user experience. Whenever I tap on the paging keys of my Kindle, I feel more connected to the content. The click of the keys revive the tactile quality of reading a book that digitalization took away. The same goes with scroll wheel on my iPod, zoom ring on my SLR lens and volume control knobs on my hi-fi receiver. Mapping control of linear contents onto virtual buttons or even worse, artificially link 2 linear controls into a 2-dimensional Touch UI* quickly degrades the experience.
*The VLC video player on OSX allows the user to control volume by scrolling left/right and scrub the video by scrolling up/down. The feature works and it is certainly better than nothing. However, the better scheme is to move the volume control to the dedicated volumes keys and leave the touch pad for scrubbing the video only.
Product testing is a serious business, but the idea of it doesn’t need to be.