After 8 months of research and interviews, writing more than 20 drafts and receiving tons support from my wife and my family, I presented my point-of-view on one of the toughest challenges that experience designers face today.
The experience and the dialogs that I started with the audience made all the hard work worth while.
A recent discussion in the client project touched upon an often debated topic – What is the value of user testing?
On the surface, the value of user testing is obvious. It invites representatives of people who will use a design to provide unbiased feedback to a proposed solution. It reduces the possibility of unrealistic expectations on users (e.g. User would put this around his neck every morning) and weeds out usability problems that designer may overlook. On the flip side, it’s often argued that companies over-rely on usability testing and defer design decisions to a limited number of randomly chosen people. Lab settings and paper prototypes do not provide enough context, or even the right context, for the user to realistically judge the experience of use. A lot of designers operate under the mentality that ‘some testing is better than no testing’, while others believe in this age of constant drive for disruptive innovation and the common practice of having a digital service in a prolonged beta phase, user testing is a waste of time. Users simply don’t know what they want until you show the design to them in its full glory as a working solution. All these arguments lead me to believe that user research, which user testing is part of, is ill-presentated in companies. Companies trapped in such kind of debate miss the value of usability testing because they mis-understand the broader role of user research in the design. This needs to be fixed.
The typical design process is divided into 4 phases – Discover, Define, Design and Deliver. These step repeat multiple times during a project, sometimes in this logical sequence, sometimes not. In a user-centered design process, understanding of the user’s needs is included in every step. User testing is most appropriate during the Design phases.
During the Design phase, user testing provides grounded evidence to improve details of the design. However, there is merit to the claim that users don’t know what they want but they know very well how to judge what is in front of them. For that reasons, designer should provide test materials that are well thought out, so the users can judge the quality of each design in their details. Sometimes it is OK to test an incomplete design, but request users to provide over-arching feedback on an incomplete design is not (e.g. Will you find this solution pleasurable to use on a daily basis?). Incomplete design in, incomplete user input out. With that said, it is important to recognize that results that stem disruptive innovations rarely come put of user testing. That is because by the time the process reaches the Design phase, designers and stakeholders have already defined the problem space of the project. Expecting users to react to a prototype and articulate an unmet need that will lead to a brand new service is just wishful thinking. In the rare cases where profound user input is captured, it will most likely documented as a footnote in the report, under ‘Further Considerations’. Most user-supported evidence for disruptive innovation result from design research during the Discover phase, when the problem space is still fuzzy, or after the Delivery phase when unexpected patterns are found in user’s operations with the solution through analytics, support calls and review comments.
Contrary to common believe, designers by themselves don’t hold the solution to most complex design problems that we are tasked to solve today. Understanding the design problem with the help of user’s input offer a much higher rate of success. User testing is not the only tool, nor it is always the right tool, to incorporate user input. To realize the value of user research, designers, especially design leaders, must education companies to use the right user research tool at the right time.
This idea has been on my mind for almost a month so I want to put up a quick post.
I’ve been reading how designer and researchers like Mike Kuniavsky and Fabien Girardin [PDF slides] view data captured within cities can be viewed as materials for designers. Taking the view point of an end-user, I think data are becoming a natural element that common people have to learn to deal with in our daily lives, much like fire and wind to our early ancestors. We understand the power of data but the concept (i.e. behaviors, appearance and properties) of data has become very elusive.
Why do most companies have no more than one innovative, ground breaking product in its history? How come only a handful of companies (Amazon.com, Apple, IBM and Nokia, to name a few) have been able to invent new business models out of it core business and ultimately transform themselves and their industries? Roger Martin gave a talk and held a panel discussion this week in London about this very topic, which is also the thesis of his new book ‘The Design of Business‘.
According to Martin, the development of innovation follows the growth path of knowledge. All knowledge begins from the discovery of a mystery. Observation and rationalization of phenomena around the mystery creates heuristics. If those heuristics are further developed to make those phenomena repeatable an algorithm, the ultimate solution, results. In the context of a business, the mystery is an unmet need in the market, the heuristic is the business idea and the algorithm is the solution that the business creates. Surprisingly, the ability to follow this path is does not explain why companies do not innovate more often. The answer to our question lies in what companies leave behind along the knowledge development process.
Pushing a body of knowledge from a mystery to an algorithm requires a iterative process of elimination. The process requires an analytical mindset and quantitative data from the past. To innovate, Martin argues, require a process of expansive thinking. To discover ground-breaking ideas requires us to step away from elimination process, observe ambiguity, look at qualitative data and perhaps most controversially, trust our intuition. The future simply cannot be told by the past. This practice is what Martin called ‘Design Thinking‘. The absence of Design Thinking is the reason why innovation is rarely done.
The rest of the panel discussion with editor-in-chief of Monocle (one of my favorite magazines) Tyler Brûlé and Oxford University professor Lucy Kimbell covered how design thinking has successfully help Monocle thrive in the dwindling printed magazine industry and how MBA programmes in Oxford and other universities are blending creative processes traditionally used by designers to creative ground-breaking products and services.
While Martin’s argument brilliantly shines a light on one of the hottest business topic and make a compelling case for new business education programs, I left the talk with a sense of emptiness. Design thinking in business has been discuss, taught and turned into consulting industry since the early part of this millennium. The crux in business innovation today is not the fact that businesses do not understand the need of innovation methods or have innovative ideas. Many design schools and consulting firms are eager to fill those gaps. The fact is that businesses lack the flexibility to shift their company culture and logistics to embrace those methods and turn those new ideas into feasible business models. The fear to let go of their proven (although never future-proof) revenue streams, established (although bureaucracy ridden) org charts and familiar customers (of recent quarters) prevent innovation from making into the market place. That is the challenge of this new decade.
I have yet to read Martin’s book but may be Clayton Christensen’s ‘The Innovator’s Dilemma‘ offers a more practical approach this innovation problem — companies optimize themselves to maximize efficiency in its core business. Any radically new idea that deviates the company from the optimization are routinely repelled and displaced. A truly innovative and ground-breaking idea requires a separate, autonomous entity to take it to the market. The sprout from a new seed, simply needs new soil to grow.