Evaluating User Experience Design: A Case for Qualitative Analysis
It’s an established fact that embracing a design mindset at the highest levels in an organization helps bring business success. A question many executives have is that once they’ve committed to placing a high value on design to the point that it is truly driving their business decisions, how do they know if their various design efforts are good or bad? While it may seem like the act of judging design quality is purely subjective, there are some evaluation methods that can remove the guesswork. And we caution you that your performance data may not be telling you the whole story.
To thoroughly evaluate the quality of a design, you need established design principles, then you need criteria to determine how well the design supports those principles. As this article will explain, at Rhythm Interactive when we conduct user experience evaluations, we start with fundamental beliefs about what makes a successful user experience, then we measure how well different aspects of the experience support those basic beliefs. The result yields tangible, actionable recommendations for improvement that are grounded in logic instead of personal taste per se.
Successful User Experience Design
Our first guiding principle is the idea that successful user experiences gracefully bridge business goals and user needs. Whether it’s a physical product, a retail environment, digital media, or what have you, the principle applies equally to all user experience design scenarios. In order to deliver on this belief, it’s important to have a clear understanding of precisely what the business needs to accomplish and what users desire in the context of the problem space you’re operating in. These things must be identified before the design begins, and before the quality of a design can be evaluated.
Our next most important principle is that a user experience consists of three main dimensions: form, meaning and behavior. In other words, how it appears, what it says, and how it works. We believe that the designer must give equal attention to how each dimension satisfies business goals and aligns with users’ needs to the point of delight. If any one dimension is neglected, the design will be off balanced and opportunities will be missed.
A third principle is the belief that a well conceived brand strategy is imperative to business success. To realize the full potential of a business, the brand must be defined and articulated, it must live upstairs at the organization and its essence must be expressed accurately at every point of customer contact.
With these basic tenants as our starting point, we then conduct evaluations applying different filters; we focus on how well the task flow supports established user goals, we examine how the design holds up to the competition, we look at whether or not the solution violates any design best practices and we analyze how well the brand is portrayed.
Task Analysis
Good user experiences are conceived with user goals at the center of the design process. Designers must know who the users are, and what they want to accomplish, then they must take it upon themselves to make it easy for users to complete their goals. When evaluating a user experience, the person conducting the analysis must go through the same process as the user. After defining the target audience and identifying the key goals that users wish to accomplish while going through the interactive experience, the evaluator must put herself through the process, making notes about her observations along the way, relative to the design’s form, meaning and behavior. She records the design’s strengths as well as areas for improvement and summarizes recommendations to make the experience more goal-directed and fluid.
For example, a user goal might be something like “register quickly and easily for a free webinar”. The task analysis may reveal that there are too many steps in the registration process and too much personal information is being requested to the point that users don’t think it’s worth the trouble, so they abandon the process. While it may be great for the business to gather as much information about their audience as possible, users might find it intrusive. In this case, the recommendation may be to shorten the process to one step and relax the data requirements up front to register, while prompting for them again later in the experience once the user is more engaged, and thus more willing to share info. Good design solutions can almost always restore balance by satisfying user needs without compromising reasonable business goals.
Competitive Analysis
We often apply a competitive filter when conducting user experience evaluations. This can be time consuming depending on how deep of a dive we take on each competitor’s design, but we’ve found that even cursory reviews can be very productive. At the end of the day it’s important to know how well you stand out in the crowd and offer superior value to your customers relative to the competition. Close competitors will have similar, if not the same, user goals which allows for a task analysis to be done for competitors’ sites using the same criteria as your own. At Rhythm Interactive, typically, we don’t spend as much time on the task analysis for our client’s competitors’ sites unless there is an unusually good reason. Instead, we breeze through, making quick observations in order to get some general bearings to navigate their business environment. We have found this method to produce the best value in most situations.
Expert Analysis - Design Best Practices
As design experts, we are sure to point out strengths and weaknesses relative to design best practices for the solutions we review. There are some commonly accepted, interactive design approaches and tactics that have emerged from several years of experimentation, trial and error, since the discipline emerged in the early 90’s. Though this part of the evaluation does require an expert’s eye to get the most out of it, there are many published works outlining well-known do’s and don’ts for various applications such as websites and email. A note of caution; some may read like the gospel but they are best thought of more as a rule of thumb or a good starting point. Rules are meant to be broken sometimes…but you have to know them before you break them, and have a compelling reason for doing so. According to our principles, that reason would have to be that it forms a better bridge between business goals and user needs. It should always come back to that simple truth.
Brand Analysis
Finally, we do a review of how well the user experience expresses the essence of the brand vision. Working from key ideas contained in a brand promise statement, documented brand attributes, personality, voice, etc., we again examine the design’s three dimensions - the form, meaning and behavior - but this time against the brand criteria. Most marketers usually don’t have a hard time seeing how the visual design and content can align or malign with their vision for the brand, but some aren’t used to thinking about how the behavior of a user experience can have the biggest impact on brand perception. Imagine if you went to test drive a new car, and when you were leaving the dealer’s parking lot the car turned left when you turned the steering wheel right. There is no amount of slick brand marketing that would restore your trust in that car brand. Just like any physical product, the behavior of interactive media has to align with your brand attributes and users’ expectations to the highest degree, otherwise risk disappointing users and/or sending inaccurate, confusing expressions of the brand.
The Trouble with Quantitative Data
Heat maps, click-throughs, registrations, purchases, etc. – it’s important to have good web analytic info. We’re all for it, but we contend that it provides the most insight after the principles outlined in this article have been applied. If traditional web analytics are relied upon solely or too heavily to measure the quality of a design, the results can be misleading. Sure, a certain area of the experience may be getting the most clicks, but that may be only because it’s the thing users could find, not necessarily because it’s the most important topic to them. When examining performance and behavioral data, remember it’s relative to the dynamics and limitations of the existing design. If the design wasn’t built on the correct foundation, the performance data won’t tell you the whole story.
