<p>Published on <a href="https://blog.prototypr.io/elevate-your-ux-with-heuristic-analysis-how-to-run-a-usability-evaluation-5ae45648045d?source=rss----eb297ea1161a---4">| Prototyping: From UX to Front End &mdash; Medium</a> Miklos Philips </p>
<p>Design is an investment, not an expense. At the risk of stating the obvious: <strong>It’s not enough to design a nice-looking product; it also has to be usable</strong>, and if you are to extract the largest ROI from a product, its usability — which generally refers to ease-of-use — takes on a vital importance.</p>
<blockquote><p>Well-designed products have great usability. Great usability is a significant contributor to product quality and a seamless user experience.</p></blockquote>
<p>There are a few ways a product’s usability can be tested: an inspection method called a heuristic analysis is one of them. This usually means running a <a href="https://en.wikipedia.org/wiki/Heuristic_evaluation">heuristic evaluation</a> on a product, whether it already exists or is brand new.</p>
<h3>What Are Heuristics and What Is a Heuristic Analysis?</h3>
<p>A heuristic analysis is used to identify a product’s common usability issues so that the problems can be resolved, consequently improving the user’s satisfaction and experience and raising the chances of a digital product’s success overall.</p>
<p>Focusing on usability, a heuristic analysis is an evaluation method in which one or more experts compare a digital product’s design to a list of predefined design principles (commonly referred to as heuristics) and identify where the product is not following those principles.</p>
<p><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*E0iKglZwqFq18zAshnqiPg.jpeg">A usability expert performs a heuristic analysis against a set of heuristics to identify usability issues.</p>
<p>A specific set of heuristics contains empirical rules of thumb, best practices, standards, rules, and conventions that have been tested or observed over long periods of time. Sticking to these heuristic standards produce <a href="https://www.toptal.com/designers/ux">UX designs</a> that simply work better.</p>
<blockquote><p><strong>“Heuristic evaluation involves having a small set of evaluators examine the interface and judge its compliance with recognized usability principles (the ‘heuristics’).”</strong> — Jakob Nielsen, <a href="https://www.nngroup.com/">The Nielsen Norman Group</a></p></blockquote>
<p>A heuristic evaluation is not a usability test (for example, a one-on-one moderated test with users). Neither is it a <a href="http://www.usabilityfirst.com/usability-methods/cognitive-walkthroughs/">cognitive walkthrough</a>, which is also a usability inspection method. With cognitive walkthroughs, the emphasis is on user tasks. The process involves identifying the user’s goals and coming up with a task list to achieve those goals. Evaluators then flag problems users may have as they use the product.</p>
<p>A heuristic evaluation expert — the evaluator — is ideally a <a href="https://www.toptal.com/designers/usability">usability testing expert</a> who has deep understanding of the chosen set of heuristics. They would typically come from the disciplines of human factors, <a href="https://www.toptal.com/designers/interactive/interaction-design-principles">interaction design</a> (IXD), HCI (human-computer interaction) and/or <a href="https://www.toptal.com/designers/ux">UX design</a>, with complementary backgrounds in disciplines such as psychology, computer science, information sciences, and commerce/business.</p>
<p>During the evaluation, individual evaluators assign a “severity rating” to each of the usability issues identified. As a rule, UX designers work their way down from the most critical issues on the backlog to the least critical. (In order to get the biggest UX bang for the buck from a heuristic evaluation, it is typical for the design team to give issues with the highest severity rating the most attention.)</p>
<p>It’s useful to note that even though a single experienced UX pro is usually adept at identifying the most critical usability issues, a group of evaluators is generally the best option. Between 5 and 8 individuals is the sweet spot: They should be able to flag over 80% of usability problems. However — as the graph below demonstrates — using more than 10 heuristic evaluators will <em>not</em> yield better results.</p>
<p><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*8nyjZ3xihC6Bselj3gezvQ.png">As the number of evaluators increases, the proportion of usability problems identified increases, the curve shows. But after 10 evaluators, the potential gains are insignificant.</p>
<h3>Why do it?</h3>
<p>The core reason to perform a heuristic analysis is to improve the usability of a digital product. Another reason is efficiency (in this context, “efficiency” is the speed with which a product can be used as a direct consequence of better usability). “Usability” refers to quality components such as learnability, discoverability, memorability, flexibility, user satisfaction, and the handling of errors. A product’s UX is greatly improved when these components are delivered at a high quality.</p>
<h3>When to do it?</h3>
<p>There are no hard and fast rules. A heuristic analysis can be performed at any advanced stage of the design process (Obviously, it would not be productive to do it too early). With new products, a heuristic analysis is usually performed later in the design phase — after wireframing and prototyping and before visual design and UI development begins. Do it too late and making changes will become costly. Existing products found to have poor usability will often have a heuristic analysis run on them before a redesign begins.</p>
<h3>What is the expected deliverable?</h3>
<p>As with other usability tests or inspection methods, the typical deliverable is a consolidated report which not only identifies usability issues, but ranks them on a scale from severe to mildly problematic. For the most part, a heuristic evaluation report doesn’t include solutions — fortunately, many usability problems have fairly obvious fixes, and once identified the design team can start working on them.</p>
<p><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yvf7zjxHn36e3OStjC3aRQ.jpeg"></p>
<h3><strong>Advantages and Disadvantages of a Heuristic Evaluation</strong></h3>
<h4>Advantages:</h4>
<ul>
<li>Uncovers many usability problems and significantly improves a product’s UX</li>
<li>Cheaper and faster than full-blown usability tests that require the recruitment of participants, coordination, equipment, running the test, recording, analyzing, etc.</li>
<li>Heuristics can help the evaluators focus on specific problems (i.e., lack of system feedback, poor discoverability, error prevention, etc.)</li>
<li>Heuristic evaluation does not carry the ethical and practical issues/problems associated with inspection methods involving real users</li>
<li>Evaluating designs using a set of heuristics can help identify usability problems with specific user flows and determine the impact on the overall user experience</li>
</ul>
<h4>Disadvantages:</h4>
<ul>
<li>Experienced usability experts are often hard to find and may be expensive</li>
<li>The value of issues uncovered by evaluators is limited by their skill level</li>
<li>At times, a heuristic analysis may set off false alarms: Issues that would not necessarily have a negative effect on the overall UX if left alone are sometimes flagged to be fixed</li>
<li>Unlike <a href="http://www.usabilitybok.org/cognitive-walkthrough">cognitive walkthroughs</a>, heuristic evaluation is based on prejudged notions of what makes “good” usability</li>
<li>If the evaluators are not part of the <a href="https://www.toptal.com/designers/all">design</a> or <a href="https://www.toptal.com/developers/all">dev team</a>, they may be unaware of any technical limitations on the design</li>
</ul>
<h3>How to Run an Effective Heuristic Analysis</h3>
<p>Preparation is key to running the analysis well. Following an established set of steps ensures that a heuristic analysis will run efficiently and yield maximum results. Here’s a heuristic analysis checklist:</p>
<ol>
<li>Define the scope.</li>
<li>Know the business requirements and demographic of the end-users.</li>
<li>Decide on which reporting tools and heuristics to use.</li>
<li>Evaluate the experience and identify usability issues.</li>
<li>Analyze, aggregate, and present the results.</li>
</ol>
<h4>Step 1: Define the scope.</h4>
<p>On both large and small projects, budgets may be limited. This may be especially the case on large eCommerce sites: For example, it may not be feasible to examine the entire site, as it could take a very long time and therefore become too expensive.</p>
<p>This is where <strong>scoping the heuristic analysis</strong> comes in.</p>
<p>Parameters may be set to examine only the most crucial areas of the site. The limited scope may only have the capacity to focus on specific user flows and functionalities, such as log in/register, search and browse, product detail pages, shopping cart, and checkout.</p>
<h4>Step 2: Know the business requirements and the users.</h4>
<p>First, the evaluators should understand the business needs of the product/system. Second, as with any typical user-centered design process, it’s crucial to know the users. To facilitate heuristic analysis, specific user personas must be established. Are the end-users novices or experts? What are the user demographics?</p>
<p>For example, although heuristics were meant to work as universal usability standards, perhaps special emphasis needs to be placed on accessibility for an older audience — or maybe diverse, multicultural audiences need to be kept in mind.</p>
<h4>Step 3: Decide on which reporting tools and heuristics to use.</h4>
<p>It’s incredibly important to decide <strong>which set</strong> of heuristics the evaluators are going to use. A selected set of heuristics will provide common guidelines against which each of the experts can make their evaluation, as well as ensure that they are all on the same page. Without it, the heuristic analysis process could fall into utter chaos — produce inconsistent, conflicting reports and ultimately become ineffective.</p>
<p>As part of the heuristic evaluation plan, a system, a format, and which tools to use should be agreed upon. This could be Google Docs, Sheets and Slides, or some other common reporting tool that everyone can use and to which the “observer” will have easy access. (More about the observer later.)</p>
<p><a href="https://en.wikipedia.org/wiki/Jakob_Nielsen_(usability_consultant)">Jakob Nielsen</a>’s <a href="https://www.nngroup.com/articles/ten-usability-heuristics/">10 Usability Heuristics for User Interface Design</a> are probably the most commonly used set of usability heuristics. There are others such as the list of six <a href="http://architectingusability.com/2012/06/28/donald-normans-design-principles-for-usability/">Design Principles for Usability</a> by <a href="https://www.jnd.org/">Don Norman</a>, and the 20 Usability Heuristics by Susan Weinschenk and Dean Barker listed below. There is even a set that contains no less than <a href="http://www.userfocus.co.uk/resources/guidelines.html">247 Web Usability Guidelines</a> by <a href="http://www.userfocus.co.uk/about/profiles.html">Dr. David Travis</a>.</p>
<p><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*yWaIPGiI9YRfs0DT4MxdTg.png"><a href="https://uxmag.com/contributors/susan-weinschenk-ph-d">Susan Weinschenk</a> and <a href="https://www.linkedin.com/in/deanbarker/">Dean Barker</a> (Weinschenk and Barker 2000) researched usability guidelines and heuristics from many sources (including <a href="https://www.nngroup.com/articles/ten-usability-heuristics/">Nielsen’s</a>, Apple and Microsoft) and generated this set of 20 Usability Heuristics to check against.</p>
<h4>Step 4: Evaluate the experience and identify usability issues.</h4>
<p>When a heuristic evaluation is performed with a group of experts, each individual evaluates the UI separately. This approach to the expert review is done in order to ensure the evaluations will be independent and unbiased. When all the evaluations are complete, the findings are then collated and aggregated.</p>
<p>In order to run the evaluation efficiently, it’s well advised to use an “observer.” It may add a little overhead to the evaluation sessions, but is definitely worth it as there are many advantages. The observer participates in every session and handles taking the notes, so is able to deliver one consolidated report at the end of the evaluation process, rather than there being a separate set of documents from each evaluator.</p>
<p><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*B3kI1u-b-AxpHswA-hEyVw.png">A heuristic evaluation example. In some cases with a single evaluator, they not only identify usability issues but also may provide recommendations.</p>
<p>During the inspection, the observer may also help answer questions from evaluators with limited domain expertise (for example, in the case of a specialized enterprise UI targeting expert users). They may also assist in guiding the session when a prototype with limited functionality is being evaluated.</p>
<p>In order to help the team move toward design solutions, findings must describe the issues precisely. Vague notes such as “this layout will slow down the registration process” are not at all productive or of any value. Notes need to be specific and clearly identify the heuristic that the issue violates. For example: “During registration the UI layout is confusing, inconsistent and violates the rules of user control, feedback and consistency (#1, #20, and #16 respectively).”</p>
<p>For the sake of speed, UIs may be marked up visually with notes that can be consolidated later (see the one below). This method helps to quickly aggregate the expert’s final notes, and the observer doesn’t have to search for the UI components being addressed. They can also be coded for easy identification by the design team.</p>
<p><img alt="" src="https://cdn-images-1.medium.com/max/1024/1*WFwBETxato4RnAAiWwir_w.jpeg">Heuristic evaluation of user interfaces. An example of a heuristic evaluator identifying usability issues (Dashboard by <a href="https://dribbble.com/shots/3655377-Dashboard-Projects-06">Guohao.w</a> on Dribbble).</p>
<h4>Step 5: Analyze, aggregate, and present the results.</h4>
<p>At the conclusion of a heuristic analysis, the evaluation manager — or observer — carries out some housekeeping and organization such as removing duplicates and collating the findings. The observer’s next step is to aggregate the heuristic evaluation reports and build a table that includes the severity ratings of usability issues and from which the design team can prioritize.</p>
<blockquote><p><strong>“For usability testing to be valuable, study findings must clearly identify issues and help the team move toward design solutions.”</strong> — <a href="https://www.nngroup.com/">The Nielsen Norman Group</a></p></blockquote>
<p>The output from a heuristic analysis should be a list of usability problems that not only identify specific problems, but reference the usability heuristics the problems violate (preferable a code number for easy reference). For example, the above screen points out that using low contrast text in the UI violates the heuristics of “visibility” and “discoverability.”</p>
<p>Using reference codes from the chosen set of heuristics will help build a data table which can then be sorted. When the design team sees that a large number of issues reference a small number of violations (identified by code), they can focus their energies on improving them. For example, there may be widespread issues of visibility and discoverability as in the example above.</p>
<p>Heuristic analysis doesn’t necessarily provide fixes to usability issues, nor does it provide a”‘success probability score” if the design improvements are to be implemented. However, because a heuristic evaluation compares the UI against a set of known usability heuristics, in most cases it is remarkably easy to identify the solution to a specific problem and come up with a more compelling design.</p>
<h3>Wrap-up</h3>
<p>Some new apps in development and many mainstream products suffer from poor usability. Most of them would benefit from a dose of heuristic analysis performed by experts and, as a consequence, see a dramatic improvement in their UX without breaking the budget.</p>
<p>A single experienced UX expert can uncover a substantial number of usability issues during a heuristic analysis. However, if time and money allows, between 5 and 8 experts seems to be the sweet spot — this option should uncover most usability issues and offers a significant ROI. This ROI would be based on the increase in user productivity as well as estimated on the expected increase in product sales due to higher customer satisfaction, better ratings, and an uptick in positive reviews.</p>
<h4>Please Note</h4>
<p>It must be mentioned that even though heuristic analyses are definitely a solid way to identify usability problems regarding digital products, they should not be relied upon as the only source of data. Studies show limitations to expert review because of psychological reasons such as cognitive bias.</p>
<p>If possible, in order to achieve optimal results, heuristic analysis should be combined with cognitive walkthroughs and one-on-one user testing. And <strong>that</strong> should produce awesome product designs.</p>
<p><strong>Further reading</strong>:</p>
<p><a href="http://asktog.com/atc/principles-of-interaction-design/">First Principles of Interaction Design</a> by Bruce Tognazzini</p>
<p><a href="https://www.nngroup.com/articles/ten-usability-heuristics/">10 Usability Heuristics for User Interface Design</a> by Jakob Nielsen</p>
<p><a href="http://www.userfocus.co.uk/resources/guidelines.html">247 Web Usability Guidelines</a> by Dr. David Travis</p>
<p><a href="http://architectingusability.com/2012/06/28/donald-normans-design-principles-for-usability/">Donald Norman’s Design Principles for Usability</a> by Don Norman</p>
<p><a href="https://twitter.com/MiklosPhilips">Follow me on Twitter</a> for juicy UX tidbits</p>
<p><em>Originally published on </em><a href="https://www.toptal.com/designers/usability/usability-analysis-how-to-run-a-heuristic-evaluation"><em>Toptal</em></a></p>
<p><img src="https://medium.com/_/stat?event=post.clientViewed&amp;referrerSource=full_rss&amp;postId=5ae45648045d" width="1" height="1" alt="stat?event=post.clientViewed&amp;referrerSou"></p>
<hr>
<p><a href="https://blog.prototypr.io/elevate-your-ux-with-heuristic-analysis-how-to-run-a-usability-evaluation-5ae45648045d">Elevate Your UX with Heuristic Analysis — How to Run a Usability Evaluation</a> was originally published in <a href="https://blog.prototypr.io">prototypr</a> on Medium, where people are continuing the conversation by highlighting and responding to this story.</p>


Elevate Your UX with Heuristic Analysis — How to Run a Usability Evaluation

Top Stories