A framework for measuring design maturity
uxdesign.cc – User Experience Design — Medium | Caio Braga
If we want design to be at the core of the business, we need to know how to evaluate its maturity.
It is a long article, ~bear~ with me
If you ask any modern CEO, they will say that design is important to their business. But, in reality, that is not always the case — or, at least, in the same dimension that they say.
- A small size business might have only one or two designers covering all aspects of the design process in multiple streams of work.
- A mid-size business can have a handful of designers, but the work is only done reactively.
- A big company can have bigger design team, but they might not have a sit at the table where the strategy is defined.
Although the work gets done, the maturity of the design team can feel stale: the budget doesn’t change much, hiring is slow, the design team doesn’t have a voice in the company’s strategy. That’s when we question how important design really is to the business.
“Why do we need another designer? We are meeting all our deadlines!”
You’ve likely heard this before. If we want to improve on it, we need to first better understand the design team’s maturity. Only then we will be able to assess whether we need to hire more people and have a solid case to pitch it to the company.
How can we translate our perception of the company’s design maturity to something more tangible, that the CEO or any other coworker can understand and act on?
Design maturity is hard to measure
The size of the design team or its budget might not be the best indicators of why the design maturity is not evolving — or at least shouldn’t be the only indicators used, because it is possible to have a successful team of one or a successful team with a limited investment.
Measuring the user satisfaction is not enough either. Although it is a good product metric, it doesn’t provide the information needed to the design team itself to improve upon.
Measuring the skills of the team can be great to identify strengths and weaknesses in the team, but it still doesn’t show how the design team work together.
To measure the design maturity, the focus needs to be on their work dynamic, in the context of the business.
Design maturity is hard to act on
Jakob Nielsen, Jared M. Spool and many others big names from design industry have written about the different stages of design maturity in a company. They might have different approaches, but they all have a similar thinking behind it: a scale of stages representing the business relationship with design.
For this article, let’s use the Nielsen Group scale as an example:
Nielsen Maturity Stage (source)
While design maturity scales like this are a great way to better understand where the design is at your company and what it could reach, its interpretation can be subjective:
- A manager might argue that the design is on stage 4 of the Nielsen’s scale above, after all, design has a dedicated budget within the company.
- A design team member might disagree and argue that it’s on stage 2 (developer-centered) because design decisions are made without enough user research.
- A designer can also disagree with other designers about whether their voices are heard by Product and Engineer or not.
This is a great discussion to have both with your team and with the big bosses. However, a design maturity scale often lacks something tangible to be measured against and to indicate if any progress has been done over time.
Increasing the maturity of the design team is not something that can be done overnight. It can take years. But it happens everyday in small steps.
Measuring these small steps can help the team to go beyond day-to-day projects and be mindful of the strategic role of design for the business and the design initiatives needed to evolve from one stage of maturity to another.
We are used to measuring the Return Over Investment (ROI) of our work and projects. Now, we need to learn how to measure more subtle aspects of design — and the design team itself.
Much ROI (source)
A proposed framework to measure design maturity
To measure the design maturity, we first need to define metrics that can be used by any design team, regardless of the context of work and team composition.
To do so, we can look at the basic ingredients of any design project. These ingredients can be grouped in three pillars:
- Design system
- Design process
- Design vision
Quoting Nathan Curtis: “A style guide is an artifact of design process. A design system is a living, funded product with a roadmap & backlog, serving an ecosystem”. It consists of a guideline and repository of patterns, components and assets a company has, but it goes beyond: it means designing, planning and maintaining a consistent experience, unique to the brand and its tone and voice. It’s a hot topic in design and you can find a few examples here.
The design process is… Well, the design process used by the team. A library of methodologies and good practices to ensure a high quality standard and consistency in the work done. Not all problems can be solved with the same approach, but having a main framework can facilitate collaboration and makes other tools and methodologies more accessible. You can find example of methods in our UX design Methods and Deliverables list.
Design vision is the direction where the team sees the design heading. It is formed by principles, values, and even their “blue sky” concept for the business, aligned with the company’s mission and goals. The team will likely fail, after long and unproductive critique sessions, if they aren’t on the same page about its vision. You can find good references of design principles in this repository.
Whether we are mindful of them or not during a project, these points are the main ingredients of any design. After all, every design project has an interface system, follows a process, and have a vision behind it.
Understanding and framing these pillars help the design team to talk about their work, their design team and its maturity.
Defining the maturity metrics
Within these three pillars in mind, we can define points to be used as metrics for each. These points can vary a lot depending on what is important for the team and for the business, from something specific and tactical to a broader satisfaction score with that pillar. Here are some examples of what could be used as metrics:
- The presence, stability, use and effectiveness of interface elements and composition, such as color, typography and iconography, meeting accessibility standards.
- A clear voice and tone guide, keeping the experience consistent and on-brand.
- The designers’ satisfaction working with the current pattern library.
- A consistent high-level process, from discovery and research to QA and launch tasks.
- Clarity for the team of all design steps and knowledge of the tools and methods available.
- Opportunity (space and time) to study and experiment different methods and cross-team collaboration.
- Resources available to help with their work, like a proper research room, lab or even a participant recruiter, resource manager or design producer.
- Clear business goals and a clear user market. Every designer should understand the company’s mission and high-level roadmap.
- Design principles and vision that the team shares and is confident to use in their projects.
- Design initiatives and projects, aligned to its vision, and not necessarily directly linked to a company’s project or product.
Setting up a process to measure it
These points are not items that you can simply cross off the list. They need to be constantly measured and discussed with the team and with the business.
Regardless of the metric points decided for each pillar, the team will need a rating scale (o to 10? Five stars? Emoji faces?) and specific weights for each point selected.
Defining what and how to measure should be done by the design team. Every design team has a different and unique context that needs to be taken in consideration. Any formula has the risk of being too generic. Discussing the metrics and the approach is a great starting point for any evaluation discussion and also makes the team feel like they are owning their self evaluation mechanism.
If a design team masters them all with a solid design system, vision and process, every project will likely unfold well and fighting for hiring a new member or for having a voice in strategic decisions should be easier.
If the design team concludes that the current state of any of these pillars is not satisfactory, they have a clear direction on what to work on and a good argument to ask for more resources for the team if needed.
One can argue that it’s impossible to do well in any of these three areas without doing well the others. It is true, but their level and quality can vary a lot. And that’s exactly why we need to measure it.
Even if the all the scores are initially low, doing this exercise with the team is the best way to check that everyone is on the same page and to measure the progress of the team over time.
On the other hand, if the team seems to have everything under control, there is always new challenges in each of these three pillars. Whether because patterns need to be updated, the team has to onboard new members, or a process needs to be reviewed for a new Virtual Reality App that is coming up, for example.
The reality is that the design industry is extremely dynamic and the context can switch quickly and it’s the same for the design team. Nothing will be perfect — everything can be perfected.
Whether the team is centralized or distributed, with these three pillars as talking points in the agenda, it is easier for the designers to assess how they, individually, perceive where they are to each one of them, starting the conversation about what needs to be done and where to invest more.
Communicating it to the world
“Can you explain what you all exactly do?”
Regardless of the design maturity of your company or how many designers you work with, the outline proposed here can help your team to organize the design initiatives, assess the design culture and pitch the design team to the rest of the company. It’s a great artifact for the team, but also a powerful tool to sell design to the company.
It is, of course, not the only way to do it. A design studio might have a completely different dynamic, a design team might prefer a more creative anarchic approach, an agency would need to adapt this framework for each client, and so on.
The most important aspect is to understand the team, its culture and way of working, its maturity and talk about it, openly and frankly. This is just one way of many ways of doing it. How are you doing it?
Caio wrote this story to share knowledge and to help nurture the design community. All articles published on uxdesign.cc follow that same philosophy.