The Evolution of UX Research Process at Kira
A few weeks ago, a group of our customers from a big law firm reached out to the design team and asked for guidance on how to do user research, after they’ve learnt about our design process.
Four years ago, when I first started out as as a solo designer at Kira who tirelessly advocated for doing user research and met with many challenges, this would not have been something I expected.
So what happened?
In this article, I will outline our user research process on a high-level and some important aspects that shape our current user research practice.
Advocacy and Education
The first thing about user research is: do user research.
This sounds so easy but is so hard in practice. User research is still not a valued and integral part of the design and product process in many companies. I have heard so many war stories of designers trying to get buy-ins from leadership teams for doing user research right but failed. There are unique challenges faced in every team, product, project, and organization. This was why, since my first day, I was determined to let the design be based on good user research. I was the advocate and educator of user research within the organization.
In a different post, I explain in detail why user research is important and some approaches to resolve challenges in establishing user research within an organization.
Forming the User Research Phases
In the early days, when the team was small and I was fairly inexperienced in user research, I relied very much on Google’s quick and dirty research sprint guide in the early days for practical guidance, while reading articles on NNG and books such as About Face. I asked every UX designer that I knew how they recruited users or planned for user research, and later on hired a UX researcher to help explore possibilities and overcome challenges in order to define our user research process.
Two books helped define the foundation of our UX research process: Just Enough Research by Erica Hall, and Interviewing Users by Steven Portigal.
When I discovered Just Enough Research, we were working on a very complex project in the exploration phase (although we didn’t officially named it that yet). And to figure out the right research methods to use for each step of the project made us realize we needed a more structured process. Just Enough Research gave us a starting point of just that.
We have since defined our research process to be these three stages:
Generative (explorative) research. During this phase, things are very ambiguous. We aim to find out the “why’s” in order to define the problem to be solved. Methods such as card sorting and concept testing would fall into this phase. The result of this phase is to have a good understanding of the problem at hand, and some “how might we” questions to take into the next phase.
Descriptive (explanatory) research. Once we formulated our design challenges, we move on to explore how these problems may be solved. This may include some more in-depth situated user interviews to gain more context, but could also be a moderated user feedback session where some early explorations of possible solutions are presented as visual cues to gather some concrete feedback instead of communicating abstract concepts. It is worth mentioning that the line between descriptive research and evaluative research can be blurred depending on the complexity of the problem at hand.
Evaluative research. At this stage, the the problems to be solved would have been very clear, design solutions have been well explored and critiqued, we are ready to zero-in on the best solution. This phase can be very iterative. For projects where the features are fairly novel to users (e.g., designing for document clustering or designing for certain part of training a machine learning model), we may have many rounds of testing and designing-tweaking, which requires us to be really responsive and move really fast as a team. For other projects, we tried to stop testing when the results are good enough. At some point, we would have to put the design to the ultimate test—release.
Building Relationships with Our Users
Despite of being mysterious and sometimes demanding, users are our fellow human beings. They are not just “people we design solutions for.” On top of that, we recognized that every time a user spend an hour with us, that was an hour of their working time where they are billing $300+.
During the days when recruiting users for user research was not easy, we decided to send special custom-made gifts such as laser-cut coasters to users who participated in our user research sessions. We also took every opportunity meeting them to talk to them and learn about them outside of their jobs. As a B2B product company with layers and layers of sales process and business negotiations, we somehow found a way to bond with our users beyond just the designer-user relationship.
The research team has collaborated with other customer-facing teams to develop a recruiting process. Some of us who were on the team long enough know a dozen of users by name and can count on them for honest and insightful feedback, as well as helping us recruit the right users for user research sessions within their firms.
Rigour and Efficiency in UX Research
When our customer reached out to us for user research guidance, they did so because they were impressed by the rigour displayed in our process.
Rigour
A defining moment in developing our user research process is to learn about how user studies are designed and conducted in academia, especially in the fields of HCI and Interactive Information Retrieval. This made us realize if we are truly to be evidence-based, we need to be more rigorous in the way we design our research, and how we analyze and synthesize the research data.
Our earliest effort in being rigorous in our research approach resulted in a publication in CHIIR (Conference of Human Interaction and Information Retrieval), an academic conference. We also recognized the noticeable difference of being rigorous in our research made when it comes to defining and solving the right problem. It forced us to be more aware of our plans when we had to articulate very clearly what our research goal was, what our assumptions were, what we chose to measure and how in according to the goals and assumptions.
Qualitative Coding and Thematic Analysis
By this time we have also gotten into the habit of video recording each user research session. We would then transcribe the recording and keep them centralized in a tool, where we subsequently clean up and code the transcripts. Once the transcripts are coded, they are synthesized to form themes which will translate to insights that form the “how might we” questions. Designers usually work closely with researchers during this step of the process to ensure a common understanding of the problems to be solved.
Qualitative coding is a common research method used in academia. We adopted it to allow us to capture as many insights with context as possible in order to define the right problems. It also allows us to document and therefore share the knowledge among team members working on different projects across times.
Efficiency
Many have challenged the approach of being rigorous in our UX research approach. Understandably being part of a fast-growing and high-demanding team, we need to move fast. Planning, designing, recruiting, conducting, analyzing and synthesizing UX research take time, especially when we aim to do it scientifically. It is a genuine concern that I can sympathize with.
In my experience the key to efficiency on a project level is collaboration between researchers, designers, product managers, sometimes even developers. As a team, we move much faster overall when more people are involved in understanding the problem. In the long term, it also makes knowledge transfer and sharing much easier.
Some of us may still remember that a few years ago, designers were advocating for “a seat at the table.” Many organization leaders were concerned that taking the time to do user-centred design was too time consuming. Fast forward to today, design is very much a valuable and integral part of the product process in most places. UX research will soon get there if there are enough of us pushing it along.