First steps for better UX research

How we are making User Research more frequent, structured and transparent at Nubank.

Better UX Research

Here at Nubank, it’s essential that we keep ourselves fine-tuned to our users’ minds and emotions. After all, every aspect of their financial lives is a very personal, delicate matter, extremely rich in details, and heterogeneous behavioral patterns.

When building for millions of diverse users, qualitative research is the tool to help us keep an empathy window open so we can design for realities, demographics, and pains that are different from our own. 

Nonetheless, as in most companies similar in size and age, there’s no team exclusively responsible for the practice of user research at Nubank. Our design team is mostly composed of process-oriented, generalist professionals.

As such, we believe that design is not only about problem-solving (executing), but a lot of times about facilitating problem-understanding. With this in mind, we’ve been acting on our responsibility to push good user research forward, with the ambitious goal of “making every Nubank employee an expert on the people they’re building for”.

Not long ago, our approach to research may have sounded familiar to many people working in agencies and consultancies. Whenever a new big project came up, we would assign one or two people to execute an extended, focused sprint of research on the topic, feature, or customer segment. They would crunch through the collected material and write a report detailing the methodology, producing user profiles, insights and action points.

This way of working, however, quickly lost sync with the pace at which things were happening at Nubank. Consultancy-like research, compared to the agile world, is slow, expensive (in terms of time and human resources), and the final outputs were often ignored by stakeholders who still trusted their guts more than our methodologies.

So, we started looking for ways of doing research that would better fit our context, and make it a more distributed, frequent practice that is consistently planned, executed, and documented, without becoming a burden for any single individual.

1. Frequent

Our user base grows and shifts extremely fast. This means that research done three or four months ago may very well not be representative of the majority of our users anymore. We need to keep up with the pace the market, technology and our users’ realities evolve. 

Our solution to accelerate the pace of sessions was to establish a schedule for user interviews – with an optimistic goal of having one member of the team talking to a user at least once a week.

The critical value of having a schedule before you even have something to test is that you get the team talking to users no matter what: if there are no new prototypes or features to test with users, we talk to them anyway.

It’s an excellent opportunity to have them show us their current apps and learn from what’s already in production. Also, publicly setting an expectation about frequency creates a healthy pressure that keeps us moving.

To lower the barriers for this to happen, we had to work on our recruiting process, which involves running database queries, crafting and sending invite emails, buying gift cards, scheduling, and keeping in touch with users. This is always going to be time-consuming. However, with the help of tools like Typeform, Doodle, and some willing Business Architects, we can now run this step much faster than before.

Also, in the beginning, we had a lot of trouble booking a place for research sessions. There was never a room available for long enough, which made us run around and re-setup with gear in between sessions, and business meeting rooms are not especially suited for making people comfortable talking about their lives.

To solve that, we’ve built a room in our design corner of the building exclusively for research use. It’s now called Engelbart, (in honor of one of the founding fathers of human-computer interaction), and we’ve substituted tables and office chairs with a sofa, TV rack, snacks, and plants to make people feel more relaxed.

2. Engaging and transparent

Writing polished reports or presentations about our research findings turned out to have very little value to us. We tried sharing PDFs on Slack, putting insight posters up on walls, and still got no sense that people were finding the information useful. Also, having stakeholders be in touch only with the results often felt frustrating, as it was easy for them to dismiss findings based on personal opinions or biases.

So, what we decided to do was to make all the action live. Now, all our research sessions are streamed to a second room where everyone interested is invited to watch, and there are no final reports or presentations to be shared.

This move was by far the most valuable of all for us, as it puts product managers, engineers, and other professionals in direct contact with the user they’re working for. It helps develop empathy both for the customer and for the design team, making our processes less obscure to professionals with more analytical and rational backgrounds.

A quick note about how we actually do this:

We didn’t want a two-way-mirror setup, since we think it’s too obvious and make people very uncomfortable. Our best solution was to plug cameras and microphones to a physical HDMI cable that goes up to a watching room, streaming all video and audio perfectly.

We tried webstreaming, but the quality was never perfect. Also, with webstreaming, stakeholders started watching sessions from their desks, which does not enable for collaborative note-taking and live discussions with others.

3. Well documented

We mentioned that we didn’t do many reports and presentations on UX research anymore, and it’s true. The beauty of having stakeholders all together watching the session is that they can do the parsing and documentation themselves, while the interview is still happening.

Furthermore, with more people involved in taking and comparing notes, we reduce bias in the conclusions, insights, and action points. (See this article on Collaborative User Testing).

We give everyone access to a fresh copy of Jessica Crabb’s incredible Trello Board and instruct them to collectively participate in the sorting of quotes, insights, and action points in real-time. This board is excellent because it helps keep all the documentation (screeners, participant list, interview guide, questionnaires, NDAs, prototypes, raw notes, insights, and action points) in one centralized place. 

As a backup, in case something goes wrong with the tech setup or in case there’s no one watching the sessions, we always have a second person in the interviewing room taking notes. In the end, we upload all testing videos to a private YouTube channel, so people can check back on them if necessary.

Challenges & next steps

This was just the beginning of our research practice evolution here at Nubank. During this period, we’ve learned a ton about processes, methodologies, and best practices from our peers and the external community. But perhaps the most critical finding was that there’s still a long way to go.

Here are a few things we’re already looking into for the future:

Diversify methods and broaden populations

We’ve done a lot to accelerate and master our in-house user interviews, and they have since added a lot of value to our teams and features. It’s evident to us now that, to keep up with the complexity of our user base, we’ll need to start expanding our vocabulary of research methods – especially by getting out of the office, the privileged neighborhood we are based on, and even start hitting the road to talk to folks and better understand their particular uses of our products.

Build a more cohesive, universal research practice across teams

Although we’ve covered a lot of ground for User Research specifically, there’s still a long way to build a company-wide, entirely connected research practice. A lot of growing companies suffer from research silos. That happens when multiple teams are doing different kinds of research (market research, A/B tests, user interviews, customer support feedback, app analytics, brand research, surveys, etc.) that are not connected, and there’s no one looking at the big picture of learnings and results.

We’ve been reading a lot and have started exploring ways to join those teams in a conversation that helps to build a shared understanding of what we know and don’t know about our users. If you can relate to the problem described above, and haven’t read the article below, please do:

Seeing the Elephant: Defragmenting User Research

Make knowledge useful and usable

One of the biggest challenges of growing research practice is the management of the also growing pool of outputs. Different teams are continually producing large quantities of insights that are well documented and shared with the immediately interested stakeholders.

Still, we are aware that there are better ways of making this content available and usable for everyone else. Our end-goal should be that any person in the company can type a few keywords somewhere and learn everything that anyone else has researched about that topic. Furthermore, anyone should be able to add their discoveries and learnings to this “insights database”.

We’re big fans of Tomer Sharon’s work on the topic of Democratizing UX and atomic research units. 👇

Democratizing UX

The atomic unit of a research insight

Make data our friend

It’s very easy to get used to the tools and processes you master. Designers will probably be more comfortable with learning from usability tests, product managers with app analytics, and data scientists with running queries on a database. The most precious insights, though, will come from the aggregate analysis of these viewpoints.

We, as designers, need to make an effort to be more comfortable with quantitative analysis and to make more data-driven people understand the value of being in a room with a unique user. It’s extremely easy to disregard someone else’s learnings or observations if they were based on a methodology you don’t understand, trust, or relate to.

Be more accountable

If we designers/researchers want the practice and its results to be taken seriously, there’s a lot we can do to make our efforts more visible and credible for everyone else (as they say: what gets measured, gets managed and improved).

Building trust from leaders and helping them take our (often subjective) point of view into consideration when making decisions means that we need to put some effort into translating our learnings to a language they can understand and navigate.

As I’ve said, we’ve been growing a lot and learning as we go. Finding the right format for user research in an agile startup environment is a very complex endeavor and one that should be approached step-by-step, as any other design problem. It’s definitely worth the effort, though.

Every new customer that we get in touch with humbles us and changes us in an unparalleled way, getting us closer to our mission of understanding their problems and building solutions that truly empower their lives.

We’ve learned a lot from the community reading your blog and medium posts. If you have more interesting resources, please contribute! We’d also love to hear your experiences in building effective research practices in your organization.

Thanks to Rachel Jordan. 

Enter your name