Why we have a research plan at Nubank

(First published in “Designing Nubank” blog on February 2019.)

Nubank has been building products without a dedicated researcher for a long time. In March 2018, I was hired as the company’s first full-time researcher. As of today, I am part of Nubank’s Design team, helping both Product and Design teams understand better how our users behave, how they are using our features, and where we envision our product to go.

Up until my arrival, designers were responsible for doing research, and they were having considerable overhead dealing with both research operation and design work. They were handling it very well, taking some first steps and evolving as fast as they could.

I saw places where I could help — such as best practices, measurement of impact or documentation, and decided to start with the latter.

Documentation is a subject I really like, and one that is often overlooked by us (designers).

Just a quick disclaimer: I love to be lean and agile. But, paraphrasing the agile manifest, “less process/documentation and more people” doesn’t mean no documentation or poor documentation – and we all know that poor documentation is worse than no documentation at all.

All documentation should exist to make your thought process clear and create a shared understanding of the problem.

Usually, everyone involved in a project has some personal opinions or thoughts on the subject being researched. A good research document is something that puts everyone on the same page and helps them discuss the research topic from the same perspective.

There are many ways to do that, but a well-designed document will help people debate, agree, and easily remember everything that was discussed and their reasoning behind it.

Also, a good document will help the group deepen its understanding of the real issue behind the subject, moving the team from a broader, open-ended question into a more specific and measurable situation.

Documentation can also work as a form of history. A good document is the best way to show the reasons behind the decisions we made and why we implemented those choices.

We like to start by distilling the problem into a list of the real and measurable questions we want to answer in the most granular way possible.

And to do that, we focus on four topics:

1. Goals
2. Questions we need to answer
3. Expected impact
4. Hypothesis

Following guidelines will make everyone in the room think harder about the problem. Perhaps some people’s expectations were just too high, or they were simply looking at the question from a wrong angle or considering a different result altogether. Again, documenting puts everyone on the same page.

Goals

The goal is, in fact, the open-ended question we began with — the broad problem that we all have at the beginning of the research. “Help us increase the number of messages exchanged inside the app”, for instance. Well, that is super broad and may mean different things to each stakeholder, right?

Questions we need to answer

These are not the actual questions we are going to ask users; these are broad questions that we want to answer during our research.

To transform this goal into something more tangible, we need to have a clear picture of the questions being asked. Asking your stakeholders what they need to achieve that goal will result in a group of more grounded questions like: “Are users able to create new conversations?”, “Is there any usability issue on the chat?”, “Do users understand that they can send multiple messages at once?”, and so on.

These questions will help you shape the methods you’ll end up using during your research. And it will also help you know if you are on the right path to answer your goal.

Expected impact

With that in mind, we can shape the expected impact that our research will have on the users. It will also help us guide the direction of the research and some of the methodologies we may choose. “Remove any usability flaws that we find.”, “Increase the number of new conversations started.”, “Increase the number of chats created.”, “Better understand why users create groups.” are some of the examples.

That is how you will measure the outcomes of your research, and it will play an essential role in shaping how you will collect your data. So, take your time and make sure that you and the stakeholders know exactly what you are expecting; otherwise, you may end up not meeting someone’s expectations.

After going through these three steps, and with a clear picture of what you are trying to achieve, you can finally start working on the final step: your hypothesis.

Hypothesis

According to Google, a hypothesis is:

“a supposition or proposed explanation made on the basis of limited evidence as a starting point for further investigation.”

A hypothesis can be written in multiple forms, but most of them are structured in an if/then situation. A well-written hypothesis will help you know what to test and give you an idea of how to test it. “Asking for the user to put a personalized avatar at the beginning of the registration process does not affect the number of messages sent.” is one example of a hypothesis.

Having everyone agree on the same hypothesis may be tough, but, again, this will make everyone think even harder about the problem that is being studied.

The more we grow, and as more people have an interest in research, the more relevant this type of document becomes. So, putting some effort into it will always bring good results.

We usually start this document during a kick-off meeting, but the meeting works more as a way to start the research process. We tend to finish the document more collaboratively, over a shared Google Docs file.

Sometimes these documents may be longer than usual to be completed, but the amount of time and effort that you put into each of these steps will change according to the size of the research.

Documentation works more like a way to make people think about what they want to uncover, what their real concerns are, and the impact they want to cause. But remember: documentation is a tool — not something carved in stone that cannot be changed.

What about you? Do you have a research plan document? What for? What does it look like?