Listening to user feedback in an engaged community

At Nubank, we are lucky to have an engaged and passionate community of users who submit hundreds of feedback and feature request messages every week through our support channels and social media.

This steady and loyal stream of inputs is like a gold mine to our lucky product teams. It is rare to have such a rich and voluntary source of insight into user’s needs and pains so readily available.

Managing this information and putting it to good use, though, has been much trickier than we had imagined. In this article, we’ll share some of the challenges we’ve faced, how we’ve dealt with them until now, and what we’re planning to do next.

Phase 1: screenshot copy and pasting

When I joined Nubank back in 2016, we were nearing the 1 million active customer mark. At that point, every feedback message we received was proactively posted by our customer support experts (Xpeers) on a Slack channel called #feedbacks.

It is interesting to note that nobody demanded that support team members report customer feedback to the product teams. This workflow was a natural result of having a highly-trained support team that feels as much ownership of the final product as designers, PMs, and engineers.

To this date, anyone in the company can join the #feedback channel, read through common and hot requests in real-time, collaboratively discuss them on threads, suggest solutions, and @ mention stakeholders to get their attention on important issues.

As I write this -in November 2018-, the #feedback channel is the 34th most popular out of 1454 channels in our Slack account. It has 232 members and is bubbling with real-time customer messages all day.

👍 Value added by this first simple solution:

• Centralized home for all customer feedback on Slack
• Threaded discussions about each problem/idea
• Bring visibility to critical bugs or urgent problems with the app

Even though this was a great start, there were evident problems that needed addressing:

👎 Downsides of the first solution:

• Screenshots are impossible to search, parse, or analyze.
• It can only be consumed as a feed. There’s no way to generate insights from an aggregate view
• A single channel for all feedback topics quickly became too noisy as we grew to a multi-product company

Check our job opportunies

Phase 2: multi-channel

In order to adapt to our growth, squads naturally started creating spin-off feedback channels that were specific to their domain.

So, beyond the main #feedback channel, which became home to more “generic” topics, we started seeing channels like #virtual-card-feedback, #chargeback-feedback, and so on, get created every now and then.

With this new approach, stakeholders now could subscribe to specific topics and understand a less-noisy subset of users’ needs.

Phase 3: Spreadsheets

The most significant change came when we decided it was time to abandon screenshots to adopt a more structured online submission form. We created a simple Google Forms and applied it to every feedback channel as a centralized submission system.

In the beginning, we worried that customer support engagement was going to drop because of the extra friction. At the same time, though, the value we’d be able to derive from structured, parsable data was definitely worth the risk.

We also made sure that when the form was submitted, a bot replicated the submission to Slack, mirroring our older model. This way, on-the-spot discussions and threads could still happen on Slack, while product teams would still be empowered with a structured database on Google Sheets.

This new workflow felt like a vast improvement, and we ran with it, unchanged, for a while. We could now categorize and rank feature requests, get in touch with specific users to understand their needs more deeply, and share more insightful reports with the company.

However, as we all know, where there’s a spreadsheet workflow, there’s an app. Very quickly, we felt that reading through a long list of requests on Google Sheets didn’t offer the best analysis experience, so we started shopping for third-party apps that could help us manage customer feedback.

Phase 4: Third-party platform

The idea first came when we landed on this video from MailChimp, where their research team explains how they built a global, searchable database of insights inside Evernote.

We tried building a similar workflow, but it didn’t feel quite right to our specific needs. We shopped around for a while until we eventually found and felt at home with productboard.

By directing our Google Form submissions to productboard, we can sort messages by product, sub-product, tags, and have in-context discussions. We can export contact information for all customers that requested feature X and get a researcher, designer, or analyst talking to them.

More importantly, a specialized tool gave us the right mindset to look at feature requests. They are very opinionated about not treating feature requests as an all-important ranking that tells us what to build next, but instead as one data point among many others that we should consider to make better decisions.

For us, this dashboard has become the starting point for many projects. It’s a place we go to get an initial pulse of customers’ needs, raise initial hypothesis for research studies, and spread awareness about low-priority but pressing issues in our products.

It has become an indispensable piece of the discovery puzzle, not merely a source of truth to roadmap decisions.

Next Steps

Our current solution gives us a much better sense of control over the information we collect, but the process is far from perfect. It is hard to estimate the cost-benefit of the enormous effort required to report, sort, and categorize all the messages we receive, and it will certainly not scale at the speed we need.

Here are some next steps we’ve been thinking about for future iterations.

Input Friction

An important step we could take involves finding ways for customers to self-report their requests without opening customer support tickets. Doing this will reduce the load created by feedback tickets that don’t require support, and give us a less-biased stream of insights, which today inevitably get filtered by the agent’s level of attention, squad, or even personal interest in the feature at hand.

Some companies solve this by creating a specific “feedback” entry point and flow on their apps, that is separate from “help” sections and knowledge bases. Others direct customers to an online form and ask them to fill it themselves.

Apps that have a separate entry-point for user feedback: Duolingo, Google Photos, Nike Training Club

Sorting

It is tough to keep up with the stream of feature requests if we have to read and sort them into categories by hand. Looking ahead, we need a model that allows customers to pre-sort their messages. This would require that we create a taxonomy that both customers can understand and that we can use internally to redirect messages to appropriate channels.

Looking even further, we might want to have machine-learning algorithms do the hard, repetitive work, the same way they currently do for routing support tickets.

Community-focused

Some companies we admire, such as Monzo, have adopted an “open roadmap” model where users can request, vote, and discuss features online. It is not clear to us yet if this model would work in our context, but it is a solution that scales well, and one that gives users an incredible sense of ownership and empowerment.

In conclusion, there are many ways to deal with customer requests, and it is yet another complex design problem we love to work on and talk about here at Nubank. It might be overwhelming to deal with this kind of information at this scale, but with the right mindset, it can become one of the cheapest, most valuable sources of data-points for better decision-making in our product roadmap.

We hope we have inspired you to look more carefully at the proactive feedback your customers are voluntarily submitting, and that you use this information wisely to build things they love.

We’d love to hear from the design and product community on this subject. Please leave your comments with any feedback, ideas, and your own experiences below 😉

Thanks to Erick Mazer Yamashita, Max Josino, Lucas Terra and Paula Rothman.