Getting Ruangguru users to find the product that fits them

This feature was released by Ruangguru. Some information is missing because the underlying data is confidential, and confidential material has been redacted from this write-up.

Context

Ruangguru is one of the largest edtech companies in Indonesia. We served a very wide range of learners, from elementary students through to high-school students preparing for university entry tests, with a matching range of products: pre-recorded video and quiz, daily learning, homework help, and exam prep.

Only one of those products was actually being used.

Role and team

I was the interaction designer on this project, and I facilitated and moderated the ideation workshops. On the team were the VP of Product, one UI designer, a business ops manager and the academics team. Because we split ideation into one session per grade, I ran the elementary-school session myself.

My work sat at the front of the project: framing the problem, running the workshop, synthesising it, designing the concept that went into test, and building the research report with the product manager, product owner and UI designer. The UI designer owned visual execution of the final design.

The problem

From data analysis we found that most of our users only used one product — pre-recorded video and quiz. Adoption of everything else on the homepage was low by comparison.

We went deeper with evaluative research and user-journey research, and found two things. Users were reluctant to explore the homepage at all, and when they did, they struggled to work out which product fitted their specific use case. Most of them did not know Ruangguru offered anything beyond pre-recorded video and quiz for their daily learning.

So the products were not being rejected. They were invisible.

Constraints

• One solution for every grade. We identified several how-might-we questions, but they spanned elementary through high school, and each grade band has genuinely different use cases. We assessed effort against impact with the product team and decided to address one problem that could apply across all grades, because that was more sensible technically and commercially than solving each grade separately.
• Research had to be combined, not sequenced. A time constraint made separate concept testing and usability testing impossible, so we combined them into one study.
• Confidentiality. The internal research plan and report contain sensitive data and are redacted here.

What I did

We started from a set of how-might-we questions and picked the one to pursue on impact: help dormant users find a product or feature that fits their use case and reach their aha moment. That framing was chosen deliberately over the alternatives for new users and early adopters, because it applied across grades.

I ran the ideation as a workshop on Miro, since all the stakeholders already knew the tool. I set the rundown and the format: five minutes for people to arrive and explore the board, ten minutes of context on the problem and how the session would run, then a vote on which of the broken-down sub-problems we would solve, fifteen minutes of lightning demos and quick benchmarks for inspiration, then a round where participants posted their own ideas and explained them briefly, then a vote for the most interesting ones. We involved every related department — business ops, marketing, content, sales, product and design — because the problem was about product visibility, not about interface design.

Because one grade is not a pattern, we ran the session again. I synthesised the output with affinity mapping to find the patterns, then compared grades to see what differed between them. That produced a set of proposed ideas and a proposed flow for the elementary grade.

From those ideas we narrowed to three concepts and tested them with 12 respondents across all grades — both active users, to see whether it changed their learning behaviour, and dormant users, to see whether the solution helped them find the features that suited them. We strengthened the qualitative findings with quantitative data from our own analytics before writing the report.

The final design was a homepage organised by learning mode rather than by product, so a student could start from what they were trying to do.

Key decisions and trade-offs

We did not build the personalised recommendation engine first. Personalised recommendation was the higher-commitment solution and the one we believed in long term. Instead we sequenced learning modes first and tested them, then decided whether to continue to the rest of the phases. Committing to the whole vision before validating any of it would have spent a quarter on an unproven assumption.

We did not solve for every user segment. New users, early adopters and dormant users all had a version of the same problem. Choosing dormant users meant leaving two segments partly unaddressed at launch in exchange for a solution that worked across grades.

We did not run concept testing and usability testing separately. Combining them saved time and produced a single study, at the cost of some depth in each. It was the right call under the constraint, and I would not repeat it without one.

We did not treat the workshop as a design activity. Half the room was not from design. That was the point: the reason the products were invisible was a framing problem, and framing problems do not get solved inside the design team.

Outcome

After stakeholder review and approval we ran an A/B test rather than committing to the full phased roadmap. Variant A was the existing homepage; variant B was the learning-mode homepage, at a 50:50 split, over roughly a month in July and August 2022.

Non-subscriber conversion to paid rose 2.9% in the learning-mode variant. Engagement came down by roughly 3% for subscribers and non-subscribers.

The learning-mode variant then shipped to 100% of users.

The honest read is that this was a net positive on conversion with a real engagement cost, not a clean win. Both numbers belong next to each other.

What I’d do differently

We designed the final flow for the elementary grade first, because that was the session I ran, and then generalised. That order was convenient for the workshop and slightly wrong for the product: the elementary flow is the least representative of the whole student base, and it shaped the concept that went into test more than it should have.

I would also have questioned the engagement drop harder before rollout. A 2.9% conversion lift next to a roughly 3% engagement decline is a trade. The right question was which users dropped engagement and what they stopped doing — if the answer had been the users closest to their aha moment, the phase plan would have needed to change, not just proceed.