Introductory Language Lesson

Link: Prototype Walk-Thru Video

Customer Goals

Start learning a language for the first time using a Rosetta Stone app.

My Role

UX designer and user researcher (solo)

Key Deliverables

  • Project plan
  • Research existing “first 15 minutes” experience and identify questions and pain points
  • Interactive prototypes (using Figma)
  • Iterative usability studies utilizing prototypes
  • Study reports and related discussions with stakeholders (esp. Product and Development)
  • Guidance and testing during sprint(s)
  • Collaboration with PM on A/B testing

Impact

  • Reduced user errors by over 20% vs. original
  • Increased user satisfaction by over 30% vs. original
  • Helped identify larger issues with the overall new user experience, extending beyond just the introductory lesson

Process

Many people using Rosetta Stone never complete the first full lesson. Before I joined, the Product team released an MVP introductory experience in the iOS phone app internally called “Lesson 0”. With the goal still being to increase the number of people completing the first lesson (original target was a 10% lift), the Product owner asked me to revise the introductory lesson in order to A/B test it. As typical, I had to work within some tight constraints – many aspects I knew from prior research were problematic,such as the prompt to select a voice type (the user’s voice type, not the voice you hear – see? confusing!), or the size of some images.

I proposed we start by looking at the experience new learners have within the first 15 minutes with the existing app, conducting a remote, asynchronous online study, identifying issues such as: confusion over some modals, feeling unsure they learned much about how to use the app, and also feeling they didn’t learn much that’s useful (though they did appreciate being able to say “hello”). Taking the findings into account, I sketched some quick ideas and discussed them with the PM.

I then created a robust, click-through prototype in Figma capturing all elements of the experience except animation and the random ordering of images, and proceeded to conduct a series of iterative user studies utilizing most of the same tasks I wrote for the original study of the existing experience, updating the prototype after each study. By the end of the process (just a couple weeks), we were feeling as though we’d resolved or minimized as many of the usability issues as we could within the scope of the project and timeframe, reducing errors over 20% and increasing satisfaction after the lesson by over 30%. I wrote up a report to share with the PM and the larger team, which, along with the designs and prototype, helped guide the implementation – though the PM decided to start with the Web instead of the mobile experience I’d been asked to design and test (for faster development and easier A/B testing).

I worked with the developer implementing Lesson 0 on the Web, walking them through the design and usability issues, answering their questions and considering their suggestions for tweaks, as well as reviewing their milestones. After implementation, the PM and I created an A/B study to test the effectiveness on the key metrics. Unfortunately, despite the new Lesson 0 being demonstrably better than the original version, there wasn’t the desired level of increase of completions of the first lesson. On the plus side, my work here helped identify and support larger issues the team had started grappling with (and for which I was conducting surveys and user interviews): there were larger navigation and information architecture issues faced by new users, as well as an overall feeling of lack of preparedness that the new Lesson 0 could only chip away at. So the project was a success within itself, but didn’t fully succeed due to larger usability issues, but also helped the team better understand those issues.

Lessons Learned/Constraints

  • My work was scoped by the PM to specifically improving the introductory lesson
  • It’s important to help solve immediate issues for the business, but it’s also important to never lose sight that a single feature usually doesn’t equate to the overall, holistic experience of the product; making one thing better may not have as much impact on the overall issue(s).

Additional Context

My work on Lesson 0, as well as strategic research outside of the specific feature work, helped identify and explore the overall new user experience – and areas that would be more impactful on key business metrics than reworking the introductory lesson. For example, through surveys and interviews, I helped the business understand that many learners felt anxious and unprepared, even as they started completing lessons. The team took a strategic “step back” to reorient and refocus.