Live TV & DVR Research


Customer Goal

I want to watch and record my favorite shows.

Role

Lead Researcher (solo)

Key Deliverables

  • Study plan (including goals, methods and tasks)
  • Implementation in remote research tool
  • Bugs/issues in github
  • Study report (including video clips)
  • Discussions with stakeholders

Impact

  • Increased focus on performance of the channel grid, leading to fewer customer complaints
  • Improved the flow of recording a show, leading to greater engagement (more recordings)

Process

On paper, this was as a fairly regular usability study, but what made it especially interesting to me was that it was the first time I tested a TV interface in a remote, asynchronous study. The study plan, and especially the instructions and tasks, had to accommodate the fact that the participants would 1) not be experienced at participating in user research, 2) would be using a remote control for their TV, and 3) have to, at the same time as using the remote control and talking aloud, record that experience with their mobile phone. The instructions and tasks were presented on their mobile device, which was recording their voice and would also, when the feature was enabled, record video for each task as needed.

After the sessions were uploaded to the vendor, I watched and annotated each, later assembling my notes into a written report accompanied by some video clips from various sessions. I filed bugs/issues in the tracking database (github) and then met with the stakeholders to review and discuss the findings.

Lessons Learned/Constraints

  • Instructions and tasks for a remote study in which the participants use their mobile phone to record their actions on a TV require writing shorter, simpler, and more discrete tasks, keeping in mind the “juggling” of phone, TV remote and possibly a laptop while talking out loud – had the study been with new users rather than existing users, that would’ve added another layer of complexity (likely one too many) as they’d have had to been learning the product as well
  • A task in this context can be a longer one, in which case it helped to prompt the participant to prop their mobile phone up where it could still record the TV but they wouldn’t then have divert attention to it until the task was completed
  • The testing software on the mobile phones obscure the screen by necessity in order to convey the text of the instructions and task – but this, then, makes it easy for participants, already managing a lot that’s taking their attention, to sometimes forget to dismiss those and focus the camera on the TV
  • Given the “homebrew” nature of the videos, the camera quality, lighting quality, and even the brightness of the backlight on their TV all interacted to sometimes make it very easy, or very hard, for me to see what they were doing when viewing the recorded sessions – what would’ve made it easy was a way to record on the TV  or streaming devices directly, but no such solution existed, especially during a remote study
  • Performance of a product or feature in someone’s own home is harder to control for (the number of things that can impact the study), but the data collected – if things DO work – can be rich

Additional Context

Watching live TV and recording a show requires hardware and comfort with technology. A user has to have purchased and connected a digital TV tuner and antenna, which, in turn, has to be connected to their home network, which needs to be robust enough to not choke on video streaming. Further, Plex requires a user have signed up for an account, and to use the DVR feature, also requires it to be a paid account. To record a show, a user must also have a Plex Server application running on a computer (or network-attached storage) with sufficient hard drive space. Despite these requirements there’s a definite segment of the “cord cutting” market (people who cancel their cable subscription) that generally have no problem making it work for them.