Highlight Reels

Problem

UserTesting’s customers have a need to find and share learnings gathered from research. UserTesting offered an incomplete feature, highlight reels, to help fulfill this need. Highlight reels enabled customers to stitch together clips of valuable insights, allowing them to then share those insights with their teams. The feature had declining usage due to technical challenges and poor user experience. My team was tasked with improving our customer’s engagement and satisfaction with this feature.

Solution

We had no shortage of external and internal feedback on ways to improve the feature. We relied on this feedback paired with customer surveys to understand the unmet needs of the highlight reel. This helped us form a job story and desired outcomes. Any solution would have to:

Minimize the time to find insights within clips
Minimize the time to build empathy among team members

Role

I led design for highlight reels from August 2017 to January 2018. I supported research by running unmoderated and moderated interviews, prototyping, and survey generation. The new highlight reel launched in early 2018.

Insight

⚡️ Instant Highlight Reels

Our customers prefer not to watch their entire videos. After all, they are using our product to get insights not to watch videos. What if we could find a way to fast track them to those insights?

Through some coding wizardry, we added a button to the product that instantly created a highlight reel based on a task. We quietly beta launched this to all our users, and in a month we had 10% of customers actively using the new feature. In six months, we saw a 350% increase using the feature. This was a great lesson in truly understanding the problem and using quick experiments to gain confidence.

🏷 Tags

Our product was always built by and for researchers. An example of this was an existing tagging feature that required customers to use [brackets] during note taking. Customers would then need to export the text to an Excel file to view these “tags”. There was a need to surface these tags in product. Tagging is not a new idea, heck it wasn’t even new to UserTesting. By staying consistent with common patterns and releasing value as it was ready, we were able to get tagging out quickly. This enabled customers to start using it providing us coverage for our next solutions: search and filters.

🔍 Search and Filters

This was the piece that brought everything together. This enabled customers to broadly search and narrowly filter a list of clips to find exactly what they were looking for. We also implemented auto-complete for tags to help assist customers when trying to remember a tag. With the improved clip content, they could search for tester names, notes, and the test they came from. If customers knew exactly where they wanted to look for clips, they could use the test filter. This helped them pinpoint the exact test or tests that contained their insights.

Design

Playlist experiment

Library experiment

Results

Enabled customers to tag clips and to search for tagged clips in the highlight reel experience

We implemented tags, search, and filter to aide our customers in identifying, sorting, and selecting clips in the highlight reel. In the three months after we released tags, our customers created 11,000 unique tags. We averaged about 500 searches per week.

Improving customer satisfaction with the highlight reel editor

We sent out surveys before, during, and after releasing the improvements above and we improved our satisfaction by 44%. Many customers commented on the enhanced clip design and dragging/dropping functionality as the reason for their positive feedback.

Increasing the number of highlight reels created

At the start of this initiative, we were seeing around 300 highlight reels created per month. Post-launch we saw that number skyrocket to 1,000, an increase of ~200%. Over fifty percent of the total highlight reels created were attributed to instant highlight reels. About seventy percent of customers who created a highlight reel came back to create a second one.

Quotes

“Love the tagging feature -- I now actually make [highlight] reels.”

Researcher @ Square

“Thanks for adding hashtags... makes it a lot easier to find items.”

Researcher @ Healthwise

Learning

Always be experimenting

During the duration of this project we had a hackathon week. The theme for our hackathon week was experimentation. We were empowered to choose a problem, write a hypothesis, track how we would validate or invalidate the experiment, and map out the appropriate next steps. This gave us a light weight framework to test out ideas that treated failure as just another step in the process. This environment gave the team the confidence and space to develop the instant highlight reel feature. We had all the necessary pieces right in front of us and using the experiment lens helped us see the bigger picture.

Understand the problem and the solution will come

Anthony Raad, a staff UX researcher was instrumental in the success of this project. He gave us a look inside the mind of our customers through filtered feedback and interviews. By the time we exited the understand phase we knew the key features that would improve the experience for a lot of our customers. Brian, our fearless PM, led the charge on defining and scoping this opportunity to ensure we were focusing on solving the right things. By the time it came to put pixels on the screen our solution so solid that it had customers asking, “when can I have this?” Working in this way made it incredibly easy for me to always map everything back to customer value.

Show your work

UserTesting is founded on the philosophy that getting feedback will make your product better. Feedback can come from anywhere and it was important for us to filter that feedback, especially internal feedback. We got a pretty strong signal from our early surveys and interviews that were were some unmet needs in the highlight reel. In a priority planning meeting a key stakeholder wasn’t so sure there was enough of a need here. The proposed solve was to do the bare minimum and move on to other pressing projects. These conversations are always challenging, when comparing a few top-level items, how does a company choose where to invest time and money?

As a team we felt this was an important initiative. We built our case around why we should make this investment and why we should make it now. We used the product as our customers do and presented clips of our interviews in the next planning meeting. Seeing the frustrated looks as our customers used the existing product and the hopefulness of their voice as they used our proposed vision was enough to convince that stakeholder that this was worth building.