What HBR.org has learned about audience engagement

Maureen Hoch

Maureen Hoch, the editor of HBR.org, writes about what it has learned about experimenting with audience engagement.

Hoch writes, “A central tenet of HBR’s experimentation and innovation practice is to write down our assumptions, goals, and hypotheses. This document is shared between product, editorial, and other teams involved in the project. We then check back on those assumptions at planned intervals to understand what we should do differently, what we’ve learned, and to help gauge whether the experiment should continue.

“We’ve done two bot projects, one on Slack and one on Facebook Messenger, that help illustrate how this works. When we started the HBR bot on Slack, we assumed that people would love to see our big beautiful images in their conversations with our bot. However, our UX research proved that was wrong. People just wanted to get to work advice we were offering.

“We also hypothesized that we wouldn’t drive any trackable subscriptions through the bot. That was right. But we also believed in driving frequent interactions through the bot should lead them to engage more deeply with us, even if we can’t draw a straight line back to their bot experience. That engagement could take a lot of shapes: Signing up for a newsletter, following a social channel, sharing an article with a friend.

“It helps to write down even your most basic assumptions. For our recent Women at Work pop-up podcast, we documented the assumption that people wanted to engage with HBR about this topic in an audio experience. For the bots, we had to test our assumptions that people wanted to consume our content in bot form (and we’re still testing whether that’s true). It seems basic, but doing these things helps your build up a store of observations that can inform your future experiment plans and pitches.”