I ran an experiment… Now What?

Once launched, the team could use the new Hightouch Performance feature seen below to measure relevant KPIs for each group to determine a champion experience.

To no one's surprise, users in Treatment 2 outperformed the other treatments and ultimately drove more revenue at the end of the day for the business.

With this audience performance view, the team has confidence that regardless of what may have shown in their email platform from direct conversions (users clicking emails), the impact was measured as a whole - even counting users who may have converted organically at a later point in time. Additionally, with these insights directly in the same platform where all their audiences and experiments are being managed, the team can move more efficiently to build, measure and continuously optimize future campaigns.

How Audience Measurement Enables Growth and Rapid Experimentation

Almost every growth team today understands experimentation is the best way to learn quickly and unlock sustainable growth. So what differentiates merely good teams from great teams? It's often the quality and speed of their experiments. Any team can run a single-channel experiment and measure it relatively quickly in downstream destinations like Facebook or Iterable, but a great team will use audience or cohort-based analysis to explore much more complicated and meaningful experiments.

Let's say the same e-commerce company discussed above wanted to take their experimentation to the next level and test new marketing channels. They will again create three splits within their cart abandonment audience for this experiment. This time, however, rather than sending all of them to only their email platform, each group will be sent to a different combination of channels.

1. The Control group will act as a holdout test and receive no additional remarketing
2. Treatment 1 will receive a personalized email similar to the one from the earlier test
3. Treatment 2 will also receive the personalized email, however, they will now additionally be targeted with paid advertising campaigns via Google Ads

Once the campaigns have launched, with audience measurement available directly in Hightouch, the team can quickly and efficiently measure the impact of the various treatments on each split. Diving into custom metrics defined in Hightouch, like LTV, the team quickly identifies how adding one or many channels impacts overall retargeting performance - and if doing so positively impacts their ROAS.

With Hightouch, our team can effortlessly experiment on and measure how certain cohorts of users are performing. This empowers us with valuable insights to understand the impacts of different campaign tactics across our growth channels

Kishan Antala

Product Manager

•

Morning Brew

A caveat: of course, we're aware audience measurement has limitations as well. There may be other factors that affect a given audience that we don't totally understand and can't see with this approach to performance measurement. It can also be challenging to weigh different factors' relative contributions and effects in a multi-channel experiment. The truth is that all attribution methods suffer from this issue, while the benefit of audience measurement is how comprehensive it is, how fast it is, and its ability to correlate real business metrics (like order value) to your campaigns. Although imperfect, we know it's an essential and under-utilized tool for marketing teams.

Wrapping Up

Modern marketing journeys are complex - they incorporate many channels with even more touchpoints, and they're not getting any simpler. As organizations continue to invest in launching new channels and more coordinated omnichannel experiments, they will need multiple measurement methods to drive insights that help them optimize campaigns. By investing in building out analytical features like Performance, we are taking another step on our journey towards building a Data Activation platform that enables every member of an organization to explore, build, manage, activate, and now measure customer data at scale - all with the data residing in the cloud data warehouse.

Interested in getting started? The Performance feature, currently live for beta customers, will be available for all advanced Customer Studio customers next week at no additional cost. If you're interested in learning more about this feature or understanding how it can help your team with experimentation or campaign evaluation, book a demo here.

Onwards,

Tejas