What is a randomized controlled trial, and why are they important in market research? Swayable CEO James Slezak joined Advertising Week CEO Lance Pillersdorf to discuss why randomized controlled trials (RCT) and pre-testing are becoming the standard for defining creative effectiveness and how prominent brand marketers and storytellers use them to achieve success.
Here are six insights he shared about randomized controlled trials.
Randomized controlled trials are the only way to prove real creative lift
In marketing and advertising, proving that your creative actually changes people's minds is the ultimate goal. You can have the catchiest copy, the most eye-catching visuals, or the wittiest tagline - but at the end of the day, if it doesn't measurably shift consumer attitudes and behaviors, did it really work?
This is where randomized controlled trials (RCTs) come in. RCTs allow you to isolate the specific impact of your marketing creative by exposing one randomized group to the content while holding out a control group.
By comparing the two groups, you can definitively measure the "lift" in key metrics like brand favorability, purchase intent, or behavior change. They are the gold standard used in scientific experiments, medical trials, and marketing.
Read More: Swayable's Guide To Digital Survey Methodologies
“We can prove the causal impact of your message on consumer opinion,’ says James. “In other words, we can measure persuasion."
RCTs give you that elusive proof of causality - the ability to say, "This specific creative element directly drove these quantifiable results."
Causal data lets you maximize the impact of creative
With causal data from RCTs, brands can finally pinpoint and double down on the creative concepts that truly resonate—no more guesswork or assumptions.
"You’ve got an unfair advantage when you know which creative works well and which doesn't move the needle at all," explains James. "You can spend a lot of money on the good stuff and make a lot more of it. When you don't know, you're just throwing darts randomly at distribution and wasting a lot of resources."
Smart creative teams are leveraging this data to explore a broader range of creative possibilities from the start. Instead of defaulting to "safe" concepts, they can objectively test even edgier or riskier directions and let the data decide what gets produced at mass scale.
Thomas' English Muffins used the Swayable platform to discover that having causal data helped them fine-tune the impact of their messaging strategy.. Their instinct was to pursue a more "edgy, youthful" brand voice to attract Gen Z. But RCT pre-testing proved their traditional, authentic messaging was over 2x more effective - saving them from a potential brand misfire.
You can now run powerful RCT experiments in just 24 hours
Traditionally, any kind of quantitative creative pre-testing involved glacial timelines. Between scheduling surveys, waiting for responses to trickle in, number-crunching, and reporting, brands were looking at months just to get directional feedback.
New technology has completely changed those slow processes. Highly powered RCTs can now be executed within 24 hours through automated cloud capabilities.
“You can automate it and do a bunch of fancy cloud computing and some sophisticated population modeling,” explains James. “You can start to see very precise effects across target segments that matter to the campaign, and you can do it in 24 hours instead of waiting for a month."
Engagement metrics can't measure persuasion
Many marketers still use vanity metrics like clicks, views, and likes as benchmarks for their campaigns' success. But as almost every modern brand has realized, just because someone watched or engaged with your content doesn't mean it actually persuaded them.
"Engagement cannot predict persuasion,” says James “That's not to say that engagement isn't important, it's just a different thing."
Executive teams invest in marketing to build demand, favorability, and purchase intent - which are all shifts happening inside the consumer's mind.
James says, "The goal is persuasion, not just clicking something." Relying solely on engagement metrics essentially leaves brands flying blind as to the true impact of their work. That's why more people are adopting persuasion data from RCTs as their new north star.
Read More: Engagement Metrics are Messing with You.
Pre-testing creative delivers the biggest increases in campaign ROI
Optimizing creative drives major increases in ROI because creative is the #1 driver of success.
While brands have gotten savvier about media optimization and audience targeting the cold, the hard truth is that any campaign's overriding success factor is the quality of the core creative idea itself.
“What makes even more of a difference is the really big choices. Is this campaign about cameras or payments? Is it funny or serious? Do we try to center a celebrity, an influencer, or a customer? What's the underlying message? How do we convey the brand voice?” James explains. “This is all the stuff that goes into the creative briefs, that's often way upstream of the actual ads themselves."
The data is clear - properly investing just 10-15% of your total campaign budget into RCT pre-testing to optimize the creative can 3-5X your overall ROI.
It does this by ensuring you're putting energy and investment behind ideas that will work.
AI-powered marketing research is the next frontier
This is just the beginning. Technological advances in generative AI ideation will continue to make it faster and easier to run experiments and discover causal data.
Since the persuasion measurement is rapidly evolving, James and the team at Swayable are constantly creating new cutting-edge capabilities for creative teams.
One recent launch is "Tracker," a tool that shows brands their current public opinion across key audience segments and overlays RCT data to recommend specific creative directions and messages for improving those perception metrics. It's like a real-time "opinion monitor" combined with an AI-driven consultation on how to move those needles.
Modern brands embracing these new technologies and data products are gaining decisive competitive advantages in the creative arena.
To learn more about how Swayable's randomized controlled trial platform and products can maximize your creative team's performance, book a demo.