Swayable @ ARF: 3 Key Takeaways About Grounding AI in Human Truth

Generative AI has emerged as a valuable tool for marketing experimentation. Teams are using machine-learning technology for everything from content generation to copywriting to analyzing market data. But how can brands determine whether AI is helping or hindering their efforts to accelerate creative development and drive lift?

AI certainly can generate value for brands, but the most effective way for marketers to confirm this is by rapidly pre-testing their ad concepts. It’s a strategy that Swayable CEO and founder James Slezak explored at the Advertising Research Foundation’s (ARF) annual AudienceXScience conference in New York. 

Slezak offered a deep dive into what startup alcohol brand Outlaw Sake learned from using AI to generate its first creative assets; pre-test those assets using Swayable’s randomized control trial (RCT) methodology; and accelerate the overall brand development process. 

Keep scrolling for a snapshot of key takeaways from our session.

1. AI predictions must be continually grounded in real consumer measurements

Swayable partners with companies to deploy rapid RCT pre-tests, serving as the gold standard for generating real-time evidence of lift. Our partnership approach is rooted in helping brands conduct sprint processes that replace traditional lift measurement studies—which can take months to complete—helping them build more agile and proactive marketing programs. 

AI has emerged as a new element that speeds up this process even more, allowing brands to quickly create and test ad concepts. Using our platform, teams can get insights back within 24 hours, with help from our AI-powered population modeling, rapid insights extraction, and analysis automation of consumer responses. 

“AI is critical to the pre-testing process because the faster you can get content out at a testable stage, the more quickly you can learn something,” said Slezak. 

However, in order for AI-powered experiments to accurately predict consumer sentiment, the research must be consistent and based on real-time measurements. Slezak noted that marketers should avoid the trap of using out-of-distribution inferences, which may result in models making unreliable predictions that lead to incorrect conclusions. 

“Consumer sentiment, and what your competitors are doing, can change from one week to the next,” said Slezak. “It’s why grounding models in real-time, active consumer measurements is necessary for success.”

2. AI can dramatically increase the efficiency of rapid prototyping 

AI has the potential to dramatically speed up a brand’s development and testing strategy. In the case of Outlaw Sake, the brand wanted to disrupt the hard seltzer market and reach a broad consumer base by positioning itself as the antithesis of traditional sake. 

To start, Outlaw Sake developed an identity designed to appeal to all Americans, using the tagline: “Who’s the genius that decided sake is only for sushi?” The brand followed up by using solely AI to create multiple ad treatments centered on humorous and irreverent concepts. Some creative leaned into the fact that the ads were completely AI-generated, while other ads doubled down on humor without explicitly stating the use of AI. 

Outlaw Sake partnered with Swayable to pre-test six video treatments (including the two shown above) and determine whether the ads could positively sway its target audience of sake and hard seltzer drinkers. While the creative garnered positive and negative lift metrics around favorability and consideration, a primary takeaway was how fast the marketing team was able to generate results from the time it launched the brand. 

The entire go-to-market timeline—from creating the brief to producing the ads to pre-testing and revisions—took four weeks as opposed to a typical timeframe of four months. This is largely due to AI being a prominent part of the process. For example, it took only three days for Outlaw Sake to create the treatments versus the three weeks it could take to manually schedule and shoot the concepts. And it took only 24 hours to pre-test them on Swayable’s platform, versus waiting an entire month to complete a traditional brand lift study. 

"By using AI to create the ads and pre-test them with Swayable, Outlaw Sake was able to dramatically speed up the ability to get the concepts in front of consumers,” said Slezak. “This approach helps the brand development process become tighter, faster, cheaper, and easier.”

3. AI-powered pre-testing generates key insights around lift, authenticity, and engagement 

By deploying rapid pre-testing through Swayable, Outlaw Sake quickly uncovered some illuminating findings about how consumers would react to its AI-generated content. 

First, and foremost, the three ads that generated the most impact in lift were the ones that leaned into funny and unserious concepts encapsulating the brand’s identity: “Sa-keh” featured a man bluntly and cheekily explaining the difference between Outlaw’s product and traditional sake; “I’m an AI” was transparent about the fact that the ad was entirely AI-generated; and “Alligator” promoted the product through a talking alligator.

Not only did those three ads generate the most lift; sake drinker respondents deemed them the most authentic, demonstrating a strong correlation between authenticity and lift. Swayable’s qualitative analysis of comments also supported the idea that the more Outlaw Sake leaned into humor or self-awareness around AI, the more likely respondents were to perceive the brand as authentic—and the less likely they were to call out AI usage in a negative light (or comment about the technology at all).  

Finally, the pre-testing results supported a classic Swayable observation: engagement metrics and upper-funnel lift metrics are almost never connected. Outlaw Sake’s ads that received higher engagement on Instagram and TikTok didn’t have the highest increases in lift. This finding drives home the idea that content generating the most views on social media isn’t necessarily the most persuasive. 

Overall, Outlaw Sake’s use case spotlights a major opportunity for marketers seeking to accelerate their content production and testing timelines. By using AI strategically and intentionally, teams can discover what works and what doesn’t faster—laying the groundwork for more agile marketing programs. 

For a complete rundown of Swayable's insights from ARF, check out Slezak's full session below:

Interested in learning about how AI-powered pre-testing can speed up your campaign development and market research? Book a demo with Swayable today.