The outcome of the 2024 U.S. election was eye opening for the progressive movement, demonstrating a need to refine campaign messaging to successfully mobilize voters. Voter Formation Project (VFP), an organization that works to increase voting in local, state, and national elections, has used last year’s results as motivation to craft more impactful messaging to empower and persuade target voting bases.
Voter Formation Project tested multiple rounds of messaging before landing on the theme of Black wealth for the final creative assets.
Specifically, VFP sought to uncover which messages worked best to keep Black voters in Mississippi politically engaged, motivated to vote, and hopeful about upcoming election cycles. To accomplish this, the organization partnered with a coalition of Mississippi organizations to research and gather findings about the state’s Black voter ideology and motivators. They applied the insights to new creative messaging, and used Swayable to deploy iterative rounds of rapid, randomized controlled trial (RCT) pre-tests to identify the most persuasive messages.
"We were excited to partner with local nonprofit groups on this message testing program in order to support the important work happening on the ground to reach Black communities with civic engagement content," said Ebonie Land, Creative Director at VFP. "The goals of VFP’s programs uniquely aligned with that of the Mississippi partners, so we were able to quickly and efficiently run message tests that would get to the core of what each organization was looking for: how do we continue to connect with underrepresented communities of color to keep them engaged in the process of civic engagement and voting at a time when politics are dividing people more than ever? What new or different types of messaging can we employ to motivate people?"
Keep scrolling for a deep dive into how VFP approached the creative testing process; what the group learned from conducting five rounds of message tests; and how other advocacy groups can apply these insights to future mobilization campaigns.
The opportunity: researching voter issues and ideology to inform new messaging
To kick off the message testing program, VFP conducted research on Black Mississippians to learn more about their political and religious ideologies, and key issues impacting the demographic. Some of the key evergreen research findings included:
- Voter ideology is complex: Pew Research Center reports that while 76% of Black Mississippians identify as Democrat, only 27% identify as ideologically liberal, with 37% conservative and 28% moderate. Additionally, 85% believe in God with certainty and say religion is very important in their lives.
- There are clear issues that resonate with voters: During the 2023 election cycle, door-to-door canvassers with One Voice conducted a survey asking 13,500 Mississippi voters about their most pressing issues: 42% said healthcare, 24% said jobs, 17% said education, 11% said policing, and 6% said infrastructure.
- Group-identity messaging boosts engagement: VFP research also shows that connecting issues to racial identity drives political engagement. For example, Mississippi’s Black maternal mortality rate is three times higher than for white women, linking public health outcomes directly to racial disparities and unjust policy.
"Black people are not a monolith and people in the South have different experiences than Black people in other parts of the country," said Land. "Before beginning our messaging development, our team pulled research from a variety of sources that would help us to build a comprehensive audience profile of the Black experience in Southern states such as Alabama, Louisiana, and Mississippi. We used this information to develop audience profiles that helped us craft messaging that we believed would emotionally resonate with this audience and best set us up for success in our testing."
With the research insights in mind, VFP and its partners crafted messages to test based on the following categories:
- Issue-based messages, focused on healthcare, reproductive rights, jobs and the economy, education, and crime and public safety
- Group-identity messages, focused on topics such as faith and education
- Empowerment messages highlighting Black voter turnout and the power of Black people’s voices and votes
- A combination of issue-based and empowerment messages
The approach: conducting multiple rounds of mobilization message testing
VFP and its partners developed multiple assets to test and iterate on, with the goal of measuring impact on the following lift metric categories: pride, hope, representation, collective fate, engaging benefits, and mobilization. The tests also measured direct reactions to whether the content was relevant, trustworthy, credible, aligned with respondents’ values, and shareable.
For the first four rounds of testing, VFP deployed simple treatments that consisted of text and audio mockups. The objective for each round was to analyze responses to these treatments to help validate the most persuasive themes to incorporate into the final creative.
The first round focused on Black-centered/empowerment (shown below), Black-centered, non-faith based, faith-based, and fact-based messages.
Black-centered/empowerment, non-faith based, and faith-based messages saw the highest increases in lift for “pride and unity: country” category, with 5.8, 4.7, and 4.1 point increases, respectively. Within the “mobilization: talk to friends and family” category, all of the messages generated positive lift, with Black-centered/empowerment, Black-centered, and faith-based seeing the highest increases, at 7.4, 7.3, and 6.4, respectively.
Based on these initial findings, the VFP team deduced that three primary messaging themes—empowerment, faith, and Black-centered messages—moved the needle the most in terms of getting Mississippi voters to feel pride about their community and willing to mobilize in some form. With this foundation, VFP experimented with different messaging tied to these categories, honing in on themes of Black wealth, hard work, and the formerly incarcerated.
"Given what we know about the current political climate and the disinformation that is rampant within online spaces, we weren’t surprised that the 'fact-based' messaging didn’t see any movement in any direction," said Land. "We did find it interesting that adding a simple empowerment message to our 'Black-centered' messages did drive different results and movement than when it wasn’t present, even though the messages were otherwise identical. We used these results to refine the way in which we demonstrated 'empowerment' to ensure it was extremely clear and concise across the board, so that we could be sure we were getting results based on the empowerment piece of the message vs. the Black-centered portion of the message."
By the fourth round of testing, VFP prioritized three concepts, all of which called out the fact that thousands of formerly incarcerated Black people are being denied their right to vote. The Black-centered/empowerment treatment (shown below) drove the most movement in the “pride and unity: community” category, with a 5.5 point increase in lift.
Interestingly, the empowerment treatment (which incorporated a “we have the power” theme), didn’t perform as well as the other two creatives in the “mobilization: learn more” categories, with faith-based messaging driving the most lift at a 4.4 point increase among registered voters.
These results suggested that incorporating calls to action—such as explicitly telling voters they have the power to make a change—could fall flat among Black voters, particularly in the wake of a U.S. presidential election where a majority of the community mobilized for a progressive outcome that didn’t happen.
The outcome: final creative spotlights the importance of Black wealth
For the final round of ad treatments, VFP shifted its messaging to focus on positive, Black-centered messaging that highlighted the dream of Black wealth, the realization that Black people deserve to thrive in the community equally, and the idea that the community deserves representatives who share the same values.
VFP tested five final treatments: an audio message; two videos narrated with the same audio message (one with stock footage, and one narrated by an influencer); and two GIF ads (shown below) that featured rotating images of the Black community paired with text mirroring the audio messages. Each of the latter ads featured different images and color schemes.
While the audio treatment generated positive movement in “pride and unity: country”, and the video treatments saw lift increases in the collective fate metric, the two GIF-based ads generated the most consistent increases in lift in the pride and unity, engaging benefits, and mobilization metric categories.
Here’s a snapshot of how the two graphics resonated with respondents:
- The “pride and unity: community” category: the first GIF ad (featuring the couple on the boat) drove a 5.9 point increase in lift, while the second drove a 3.7 point increase among all respondents. Among male respondents, the first ad saw the highest overall lift increase at 6.6 points.
- The “engaging benefits: can’t vote but create change” category: Among all respondents, the first graphic outperformed with a 5.7 point increase in lift. This increase was even higher among women respondents (+6.2) and among Gen Z, millennials, Gen X respondents (+5.9).
- The “mobilization: talk to friends and family” category: Both ads saw consistent boosts in lift across all segments. The second graphic in particular drove the highest increases, with a 6.9 point increase among all respondents and a 7 point increase among women.
- The qualitative comments: the majority of respondent feedback was positive when asked to share reactions to the ads. Comments praised the ads for depicting togetherness within the community, callouts to ancestors paving the way for a better future, and the importance of Black wealth.
A snapshot of qualitative reactions to the final creative.
The Mississippi partners plan to use these results and best practices in the accompanying creative in their programs to persuade Black voters in the South to remain active and energized ahead of upcoming elections. VFP credits the ability to continuously pre-test message concepts using Swayable—which helped the team rapidly extract evidence of positive or negative lift—as key to landing on the most effective Black-centered ads to take live.
"We need to move away from the idea that there’s one 'best practice' or one 'universal message' that will move large audiences," said Tatenda Musapatike, CEO and Founder at VFP. "People in different demographics and intersectionalities of demographics are inspired to take action based on different motivating factors. If financially available, it’s critically important to perform message testing with your audiences to build the most efficient program for your goals. We recommend throwing out the old playbook for traditional messaging—even down to including calls to action—and building modern messaging frameworks for agile and efficient campaigns."
Our final takeaways on VFP’s message testing campaign:
- Creative pre-testing can help advocacy groups quickly understand what messages positively sway target voter bases to mobilize—and what messages miss the mark.
- The results of VFP’s testing showed that Black voters in the South, in particular, are growing tired of messages that include explicit calls to action. Messages that acknowledge community history and issues tied to voter needs are likely more effective to persuade people to take action.
- Responses to Black-centered messages highlight the fact that mobilization can start with communication among family members and friends. Incorporating visuals and copy that highlighted intimate and personal moments directly correlated to voters being more likely to spread the message to people closest to them.
Looking to quickly test and confirm whether your campaign messaging is on the right track? Book a demo with Swayable today.
Methodology
These 3 monadic RCT survey experiments were conducted on Swayable over 124 hours, 125 hours, and 46 hours, respectively, between February 28th and April 30th, 2024, with responses from a total of 3,036 U.S. consumers across the 3 tests. Questions were asked of a general population sample using Swayable’s proprietary online platform. Responses were screened using industry-standard quality control measures, including checks for attention, speeding, location verification, and demographic consistency, along with reCAPTCHA verification. Duplicate respondents are removed based on IP address and device fingerprinting.
The sample frame is U.S. smartphone users in Mississippi, Alabama, and Louisiana with active internet connections who are users of popular mobile and web apps that make up Swayable’s network of respondent partners. Respondents are solicited from partner apps with non-monetary reward offers for their participation. Respondents were recruited with an approximately even ratio of men and women imposed via separate quotas for each. This is a “non-probability sample” (in the conventional terminology of public opinion research, although this team believes this concept is not meaningful since truly random sampling of the population is not possible via any methodology). To correct for over/under-sampling, all samples are post-stratified to the general U.S. population using cross-tabulations accounting for factors including age, ethnicity, gender, educational attainment, and geography, based on the latest available data from the U.S. Census Bureau and Swayable’s proprietary population modeling. Margins of error quoted are based on response distribution statistics and sample sizes and are calculated independently for each result. Access the survey questions here.
This research was conducted and self-funded by Swayable. Like all public opinion research, this study is subject to unmeasured sources of error that should be considered when interpreting the findings.