Introduction
Videos are an advocacy tool frequently used by Mercy For Animals and other groups to educate the public about farmed animal cruelty and to move them toward vegan eating. But what makes a video successful?
How long should the video be? Should the narrator be male or female? Should there be text on the screen? What kind of music should it have? To answer some of these questions, we ran two studies with 13 versions of What Came Before, a video about farmed animal cruelty narrated by celebrity Steve-O. Each video version differed from the others with respect to one element so that we could test the effect of that element.
The Studies
In the first study, we recruited 383,000 participants through a Facebook advertising campaign. When users clicked on the ad, they were taken (at random) to a landing page for one of the 13 video versions. Each landing page also featured a request form for a free Vegetarian Starter Guide. “Vegetarian Starter Guide requests was the main variable used to determine which video element worked best. When an element prompted more Vegetarian Starter Guide requests, we considered it more effective at influencing individual dietary change.
The second study was a survey experiment whereby we randomly assigned 1,600 participants to watch one of the 13 video versions. After participants watched the video, we asked them whether they thought their meat consumption would change over the next two months and whether they wanted us to send them a free Vegetarian Starter Guide.
What did these studies find?
The studies suggest that when making videos about the cruelty inflicted on farmed animals, the following three guidelines may make your video more effective at inspiring people to eat fewer animal products:
- Use a celebrity narrator (even if the narrator doesn’t appear on screen)
- Use a narrator rather than just text or subtitles
- Talk about farmed animal intelligence.
And here are six things that did not seem to make a difference in our studies:
- Removing slaughterhouse and killing scenes
- Focusing cruelty footage on direct abuse (e.g., hitting, kicking)
- Including stories of individual animals
- The volume of the background music, or using eerie music instead of aggressive music
- The sex of the narrator
- Promoting veg eating at the end of the video rather than toward the beginning
The Analysis
We synthesized the results from both studies, but put significantly less weight on the survey experiment due to its smaller sample size, the unreliability of self-reported dietary measures, and the artificiality of the survey environment relative to the Facebook campaign study.
In fact, in our synthesis of the two studies, the survey experiment was weighted up to 250 times less reliable than the Facebook study. Due to this, our final conclusions were almost identical to those of the Facebook study alone. This blog only reports results from the Facebook study to simplify and better articulate the main conclusions.
To clarify the conclusions we present in this blog, when we refer to a result as “not statistically significant, we are saying that it does not provide strong enough evidence for us to come to a strong conclusion.
On a more technical note, when determining statistical significance, we use the p-value cutoff of 0.15 (85% level) because our intention is to make decisions on evidence that will likely make a video better, rather than to use the strict (and arbitrary) cutoff of 0.05 (95% level). In addition to reporting p-values in this blog, we report standardized effect sizes (g) and 95% confidence intervals.
There is a more detailed description of the analysis in the supporting documentation at the end of the page.
The Videos
What Came Before (WCB) is a video created by Nick Cooney and Bruce Friedrich during their time at Farm Sanctuary. The video features the celebrity Steve-O talking about the shocking realities of modern farms. WCB is 11 minutes long, but we used a roughly five-minute adaption. We created 12 new versions of the shortened video by altering various characteristics, such as the narrator’s sex, the music volume and type, and inclusion of footage from slaughterhouses. We wanted to determine how changing different video elements influenced viewers’ interest in changing their diets.
Results
Celebrity Narration
If you can, have a celebrity narrate your video. The video version with a celebrity narrator generated significantly more Vegetarian Starter Guide requests than the versions with a non-celebrity narrator (g = 0.03, p < 0.002, 95% CI [0.01, 0.05]).
It also doesn’t seem to make much difference whether the celebrity physically appears in the video. In fact, the video version in which the celebrity is off camera prompted slightly more Vegetarian Starter Guide requests than the version with the on-camera celebrity, but this difference was not statistically significant (g = 0.01, p = 0.18, 95% CI [-0.01, 0.03]). This is good news if you’re unable to film a celebrity. Having a celebrity’s voice alone in the video can be just as good, if not slightly better.
Voiceover vs Subtitles
Try to narrate a video with a voiceover—especially a celebrity voiceover—rather than just using subtitles. Viewers who saw the video with the celebrity voiceover were significantly more likely to order a Vegetarian Starter Guide than viewers who saw the version with subtitles and no celebrity voiceover (g = 0.03, p < 0.001, 95% CI [0.01, 0.05]).
Would using a non-celebrity voiceover still be better than using subtitles? Yes, but just slightly. When the subtitled video was compared to two videos narrated by non-celebrities, viewers who saw the narrated videos were slightly more likely to order a Vegetarian Starter Guide than viewers who saw the subtitled video, but the differences were not statistically significant (male voiceover: g = -0.009, p = 0.28, 95% CI [-0.03, 0.01], female voiceover: g = -0.01, p = 0.16, 95% CI [-0.03, 0.01]).
Our results indicate that using a celebrity voiceover is definitely better than using subtitles and that using a non-celebrity voiceover is only slightly better than using subtitles. So when you can, use a celebrity voiceover. A non-celebrity voiceover may yield slightly more effective videos than subtitles, but will not create as great a difference as a celebrity voiceover.
Farmed Animal Intelligence
The original shortened video details the intelligence of farmed animals, which we know is equal to that of cats and dogs. We created a video version without the sections on farmed animal intelligence to see whether it influenced viewers’ intentions to change their diets.
The video with sections about farmed animal intelligence prompted more Vegetarian Starter Guide requests than the version without sections about farmed animal intelligence (g = – 0.01, p = 0.14, 95% CI [-0.01, 0.03]). Therefore, we conclude that including a discussion of farmed animal intelligence could increase the persuasiveness of a video.
Cruelty Footage
Determining how much and which animal abuse footage to show viewers can be a challenge. The original shortened video contains a lot of cruelty content focusing almost equally on confinement (e.g., animals suffering in small cages) and direct abuse (e.g., hitting, kicking). We created three variations, each of which altered the cruelty footage in a different way.
In the first variation, we focused the cruelty footage more on direct abuse. This version generated slightly more Vegetarian Starter Guide requests, but this difference was not statistically significant (g = 0.008, p = 0.35, 95% CI [-0.01, 0.03]).
The second variation consisted entirely of cruelty footage until the last 30 seconds with little to no positive content. Viewers who saw this version were also slightly more likely to order a Vegetarian Starter Guide, but again, this difference was not statistically significant (g = 0.01, p = 0.21, 95% CI [-0.01, 0.03]).
In the final variation, we removed the slaughterhouse and killing scenes so that all of the cruelty footage focused on confinement, but doing so did not make a difference (g = 0.0003, p = 0.97, 95% CI [-0.0157, 0.0163]).
These three cruelty footage alterations did not seem to impact Vegetarian Starter Guide requests, which is good news if you’re unsure of which type of cruelty footage to focus on. If we had to guess, focusing more on direct abuse and removing positive content may increase the persuasiveness of a video, but since the differences were not statistically significant, we can’t be confident.
Stories About Individual Animals
The original shortened video tells the stories of individual farmed animals who are subjected to a life of misery. In theory, stories about individual animals should help viewers connect emotionally with the animals rather than see them as just a number.
To test whether individual animal stories is an effective element, we created a variation of the original shortened video that did not include the individual animal stories. The variation without individual animal stories yielded slightly more Vegetarian Starter Guide requests, but this difference was not statistically significant (g = 0.01, p = 0.22, 95% CI [-0.01, 0.03]).
Background Music
There are at least two things to consider when selecting the background music of a video: the type of music and the volume. We tested both of these elements.
Lowering the volume of the background music didn’t alter Vegetarian Starter Guide requests (g = 0.0005, p = 0.96, 95% CI [-0.0155, 0.0165]). Similarly, changing the music from upbeat and fast-paced to mellow and eerie did not make a difference (g = -0.002, p = 0.78, 95% CI [-0.018, 0.014]).
As long as the music isn’t blaring, undetectable, or completely mismatched with the context of the video, it probably doesn’t impact the success of the video at inspiring dietary change.
Sex of Narrator
The sex of the narrator does not seem to matter. The video versions narrated by a non-celebrity male and a non-celebrity female generated a similar number of Vegetarian Starter Guide requests (g = 0.003, p = 0.74, 95% CI [-0.013, 0.019]).
Promotion of Veg Eating
Early in the original shortened video, there is a break to promote veg eating. To see how this impacted viewers’ intentions to change diets, we created a version that promoted veg eating only at the end.
The video version without the early promotion of veg eating generated slightly more Vegetarian Starter Guide requests, but this difference was not statistically significant (g = 0.007, p = 0.42, 95% CI [-0.009, 0.023]).
Future Research
The research presented here represents initial steps toward learning which elements make a video more effective at inspiring people to eat fewer animal products.
However, more research is needed. We used Vegetarian Starter Guide requests and self-reported intention to change diet to determine which video elements worked best, but as noted earlier, self-reported measures can be unreliable. Unfortunately, measuring actual dietary change, which is ultimately of greatest concern, is very difficult. For that reason, we used intention to change behavior as a proxy for real behavioral change.
The data collected by these two studies provide a starting point upon which future research can build and improve.
Supporting Documents
- Video descriptions
- The survey
- Data from survey experiment
- Data from Facebook campaign study
- Tables from analysis
- Description of analysis
…
If you want to hear about our research blog posts as they’re published, see here for instructions on how to do so: How to Subscribe to the MFA Research Blog.
To read more of our research blogs, click here: MFA Research Blog.