Skip to main content

An (un)healthy social dilemma: a normative messaging field experiment with flu vaccinations



Influenza seasons can be unpredictable and have the potential to rapidly affect populations, especially in crowded areas. Prior research suggests that normative messaging can be used to increase voluntary provision of public goods, such as the influenza vaccine. We extend the literature by examining the influence of normative messaging on the decision to get vaccinated against influenza.


We conduct a field experiment in conjunction with University Health Services, targeting undergraduate students living on campus. We use four posters, randomized by living area clusters to advertise flu vaccination clinics during the Fall. The wording on the posters is varied to emphasize the individual benefits of the vaccine, the social benefits of the vaccine or both benefits together. We collect survey data for those vaccinated at the vaccination clinics, and for those not vaccinated via an online survey.


We find that any normative message increases the percentage of students getting the flu vaccine compared with no message. In terms of the likelihood of getting the flu vaccine, emphasizing both the individual and social benefits of vaccination has the largest increase in the vaccination rate (19–20 percentage point increase). However, flu vaccinations did not reach the herd immunity threshold (70% of students vaccinated).


This study provides evidence that there is a pro-social component that is relevant in individual vaccination decisions which should be accounted for when designing vaccination campaigns. The results of this normative, pro-social messaging experiment could be extended to other at-risk communities where the number of background risks is much larger. This is especially relevant nowadays, as other seasonal vaccines are being rolled out and younger adults are the ones with the lowest uptake.


Yearly seasonal influenza (flu) epidemics can seriously affect populations through rapid spreading of the disease, particularly in crowded areas such as schools or college residences [1]. Flu seasons can be unpredictable, and vaccination is one of the ways to prevent infection and related illnesses, as well as to reduce the severity of the symptoms and the likelihood of hospitalization if one catches the flu. The Centers for Disease Control (CDC) recommends the flu vaccine for everyone six months or older. Vaccination coverage in the United States, however, is far below the public policy objectives, as more than half of the population is typically unvaccinated (target is 70%). With the aim of increasing vaccination coverage, we analyze whether a flu vaccination campaign is successful at increasing flu vaccinations depending on the message transmitted to different groups of individuals.

Vaccines protect the individual who gets immunized and helps create a barrier for those who cannot get vaccinated, building herd immunity.Footnote 1 Flu vaccines can therefore be considered a joint product, as vaccines have both individual and social benefits. However, when it comes to getting vaccinated, individuals have an incentive to free ride, avoiding the cost of the vaccine and indirectly benefiting from others getting vaccinated. Thus, herd immunity is a social dilemma, as vaccinations help the individual and others, but it is costly for someone to get the vaccine. One non-pecuniary mechanism which could influence vaccination decisions while highlighting the benefits of the vaccine is normative messaging [3, 4]. The present paper incorporates normative messages that appeal to these benefits (individual and social) into an established University campus-wide flu vaccination campaign. Normative messaging is done by incorporating social norms into the posters used to advertise the location and times of flu clinics on campus. We examine whether emphasizing each benefit of the vaccine (in terms of a reduction in risk) influences the decision to get vaccinated.

This paper contributes to the experimental vaccination literature by examining actual flu vaccination decisions as opposed to intention to vaccinate [5,6,7] We focus on individual demand-side decisions. This complements a significant body of experimental work related to the supply side of health markets and services, such as insurance markets [8] and physician behavior [9]. We gather data on both vaccinated and non-vaccinated individuals on campus to examine the influence of the campaign on student vaccination levels. We also contribute to the analysis of the influence of social norms on individual decision-making [10,11,12,13], as our experimental design allows us to expose large numbers of students to different normative messages. Understanding if reminders about benefits of the vaccine throughout the flu season can address part of the individual processes behind the decision to get vaccinated serve as an input for more effective and efficient interventions to tackle vaccine hesitancy [5, 6].

Our findings show that emphasizing both the individual and social benefits of vaccination has the largest increase in the vaccination rate, compared to emphasizing one benefit or when no benefit of the vaccine was highlighted (our baseline). This shows that there is synergy between both the self-interest and the pro-social or altruistic components of individual vaccination decisions. Our finding is also evidence that there is a pro-social, community component that is relevant in individual vaccination decisions which should be accounted for when designing vaccination campaigns by health services providers.


There are two main strands of literature relevant to our study: the influence of social norms on health-related behaviors and voluntary public goods provision in the case of vaccinations. We briefly discuss these literatures below.

Influencing health-related behaviors

Many public health interventions attempt to influence the way people consciously think about their health behaviors. Inducing beneficial habits, such as exercising or quitting to smoke, by adjusting perceptions of social norms could significantly reduce health care provision costs [14, 15].

Normative influence is one avenue to affect behavior. Normative influence can be defined as a form of influence that uses perceived behavioral patterns as well as approval and disapproval of social norms to modify behavior [13]. Norms can be used to induce both individual and collective change, particularly when nudges focus on social comparisons [16, 17]. Norms are expected to affect behavior when salient at the time of the decision: that is, when peoples’ attention can be drawn to the norms [18]. Norms are also most influential when they are common to social groups, especially for health-related behaviors [19,20,21]. Normative nudges should increase welfare: anyone who is already behaving according to the norm used for the nudge will not be affected and those who are not will make decisions that improve their welfare if the nudge is successful in changing behaviors [16, 22]. In addition, health-based nudges have been shown to have high levels of acceptance, more so when there is high trust in the authorities [23, 24].

In the case of health-related behaviors, normative influence has been previously shown have short-run effects. These effects include increases in physical activity [25], healthier food consumption [14, 26] and reductions in alcohol consumption [27]. However, the decision to exercise, eat healthy food or drink alcohol are daily decisions with short-lived effects. The flu vaccine, in comparison, is a yearly one-time decision with longer-term effects (protection from the virus). Getting a flu vaccine is therefore a promising avenue for a normative-based intervention.

Normative influence can occur through social interactions and visual cues, such as signs, posters or electronic communications. Visual cues are particularly useful when the target audience is large and harder to reach and where interactions are repeated, such as hotel guests or college students. In the past, descriptive and injunctive messages have been used to incentivize different types of behavior, such as healthy food choices (descriptive norms provide information about how to act in a certain situation while injunctive norms refer to conducts that the majority of individuals approve or disapprove of) [28]. The studies on food choices finds that healthy descriptive messages result in healthier food choices compared to no message, an unhealthy descriptive or an injunctive message. Similar results have been found by using descriptive norm manipulations to analyze the choice between taking the elevator and using the stairs in university buildings [25]. However, for alcohol consumption, both the perception of the individual’s drinking (injunctive norm) and average drinking of each individual’s reference group (descriptive) predict alcohol consumption [29]. The same has been shown for flu vaccinations, where both injunctive messaging (whether the people closest to the parents to them think that the child should be vaccinated) and descriptive messaging (what other parents do) have an impact on parents’ decision to vaccinate their children [30]. Similar effects have been shown for front-line staff: flu vaccination uptake is impacted by both injunctive and descriptive norms [31]. In other vaccination campaigns, such as for COVID-19 vaccinations, evidence of the impact of nudges is mixed [32,33,34]: emphasizing the benefits of the vaccine has a positive impact on actual vaccinations, while just targeting intention to vaccinate is not enough to ensure a vaccination decision.

Given that both types of normative nudges are successful in changing decisions in the health domain, we used messaging that was injunctive and prescriptive to emphasize the individual and social benefits of the flu vaccine, keeping the messages short and to the point. This decision was recommended by the University Health Services (UHS) at our target university.

While our study was conducted prior to COVID-19, the use of these messages is consistent with the current literature on COVID-19 vaccinations, where emphasizing the benefits of the vaccines increase vaccination rates, both for individual benefits [33], social benefits (combined with different vaccine reminders) [34] and a combination of both [35]. We focus on prescriptive messages that emphasize what the individual should do (protect themselves and their community through getting the flu vaccine) and what other people do [36]. Moreover, and consistent with our study on flu vaccines for young adults, vaccine hesitancy for COVID-19 has been shown to be lower for injunctive norm treatments (social approval of a behavior compared to neutral interventions, and the use of behavioral cues for increasing vaccine acceptance are recommended for policy-making purposes [37].

Our normative messages are part of an established flu campaign at the university-level that uses posters placed in university residence halls. More specifically, flu vaccinations have also been studied when encouraged from a point of authority, such as the firm the individual is working in or the university the student attends. Types of low-cost nudge interventions such as emails done through university campuses and university health centers have been previously used to tackle vaccinations, but the effects have not been significant [38]. Implementation intentions and defaults have been used to tackle flu vaccines among employees with better results [39, 40].

For the case of vaccines, normative interventions could be used where vaccination mandates cannot be put into place for ethical reasons [41]. Vaccine normative interventions can be leveraged by highlighting aspects of the social norm and informing the public about who gets vaccinated, why they get vaccinated, and the level of protection provided by vaccines. The SAGE group on immunizations has highlighted individual and social group influence as one of the main drivers of vaccine hesitancy [42]. Therefore, positive reinforcement of vaccination social norms through nudges that highlight the individual and community effects of the vaccine have been shown to be successful in influencing vaccination decisions [43].

Voluntary public goods provision: the case of vaccinations

From a standard economic perspective, an individual deciding to vaccinate would go through a cost-benefit analysis, where they get vaccinated if the perceived costs are lower than the perceived benefits of the vaccine [44]. An individual getting a flu vaccine does so with the aim of protecting themselves and others from the illness. In addition, an individual who gets vaccinated would generate a positive externality on other individuals, reducing the chances of contagion [5]. Vaccinations contribute to building herd immunity—protecting children, pregnant women, older adults or immunocompromised individuals and is an individual decision, and it can be interpreted as the contribution towards building herd immunity in a community [45]. Given that nobody (both vaccinated and unvaccinated) can be excluded from benefiting from herd immunity (non-excludable) and everyone receives the benefits (non-rival) from everyone’s decision to vaccinate themselves, herd immunity can be defined as a public good (For the theoretical model behind herd immunity as a public good see [45]).

Although some individuals might voluntarily seek to get a vaccination, there is an incentive to free ride and not get vaccinated [7, 11]. For example, several studies find that some parents believe that their children do not need a vaccine because other children are vaccinated [46, 47]. Other determinants of vaccine uptake found in the literature include perceptions of the social norms, beliefs about vaccine effectiveness and misperceptions of vaccine safety. Risk perception about the disease (the belief about potential harm) affects the vaccination decision, as individuals who believe they are at greater risk of getting the flu are more likely to get vaccinated [48, 49]. Perceived risk is associated with a range of anticipated emotions before getting the vaccine, such as fear, regret or worry [50, 51].

In addition, some misconceptions regarding the flu vaccine found in the literature include that it will: give people the flu, get people sick or not prevent the flu. These misconceptions are negatively associated with vaccination rates [44, 52]. Moreover, several studies focus on correcting misinformation and using norms to tackle vaccine hesitancy and observe a positive relationship between norms and vaccination decisions [53,54,55,56]. We thus incorporate individual risk attitudes and beliefs about the effectiveness and safety of the vaccine into our analysis.

Empirical questions

Our empirical questions are inspired by the model of impure altruism [57], a special case of the joint products model [58]. Receiving a flu vaccine has both private and public benefits, as it protects the individual and helps build herd immunity. Our outcome variable, the vaccination decision, is discrete [59]. As the vaccination against the flu is recommended, the decision of whether to vaccinate or not is left to the individual.

The focus of this analysis is on testing the influence that normative messaging has on the individual decision to get a flu vaccine [13, 28]. As the two main benefits of the vaccine are protecting the individual and the individual’s community, we target the individual’s self-interest and the individual’s “community-mindedness” in the construction of the normative messages. The concept of “community-mindedness” has been used in the past to refer to the idea of helping others through priming social cohesion and affinity [60].

Therefore, we have four potential interventions depending on which characteristic of the vaccination decision is emphasized: (i) neither component of the vaccination decision is emphasized (Baseline), (ii) only the individual’s self-interest is emphasized (Self), (iii) only the individual’s community-mindedness is emphasized (Others) and (iv) both messages are emphasized at the same time (Both). Our empirical questions are directly related to individual behavior when exposed to the messages. Our first two empirical questions focus on the overall influence of the messaging.

  • Empirical Question 1: Does messaging that highlights any benefits from the vaccine (individual, community or both) have a positive influence on the decision to get vaccinated?

If normative messaging of any type is successful at increasing the likelihood of getting vaccinated compared to no messaging, we next need to know how the type of message influences the likelihood of getting vaccinated.

The decision to vaccinate will be influenced by the individual’s pro-social preferences. But individual preferences could be influenced by the strength of the norm. In the margin, individual choices could be affected by both their tendency to conform to the norm as well as how much weight the individual places on the community (“others”) versus themselves in their utility function [45]. This leads to our second empirical question.

  • Empirical Question 2: Does messaging that highlights only the social benefits of the vaccine have a larger positive influence on the likelihood of vaccination compared to messaging that highlights only the individual benefits?

Messaging that emphasizes both the individual and community benefits of the vaccine together could have a stronger, weaker, or the same influence depending on the strength of both self-interest and pro-social preferences. This brings us to our third empirical question.

  • Empirical Question 3: Does messaging that simultaneously highlights both the individual and social benefits of the vaccine have a larger positive influence on the decision to get vaccinated compared to messages that compare one benefit?

We conduct a field experiment to answer these empirical questions.

Design and implementation

In this section, we discuss the assignment of participants and treatments, then describe the design of the experiment, the associated posters and the data collection process.


The experiment was conducted during the Fall 2016 semester. We focus on undergraduate students living on campus in six different residential areas at the University of Massachusetts Amherst. Over 11,100 students were included in the study (Table 1). Participants did not know that they were part of an experiment, they only saw posters promoting the vaccination campaign, which are the standard tool used by UHS to provide information about the campaign. This reduced any potential social desirability bias [2, 61]. Posters were placed to maximize visualization within a residential area but minimize cross-contamination across treatments.

Table 1 Intention to treat. Number of individuals per treatment


The study was conducted in conjunction with the UHS vaccination campaign. Four different informational posters were used, which differed in the normative message and the flu fact included. The posters are designed to meet the MacDonald, World Health Organization and SAGE working group recommendations to reduce vaccine hesitancy [42]. The recommendations are summarized in the 3C model and include confidence (trust in the effectiveness and safety of vaccines, the system that delivers them and the health professionals behind it), complacency (where the perceived risks of vaccine are low, thus vaccination is deemed not necessary) and convenience (including physical availability of vaccines, affordability and geographical accessibility). We follow confidence as the campaign has been consistently and successfully ran over the years with trusted professionals who have been the face of UHS as well as nurses and nursing students from the university itself. We also follow complacency, as the campaign and the messages target the need to protect oneself and the community, and convenience, as the posters highlight that most flu clinics happen through the semester in different places on campus and that vaccines are available and covered by insurance. The posters and the campaign itself is described in more detail below.

The poster campaign, which includes our nudging prompt is consistent with evidence that young adults prefer flu vaccination campaigns that rely on (1) quality and balanced information from (2) credible information sources, positioned in the (3) relevant health contexts, (4) emphasize actionable messages, and incorporate (5) persuasive campaign design [62]. Our experiment can be defined as a natural field experiment, as subjects are not informed that they are in an experiment (the campaign is part of University Health Services), the stimuli the students face their own context and environment in which they live and make decisions, such as their residential halls, study and work schedules as well as the University campus (the “real world”) [63].

The Baseline posters only included common information about the time and place of the flu vaccination clinics but did not include normative messages or flu facts, aiming to be a neutral poster.

The intervention posters include the information in the Baseline poster and add a normative message and a flu fact. The normative message was positioned prominently in the poster. The normative messages used in this experiment are injunctive and prescriptive. That is, they explicitly encourage an action to protect the subject and/or other individuals in the community from the flu [13].

The nudge-related prompt was designed to help modify risk perception about the flu and the flu vaccine, using short and concise messaging (but not focusing on clarifying vaccine misconceptions) [44, 50, 52]. The Self poster included the phrase “Protect yourself!”, the Others poster the phrase “Protect the UMass community!” and the Both poster the phrase “Protect yourself and the UMass community!”. Thus, the Both poster includes both Self and Others messages together. In the case of Others and Both posters, we present the social dilemma in a language that states what would be desirable in terms of social norms for cooperation [64].

In the design of the intervention, information on the vaccination clinics, always provided by UHS in the previous years, needed to be communicated. This limited the space available to run the three treatment arms. We used prominent placement and a font that was large relative to the administrative and clinical information to maximize the saliency of the treatment, given the space constraint. Note that similar constraints will likely exist for other vaccination or testing clinics due to the amount of information that UHS needs to communicate [65, 66].

Given the space limitations imposed upon us, our treatments could be considered relatively weak, providing potential lower bounds on the influence on the likelihood of getting vaccinated.

The flu fact is positioned below the normative message. The fact is informational and emphasizes the same information in all treatments (that a person can catch the flu from someone up to six feet away from them). It is worded to complement the normative message in terms of the protection (in terms of risk reduction) that the vaccine offers. An example of the poster for the Both treatment is in Fig. 1. The Baseline, Self and Others posters are shown in Additional file 1.

Fig. 1
figure 1

Poster for Both treatment

Whenever possible, a complete residential area was assigned a single treatment. This was possible for four of the six residential areas. One residential area, Southwest, due to the number of students it houses, was split into two using accessibility to two dining halls (Hampshire and Berkshire Dining, see Fig. 2, bottom left for Southwest residential area). The Central residential area was naturally split into two separate areas, downhill and uphill (see Fig. 2, center-right for Central residential area).

Fig. 2
figure 2

Distribution of treatments and flu clinics among residential halls on campus

The treatments were randomly assigned to each group of residence halls, with the aim of avoiding as much contamination between residence halls and areas as possible. We further confirmed with UHS that treatments and dates were not geographically crowded on specific weeks: that is, no two flu clinics geographically were tied to the same treatment in the same week and no two flu clinics were in the same geographical area of the campus in the same week. Further, we ensured that students residing on campus had access to a variety of locations to get vaccinated, including locations in the center of campus and UHS main clinic.

In addition, we confirmed with University Residential Services that the assignment of the majority students into residences and residential areas is random. This means that there should not be any unobserved characteristics varying by building. The location of each treatment among the residential areas on campus is specified in Fig. 2. For this reason, there are not equal numbers of students intent to treat per treatment (see Table 1).

Each treatment corresponded to a set of contiguous buildings, and each floor and entrance board had the same poster. Posters were placed in common areas of each residential hall floor and were located to maximize visibility by residents of the halls, as common areas are usually close to stairs elevators and kitchens (as per standard practice of all of UHS posters). The positioning of the posters is in line with norm saliency: the norm should be made salient as close in time as possible with the time of the decision [18]. Posters were placed in the residences the week before the flu clinics started and were checked weekly to make sure they were not (and should not be) taken down, as they are part of a University information campaign. Regarding potential contamination, entrance to residence halls by students who do not live there is subject to strict guidelines, with specific times of entrance as well as identification procedures. If contamination occurs and it is significant, it will bias the results against finding that the messages influenced vaccination decisions. Baseline posters and table tents were also used in common areas of the University, such as the campus center dining area, to avoid contamination between messages. A mass email to the whole University population (staff, students who live on and off-campus) was sent the day before the first clinic, with a reminder of the dates and times of the clinics and insurance information, similar to the Baseline information. Therefore, everyone got the same exposure to the information but differential treatment exposure depending on which poster they were assigned to.

Flu clinics and data collection

UHS held drop-in flu clinics during the Fall 2016 semester. All flu clinics were held by UHS and were located around campus during the semester, allowing students and staff to get vaccinated closer to where they live, work or go to class. Clinics spanned from September to December 2016. Most clinics occurred before Thanksgiving break and avoided holidays and long weekends (see Additional file 1 for the dates and type of flu clinic). The times and locations of the clinics were determined by UHS, this decision being exogenous to the experimenters. Clinics were held in different areas of the University, such as dining commons, residential areas and the UHS building. This allows the University to reach as many people as possible (see Fig. 2 for the location of each clinic around campus). During the Fall, there is no other flu vaccination campaign other than the UHS campaign being rolled out on campus. Flu clinics were located in accessible areas, such as lobbies and ground floor meeting rooms. Location signs were put up outside the buildings the day of the clinics to direct students.

Students receive the vaccine at no cost as flu vaccines are covered by most insurance plans.Footnote 2 The State of Massachusetts requires all college students to have comprehensive health insurance. For this University, students taking five or more credits are automatically enrolled in a Student Health Benefit Plan (SHBP) each semester and charged on the semester’s tuition bill or they can waive the SHBP against private health insurance. Vaccinations are billed towards the student’s health insurance and this information is already associated with their student ID in the University database, as it is a requirement.

Once students arrive to the flu clinic, they were greeted by one of the researchers or research assistants of the project and given a clipboard with the insurance forms and a survey. The survey was collected to gather individual data which we could not collect from UHS, given the confidentiality protocols at the University. The survey was voluntary for the students to fill while waiting to get vaccinated. We used a script with neutral language. The 5-minute survey was paper-based and included informed consent plus questions on health, socio-demographics, beliefs about the vaccine and its effectiveness as well as social preferences. The survey is independent from the flu vaccination campaign (the field experiment), and we clarified in the consent form that it was part of a study on healthy behaviors, not a flu vaccination study. We focused on collecting data from regular undergraduate students at the University.Footnote 3

In addition to the surveys at the flu clinics, we also surveyed students who decided not to get a flu vaccine during the semester with an online survey. All students in the residence halls were mailed invitations to complete the survey through the University’s internal mailing procedures by mid-October. We incentivized completing the surveys by raffling two $100 Amazon gift cards. Once the surveys and the flu clinics were completed for the semester, we discarded the online surveys of those students who went to the flu clinics and filled the online survey.


We begin by examining vaccination turnout by treatment. Next, we discuss the influence of each treatment on the likelihood of getting the flu vaccine. We focus on the Fall 2016 data, unless otherwise specified. Statistical tests of balance across treatments are presented in the Additional file 1 to cross check if the assumption of independence of potential outcomes across treatments holds. Overall, and consistent with the random assignment of students to residence halls, we find that for our vaccinated participants tests are balanced for our relevant variables across treatments. For our pooled sample, we find balance across most but not all variables: to account for this lack of balance, we detail below the covariates we introduce in our analysis. More detail is provided in the Additional file 1, including balance tests for our vaccinated + unvaccinated sample.

Vaccination turnout by treatment

The highest social benefit from vaccination occurs when herd immunity is reached. The vaccination rate in our sample is small, with 5.3% of the total intent to treat students getting vaccinated (590 undergraduate students of the 11 thousand intent to treat). The vaccination rate is significantly below what the CDC reports for the 18–24-year-old cohort, which is almost 30% [67], but it is slightly lower than other studies, reporting a rate of 7.2–7.9% [68]. For our target community, although individuals who receive the vaccine are protected against the flu, the vaccination level is low. This means that students who are at risk during the start and take up of the flu season were not protected.

In addition, it is unlikely that students are getting vaccines anywhere else during the academic semester. Although during the Fall 2016 wave we did not account for alternative vaccination locations outside the University, survey data from Fall 2017 indicates that only 7.5% of the total vaccinated surveyed students received the vaccine from somewhere other than the University flu campaign. Thus, vaccination turnout numbers are likely an accurate representation of the vaccinated undergraduate students on campus for the Fall semester. Another reason for the low vaccination turnout for the Fall semester is that students could be receiving the flu vaccine during the Winter period, when they return home. This could potentially bring our vaccination rates closer to the CDC average. However, the recommendation is to get vaccinated before the flu activity begins. Specifically, the CDC recommends receiving the flu vaccine by the end of October if possible, as it takes two weeks for the body’s immune response to fully respond [69]. In the case students are receiving the vaccine during the Winter period, the decision is not influenced by our treatments and could be a family-based decision. In addition, receiving the vaccine after the Fall term ends leaves students fully exposed to the virus during the take-up of flu activity (November to January).

Compared to the Baseline, all treatments are significantly different in terms of the proportion of vaccinated students (proportion tests: Self = Baseline, p < 0.01; Others = Baseline, p < 0.01; Both = Baseline, p < 0.01). Comparing the different treatments in terms of probability of vaccination (number of vaccinated students over intent to treat in each treatment, Fig. 3), the Both treatment achieves a higher response to the treatment. These percentages are also different from each other (proportion tests: Self = Others, p < 0.01; Self = Both, p < 0.01; Others = Both, p < 0.01). Thus, highlighting the vaccination’s social and individual benefit together helps the most in incentivizing students’ decision to receive the vaccine at a flu clinic during the semester.

Fig. 3
figure 3

Proportion of vaccinated by treatment and gender

The prior literature on vaccination and gender for general populations shows that a higher percentage of females seem to accept and get vaccinated compared to males [70, 71]. If this is not the case, females tend to give more thought-out arguments (such as expressing hesitancy openly) for whether they want to get the vaccine or not [72]. Given that there could be a gender gap in our experiment, we test for these differences. In our case, and consistent with the prior work on vaccination and gender, a higher percentage of females seem to be getting vaccinated compared to the baseline regardless of the treatment (Fig. 3, Proportion tests: Baseline = Self, p < 0.001; Baseline = Others, p = 0.001; Baseline = Both, p < 0.001). For men however, the increase in vaccinations happens only for the Both treatment, which incorporates both the individual and community benefits of the vaccine, while for the Others treatment, the association is marginally significant (Proportion tests: Baseline = Self, p = 0.433; Baseline = Others, p = 0.095; Baseline = Both, p < 0.001). Thus, initial evidence suggests that our empirical question 1, which stated that any message increases vaccinations holds for females but not for males.

One explanation for the differential influence of the normative messages depending on gender could be related to risk perceptions. Prior literature argues that women could be more sensitive to social cues than men. An alternative explanation for this result is related to health care usage [73]. Women usually report a greater use of health care services, worse levels of self-perceived health as well as greater levels of morbidity and disability [74, 75].

Likelihood of getting a flu vaccine

We next analyze the influence of the normative messaging intervention and individual characteristics on the likelihood of getting a flu vaccine. A total of 828 undergraduate students participated in this study, 360 getting the flu vaccine in the clinics and 468 not getting the vaccine. Characteristics for both sub-populations are reported in Table 2. Vaccinated and non-vaccinated students are on average similar in terms of gender (means test, p = 0.82) and age (means test, p = 0.13) and race (means test, p = 0.16). Students who decided to receive the vaccine self-report overall better health status (means test, p < 0.001). However, the question might be biased as those who get vaccinated respond to the question when arriving to get the vaccine.

Table 2 Descriptive statistics of survey respondents, vaccinated versus non-vaccinated

The results of logit specifications for the full sample of vaccinated and non-vaccinated students are reported in Table 3. We define our dependent variable as equal to 1 if the subject received a vaccine in a flu clinic and 0 otherwise. To be consistent with the way the sample is selected and treated (as our aim is to understand which poster yields a higher vaccination turnout), errors in each specification are clustered by treatment (as our poster assignment is based on building groups) [76, 77] and bootstrapped with 10 thousand replications. In addition, given that the number of treatments (clusters) is small and to provide asymptotic refinement, we provide the results of score-based wild cluster bootstrap tests for our regressions [78, 79]. Regression results are found in Additional file 1, including the p-values from the score wild cluster bootstrap. Table 3 shows the marginal percentage changes (from a linear probability model) for the relationship between the normative message treatments and receiving a flu vaccine. Other specifications can be found in Additional file 1. We have five econometric specifications. The marginal percentage changes are interpreted as the percentage point change in the likelihood of an individual getting a flu vaccine. Table 4 presents the probability of getting vaccinated under different econometric specifications.

Table 3 Influence of treatment on likelihood of getting a flu vaccine (marginal percentage change, linear probability model)
Table 4 Measures of prediction

Our first empirical question asks whether messaging emphasizing the benefits of the vaccine positively affects the likelihood of vaccination. The second and third empirical questions compare normative treatments. If the individual and community benefits of the vaccine are complements, (see [45] for an associated model on crowding of the benefits of the vaccine), we expect that when both benefits of the vaccine are highlighted together, the final value of the likelihood of getting vaccinated compared to highlighting only one of the benefits might depend on whether the messages crowd-in or -out. We discuss this briefly below.

To answer these questions, we include dummy variables associated to the normative messages highlighted in the experiment. The variables of primary interest are Self, Others and Both, which are binary variables that correspond to whether the subject was assigned to a specific poster. Our Baseline poster had no message and no flu fact.

The estimated marginal percentage change of the normative messaging treatments is significant for the Others treatment and for the Both treatment. Compared to the Baseline (no message), being exposed to the messages that emphasize only the social benefit of the vaccine increases the likelihood of getting a flu vaccine by 11–12 percentage points, while being exposed by the message that highlights the individual and social benefits together increases this likelihood by 19–21 percentage points (Table 3). One concern which arises is proximity-related: the UHS building which holds some of the clinics are closest to the Self and Both treatments (see Fig. 2). If proximity influenced the decision to vaccinate in combination with the availability of UHS clinics, one would expect to find the same percentage change on those two treatments regardless of the message in the posters. However, we do not find any significant change on vaccination decisions when participants are exposed to the Self posters.

Regarding the second question, we find that there is no significant difference between the treatments with messages that emphasize only one benefit of the vaccine (either individual or social, Self = Others, χ2 = 0.69, p = 0.407 for specification (1)).

Regarding our third question, we find that emphasizing both the individual and community benefits of the flu vaccine has the strongest association with the likelihood of getting vaccinated (χ2 = 7.37, p = 0.025 for specification (1)) compared to no benefits. Also, it is important to know whether using both individual and social prompts results in crowding-in or crowding-out. If they crowd-in, the benefits of both aspects (Both) should be larger than the sum of the parts (Self + Others). To test this, we test whether Both ≥ Self+Others in the Table 3 specifications. The associated p-values for this test suggest that we fail to reject an equal or greater effect of our vaccine benefits for all our specifications. We can thus conclude that emphasizing both aspects together is at least neutral (additive) and could potentially result a larger benefit than emphasizing them separately: crowding-in the benefits of the flu vaccine and self-interest and pro-social preferences work together to increase vaccination turnout.

Lastly, and as a way to understand the motives behind the decision to get the flu vaccine, we incorporate unincentivized measures of (general) individual preferences. We start by including a variable to account for general altruistic motives in our specifications. We assume that altruism is a continuum, and that the strength of the preference is what we look at in our analyses first. Following the prior literature [81], we expect that if altruism influences the decision to be vaccinated, we would see a positive coefficient, as altruism shifts vaccination decisions away from individual self-interest and towards community welfare. Altruism in our main regressions (incorporated as a scale from 1 to 10) is not statistically significant. However, a more detailed analysis (provided in the Additional file 1) shows that very altruistic individuals (8–10 points in the altruistic scale) are more likely to get a flu vaccine, regardless of the treatment they were assigned to. We also tested for reciprocity and selfishness with unincentivized measures but did not find consistent or significant results. Full results of these specifications are provided in the Additional file 1.

Prediction of vaccination outcomes

In this section we briefly discuss the predictive power of our natural experiment in terms of the probability of getting vaccinated. Table 4 shows the predicted probabilities of getting vaccinated. In our sample, a participant exposed to the treatment which highlights both benefits of the vaccine has a much higher chance of getting the vaccine (around 50%) than when being exposed to a single benefit or when not being exposed to any poster (between 31 and 45% depending on the treatment and the specification). In addition, our model including beliefs and prior history of vaccination has an acceptable predictive power when including controls related to beliefs and prior vaccination decisions (using cross-validation measures of accuracy applied in medicine and social science, Table 4). Therefore, our intervention be used to further study the influence of our campaign on vaccination outcomes provided it is properly controlled by beliefs and decision-making-related variables.


In this section, we discuss some of the factors which should be considered when interpreting the results and compare our results with the prior literature on non-pecuniary flu vaccine experiments.


We have three main caveats to be considered for this experiment. First, although we do see students coming in groups to the flu clinics, we were not able to gather information to elicit peer or family history during the Fall 2016. However, there is prior evidence on the influence of peers on flu vaccination decisions, as students coordinate vaccination decisions with their friends at the college level, with friends exchanging information and shaping each other’s beliefs about the flu virus and the vaccine [68]. Family also exerts a similar influence, either by recommending the vaccine to other family members or by setting an example and getting the vaccine [82].

Second, a bad prior season in terms of flu contagion and hospitalizations could also increase the likelihood of getting vaccinated during our flu clinics (2016–2017 season). However, the prior flu seasons to our vaccination campaign were designed by the CDC as moderate or mild [83]. Therefore, based on the information about prior seasons, we would not expect external factors related to prior flu season severity to affect vaccination decisions at the beginning of the (Fall) 2016–2017 season, when our study took place.

Third, there could be response bias in our surveys, as the number of people getting vaccinated and surveyed is much higher in proportion than those who did not. However, from the survey analysis we get similar treatment results as from the vaccinated sample, and our vaccinated and non-vaccinated samples are similar in terms of socio-economic characteristics.

Comparison with prior literature on flu vaccines

In this section, we compare our general poster campaign to other types of interventions that can operate through social incentives to change behavior. In particular, we focus on key health-based economics literature employing low-cost strategies: namely, network analysis (including peer influence) and text messaging.

Networks and peer influence have been used to increase vaccination rates. As an example, researchers have examined the impact of identifying the “best” people to spread information about vaccination clinics [84]. In this sample of Indian villages, the villages where information was spread through those nominated by the village experienced an increase of 22% in the number of vaccinated children every month compared to villages which did not use nominated leaders to spread information. While this approach is very effective in improving vaccination rates, the cost associated with assessing who the “best” people to convey information can be high and, in low-trust environments, identifying these individuals may be infeasible.

Another approach is to improve ease-of-access to clinics. In a University setting similar to ours, as the percentage of residential halls with flu clinics grows (from one third to two thirds of the halls), the vaccination rate among the student body rises by a range of 7.2–7.9 percentage points. Over 25% of this increase can be attributed to social (peer) influence impacting vaccination decisions [68]. This can be an effective strategy if current coverage is low, although it would increase staffing costs. It is also not feasible if all halls are already covered by clinics.

Another successful, low-cost strategy for improving flu vaccine take-up is to design the campaign as opt-out rather than opt-in [40, 85]. In one of the campaigns, there was an increase in the probability of a flu shot appointment when using the opt-out strategy: 45% of the opt-out participants were vaccinated, compared to 33% in the opt-in strategy [40]. It is important to highlight that this campaign was focused on University staff, indicating that it could functionally be translated to a student population. In a second study [85] done with health care workers, there was no statistically significant differences between the opt-in and opt-out interventions. However, workers in the opt-out condition were more likely to have an appointment to get a flu vaccine, which could predict the probability of getting the vaccine.

In the United States, the use of nudging prompts through text messaging has been successful in increasing flu vaccination uptake by 5%, particularly when the prompt directly assigns a flu vaccine appointment to the individual [86]. This last study is comparable to our vaccination 5.3% intent to treat uptake results in our comparatively low-cost design focused on a large population of students.


In this study, we set up a field experiment to investigate the influence of normative messaging on individual decisions to get a flu vaccine. We aim at building on the literature that uses normative influence to study how individuals’ behavior could be influenced to make healthier decisions [25, 27, 28]. We incorporate normative messaging to an already established flu vaccination campaign at a University. The posters promoting the campaign had normative messages that highlighted the individual and the social benefits of the flu vaccine. Our field experiment allows us to look at actual vaccination decisions instead of the intention to receive the vaccine. Our analysis tries to answer three main questions: whether the normative messages have a positive influence on vaccination decisions, whether messages that highlight the social benefits of the vaccine have a larger positive influence than those that highlight the individual benefits.

We find that normative messaging increases the number of students getting the flu vaccine when compared to no message. A larger number of individuals getting vaccinated means a higher level of community protection, which in the end contributes to protecting populations at risk. In line with prior studies [60], we find that strategically selected messages can increase public goods provision. When the individual and community benefits of the vaccine are emphasized together, the likelihood of getting a vaccine significantly increases.

Our results suggest that normative messaging done through a low-cost campaign could be used as a non-pecuniary policy instrument for increasing the number of vaccinated individuals within a large community [87]. In our case, messaging was strategically designed to be as salient as possible in a continuous manner, exposing students to the messages every time they are in the common areas of their residential hall. Emails and email reminders have the risk of not being open but using posters in strategic areas exposed students to the messages and information repeatedly. Normative messaging might have behavioral and health consequences, as it could help modify individual behavior. A campaign highlighting healthy behaviors (getting a vaccine) and making salient the outcome of the behavior from a social point of view (protecting the individual and the community) could be an effective and low-cost strategy to promote behavior changes [25, 28].

The experiment was successfully implemented within a University community, with undergrad students being repeatedly exposed to the messages posted in each floor of a residential area. The intervention was easy to implement and a low-cost strategy to modify behaviors, but we still see a small percentage of students receiving the vaccine outside the university clinics. It could happen that students could receive the vaccine at home during the Winter period, but that falls outside the recommended timeframe to receive the flu vaccine to be fully protected before the start of the flu season. However, we found that a low percentage of students got vaccinated after Thanksgiving. Therefore, while capturing most vaccinated students in our sample, this means that more work must be done to reach herd immunity in terms of public campaigns before the flu season peaks. Potential future work could include other non-pecuniary strategies such as “days-off” for students to get vaccinated in a one-day vaccination clinic on campus right before the flu season starts. In addition, as our target is a young adult population with widespread access to social media, new and innovative ways of transmitting information are available and should be used to increase vaccination uptake. The use of emojis, for example, which is essential in communication nowadays, is gaining traction for health uses [88, 89], and there is a current discussion on the representation of vaccinations as emojis (instead of syringe or needle emojis) [90] which should be taken forward to both in-person and online vaccination nudging campaigns.

Further research on flu vaccinations at the campus level should be directed at teasing out the influence of peer and family background (family history of chronic illnesses, vaccinations and at-risk population) from the influence of normative messaging. Contamination between residence halls should be addressed further to tease the influence of visiting another residence hall and the frequency of those visits, as students could go from one residence hall to another and be exposed to different messages. In addition, given that the normative messages influenced women (compared to men), it would be necessary to come up with new strategies or even new normative messages that can have a significant influence on male vaccination rates.

More broadly, with flu seasons being difficult to predict and ranging from mild to severe, flu vaccinations are the first barrier of protection against the flu as well as against complications from the illness. A 70% of the population vaccination target for the current year [91, 92] compared to less than half of the United States population actually getting the vaccine implies that the level of coverage against the illness was low and populations at-risk could be negatively affected by the low level of community protection, below herd immunity levels. From a public policy point of view, developing strategies and campaigns to increase vaccination on a yearly basis should be a priority. These campaigns should tackle the perception of risk related to the vaccine as well as the myths and lack of education and information surrounding flu vaccination. Moreover, the protection of others who are vulnerable or at-risk is one of the key elements of vaccination. If normative messages which include pro-social messaging can be used to increase vaccination rates in a population of young, healthy and insured individuals, a natural step should be to extend low-cost but high-impact campaigns to other at-risk communities where the number of background risks is much larger (chronic health issues, job and income instability and without health insurance). This is especially relevant nowadays for example, for the case of the coronavirus disease (COVID-19), where vaccination has been extended to whole populations and younger adults have been overall more hesitant to vaccinate than other adult groups.

Availability of data and materials

The data (anonymized) and code to replicate this study is available upon request. Questionnaires and experiment materials are available as an additional file jointly with this manuscript.


  1. Herd immunity happens when a significant number of people in a community are protected from getting a disease because they have either had the disease or have been vaccinated [2].

  2. For nursing students and students taking courses which includes visits to hospitals and laboratory students, flu vaccinations are mandatory, so they are not part of our target population. This is a total of 60 students.

  3. Administrative data on the whole University student population is always preferrable in these cases, but by University regulations we were only permitted to collect individual-specific data from our survey participants.



University Health Services


  1. World Health Organization. Measures in school settings: Pandemic (H1N1) 2009 briefing note 10. 2009. (Accessed 11 Dec 2018).

    Google Scholar 

  2. Levitt SD, List JA. Homo Economicus Evolves. Science. 2008;319(5865):909–10.

    Article  CAS  PubMed  Google Scholar 

  3. Allen JD, Mohllajee AP, Shelton RC, Othus MK, Fontenot HB, Hanna R. Stage of adoption of the human papillomavirus vaccine among college women. Prev Med. 2009;48(5):420–5.

    Article  PubMed  Google Scholar 

  4. Krupka EL, Croson RT. The differential impact of social norms cues on charitable contributions. J Econ Behav Organ. 2016;128:149–58.

    Article  Google Scholar 

  5. Böhm R, Betsch C, Korn L. Selfish-rational non-vaccination: experimental evidence from an interactive vaccination game. J Econ Behav Organ. 2016;131:183–95.

    Article  Google Scholar 

  6. Betsch C, Böhm R, Korn L. Inviting free-riders or appealing to prosocial behavior? Game-theoretical reflections on communicating herd immunity in vaccine advocacy. Health Psychol. 2013;32(9):978.

    Article  PubMed  Google Scholar 

  7. Chapman GB, Li M, Vietri J, Ibuka Y, Thomas D, Yoon H, et al. Using game theory to examine incentives in influenza vaccination behavior. Psychol Sci. 2012;23(9):1008–15.

    Article  PubMed  Google Scholar 

  8. Janssens W, Kramer B. The social dilemma of microinsurance: free-riding in a framed field experiment. J Econ Behav Organ. 2016;131:47–61.

    Article  Google Scholar 

  9. Godager G, Hennig-Schmidt H, Iversen T. Does performance disclosure influence physicians’ medical decisions? An experimental study. J Econ Behav Organ. 2016;131:36–46.

    Article  Google Scholar 

  10. Böhm R, Meier NW, Korn L, Betsch C. Behavioural consequences of vaccination recommendations: an experimental analysis. Health Econ. 2017;26:66–75.

    Article  PubMed  Google Scholar 

  11. Ibuka Y, Li M, Vietri J, Chapman GB, Galvani AP. Free-riding behavior in vaccination decisions: an experimental study. PLoS One. 2014;9(1):e87164.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  12. Vietri JT, Li M, Galvani AP, Chapman GB. Vaccinating to help ourselves and others. Med Decis Mak. 2012;32(3):447–58.

    Article  Google Scholar 

  13. Cialdini RB, Reno RR, Kallgren CA. A focus theory of normative conduct: recycling the concept of norms to reduce littering in public places. J Pers Soc Psychol. 1990;58(6):1015.

    Article  Google Scholar 

  14. Burger JM, Bell H, Harvey K, Johnson J, Stewart C, Dorian K, et al. Nutritious or delicious? The effect of descriptive norm information on food choice. J Soc Clin Psychol. 2010;29(2):228–42.

    Article  Google Scholar 

  15. Zimmerman FJ. Using behavioral economics to promote physical activity. Prev Med. 2009;49(4):289–91.

    Article  PubMed  Google Scholar 

  16. Bicchieri C, Dimant E. Nudging with care: the risks and benefits of social information. Public Choice. 2019;191:1–22.

    Google Scholar 

  17. Allcott H. Social norms and energy conservation. J Public Econ. 2011;95(9–10):1082–95.

    Article  Google Scholar 

  18. Lawrence NK. Highlighting the injunctive norm to reduce phone-related distracted driving. Soc Influ. 2015;10(2):109–18.

    Article  Google Scholar 

  19. Abrams D, Wetherell M, Cochrane S, Hogg MA, Turner JC. Knowing what to think by knowing who you are: self-categorization and the nature of norm formation, conformity and group polarization. Br J Soc Psychol. 1990;29(2):97–119.

    Article  PubMed  Google Scholar 

  20. Centola D. An experimental study of homophily in the adoption of health behavior. Science. 2011;334(6060):1269–72.

    Article  CAS  PubMed  Google Scholar 

  21. Bavel JJ, Baicker K, Boggio PS, Capraro V, Cichocka A, Cikara M, et al. Using social and behavioural science to support COVID-19 pandemic response. Nat Hum Behav. 2020;4(5):460–71.

    Article  PubMed  Google Scholar 

  22. Galizzi MM. Label, nudge or tax? A review of health policies for risky behaviours. J Public Health Res. 2012;1(1):14.

    Article  PubMed  PubMed Central  Google Scholar 

  23. Sunstein CR, Reisch LA, Kaiser M. Trusting nudges? Lessons from an international survey. J Eur Public Policy. 2019;26(10):1417–43.

    Article  Google Scholar 

  24. Sunstein CR, Reisch LA, Rauber J. A worldwide consensus on nudging? Not quite, but almost. Regul Governance. 2018;12(1):3–22.

    Article  Google Scholar 

  25. Burger JM, Shelton M. Changing everyday health behaviors through descriptive norm manipulations. Soc Influ. 2011;6(2):69–77.

    Article  Google Scholar 

  26. Smith-McLallen A, Fishbein M. Predictors of intentions to perform six cancer-related behaviours: roles for injunctive and descriptive norms. Psychol Health Med. 2008;13(4):389–401.

    Article  PubMed  Google Scholar 

  27. Neighbors C, Larimer ME, Lewis MA. Targeting misperceptions of descriptive drinking norms: efficacy of a computer-delivered personalized normative feedback intervention. J Consult Clin Psychol. 2004;72(3):434.

    Article  PubMed  Google Scholar 

  28. Mollen S, Rimal RN, Ruiter RA, Kok G. Healthy and unhealthy social norms and food selection. Findings from a field-experiment. Appetite. 2013;65:83–9.

    Article  PubMed  Google Scholar 

  29. Neighbors C, Lee CM, Lewis MA, Fossos N, Larimer ME. Are social norms the best predictor of outcomes among heavy-drinking college students? J Stud Alcohol Drugs. 2007;68(4):556–65.

    Article  PubMed  Google Scholar 

  30. Daley MF, Crane LA, Chandramouli V, Beaty BL, Barrow J, Allred N, et al. Misperceptions about influenza vaccination among parents of healthy young children. Clin Pediatr. 2007;46(5):408–17.

    Article  Google Scholar 

  31. Schmidtke KA, Nightingale PG, Reeves K, Gallier S, Vlaev I, Watson SI, et al. Randomised controlled trial of a theory-based intervention to prompt front-line staff to take up the seasonal influenza vaccine. BMJ Qual Saf. 2020;29(3):189–97.

    Article  PubMed  Google Scholar 

  32. Chang T, Jacobson M, Shah M, Pramanik R, Shah SB. Financial incentives and other nudges do not increase covid-19 vaccinations among the vaccine hesitant. Cambridge: National Bureau of Economic Research; 2021.

  33. Ashworth M, Thunström L, Cherry TL, Newbold SC, Finnoff DC. Emphasize personal health benefits to boost COVID-19 vaccination rates. Proc Natl Acad Sci. 2021;118(32):153–67.

    Article  CAS  Google Scholar 

  34. Dai H, Saccardo S, Han MA, Roh L, Raja N, Vangala S, et al. Behavioural nudges increase COVID-19 vaccinations. Nature. 2021;597(7876):404–9.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  35. Sasaki S, Saito T, Ohtake F. Nudges for COVID-19 voluntary vaccination: how to explain peer information? Soc Sci Med. 2022;292:114561.

    Article  PubMed  Google Scholar 

  36. Bergquist M, Nilsson A. I saw the sign: promoting energy conservation via normative prompts. J Environ Psychol. 2016;46:23–31.

    Article  Google Scholar 

  37. Duong HT, Nguyen TT, Trieu LT. When vaccine uncertainty prevails: association between online social influence and COVID-19 vaccine intentions. Int J Commun. 2022;16:25.

    Google Scholar 

  38. Bronchetti ET, Huffman DB, Magenheim E. Attention, intentions, and follow-through in preventive health behavior: field experimental evidence on flu vaccination. J Econ Behav Organ. 2015;116:270–91.

    Article  Google Scholar 

  39. Milkman KL, Beshears J, Choi JJ, Laibson D, Madrian BC. Using implementation intentions prompts to enhance influenza vaccination rates. Proc Natl Acad Sci. 2011;108(26):10415–20.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  40. Chapman GB, Li M, Colby H, Yoon H. Opting in vs opting out of influenza vaccination. JAMA. 2010;304(1):43–4.

    Article  CAS  PubMed  Google Scholar 

  41. Dubov A, Phung C. Nudges or mandates? The ethics of mandatory flu vaccination. Vaccine. 2015;33(22):2530–5.

    Article  PubMed  Google Scholar 

  42. MacDonald NE. Vaccine hesitancy: definition, scope and determinants. Vaccine. 2015;33(34):4161–4.

    Article  PubMed  Google Scholar 

  43. Abhyankar P, O'connor DB, Lawton R. The role of message framing in promoting MMR vaccination: evidence of a loss-frame advantage. Psychol Health Med. 2008;13(1):1–6.

    Article  PubMed  Google Scholar 

  44. Chen F, Stevens R. Applying lessons from behavioral economics to increase flu vaccination rates. Health Promot Int. 2017;32(6):1067–73.

    PubMed  Google Scholar 

  45. Mussio I. Three essays on health, risk and behavior [doctoral dissertation]. Amherst: University of Massachusetts Amherst; 2018. Available from: Accessed 14 June 2022.

  46. Benin AL, Wisler-Scher DJ, Colson E, Shapiro ED, Holmboe ES. Qualitative analysis of mothers' decision-making about vaccines for infants: the importance of trust. Pediatrics. 2006;117(5):1532–41.

    Article  PubMed  Google Scholar 

  47. Meszaros JR, Asch DA, Baron J, Hershey JC, Kunreuther H, Schwartz-Buzaglo J. Cognitive processes and the decisions of some parents to forego pertussis vaccination for their children. J Clin Epidemiol. 1996;49(6):697–703.

    Article  CAS  PubMed  Google Scholar 

  48. Brewer NT, Chapman GB, Gibbons FX, Gerrard M, McCaul KD, Weinstein ND. Meta-analysis of the relationship between risk perception and health behavior: the example of vaccination. Health Psychol. 2007;26(2):136.

    Article  PubMed  Google Scholar 

  49. Conner M, Sparks P. Theory of planned behaviour and health behaviour. Predicting Health Behav. 2005;2(1):121–62.

    Google Scholar 

  50. Betsch C, Schmid P. Angst essen Impfbereitschaft auf? Bundesgesundheitsblatt-Gesundheitsforschung-Gesundheitsschutz. 2013;56(1):124–30.

    Article  CAS  PubMed  Google Scholar 

  51. Weinstein ND, Kwitel A, McCaul KD, Magnan RE, Gerrard M, Gibbons FX. Risk perceptions: assessment and relationship to influenza vaccination. Health Psychol. 2007;26(2):146.

    Article  PubMed  Google Scholar 

  52. Nyhan B, Reifler J. Does correcting myths about the flu vaccine work? An experimental evaluation of the effects of corrective information. Vaccine. 2015;33(3):459–64.

    Article  PubMed  Google Scholar 

  53. Verelst F, Willem L, Kessels R, Beutels P. Individual decisions to vaccinate one's child or oneself: a discrete choice experiment rejecting free-riding motives. Soc Sci Med. 2018;207:106–16.

    Article  PubMed  Google Scholar 

  54. Verelst F, Kessels R, Delva W, Beutels P, Willem L. Drivers of vaccine decision-making in South Africa: a discrete choice experiment. Vaccine. 2019;37(15):2079–89.

    Article  PubMed  Google Scholar 

  55. Thorpe EL, Zimmerman RK, Steinhart JD, Lewis KN, Michaels MG. Homeschooling parents’ practices and beliefs about childhood immunizations. Vaccine. 2012;30(6):1149–53.

    Article  PubMed  Google Scholar 

  56. Gidengil CA, Parker AM, Zikmund-Fisher BJ. Trends in risk perceptions and vaccination intentions: a longitudinal study of the first year of the H1N1 pandemic. Am J Public Health. 2012;102(4):672–9.

    Article  PubMed  PubMed Central  Google Scholar 

  57. Andreoni J. Impure altruism and donations to public goods: a theory of warm-glow giving. Econ J. 1990;100(401):464–77.

    Article  Google Scholar 

  58. Cornes R, Sandler T. Easy riders, joint production, and public goods. Econ J. 1984;94(375):580–98.

    Article  Google Scholar 

  59. Deacon R, Shapiro P. Private preference for collective goods revealed through voting on referenda. Am Econ Rev. 1975;65(5):943–55.

    Google Scholar 

  60. Kessler JB, Milkman KL. Identity in charitable giving. Manag Sci. 2018;64(2):845–59.

    Article  Google Scholar 

  61. List JA. Introduction to field experiments in economics with applications to the economics of charity. Exp Econ. 2008;11(3):203–12.

    Article  Google Scholar 

  62. Su Z, McDonnell D, Wen J, Cheshmehzangi A, Ahmad J, Goh E, et al. Young adults’ preferences for influenza vaccination campaign messages: implications for COVID-19 vaccine intervention design and development. Brain Behav Immun Health. 2021;14:100261.

    Article  PubMed  PubMed Central  Google Scholar 

  63. Harrison GW, List JA. Field experiments. J Econ Lit. 2004;42(4):1009–55.

    Article  Google Scholar 

  64. Rege M, Telle K. The impact of social approval and framing on cooperation in public good situations. J Public Econ. 2004;88(7–8):1625–44.

    Article  Google Scholar 

  65. Wisdom J, Downs JS, Loewenstein G. Promoting healthy choices: information versus convenience. Am Econ J Appl Econ. 2010;2(2):164–78.

    Article  Google Scholar 

  66. Roberto CA, Larsen PD, Agnew H, Baik J, Brownell KD. Evaluating the impact of menu labeling on food choices and intake. Am J Public Health. 2010;100(2):312–8.

    Article  PubMed  PubMed Central  Google Scholar 

  67. Centers for Disease Control and Prevention. Flu vaccination coverage, United States, 2016–17 influenza season. 2017. (Accessed 11 Dec 2018).

  68. Rao, N, Kremer, M, Mobius, M, Rosenblat, T. Social Networks and Vaccination Decisions. 2017. (Accessed 13 Apr 2022).

  69. Centers for Disease Control and Prevention. Frequently asked flu questions 2017–2018 influenza season. 2018. (Accessed 11 Dec 2018).

  70. Applewhite A, Stancampiano FF, Harris DM, Manaois A, Dimuna J, Glenn J, et al. A retrospective analysis of gender-based difference in adherence to influenza vaccination during the 2018-2019 season. J Prim Care Community Health. 2020;11:2150132720958532.

    Article  PubMed  PubMed Central  Google Scholar 

  71. Davis CJ, Golding M, McKay R. Efficacy information influences intention to take COVID-19 vaccine. Br J Health Psychol. 2022;27(2):300–19.

    Article  PubMed  Google Scholar 

  72. Velan B, Kaplan G, Ziv A, Boyko V, Lerner-Geva L. Major motives in non-acceptance of a/H1N1 flu vaccination: the weight of rational assessment. Vaccine. 2011;29(6):1173–9.

    Article  PubMed  Google Scholar 

  73. Croson R, Gneezy U. Gender differences in preferences. J Econ Lit. 2009;47(2):448–74.

    Article  Google Scholar 

  74. Salganicoff A, Ranji U, Beamesderfer A, Kurani N. Women and health care in the early years of the ACA: key findings from the 2013 Kaiser Women’s health survey. Menlo Park: The Henry J. Kaiser Family Foundation; 2014.

    Google Scholar 

  75. Redondo-Sendino Á, Guallar-Castillón P, Banegas JR, Rodríguez-Artalejo F. Gender differences in the utilization of health-care services among the older adult population of Spain. BMC Public Health. 2006;6(1):1–9.

    Article  Google Scholar 

  76. Abadie A, Athey S, Imbens GW, Woolridge JM. When should you adjust standard errors for clustering? Cornell University arXiv, NBER Working Papers. 2022. (Accessed 13 Jun 2022).

  77. Cameron AC, Miller DL. A practitioner’s guide to cluster-robust inference. J Hum Resour. 2015;50(2):317–72.

    Article  Google Scholar 

  78. Kline P, Santos A. A score based approach to wild bootstrap inference. Journal of Econometric Methods. 2012;1(1):23–41.

    Article  Google Scholar 

  79. Roodman D, Nielsen MØ, MacKinnon JG, Webb MD. Fast and wild: bootstrap inference in Stata using boottest. Stata J. 2019;19(1):4–60.

    Article  Google Scholar 

  80. Luque-Fernandez MA, Redondo-Sánchez D, Maringe C. Cvauroc: command to compute cross-validated area under the curve for ROC analysis after predictive modeling for binary outcomes. Stata J. 2019;19(3):615–25.

    Article  Google Scholar 

  81. Shim E, Chapman GB, Townsend JP, Galvani AP. The influence of altruism on influenza vaccination decisions. J R Soc Interface. 2012;9(74):2234–43.

    Article  PubMed  PubMed Central  Google Scholar 

  82. Takahashi O, Noguchi Y, Rahman M, Shimbo T, Goto M, Matsui K, et al. Influence of family on acceptance of influenza vaccination among Japanese patients. Fam Pract. 2003;20(2):162–6.

    Article  PubMed  Google Scholar 

  83. Center for Disease Control and Prevention. Past Seasons Estimated Influenza Disease Burden 2020. (Accessed 16 Jun 2021).

  84. Banerjee A, Chandrasekhar AG, Duflo E, Jackson MO. Using gossips to spread information: theory and evidence from two randomized controlled trials. Rev Econ Stud. 2019;86(6):2453–90.

    Article  Google Scholar 

  85. Lehmann BA, Chapman GB, Franssen FM, Kok G, Ruiter RA. Changing the default to promote influenza vaccination among health care workers. Vaccine. 2016;34(11):1389–92.

    Article  PubMed  Google Scholar 

  86. Milkman KL, Patel MS, Gandhi L, Graci HN, Gromet DM, Ho H, et al. A megastudy of text-based nudges encouraging patients to get vaccinated at an upcoming doctor’s appointment. Proc Natl Acad Sci. 2021;118(20).

  87. Brito DL, Sheshinski E, Intriligator MD. Externalities and compulsary vaccinations. J Public Econ. 1991;45(1):69–90.

    Article  Google Scholar 

  88. Boender TS, Louis-Ferdinand N, Duschek G. Digital visual communication for public health: design proposal for a vaccinated emoji. J Med Internet Res. 2022;24(4):e35786.

  89. Lotfinejad N, Assadi R, Aelami MH, Pittet D. Emojis in public health and how they might be used for hand hygiene and infection prevention and control. Antimicrob Resist Infect Control. 2020;9(1):1–6.

    Article  Google Scholar 

  90. O’Reilly-Shah VN, Lynde GC, Jabaley CS. Is it time to start using the emoji in biomedical literature? BMJ. 2018;363:k5033.

  91. Office of Disease Prevention and Health Promotion, healthy people 2030, goal: increase vaccination rates, increase the proportion of people who get the flu vaccine every year — IID-09. 2022. Accessed 31 May 2022.

  92. Center for Disease Control and Prevention. Key Things to Know about COVID-19 Vaccines. 2021. Accessed 16 June 2021.

Download references


We would like to thank Dr. Ann Becker at University Health Services (UMass Amherst) for allowing us to run our experiment around the flu vaccination campaign. Thanks to Collin B. Raymond and Nathan W. Chan for their advice and to Onupurba Das, Emma Grazier, Jane Peng and Anthony Lorencette who provided exceptional assistance.


No funding was received from any private or public institution.

Author information

Authors and Affiliations



All of the authors (IM and AdO) fulfilled the authorship criteria and contributed to the study. The first author IM designed and implemented the study, analyzed and interpreted the experimental data and wrote the manuscript. AdO supervised, reviewed and contributed to design, implementation, analysis, interpretation and editing of the manuscript. All of the authors are aware of the submission and are in agreement with the manuscript. The author(s) read and approved the final manuscript.

Corresponding author

Correspondence to Irene Mussio.

Ethics declarations

Ethics approval and consent to participate

This research was approved by the University of Massachusetts Internal Review Board (protocol number 2016–3333). Participants had to sign an informed consent prior to the experiment.

Consent for publication

All authors provided consent for their names to be included for publication.

Competing interests

The authors declare that they have no competing interests with regard to the content of this article.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Supplementary Information

Additional file 1.

Posters, flu clinics calendar, full regression results, leaflet for online survey recruitment and questionnaires.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit The Creative Commons Public Domain Dedication waiver ( applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Mussio, I., de Oliveira, A.C.M. An (un)healthy social dilemma: a normative messaging field experiment with flu vaccinations. Health Econ Rev 12, 41 (2022).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: