Skip to main content
Download PDF
Download ePub

Multiple effects health economic evaluation of the Ahead of The Game Study for mental health promotion in sporting club communities

Health Economics Review volume 11, Article number: 28 (2021) Cite this article

Abstract

Background

This study evaluates the Ahead Of The Game (AOTG) mental health promotion strategy for adolescent males relative to usual practice in team based sporting club community settings, allowing for joint incremental effects across 13 dimensions and 5 domains alongside intervention implementation costs.

Methods

Analysis is undertaken between matched communities with difference in differences analysis of joint multiple pre-post effect changes alongside implementation costs employing radar plots in cost-disutility space. A robust bootstrapping method allowed including all observed change in effect data from 343 AOTG and 273 control arm participants across 13 effect dimensions.

Results

Triangulation across joint evidence shows mean incremental effects favoured AOTG in all dimensions (10/13 significantly at 5% level) and in simple aggregation to each of five pre-specified 5 domains (each significant at < 1% level) and global measures (significant at 0.001% level), while mean AOTG implementation costs were conservatively estimated as $37.47 per participant.

Conclusion

The AOTG strategy was found to represent an effective mental health promotion strategy across all domains and globally with associated significant potential for downstream health system cost savings to offset against modest implementation costs. Evaluation methods extend conventional cost-effectiveness analysis to enable robust joint presentation and triangulation under uncertainty of multiple effect dimensions alongside costs.

Trial registration

ANZCTR, ACTRN12617000709347. Registered 17th May 2017.

Background

Evaluating health promotion in community settings for adolescent mental health: the ahead of the game (AOTG) program and study

Health promotion programs in community settings (e.g. school, workplace or sports club) adopt socio-ecological approaches considering multiple levels of influence on populations health behaviours, including intra-personal, proximal interpersonal, organisational, community, and policy levels [32] and across social, cultural, environmental, and economic determinants [24]. Jointly influencing multiple determinant and health behaviours across these multiple levels are key to such programs creating and supporting pathways and policies for meaningful community ownership, and behaviour change across target populations in community settings [5, 23, 28, 34, 35] and particularly for programs supporting mental health of adolescent male populations in community sports club settings [14, 39, 40].

Ahead of the Game (AOTG) is a comprehensive multi-level, multi-component program for adolescent males delivered through community sporting clubs and organisations [39] promoting early intervention for, and prevention of, mental illness and improved knowledge of and help seeking in use of social support networks and services. Sports clubs were chosen as an appropriate setting for the implementation of a mental health program for adolescent males because they provide access to a large proportion of the population in an engaging context. Over two-thirds of all adolescent males participate in organised sports in any given year [2], and on average spend over 8 h in organised sports each week [38]. Adolescent males also report that sports clubs are a motivating and engaging vehicle for the delivery of mental health literacy programs [37]. This is a view shared by both parents and coaches of adolescent sport participants [15, 16]. In addition, the Ahead of the Game program specifically targeted adolescent males because they are the group at highest risk of experiencing psychological difficulties, and are also the least likely to seek help [36]. For example, only 13.2% of young Australian men with a need for mental health services actually accessed the available services [36].

The aims of the program are to: (1) increase mental health literacy among adolescents and their social support systems; (2) increase help-seeking intentions and attitudes that facilitate help-seeking among adolescent male sport participants; and (3) increase resilience and factors which prevent the onset of mental health problems, including wellbeing and self-determined motivation. To address these aims, the program developed four components, two interventions for adolescents targeting mental health literacy and resilience, a parent mental health literacy intervention, and an intervention for sports club coaches which aims to help them facilitate self-determined motivation.

The AOTG study [39, 40] was designed in three phases to:

  1. (i)

    develop an AOTG health promotion intervention strategy in sports club community settings that aimed to maximise club, team and participant ownership (phase one);

  2. (ii)

    test and optimise implementation strategies to maximise community engagement and effectiveness while minimising implementation costs (phase two); and,

  3. (iii)

    prospectively compare the optimised AOTG implementation strategy developed in the first two phases relative to usual practice at a community level evaluating across joint incremental effects and AOTG implementation costs (phase three).

The first and second phases are important in developing community health promotion strategies for complex community settings, given strategies with community ownership both enable program integration into practice and optimise potential for community level multiplier impacts over time and across community networks [7, 9, 17,18,19,20, 28, 34, 35]. Unlike individual based therapies (medications, hospital procedures etc.), community health promotion strategies where they have community ownership provide potential for community population level impacts to continue to grow in their population scope and duration of impact with integration into community practice, behaviours and lifestyle. Consequently, the community ownership and integration of health promotion strategies are key factors in assessing whether health promotion strategies and interventions in community settings are effective and cost effective at a population level in practice [7].

While optimisation of the AOTG model and the efficacy of its implementation strategy is reported elsewhere [40], this paper focuses on within study matched community level evaluation in phase three. That is, evaluation of the optimised AOTG implementation strategy relative to usual practice, with joint evaluation and triangulation of evidence across multiple effects alongside AOTG implementation costs and consideration of community level ownership, potential for broader effects and associated health system cost offsets. We use the term ‘usual practice’ instead of the term ‘usual care’ to describe the experience of participants within the control group because no explicit care regarding mental health is usually given within recreational sporting clubs. Sport participants were exposed to the standard practice of the sporting clubs to which they belonged during the study period.

Methods

Evaluation of the AOTG study as a health promotion strategy has been designed [39] to jointly consider effect measures for intervention and control arm effects across 5 domains and 13 dimensions: global mental health (2 dimensions), resilience and preventative factors (3 dimensions), mental health literacy (4 dimensions), help seeking intentions (2 dimensions) and joint sport and mental health (2 dimensions). Variables in the global mental health domain reflect current mental health, while resilience, mental health literacy and help seeking intentions are constructs for protective factors predictive of future mental health [4, 22, 31]. The mental health sport domain captures sport specific dimensions associated with mental health. The coverage of domains, their associated dimensions measured in the AOTG study and dimension score ranges are shown in Table 1.

Table 1 AOTG evaluation mental health domains, variable dimensions and score ranges
Full size table

To estimate joint incremental effects between AOTG and usual care matched communities observed changes in pre-post effects was undertaken with difference of differences analysis [21]. Communities were matched on ARIA (Accessibility Remoteness Indicator Australia) location, geographic size, socioeconomic status, number of regions sporting clubs and participating sports. Participants were recruited from within those communities, while nested at a club and team level within their communities. Adolescent males aged 12–17 years, who participated in any community-based non-elite organised sports club from within the intervention or control communities were eligible for participation. Parents/caregivers and coaches of any adolescent participants were also invited to participate. All sports and clubs were eligible regardless of sport played. Eligible teams included those competing in the ‘Under 13′ through to ‘Under 18′ age groups. Details of the final sample can be found elsewhere [40], as can details of the recruitment and engagement strategies [39]. All participating clubs in the control community were offered a mental health literacy program following completion of the study.

To robustly consider net incremental effects and undertake health economic analysis for community based health promotion interventions such as the AOTG program requires flexible methods for triangulating evidence [41] which respectively cover and facilitate: (i) Triangulation of prospective quantitative and qualitative evidence of program community level incremental pre-post effect change between randomised or matched communities [7]; and (ii) Robust evaluation under uncertainty of joint incremental effects with multiple dimension and domain comparisons across populations, and alongside costs for incremental cost effectiveness analysis [25, 26].

Both are required for joint decision analytic coverage and comparability principles to be met and avoid expected biases for decision making that otherwise arise [6].

In relation to comparability, the AOTG study was prospectively designed to combine pre-post and matched community level evidence of AOTG effects relative to usual practice across dimensions at club, team and individual level, reflecting participant effects and community level evidence of program ownership and integration into club policies and practice [39]. Quantitatively, measuring each dimension’s score for AOTG and usual practice control arms with changes relative to baseline enables a primary mechanism to control for potential population confounders of differences between AOTG and usual practice matched communities that matching fails to control for.

In relation to coverage the net benefit correspondence theorem (NBCT) has been shown to have unique advantages in enabling flexible while robust joint multiple effect and multiple strategy cost effectiveness analysis under uncertainty. Theses advantages arise with radial properties in framing effects from a disutility (DU) bearing perspective and considering them alongside cost [6, 10,11,12,13, 25, 26]. Never-the-less previous NBCT methods in orthogonal cost-disutility (C-DU) space have been limited to 3-dimensional representation in joint consideration of 2 DU framed effects alongside costs. Hence the previous NBCT multi-dimensional cost effectiveness methods needed to be extended to allow robust joint evaluation of the AOTG studies 13 effect dimensions alongside implementation costs.

To enable robust joint comparison of 13 effect dimensions alongside costs we extend the orthogonal NBCT methods to enable joint comparison and evidence triangulation across more than 3 dimensions with use of non-orthogonal radar plots in C-DU space, while retaining the natural radial direction for comparing performance across dimensions. Joint comparison of cost and these multiple DU framed effects between AOTG and usual practice can then be robustly undertaken with radar plots in C-DU space where performance and net benefit unequivocally improve in contracting towards the origin.

In practice for the AOTG study health economics evaluation this simply involved framing observed pre-post changes in effects in AOTG and usual practice matched communities from a DU perspective as:

  1. (i)

    Psychological distress;

  2. (ii)

    Reduction in mental health;

  3. (iii)

    Reduction in resilience;

  4. (iv)

    Reduction in perceived familial support;

  5. (v)

    Social Distance;

  6. (vi)

    Reduction in confidence in seeking mental health information;

  7. (vii)

    Depression literacy deficit;

  8. (viii)

    Anxiety literacy deficit;

  9. (ix)

    Reduction in confidence about where to seek mental health information;

  10. (x)

    Reduction in Intention to seek help from informal sources;

  11. (xi)

    Reduction in Intention to seek help from formal sources;

  12. (xii)

    Reduction in athlete engagement deficit; and

  13. (xiii)

    Athlete burnout.

Comparing between arms across 13 dimensions, relative disutility effect measures and cost are, as with previous NBCT methods, best presented using flexible axes for each dimension [11, 25, 26]. That is, for each strategy, each of their multiple disutility framed effects or cost dimensions can be measured relative to the lowest disutility or lowest cost strategy in comparing across strategies mean DU effect measures, or across replicates in uncertainty analysis.

These flexible axes ensure that each strategy for each dimension (DU effects measures or cost) have non-negative measures, the same common comparator and unequivocally improve when reducing, enabling key radial contraction properties in comparing relative performance improvement. Where a strategy is the best strategy for that dimension with lowest disutility or cost across strategies they will have a 0 value, while otherwise they will have an incremental DU value or cost greater than 0.

For the AOTG study analysis where variables represent pre-post change scores for the AOTG and control arm, change scores from a utility bearing perspective are simply reframed to a disutility bearing perspective as the negative of their change value from a utility bearing perspective. Hence, for example, resilience reduction for each strategy are simply the negative of resilience improvement. In comparing between strategies in DU space with flexible axes comparison for each DU axis will be relative to the strategy with lowest disutility strategy. If AOTG had a higher improvement in resilience than control or equivalently a lower incremental resilience deficit then presented with flexible DU axes incremental change in resilience deficit will be relative to that of AOTG. For AOTG incremental resilience deficit would then be relative to itself as the most effective strategy and hence have a 0 value, while the control arm has an incremental resilience deficit greater than 0 reflecting the improvement possible.

Method for standardising absolute DU (effect) difference scores across dimensions

In interpreting incremental difference across the AOTG study Table 1 shows that the 13 effect dimensions vary considerably in their potential score ranges. Consequently, while dimension score changes can be presented as raw differences, interpreting differences in scores across dimensions or their triangulation and aggregation requires standardising for different absolute score ranges across dimensions. In comparing across the 13 dimensions incremental pre-post and case-control adjusted differences between arms, standardised incremental scores allowing for different score ranges across dimensions are calculated proportional to each variables score range. Standardised incremental differences are calculated for each dimension relative to that variables score range as:

$$ \frac{Raw\ score\ difference}{variable\ range}\times 100 $$

Hence for example if there were a 2 point improvement on:

  1. (i)

    the psychological distress dimension with a variable range of 24 (0–24) this would correspond to a standardised incremental improvement of \( \frac{2}{24}\times 100=8.33; \)

  2. (ii)

    the social distance dimension with a variable range of 4 (1–5) this would correspond to a standardised incremental improvement of \( \frac{2}{4}\times 100=50.00 \).

For incremental effects of any dimension this standardisation is simply interpreted as a percentage change relative to that dimensions range, while also enabling a robust basis for triangulation and simple aggregation across the 13 effect dimensions to joint consideration of domain and global levels of effect. Triangulation in aggregating from standardised dimensions to standardised domain level and from domain level to overall global effects in the base case analysis employ simple and intuitive equal weighting across dimensions to domain level effects and from domains to global effects. At a dimension or more aggregate level such simple standardisation retains advantages in representing the average percentage change and nested structure in aggregating from dimension to domain and global measures.

A priori an alternative form of standardisation in interpreting incremental effects across dimensions such as relative to population standard deviations in each case would in contrast create a black box to the extent they loose direct interpretability relative to instruments underlying each dimension in their aggregation to domain and or global levels as well as in comparison beyond the study population. Never-the-less directly interpretable alternative structural aggregation from across dimensions to a global levels is undertaken in sensitivity analysis, in triangulating evidence directly from a dimension level to a global level as the average of dimensions, ignoring domain level of analysis.

AOTG resource use and costing methods

In providing and implementing the AOTG intervention, resources used for staff training and implementation time and disposables were measured at a club, team and individual participant level. Project staff time included time:

  1. (i)

    for training (6 h);

  2. (ii)

    attending meetings (12–15 h per staff member);

  3. (iii)

    to invite a club to participate and undertake the recruitment process (2 h per club);

  4. (iv)

    to recruit and deliver coach education sessions run at a club level (variable time per club);

  5. (v)

    to inform and recruit team participants (variable time per team) and;

  6. (vi)

    to deliver the AOTG intervention to teams (variable time per team).

Staff costs were calculated applying hourly wage rates including on-costs to each staff member’s time attributed to the club and team level where they accrued.

Staff program level costs were equally distributed across the clubs for which the staff member was responsible. Staff costs attributable to a club level included time taken to invite a club to participate and undertake the recruitment process and deliver the coach education sessions which were run at a club level. These club level staff costs were then added to disposable cost of banners and posters displayed within clubs to estimate total club costs which were in turn distributed equally between clubs participating teams. Direct team level costs included the time taken to inform and recruit team participants and to deliver the intervention to teams. Participant level costs covered the disposable merchandise package provided to each participant and was microcosted.

Total costs for each AOTG participant added the disposable per participant costs to club and team level costs, which were conservatively attributed down only to participants who provided any pre-post effect change data consistent with analysis on effects. While this is conservative in attributing AOTG costs from a team and club level down to a participant level it appropriately includes all AOTG club and team level costs in undertaking both cost effectiveness analysis and bootstrapping under uncertainty across individuals who had any pre-post change in effect data across dimensions, consistent with evaluation of effects.

Robust bootstrapping method allowing for joint uncertainty with missing data

To enable robust analysis under uncertainty of joint incremental effects across 13 dimension (and 5 domain) effect measures with different levels of missing data across individuals by dimension, bootstrapping was undertaken on joint participant level effects and AOTG costs to support consistency and retaining covariance between effects and costs in synthesising evidence [3]. Given multiple effect measures of pre-post change with different levels of missing data and an inability to structurally undertake imputation at club, team and individual level a robust bootstrapping approach was developed which allowed inclusion of all observed effect and associated conservatively allocated AOTG implementation cost data at an individual level.

Hence, robust bootstrapping across individuals’ joint effects and cost required retaining all observed evidence while allowing for the appropriate numerator and denominator in each replicate for each variable. This was achieved separately tracking numerator and denominator sums for each of the joint 13 effect dimensions alongside intervention costs for the AOTG arm across bootstrap populations, resampled with replacement in each replicate in an inner loop for AOTG; while repeating this process 1000 times in the outer loop. The same procedure was undertaken in the control arm for the 13 effect dimensions.

Specifically, the bootstrapping method randomly resampled with replacement for each of 1000 replicates with the same number of individuals as in the study in each arm who had any change in effect data. This included 343 participants in the AOTG arm and 273 in the control arm for base case analysis, while a football only sensitivity analysis was undertaken in a somewhat more restricted population.

Sensitivity analysis comparing AOTG vs control in football clubs alone

Base case analysis compares pre-post effects in AOTG relative to control communities over the same winter season in 2017. Clubs, their teams and participants approached over that season within these communities were predominantly football (soccer) clubs. Never-the-less 30/343 participants in the AOTG arm were from a swimming and a rugby league club while in the control arm an Australia rules football and a basketball club contributed 13/273 participants. While analysis of pre-post effects can control for baseline differences that might arise, different sports potentially have different amenity to mental health programs for adolescent males. To allow for this potential a sensitivity analysis restricting analysis to football clubs (called soccer clubs in Australia), teams and participants alone was undertaken to see whether the inclusion of a limited number of participants from other sports clubs in baseline analysis may have influenced results.

Results

Pre-post DU changes across the 13 mental health dimensions for the AOTG program and usual practice arms are shown in Fig. 1a. Across strategies performance with disutility framed effects intuitively improves in moving for any dimension directly (radially) towards the origin. Hence in Fig. 1a, AOTG has lower DU (equivalently higher effect) that usual practice on each of the 13 dimensions across 5 domains.

Fig. 1
figure 1

a: Pre-post changes in DU effects across 13 dimensions for AOTG and usual practice. b: Incremental DU effects with flexible axes relative to lowest DU (highest effect) strategy across 13 dimensions for AOTG and usual practice. c: Standardised incremental DU with flexible axes relative to lowest DU (highest effect) strategy across 13 dimensions

Full size image

This is more clearly seen in relative terms between arms across dimensions in Fig. 1b where incremental effects are presented with flexible axes measured relative to the lowest DU (highest effect) strategy on each dimension. Using these flexible axes the AOTG strategy is highlighted as the most effective (lowest DU strategy) on each dimension and hence remains at the origin with 0 incremental DU relative to itself the most effective strategy. The usual care program with current practice has positive incremental DU on each dimension relative to the AOTG program and hence has incremental DU relative to AOTG as the lowest DU (most effective) strategy on each of the 13 dimensions.

The absolute pre-post differences between arms presented in their natural scoring units for each of the 13 dimensions in Fig. 1b are standardised in Fig. 1c to allow interpretation of the differences in strategies as the percentage change relative to dimension score range.

In considering aggregated triangulation to standardised domain scores across dimensions, range standardised scores across domains as per Fig. 1c provide a common standard basis for combining dimensions to a domain level. As described in methods a priori the simplest form of aggregated domain level standardised incremental comparison is to equally weight range standardised incremental dimension scores from Fig. 1c within each domain. Standardised domain level incremental differences then represent the average percentage change across relevant standardised dimension score ranges, as shown in Fig. 2.

Fig. 2
figure 2

Standardised domain level incremental DU effects with flexible axes relative to lowest DU strategy for the AOTG and usual practice arms

Full size image

Figures 1 and 2c show that for each of the 13 mental health effect dimensions and their aggregation to 5 domains the AOTG study arm has lower expected disutility or equivalently higher expected effect than usual care. In standardised terms, the greatest expected improvement with AOTG is an incremental improvement of almost one tenth of the range or 9.92 points over a 100 point standardised range for mental health literacy domain, followed by 5.75 points for resilience and preventive factors and 4.55, 4.35 and 3.32 points for help seeking intentions, mental health and sport and global mental health domains, respectively.

Whether the dimensions (Fig. 1c), domains (Fig. 2) or weighted global measures from dimension level are statistically significant requires robust joint consideration across dimensions under uncertainty. Bootstrapped analysis allowing for uncertainty across the 13 effect variable dimensions jointly is summarised in Table 2 for both incremental raw scores changes and standardised score changes adjusting for different score ranges for dimensions.

Table 2 Mean and bootstrapped 95% CIs for raw and standardised incremental effect difference for AOTG vs control effect across 13 dimensions
Full size table

Table 2 shows that at a dimension level the mean incremental (AOTG vs control) and pre-post change controlled change in effect favoured AOTG in all dimensions, statistically significant at a 5% level of type I error for 10/13 if they had been considered partially. Key to robust coverage and comparability is triangulation for joint consideration across effects under uncertainty. Table 3 triangulates across joint dimension level evidence with simple (equal dimension and/or domain weighted) aggregation to estimate incremental (AOTG vs control) and pre-post change controlled range standardised domain and global effects (for a base case domain aggregation and sensitivity analysis direct dimension aggregation) and their bootstrapped distributions under uncertainty.

Table 3 AOTG vs Control pre-post change controlled and range std. mean domain and global effects with parametric bootstrapped 95% CIs
Full size table

In interpreting domain level incremental findings (Fig. 2), Table 3 shows that standardised differences as a percentage of their scales range are significant with a lower than 1% level of type I error for each domain, indeed less than 0.1% for resilience and preventative factors and less than 0.001% for mental health literacy. Aggregate global effect measures for a base case with equal weights for domains and dimensions within domains or sensitivity analysis with equal weights for all dimensions ignoring domains are also both significant at a < 0.001% level.

The very high levels of certainty of standardised domain and global incremental effect change benefits for AOTG relative to control reflect orders of magnitude less variability within and between arms than for their equivalent individual dimensions and that across dimensions in each case the mean incremental change between arms favoured AOTG, while 10/13 significantly at a 5% level. Hence, triangulation with joint aggregate consideration of effects is key, given partial consideration of a single effect would have not adequately covered effects, lead to greater uncertainty and effectively a lottery (10/13 chance) as to whether AOTG was considered effective depending on which dimension was considered.

Such triangulation is particularly key in assessing joint evidence from health promotion strategies [41] in community settings, and more generally in satisfying coverage conditions for robust health economic analysis across community and individual level strategies and interventions [7], as considered in discussion. While triangulated evidence makes clear the statistically significant incremental improvement across domains or globally and hence net clinical benefit with AOTG relative to usual care effects changes at a community level, to move towards net benefit consideration implementation costs of the AOTG program also need to be considered alongside potential for downstream costs impacts associated with effects.

Resource use and cost results

The AOTG strategy was implemented across 8 clubs and 22 teams utilizing 6 AOTG staff as facilitators with the amount of staff time per club ranging from 10.5 to 35 h (mean 24.5 h), the time per team ranging from 5.25 to 14.5 h and the total staff time per participant ranging from 0.31 to 0.97 h. Including all on-costs, staff wages ranged from $40.74 to $47.11 per hour, depending on salary level. The mean hourly cost for staff members was $43.79 (SD = $3.49).

Club level mean cost was $1223.05 (SD = $478.26) and ranged from $556.79 to $1773.11. When attributed down to the team level, costs ranged from $278.39 to $779.13 per team with mean of $473.83 (SD = $162.91). When those costs were attributed down to an individual level and microcosted merchandise costs of $8.94 per participants were added, AOTG implementation cost per participant ranged from $24.85 to $64.43, with mean cost of $37.47 (SD = 10.44).

The mean AOTG cost per participant across 8 clubs and 22 teams of $37.47 were in turn attributable to club level costs (staff costs of club liaison and training and promotional materials) of $21.03 (range $8.05 to $45.35), team level costs (staff costs associated with presenting AOTG strategies to each team) of $7.49 (range $4.42 to $13.58) and participant level costs (merchandise cost) of $8.94, as shown in Table 4. Joint consideration of AOTG implementation costs alongside standardised effect differences for 13 domains are presented in Fig. 3.

Table 4 AOTG attributable implementation costs by team per participant for club, team and individual level (merchandise) costs
Full size table
Fig. 3
figure 3

Incremental cost and 13 standardised incremental DU effects with flexible axes relative to lowest cost and lowest DU (highest effect) strategy

Full size image

Interpreting Fig. 3 the AOTG program is shown to have the lowest expected DU and hence greatest benefit for each effect dimension or domain while direct implementation costs of the AOTG program are $37.47 per person.

Sensitivity analysis with football only comparison AOTG vs control

Sensitivity analysis restricting participants to those in football clubs and teams removed 30 participants in the AOTG arm (14 from a swimming club and 16 from a NRL club) and 13 participants in the control arm (8 from a basketball club and 5 from an AFL club). Hence, a football (soccer) only comparison (Table 4) is based on 313 AOTG and 260 control arm participants.

Comparing football (soccer) only participants, each of the 13 effect dimensions remain in favour of AOTG over control with higher mean effect change or equivalently lower disutility, 11/13 statistically significant with 5% type I error and cost of implementing AOTG per participant reduced to $36.57 per participant (Table 5).

Table 5 Sensitivity analysis with football only participants and AOTG vs control raw and range standardised effects across 13 dimensions and implementation cost
Full size table

Hence, sensitivity analysis restricting comparison between AOTG and usual practice to football only clubs, teams and participants further reinforces the findings of AOTG being both an effective and low cost mental health promotion strategy.

More generally, joint consideration of effects and their associated long term health system cost implications, whether in base case or sensitivity analysis in Fig. 3 and Tables 4 and 5, point to the potential for long term net cost savings if there were community ownership of AOTG as health promotion strategy, as considered in greater detail in discussion.

Discussion

This paper has undertaken health economic multi-dimension cost effectiveness analysis in the third phase of the AOTG study evaluating the Ahead of the Game mental health promotion strategy for adolescence males and its implementation in practice in community sporting club settings developed in the first two phases [39, 40]. Methods have been developed to enable robust evaluation of the AOTG strategy and its combined implementation impacts in practice triangulating on evidence across 13 mental health dimensions, 5 domains and AOTG implementation costs.

Within study triangulated analysis of jointly estimated differences of effects under uncertainty has demonstrated that the AOTG strategy is associated with mental health improving across all 13 dimensions with 10/13 significantly at 5% level type 1 error, across all 5 mental health domains (less than 1% level type 1 error in each case) and with simply weighted overall aggregate measures from domain or dimension level (less than 0.001% level type 1 error in each case), while having conservatively estimated per participant expected implementation cost of $37.47.

Extrapolating beyond the within study analysis presented in this paper, AOTG would be net cost savings to the health system in the long term if the net present value of cost saving to the health system from downstream improved mental health effects were greater than $37.47 per participant. That is, from a long-term health system perspective integration of AOTG program into clubs would be expected to dominate current practice if the net present value of downstream cost savings associated with improved effects were $37.47 per individual or greater. If there is long term community ownership of AOTG community strategies the downstream potential cost savings from successful community level health promotion and disease prevention strategies such as AOTG are significant given community network and multiplier effects that are then expected to arise in increasing population scope and duration of effects [7, 9, 17, 19, 20, 34, 35].

That is, combined population improvements with AOTG across the mental health literacy domain by almost one tenth of its range (9.92/100), resilience and preventative factors by 5.75/100, health seeking intentions by 4.55/100, Mental health and sport by 4.35/100 and global health by 3.32/100 would be expected to lead to long term population level cost savings where the AOTG program had sports community (club, team and participant) ownership with policy and practice integration. The development of the AOTG strategy and model of implementation to optimise ownership, implementation and integration in phase one and two of the AOTG study alongside participant dimensions and domains of effect that reflect community ownership and help seeking intentions support such potential cost savings with AOTG.

Extending within study analysis in Fig. 3 to such longer-term consideration of cost and effect dominance of AOTG would be simply represented in C-DU space with incremental cost and each effect framed from a disutility (DU) perspective for AOTG being at the origin. Importantly then, the AOTG strategy has significant potential to be net cost saving as well as effective as a community health promotion strategy with scope for multiplier effects across community networks and with pricing at factor costs. This potential is precluded with individual level interventions with ‘value based’ pricing such as pharmaceuticals both because they are priced up to ‘value thresholds’ and as interventions targeted at individuals do not enable multiplier effects beyond that target population [7, 8, 29, 30].

Never-the-less, while beyond the scope of this paper, to better assess and model the potential for population level community level effect of community health promotion strategies such as AOTG and the extent of associated downstream health system cost savings quantitative analysis would ideally assess community multiplier and more generally network effects over time [7, 9, 20, 34]. Future research in that respect would be valuable in quantifying the extent of long-term effects and downstream cost savings with AOTG as with any community level health promotion strategy. Such research is more generally key to identifying effective and net cost saving community health promotion strategies that health systems should be prioritising for investment [33].

Overall, AOTG study findings demonstrate an inexpensive health promotion strategy which in addition to having expected benefits in promoting mental health having significant potential to be net cost saving to the health system relative to current practice with integration into sporting clubs and organisation supported by community ownership.

Study strengths and weaknesses

One aspect of that study analysis that could be considered a limitation is a lack of consideration of multiple effect adjustments. However, any such conventional consideration of multiple effect adjustment only serves to reinforce the central finding of the study of the perils of partialisation with fragmented consideration of multiple effects and key importance of the robust methods illustrated for jointly considering effects. For example, with the most conservative conventional Bonferroni correction for multiple comparisons the appropriate p value with a one sided test for a global level is p < 0.05, while at a domain level it would be p < 0.01 = 0.05/5, and a single effect level p < 0.0076 = 0.05/13 or half that with a 2 sided test. Hence, undertaking multiple comparisons the overall global improvement with AOTG remains highly significant (p < 0.00001), while each of the domains remain significant with two or one sided tests. However at an individual effect level only 6 of 13 would remain significant with a multiple effect adjusted two sided test (wellbeing, resilience, dealing with problems, depression literacy, anxiety literacy) and 7/13 with a one sided test (athlete burnout also significant). Consequently, multiple effect comparison methods reinforce the perils of partialisation, making even clearer the lottery that conventional partial consideration of effects leads to and the need for their joint consideration.

More generally, robustly informing public sector decision making with health economic analysis requires evidence synthesis with joint consideration of decision analytic principles of coverage and comparability to avoid biases in estimating incremental effects and costs and consequently undertaking cost effectives or net benefit analysis [6].

Coverage alongside comparability is particularly key for evaluating health promotion strategies such as AOTG in community settings where triangulation across quantitative as well as qualitative evidence for individual and community (e.g. team, club and wider community) ownership, attitudes, lifestyle and behaviour factors are key to continuation of the program, expected individual effects, multiplier effects and subsequently the effectiveness and cost effectiveness of the strategy [7, 9].

In evaluating AOTG as a health promotion strategy triangulation across community level pre-post matched evidence for each of 5 domains and 13 dimensions were each important to assessing the success and net effect of the program. The extended NBCT methods presenting multiple effects with radar plots in C-DU space and bootstrapping methods for triangulating evidence have been illustrated to combine and summarise evidence in enabling robust joint consideration of AOTG multiple effects alongside implementation costs under uncertainty.

Importantly the adapted and extended NBCT methods enable informed decision making with intuitive joint comparison of multiple effects alongside costs consistent with maximising net benefit with natural improvement where cost or effects presented from a disutility perspective are minimised. By jointly presenting dimension level analysis such as that in Fig. 1b and c prior to any aggregated analysis such as in Fig. 2 and Table 3 under uncertainty the method also flexibly enables alternative weightings in aggregating to domain or global level to reflect preferences across effects of any given jurisdiction at any point in time. Hence, while triangulation to domain and global level in Fig. 2 and Table 3 a priori employed equal dimension and/or domain weights, if alternative weights reflecting preferences in any given jurisdiction across dimensions and or domains were known, they naturally can be simply applied.

In relation to comparability, for community level evidence of interest in evaluating health promotion strategies ideally randomising allocation to AOTG or usual care across many communities would be undertaken. Never the less as a prospective study of community level effects where trial funding limited the feasibility of randomising across communities, undertaking differences in differences analysis triangulating between pre-post and matched community level evidence represented the strongest feasible quantitative study design. Undertaking pre-post and matched analysis together addresses many of the weaknesses of undertaking each separately. Pre-post analysis i.e. controlling for baseline effects addresses limitations in controlling for population differences between arms. Undertaking matched analysis over the same timeframe addresses the weakness of pre-post analysis in not controlling for impacts of external environment beyond the study intervention across intervention communities e.g. legislative or peak sporting body policy changes arising across communities and their clubs over time. Naturally, further research randomising between the AOTG implementation strategy and other implementation strategies across multiple communities would be valuable to further improve comparability, but should not be at the expense of compromising coverage.

In relation to coverage, adapting and extending NBCT methods with radar plot presentation in C-DU space and triangulation under uncertainty with bootstrapping methods have been shown to facilitate robust joint consideration across multiple effect domains alongside AOTG implementation cost. Importantly radar plot presentation in C-DU space retains the key advantages of the NBCT in enabling radial comparison consistent with maximising net benefit [10,11,12,13, 25, 26] while extending multiple effect presentation and triangulated uncertainty analysis to more than 3 dimensions. Like previous multiple domain of effect health economic research methods [25, 26], health economic analysis undertaken in this paper extends deterministic cost consequences and single effect cost effectiveness categories to enable robust multiple domain cost effectiveness analysis.

In many complex community settings such as health promotion or palliative care key incremental effects cannot be integrated into a single effect such as patients’ quality-adjusted life years (QALYs) and hence health economic analysis to robustly inform decision making requires coverage of multiple effects alongside costs under uncertainty, to enable robust unbiased analysis [25,26,27]. For example, allowing for palliative patient primary domains of finalising affairs, carer and family effects and being in their community of choice with who they want to be with; each of which cannot be integrated with patient survival in patient QALY analysis [27]. Similarly, in community health promotion settings such as this study, multiple dimensions across mental health resilience and preventative factors, help seeking intentions, use of network support and community ownership and network impacts at a sporting organisation, club and team level cannot be integrated into a single effect measure.

Unlike previous applications of the NBCT to comparison of multiple effects with orthogonal presentation in cost-disutility space, comparison in cost disutility space for AOTG intervention relative to usual care across 13 effects and cost shown in this paper employ non-orthogonal methods. These non-orthogonal methods enable both simple and feasible joint presentation of the AOTG studies 13 effects and costs data not possible with orthogonal methods, and for deterministic analysis can graphically summarise all data in one diagram.

The non-orthogonal methods presented here are otherwise more constrained than orthogonal methods in scope of analysis for informing societal decision making allowing for joint effect and cost uncertainty and in extending to summary measures such as expected net loss curves, frontiers, planes and surfaces [1, 11, 12, 25, 26]. Hence, use of orthogonal methods to complement analysis presented in this paper are suggested for further research in extending associated summary measures. Never-the-less, the joint consideration of 13 effects and costs and hence 14-dimensional analysis in evaluating the AOTG strategy as a health promotion strategy extends dimensionality of multiple domain cost effectiveness analysis from three-dimensional analysis previously considered employing orthogonal NBCT methods in palliative settings.

Robust methods for undertaking multiple domain cost-effectiveness analysis are important to allow appropriate coverage in many areas where multiple effect domains cannot be collapsed into a single effect measure such as QALYs. Indeed, even for cases where QALYs enable a single effect measure integrating survival and quality of life aspects, the weights of multiple effects and dimension contributing to QALY measurement are rarely transparent and can be a black box for decision makers. Consequently, further research is also suggested as valuable in extending the multiple dimension evaluation methods presented here to enable transparent QALY analysis. That is, to allow explicit joint consideration of multiple dimensions and domains of effects in QALY analysis where weights can differ with different preference sets across jurisdictions but also over time.

Conclusion

The methods identified for undertaking multiple effect comparison and cost effectiveness analysis for the AOTG study in this paper have adapted and extended robust net benefit correspondence theorem methods enabling joint presentation with radar plots of 13 effect dimensions alongside costs in C-DU space and their triangulation under uncertainty with robust bootstrapping methods. Employing these methods statistically significant benefits were found for AOTG effect changes incremental to usual care on 10/13 dimensions and all 5 domains (p < 0.01) or global measures (p < .0001) with simple dimension and/or domain aggregation. This analysis demonstrates AOTG effectiveness as a mental health promotion strategy and points to AOTG dominating current practice if down streaming cost savings associated with better mental health management are greater than the estimated $37.47 direct costs per individual of implementing the AOTG program. The multidimensional extension of net benefit correspondence theorem methods with radar plot presentation for their joint consideration in cost-disutility space, triangulation and bootstrapping methods developed and illustrated for the AOTG study have distinct advantages over cost-consequences analysis in flexibly allowing for joint cost and multiple effect dimensions and domains comparison and their aggregation and triangulation under uncertainty consistent with net benefit maximisation. This approach improves triangulation and coverage of multiple domains key for evaluation in settings such as health promotion without compromising comparability and avoid perils of partialisation that otherwise arise with multiple single effect cost effectiveness analyses.

Availability of data and materials

The datasets analysed during the current study are available from the corresponding author on reasonable request.

Abbreviations

AOTG:

Ahead of the Game (intervention and study)

C-DU space:

Cost-disutility space

DU:

Disutility (effect measure and effect framing)

NBCT:

Net benefit correspondence theorem

QALY:

Quality adjusted life year

References

  1. Alarid-Escudero F, Enns EA, Kuntz KM, Michaud TL, Hawre J. “Time traveling is just too dangerous” but some methods are worth revisiting: the advantages of expected loss curves over cost-effectiveness acceptability curves and frontier. Value Health. 2019;22(5):611–8 https://doi.org/10.1016/j.jval.2019.02.008.

    Article  PubMed  PubMed Central  Google Scholar 

  2. Australian Bureau of Statistics. Participation in Sport and Physical Recreation, Australia, 2013-2014. Canberra: Australian Bureau of Statistics; 2015.

    Google Scholar 

  3. Briggs AH, O’Brien BJ, Blackhouse G. Thinking outside the box: recent advances in the analysis and presentation of uncertainty in cost-effectiveness studies. Annu Rev Public Health. 2002;23(1):377–401 https://doi.org/10.1146/annurev.publhealth.23.100901.140534.

    Article  PubMed  Google Scholar 

  4. Davydov DM, Stewart R, Ritchie K, Chaudieu I. Resilience and mental health. Clin Psychol Rev. 2010;30(5):479–95 https://www.sciencedirect.com/science/article/pii/S0272735810000437. https://doi.org/10.1016/j.cpr.2010.03.003.

    Article  PubMed  Google Scholar 

  5. Durlak JA, Dupre EP. Implementation matters: a review of research on the influence of implementation on program outcomes and the factors affecting implementation. Am J Community Psychol. 2008;41(3-4):327–50 https://doi.org/10.1007/s10464-008-9165-0.

    Article  PubMed  Google Scholar 

  6. Eckermann S. Health economics from theory to practice: optimally informing joint decisions of research, Reimbursement and Regulation with Health System Budget Constraints and Community Objectives. London: Springer; 2017. https://doi.org/10.1007/978-3-319-50613-5. https://www.springer.com/gp/book/9783319506111

    Book  Google Scholar 

  7. Eckermann S, McCaffrey N. Beyond the Individual: Evaluating Community-Based Health Promotion and Prevention Strategies and Palliative Care. Chapter 4, p. 91–106. In: Health Economics from Theory to Practice: Optimally Informing Joint Decisions of Research, Reimbursement and Regulation with Health System Budget Constraints and Community Objectives. London: Springer; 2017. https://doi.org/10.1007/978-3-319-50613-5.

    Chapter  Google Scholar 

  8. Eckermann S, Pekarsky B. Can the real opportunity cost stand up: displaced services the straw man outside the room. PharmacoEconomics. 2014;32(4):319–25 http://www.springerlink.com/openurl.asp?genre=article&id=doi:10.1007/s40273-014-0140-3.

    Article  PubMed  Google Scholar 

  9. Eckermann S, Dawber J, Yateman H, Quinsey K, Morris D. Evaluating return on investment in a school based health promotion and prevention program: the investment multiplier for the Stephanie Alexander Kitchen Garden National Program. Soc Sci Med. 2014;114:103–12 https://doi.org/10.1016/j.socscimed.2014.05.056.

    Article  PubMed  Google Scholar 

  10. Eckermann S, Coelli T. Including quality attributes in efficiency measures consistent with net benefit: creating incentives for evidence based medicine in practice. Soc Sci Med. 2013;76(1):159–68 https://doi.org/10.1016/j.socscimed.2012.10.020.

    Article  PubMed  Google Scholar 

  11. Eckermann S, Willan A. Presenting evidence and summary measures to best inform societal decisions when comparing multiple strategies. PharmacoEconomics. 2011;29(7):563–77 https://doi.org/10.2165/11587100-000000000-00000.

    Article  PubMed  Google Scholar 

  12. Eckermann S, Briggs A, Willan A. Health technology assessment in the cost-disutility plane. Med Decis Mak. 2008;28(2):172–81 https://doi.org/10.1177/0272989X07312474.

    Article  Google Scholar 

  13. Eckermann S. Hospital performance including quality: creating incentives consistent with evidence-based medicine. Sydney: PhD dissertation, UNSW; 2004. http://unsworks.unsw.edu.au/vital/access/manager/Repository/unsworks:716

    Google Scholar 

  14. Finch CF, Donaldson A. A sports setting matrix for understanding the implementation context for community sport. Br J Sports Med. 2009;44:973–8.

    Article  PubMed  Google Scholar 

  15. Ferguson H, Swann C, Liddle SK, Vella SA. Investigating youth sports coaches’ perceptions of their role in adolescent mental health. J Appl Sport Psychol. 2019;31(2):235–52 https://doi.org/10.1080/10413200.2018.1466839.

    Article  Google Scholar 

  16. Hurley D, Swann C, Allen MS, Okely AD, Vella SA. The role of community sports clubs in adolescent mental health: the perspectives of adolescent males’ parents. Qual Res Sport Exer Health. 2017;9(3):372–88 https://doi.org/10.1080/2159676X.2016.1275751.

    Article  Google Scholar 

  17. Hawe P, Ghali L. Use of social network analysis to map the social relationships of staff and teachers at school. Health Educ Res. 2008;23(1):62–9 https://doi.org/10.1093/her/cyl162.

    Article  PubMed  Google Scholar 

  18. Hawe P, Shiell A. Social capital and health promotion: a review. Soc Sci Med. 2000;51(6):871–85 https://doi.org/10.1016/S0277-9536(00)00067-8.

    Article  CAS  PubMed  Google Scholar 

  19. Hawe P, Shiell A, Riley T. Complex interventions: how far ‘out of control’ should a randomised controlled trial be? Br Med J. 2004;328(7455):1561–3 https://doi.org/10.1136/bmj.328.7455.1561.

    Article  Google Scholar 

  20. Hawe P, Shiell A, Riley T. Theorising interventions as events in systems. Am J Community Psychol. 2009;43(3e4):267–76.

    Article  PubMed  Google Scholar 

  21. Husereau D, et al. Consolidated Health Economic Evaluation Reporting Standards (CHEERS) statement. BMJ. 2013;346:f1049 https://doi.org/10.1136/bmj.f1049.

    Article  PubMed  Google Scholar 

  22. Jorm AF. Mental health literacy: empowering the community to take action for better mental health. Am Psychol. 2012;67(3):231–43 https://doi.org/10.1037/a0025957.

    Article  PubMed  Google Scholar 

  23. Kokko S. Sports clubs as settings for health promotion: fundamentals and an overview of research. Scand J Public Health. 2014;42(Supp. 15):60–5.

    Article  PubMed  Google Scholar 

  24. Kokko S, Green LW, Kannas L. A review of settings-based health promotion with applications to sports clubs. Health Promot Int. 2013;29:494–509.

    Article  PubMed  Google Scholar 

  25. McCaffrey N, Agar M, Harlum J, Karnon J, Currow D, Eckermann S. Better informing decision making with multiple outcomes cost-effectiveness analysis under uncertainty in cost-disutility space. PLoS One. 2015;10(3):e0115544 https://doi.org/10.1371/journal.pone.0115544.

    Article  PubMed  PubMed Central  CAS  Google Scholar 

  26. McCaffrey N, Eckermann S. Multiple Effects Cost Effectiveness Analysis in Cost-Disutility Space. Chapter 10. In: Health Economics from Theory to Practice: Optimally Informing Joint Decisions of Research, Reimbursement and Regulation with Health System Budget Constraints and Community Objectives. Cham: Springer; 2017. https://doi.org/10.1007/978-3-319-50613-5.

    Google Scholar 

  27. McCaffrey N, Eckermann S. Raise the bar, not the threshold value: meeting patient preferences for palliative and end-of-life care. PharmacoEconomics Open. 2018;2(2):93–5 https://doi.org/10.1007/s41669-017-0039-y.

    Article  PubMed  Google Scholar 

  28. Moore S, Haines V, Hawe P, Shiell A. Lost in translation: a genealogy of the “social capital” concept in public health. J Epidemiol Community Health. 2006;60(8):729–34 https://doi.org/10.1136/jech.2005.041848.

    Article  CAS  PubMed  PubMed Central  Google Scholar 

  29. Pekarsky BAK. The new drug reimbursement game: a regulators guide to playing and winning. London: Springer; 2015.

    Book  Google Scholar 

  30. Pekarsky BAK. Trusts, constraints and the counterfactual: reframing the political economy of new drug price. Dissertation: University of Adelaide; 2012.

    Google Scholar 

  31. Rickwood D, Deane FP, Wilson CJ, Ciarrochi J. Young people’s help-seeking for mental health problems. Austr e-J Adv Mental Health. 2005;4(3):218–51 https://doi.org/10.5172/jamh.4.3.218.

    Article  Google Scholar 

  32. Sallis JF, Owen N. Ecological models of health behavior. In: Glanz K, Rimer BK, Viswanathan M, editors. Health behavior: theory, research, and practice. San Francisco: Jossay-Bass; 2015. p. 43–64.

    Google Scholar 

  33. Shiell A, Jackson H. How much does Australia spend on prevention and how would we know whether it is enough? Health Promot J Austr. 2018;29(Suppl 1):7–9 https://doi.org/10.1002/hpja.165.

    Article  PubMed  Google Scholar 

  34. Shiell A, Hawe P, Gold L. Complex interventions or complex systems? Implications for health economic evaluation. Br Med J. 2008;336(7656):1281–3 https://doi.org/10.1136/bmj.39569.510521.AD.

    Article  Google Scholar 

  35. Shiell A, Hawe P. Health promotion community development and the tyranny of individualism. Health Econ. 1995;5(3):241–7.

    Article  Google Scholar 

  36. Slade T, Johnston A, Oakley Browne MA, Andrews G, Whiteford H. 2007 National Survey of mental health and wellbeing: methods and key findings. Aust N Z J Psychiatry. 2009;43(7):594–605 https://doi.org/10.1080/00048670902970882.

    Article  PubMed  Google Scholar 

  37. Swann C, Telenta J, Draper G, Liddle SK, Fogarty A, Hurley D, et al. Youth sport as a context for supporting mental health: adolescent male perspectives. Psychol Sport Exerc. 2018;35:55–64 https://doi.org/10.1016/j.psychsport.2017.11.008.

    Article  Google Scholar 

  38. Vella SA, Cliff DP, Okely AD, Scully ML, Morley BC. Associations between sports participation, adiposity and obesity-related health behaviors in Australian adolescents. Int J Behav Nutr Phys Act. 2013;10(1):113 https://doi.org/10.1186/1479-5868-10-113.

    Article  PubMed  PubMed Central  Google Scholar 

  39. Vella S, Swann C, Batterham M, Boydell KM, Eckermann S, Fogarty A, et al. Ahead of the game protocol: a multi-component, community sport-based program targeting prevention, promotion and early intervention for mental health among adolescent males. BMC Public Health. 2018;18(1):390 https://bmcpublichealth.biomedcentral.com/track/pdf/10.1186/s12889-018-5319-7.

    Article  PubMed  PubMed Central  Google Scholar 

  40. Vella SA, Swann C, Boydell KM, Eckermann S, Fogarty A, Hurley D, et al. Sports-based mental health promotion in Australia: formative evaluation. Psychol Sport Exerc. 2019;45:101560 https://doi.org/10.1016/j.psychsport.2019.101560.

    Article  Google Scholar 

  41. Zaza S, Briss PA, Harris KW. The guide to community preventive service. In: What works to promote health? New York: Oxford University Press; 2005.

    Google Scholar 

Download references

Acknowledgements

We would like to thank and acknowledge all participants in the AOTG study.

Funding

The Ahead of the Game study was funded by the Movember Foundation through an Australian Mental Health Initiative grant. The project also received funding from the University of Wollongong Global Challenges Program. The findings and views reported in this paper are those of the authors and independent of funding organisations. SV is supported by a National Heart Foundation of Australia Postdoctoral Fellowship (100974).

Author information

Authors and Affiliations

  1. School of Health and Society, University of Wollongong, Wollongong, Australia

    Simon Eckermann

  2. Deakin Health Economics, Institute for Health Transformation, Deakin University, Geelong, Australia

    Nikki McCaffrey & Utsana Tonmukayakul

  3. Faculty of Health, Southern Cross University, Coffs Harbour, Australia

    Christian Swann

  4. School of Psychology, University of Wollongong, Wollongong, Australia

    Stewart Vella

Authors
  1. Simon Eckermann
    View author publications

    You can also search for this author in PubMed Google Scholar

  2. Nikki McCaffrey
    View author publications

    You can also search for this author in PubMed Google Scholar

  3. Utsana Tonmukayakul
    View author publications

    You can also search for this author in PubMed Google Scholar

  4. Christian Swann
    View author publications

    You can also search for this author in PubMed Google Scholar

  5. Stewart Vella
    View author publications

    You can also search for this author in PubMed Google Scholar

Contributions

SE contributed to conception and designing the AOTG health economic evaluation, undertaking analysis and interpretation of data, creating robust bootstrapping software, drafting and revising the manuscript. UT contributed to undertaking analysis and interpretation of data, creating robust bootstrapping software and in drafting and revision of the manuscript. NM contributed to designing the health economic evaluation, interpretation of data and in drafting and revision of the manuscript. CS contributed to acquisition and interpretation of data and revision of the manuscript. SV contributed to conception of the health economic evaluation of the AOTG study, acquisition and interpretation of data and in drafting and revision of the manuscript. All authors have read and approved the manuscript.

Corresponding author

Correspondence to Simon Eckermann.

Ethics declarations

Ethics approval and consent to participate

The AOTG study was approved by the University of Wollongong Health and Medical Human Research Ethics Committee (HE15/423) on 18th December 2015. All participants provided written informed consent to participate. Participants aged 15 years and under also received written informed consent from their parent or caregiver.

Consent for publication

NA. No consent was required for publication of statements from individuals.

Competing interests

The authors have no competing interests.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is licensed under a Creative Commons Attribution 4.0 International License, which permits use, sharing, adaptation, distribution and reproduction in any medium or format, as long as you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons licence, and indicate if changes were made. The images or other third party material in this article are included in the article's Creative Commons licence, unless indicated otherwise in a credit line to the material. If material is not included in the article's Creative Commons licence and your intended use is not permitted by statutory regulation or exceeds the permitted use, you will need to obtain permission directly from the copyright holder. To view a copy of this licence, visit http://creativecommons.org/licenses/by/4.0/. The Creative Commons Public Domain Dedication waiver (http://creativecommons.org/publicdomain/zero/1.0/) applies to the data made available in this article, unless otherwise stated in a credit line to the data.

Reprints and permissions

About this article

Check for updates. Verify currency and authenticity via CrossMark

Cite this article

Eckermann, S., McCaffrey, N., Tonmukayakul, U. et al. Multiple effects health economic evaluation of the Ahead of The Game Study for mental health promotion in sporting club communities. Health Econ Rev 11, 28 (2021). https://doi.org/10.1186/s13561-021-00323-1

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI: https://doi.org/10.1186/s13561-021-00323-1

Keywords

Health Economics Review

ISSN: 2191-1991

Contact us