Table 3 Appraisal of included studies
Author (year) | Was a clear relation between costs & consequences established? | Which alternative intervention was considered? | The costs (or inputs) and consequences were measured over which time frame? | Were important costs (or inputs) and consequences omitted? | What were the main limitations? | Are there potential conflicts of interest? |
|---|---|---|---|---|---|---|
Bowser (2013) [19] | No | Status quo (traditional salaries and line-item budgets). | 2006 to 2010 | No clear description of the included and omitted costs. | 1) absence of pre-intervention data; 2) possibility that other factors influenced the costs per capita; 3) difficulty of teasing apart the effects due to the incentives from those related to other components of the reform. | - None declared - 1 author affiliated with the organisation involved in the implementation |
Gok (2015) [18] | No | Status quo (before vs after P4P). | 2001-2008 | Yes, for example, the costs of implementing the program were not included. | 1) the absence of randomization; 2) the lack of a control group; and 3) the use of aggregate input and output variables. | - None declared |
Zeng (2013) [21] | No | International support (including procurement procedures, minor renovations, advice on community mobilization, communication, public relations & promotion of family planning). | 2008-2010 | No clear description of the included and omitted costs for the "international support". | 1) absence of randomization; 2) the absence of pre-intervention data; 3) the lack of control for the quality of the data in the 202 health centers without PBF. | - Declaration that one co-author was employed by an organisation involved in PBF. |
Basinga (2011) [16] | No | Input-based budgets in the control group were increased by the average P4P payments made to the intervention group | June, 2006 to Avril 2008 (~23 months) | Lack of detailed information on the costs of PBF. Health outcomes were not included. | 1) the absence of pre-intervention data; 2) problems identified with allocation to treatment and control-groups (see Witter et al., 2013). | - None declared - Some authors affiliated with organisations involved in the funding and implementation. |
Rusa (2009) [22] | No | Status quo for performance data (3 months of pre-intervention data). No alternative intervention was used to compare costs. | 2005 to 2007 for costs. October 2014 to December 2007 for performance. | Includes subsidies and administration costs. No detailed description of the included and omitted costs. Health outcomes were not included. | 1) insufficient use of pre-intervention data; 2) the lack of a control group without PBF during the entire time period; and 3) the possibility that other interventions (e.g., mutual health organizations, sensitization campaigns) influenced the results. | - None declared - 5 of 6 authors affiliated with organisations involved in the implementation |
Sabri (2007) [20] | No | Comparison of three different PBF programs. | 2002 to 2006 | No clear description of the included and omitted costs. Limited data on healthcare services and health outcomes. | 1) the lack of information on the methodology used; and 2) the absence of links between the costs and outcomes. | - None declared - At least one author employed by an organisation involved in the implementation |
Soeters (2006) [23] | No | Comparison of PBF in early vs later stages. | 2003 to 2005 | Lack information on how PBF affects total health expenditures (only focuses on out-of-pocket health spending). | 1) absence of pre-intervention data; 2) absence of a control group; and 3) possibility that other interventions occurring simultaneously reduced catastrophic user fee payments. | - None declared - Authors worked for an organisation involved in the implementation |