Micro-costing in health and medicine: a critical appraisal

Abstract

Background

Concerns about rising health care costs require rigorous economic study to inform clinical and policy decision-making. Micro-costing is a cost estimation methodology employing detailed resource utilization and unit cost data to generate precise estimates of economic costs. Micro-costing studies have not been critically appraised.

Methods

Critical appraisal of micro-costing studies in English. Studies fully or predominantly employing micro-costing were appraised for methodological and reporting quality through economic evaluation guidelines (Evers, Drummond, Consolidated Health Economic Evaluation Reporting Standards (CHEERS), Fukuda and Imanaka checklists). Following the Panel on Cost Effectiveness in Health and Medicine, micro-costing studies were defined as involving “direct enumeration and costing out of every input consumed in the treatment of a particular patient.”

Results

Full or predominant micro-costing studies included  neoplasms (18.5%), infectious and parasitic diseases (17.9%), and diseases of circulatory systems (10.8%) as the  most studied diseases. 36.9% were in the United States and 34.9% were in Europe. 33.8% did not report analytic perspective, 32.8% did not report price year, 3.6% did not inflation adjust cost data, and 44.1% did not specify inflation adjustment. 86.2% did not separately report unit costs and resource utilization quantity, 14.9 and 19.5% did not provide sufficient detail to assess appropriateness of measured physical units or valued costs.

Conclusions

Micro-costing studies vary widely in methodological and reporting quality, highlighting the need to standardize methods and reporting of micro-costing studies and develop tools for their evaluation.

Key messages

(1) Micro-costing studies vary widely in methodological and reporting quality; (2) There is a need to standardize methods and reporting of micro-costing studies and develop tools for their evaluation.

Introduction

Concerns about rising health care costs require rigorous economic study to inform clinical and policy decision-making. Advances in medical knowledge and technology results in rapid diffusion of new health interventions that cannot be precisely costed by existing estimates or prices. Micro-costing is particularly relevant for estimating costs of new interventions, but existing interventions, too, benefit from this methodology. By measuring detailed resource utilization and unit costs, micro-costing methodology generates precise cost estimates and has been considered the preferred method for estimating costs of health interventions [1].

The conduct, appraisal and reporting of micro-costing studies can be complex due to the detailed cost categorization and data collection involved. Existing guidelines for economic evaluations, the Drummond checklist [2], Evers checklist [3], and Consolidated Health Economic Evaluation Reporting Standards (CHEERS) checklist [4], have focused on full economic evaluations. Although these guidelines have been instrumental in enhancing the quality, transparency, and comparability of economic evaluation studies, they do not provide sufficient guidelines regarding appropriate techniques and standards to use in micro-costing studies. Moreover, while the recently convened Second Panel on Cost-Effectiveness in Health and Medicine [5, 6] updated recommendations concerning the conduct of economic evaluations, there remains no standardized requirement about how to conduct, appraise, or report micro-costing studies [7].

Given this gap in the literature, current practices in conducting and reporting micro-costing studies likely vary substantially. This may hinder comparison of research findings across studies and interpretation of study results. The objective of this study was to critically appraise micro-costing studies to assess their methodological and reporting quality and the appropriateness of existing economic evaluation guidelines for this purpose.

Methods

A critical appraisal of micro-costing studies in English was conducted. Ovid MEDLINE, EconLit, BIOSIS Previews, Embase, Scopus, and National Health Service Economic Evaluation Database were searched for records through July 2015. Following Gold and colleagues [1], micro-costing studies were defined as those that involved “direct enumeration and costing out of every input consumed in the treatment of a particular patient”. To inform future design and reporting of micro-costing studies, analysis was limited to studies that fully or predominantly used micro-costing in their cost estimation and studies that only partially or minimally involved micro-costing were excluded. To inform future research, the focus was on research studies with a defined patient sample and research question, studies with a primary goal to inform hospital financial management or estimate unit cost per medical service were excluded.

A detailed data collection form was developed to extract comprehensive data on study design and reporting characteristics for each micro-costing study. Data collected included study population, study intervention, analytical perspective, type of economic evaluation, economic and health outcomes evaluated, time horizon of analysis, discounting, price and currency, method of data collection for quantity and unit cost information, cost components included, and use of sensitivity analysis. These data elements captured sufficient specifics to facilitate a detailed characterization of studies.

To evaluate study quality, validated economic evaluation checklists were employed, including the Drummond checklist [2] and Evers checklist [3]. The Drummond checklist contained 35 items evaluating the quality of an economic evaluation regarding its study design, data collection, analysis, and interpretation of results [2]. The Evers checklist included 19 items covering a core set of methodological standards for economic evaluations [3]. For each checklist, the percentage of applicable items that a study scored “Yes” was calculated. Since not all checklist items were applicable to a given study, inapplicable items were excluded when calculating the percentage of Drummond checklist and Evers checklist items that the study scored “Yes”. Higher percentage indicates better quality.

A single item classification scheme developed by Fukuda and Imanaka [8] was employed to further evaluate reporting quality. This scheme categorized each study into four levels based on transparency of their reporting: “All components of costs were described and data for both quantity and unit price of resources were reported for each component”, “All components of costs were described and data for costs in each component were reported”, “All components of costs were described but data for costs in each component were not reported”, and “Only scope of costing was described but components of costs were not described” [8].

Two researchers independently screened studies and performed data extraction. Disagreements were discussed and resolved by consensus as well as consultation with a third researcher. Descriptive statistics (e.g., frequency and percentages) were used to summarize key study design and reporting characteristics. SAS version 9.4 was used for all data analysis (SAS Institute Inc., Cary, NC).

Results

195 studies were critically appraised. The number of studies that fully or predominantly applied the micro-costing methodology increased over time. General characteristics of the 195 micro-costing studies are summarized in Table 1. Neoplasms, infectious and parasitic diseases, and diseases of the circulatory systems were the most commonly studied disease areas, accounting for 18.5, 17.9, and 10.8% of the studies, respectively. In terms of geographic location, 36.9% were conducted in the United States, and 34.9% were performed in European countries. Most were observational studies (67.2%), while 32.8% were based on randomized controlled trials. When categorized by type of economic evaluation, a small proportion were full economic evaluations, including 24.1% cost-effectiveness analyses, 5.6% cost-utility analyses, 1.5% cost-benefit analyses, and 5.6% cost-minimization analyses. In contrast, 19.0% compared costs between two or more interventions without evaluating effectiveness and 23.1% estimated costs for a particular intervention without a comparison group or evaluating effectiveness.

Table 1 General characteristics of micro-costing studies

Key design features were poorly reported (Table 1). Approximately one third of studies did not specify perspective, and only 20.5% adopted a societal perspective. 32.8% of studies did not specify price year, while 44.1% did not state whether costs from different years were inflation adjusted. 3.6% did not perform inflation adjustment when cost data were collected across multiple calendar years and did not provide justification. 48.2% of studies did not perform any sensitivity analysis to account for uncertainty in cost estimation. A large proportion of micro-costing studies did not report nature of funding (28.2%) or conflict of interest (54.9%).

Study characteristics related to cost measurement are summarized in Table 2. 49.7% referred to their own methodology as micro-costing. When assessed by the Fukuda and Imanaka [8] transparency scale, 13.8% of studies described all cost components and reported both quantity and unit cost of resources for each component, whereas 80.5% described cost components and reported data for cost in each component but did not separately report quantity and unit cost of all resources used. In terms of specific cost components estimated, virtually all studies (95.4%) included personnel costs, 88.2% included material, supplies and consumables, and 68.2% included overhead costs. Provider/staff interview, hospital administrative or cost/accounting databases, and time-motion study were the most common methods of collecting data on quantity of resource utilization, used by 36.4, 29.2 and 21.5% of studies, respectively. Invoice price, hospital administrative or cost/accounting database, and fee schedules were most commonly used to obtain unit cost data, accounting for 31.8, 30.8, and 24.1% of studies, respectively.

Table 2 Micro-costing related characteristics of studies

Evaluation based on the Drummond checklist [2] and Evers checklist [3] showed moderate quality of these studies. Among the checklist items applicable to each study, the average proportion of items that scored “Yes” was 72.5% on Drummond checklist and 69.6% on Evers checklist. 72.8% did not specify analytical perspective or did not use a societal perspective and did so without justification, 83.6% did not explain why costs and benefits were not discounted, and 68.7% did not perform sufficient sensitivity analysis on important variables. While most studies measured physical units and unit costs appropriately, 14.9 and 19.5% did not provide sufficient detail to inform whether they appropriately measured physical units or valued costs, respectively.

Four studies compared the impact of micro-costing versus other costing methods, such as gross-costing, on cost estimation [9,10,11,12]. Their findings showed that, depending on services type, use of different costing methods could result in large differences in cost estimates and may affect decision-making regarding the cost-effectiveness of alternative interventions [9, 11, 12]. The micro-costing method was considered more transparent [9] and provided more detailed information to help identify cost drivers [11]. In addition, three studies compared bottom-up (i.e., assessed resource utilization for each cost item and then aggregated for a total system) and top-down (i.e., assessed resource utilization for a total system and then allocated to individual cost items) [13] costing. They demonstrated that bottom-up costing generated more accurate estimates for labor intensive services [12, 14, 15].

Discussion

This critical appraisal of micro-costing studies in health and medicine suggests that use of micro-costing methods, while increasing, has largely varied in the quality of conducting and reporting of micro-costing analyses. Studies often did not adequately address key design issues and lacked sufficient detail or transparency in explaining study design. The few studies available that compared micro-costing with other costing methods suggest that micro-costing is the preferred method for generating accurate cost estimates, especially for studies that involve services that are labor intensive.

These findings highlight the importance and need for rigorous guidelines to help standardize the conduct and reporting of micro-costing studies. Future efforts to develop a checklist specifically tailored to micro-costing studies [7] will be instrumental. Prior checklists have evaluated full economic evaluations such as cost-effectiveness analysis and many of the checklist questions are not applicable to micro-costing studies. Moreover, many specific aspects of micro-costing studies – such as detailed enumeration of resource utilization quantity and unit costs – are not fully assessed by existing checklists. Closer attention to the design and reporting of micro-costing studies is needed from the research community, and quality standards deserve consideration by relevant professional organizations, such as future Panels on Cost-Effectiveness in Health and Medicine, the International Society for Pharmacoeconomics and Outcomes Research, and the Society for Medical Decision Making. Our study identified several limitations in current micro-costing studies, which may assist future efforts. Lack of sufficient detail about measurement of resource utilization quantity and unit costs requires particular attention in development of quality standards or checklists for micro-costing studies.

Variability existed in how studies have referred to their micro-costing method. Some studies stated bottom-up approach or did not name their method, while other studies used different terminologies such as ingredient based approach. Micro-costing methods need to be more systematically reported and consistently referred to when applied. This can further improve comparability and transparency of future studies.

This study has several limitations. First, our selection of micro-costing studies focused on those that involved systematic collection of quantity and unit costs data for a defined patient cohort. Other researchers may have used broader definitions for micro-costing and may include other types of studies such as those focusing on hospital accounting or financial interest or estimating unit costs of a particular procedure or laboratory test without a defined patient sample. Second, studies that only cited use of hospital accounting databases without providing sufficient details regarding the collection or tracking of quantity and unit cost data were excluded because lack of such information precluded verifying whether the study used micro-costing methods. As hospitals are increasingly adopting micro-costing or activity-based cost allocation systems, researchers may be able to draw on such electronic data to facilitate data collection. However, these studies should provide sufficient detail regarding key elements of micro-costing methodology in order to enhance the transparency and usefulness of information. Third, studies that claimed micro-costing but provided little information to verify their costing methodology, were excluded, thus the appraisal of included studies may overestimate the quality of existing micro-costing studies.

Although micro-costing may not always be practical, it has unique advantages compared with the more commonly used gross-costing method. It allows more accurate cost estimation, and is particularly useful for evaluating the economic impact of new interventions that lack established cost estimates [16, 17] although existing interventions, too, benefit from this methodology. Micro-costing should be the preferred method in all areas of health and medicine when such data are available. It is also increasingly a preferred method for policy purposes [18] and a measure of accountability [19]. Given the complexity of micro-costing methodology and our findings of poor quality in some key aspects of study design and reporting, more attention and effort are needed to better guide future studies. Development of a checklist specifically for micro-costing studies will be instrumental to establishing quality standards and promoting comparability and transparency in future research [7]. There is a need especially for standardizing terminology in micro-costing studies.

Conclusion

Micro-costing studies vary widely in methodological and reporting quality, highlighting the need to standardize methods and reporting of micro-costing studies and develop tools for their evaluation.