Skip to main content

Patient-reported data informing early benefit assessment of rare diseases in Germany: A systematic review



Since the implementation of the Regulation on Patient Integration (2003), the Act on the Reorganization of the Pharmaceutical Market (2011), and the Patient Rights Law (2013), the inclusion of patient perspectives has been further anchored in the German early benefit assessment process. During the assessment of rare disease interventions, patient perspectives are particularly important, as clinical studies are often designed acknowledging small samples and patients suffering from severe symptoms and the chronic course of the disease. Therefore, our research question is whether patient perspectives are considered as part of early benefit assessments for rare diseases. We also strive to examine how patient perspectives are methodologically elicited and presented.


Our empirical evidence comes from a systematic review of orphan drug value dossiers submitted to the German Federal Joint Committee as well as the corresponding evaluations conducted between January 1, 2011 and March 1, 2019 (n = 81). Data on patient perspective integration were extracted using the following patient-reported outcome subcategories: clinical patient-reported outcomes, health-related quality of life, patient preferences, and patient satisfaction.


The analysis demonstrates the specific relevance of patient-reported outcomes raised as part of the medical data set and presented during the early benefit assessment process. They are predominantly presented in the form of health-related quality of life data (n = 75%) and clinical outcomes (n = 49%). Preferences (n = 2%) and satisfaction (n = 1%) are still rarely presented, although the heated methodological discussion in Germany would suggest otherwise. While various methodologies for the integration of clinical outcomes and quality of life data were found, presenting data on satisfaction and preferences still lacks methodological rigor. The German Federal Joint Committee has not yet integrated these data in their decision text. Clinical outcomes and quality of life have been included in 46% and 73% of the cases, respectively.


The underlying analysis demonstrates that there is still a relative high potential for the regular and systematic inclusion of patient perspectives within the early benefit assessment process for rare diseases. In particular, patient preferences and patient satisfaction are still rarely included suggesting the need for a clear-cut methodological foundation and incentives.


The relevance of patient perspective integration in health care

In Germany, the 2003 Patient Participation Regulation [1] as well as the 2013 Patient Rights Law [2] form the most important basis for the integration of patient perspectives within the health care system. The Patient Participation Regulation, which is linked to §140f of the German Social Insurance Code Book Volume V, regulates the mandatory involvement of patient organizations in health care decision making. Within the German Federal Joint Committee, patient organizations now have the right to advise and request, but not to vote. In 2013, patients’ position was further strengthened in terms of involvement and rights [1, 2].

In the extant literature, patient-oriented health care systems are created to extend traditional health care models using patient empowerment. Therefore, important attributes of patient-oriented health care systems are identified as: high-quality information generation and transparency, patient perspective integration through customization and collaboration, as well as the integration of patient choice and responsibility. To these, predictive and preventive instruments can also be added [3]. Consequently, empirical health economics have turned towards the collection of evidence regarding patient views. Patients should be effectively integrated during the health technology assessment process, beginning with evidence generation and value measurement and concluding with recommendations and communication of results, e.g., in the context of health policy [4].

Patient perspectives can be integrated in different ways. While summarizing data contributed by patients or their representatives is most common, this approach must be differentiated from the studies carried out from the patient perspective but contributing data, for example, collected by physicians or other health care specialists [5]. As a methodologically-grounded approach, patient-reported outcomes (PRO) stand for the reports directly originating from patients [5, 6] without the involvement of a physician or other communicators [5]. For instance, Klose et al. [7] report that the terms of PRO and outcome sometimes diverge in their interpretation. Used as a medical term, an outcome indicates an end result or intervention consequence in terms of symptoms and functioning, as well as the health-related quality of life (HRQoL). However, as reported by the PRO Harmonization Group, the discussion expanded from only including HRQoL outcomes to considering any outcome based on data provided by the patient or the patient’s proxy [8]. In this study, we follow the broader interpretation of the term PRO, as provided by the PRO Harmonization group.

Since the definition of subcategories also varies broadly, we follow the nomenclature of Klose et al. [7]. This meant that clinical PRO, HRQoL, patient satisfaction, patient experiences, and patient preferences are differentiated. Within our analysis, it seemed preferable to differentiate between the clinical PRO following the traditional medical interpretation and HRQoL due to differences in mortality, morbidity, and quality of life endpoints within the process of early benefit assessment [2]. Furthermore, patient experiences were not acknowledged separately, as they are predominantly reported in the context of patient satisfaction and patient preferences. Preferences describe whether one item is favored over another [9], meaning they withhold information regarding preferable treatment options from the affected individuals. “Preference” is often used as an umbrella term. As such, a preference measurement can result in either utility or value depending on the measurement approach [10]. There may be various reasons for the consideration of patient preferences in health care, such as improved therapy uptake or efficiency of health care interventions in practice, thus facilitating patient involvement and promoting shared decision-making in medicine. Medical decisions that are consistent with patient preferences may facilitate patient acceptance [9, 11, 12]. Therefore, patient satisfaction is also considered an important component of medical intervention assessments. However, it is a rather subjective assessment of the quality of care and is often used to incorporate the patient’s perspective on the quality of care as part of medical evaluations [12]. Patient satisfaction has not yet been well defined but is generally considered to describe a subjective assessment of medical care by patients [13]. Further, the concept can contain various elements such as medical therapy, nonmedical aspects of treatment, as well as health care infrastructure [14].

In general, the PRO can be raised via qualitative, mixed, and quantitative methodologies. Within the field of clinical PROs and HRQoL generic and disease-specific instruments are differentiated by incorporating symptom-specific modules. Patient preferences can be analyzed using contingent evaluation approaches, self-explication approaches, analytic hierarchy process, conjoint analysis, standard gamble, time trade-off approaches, as well as rating scales. For the examination of patient satisfaction, various approaches exist, such as the Patient Satisfaction Questionnaire and European Project on Patient Evaluation of General Practice Care Questionnaire [7].

Patient perspectives in the field of rare diseases

The term “rare diseases” summarizes between 5000 and 8000 different diseases that are characterized by their severity, by their genetic origin and status as life threatening, or by the chronically debilitating course of the disease. Within the European Union, a disease is called “rare” when fewer than 5 out of 10,000 people are affected. Despite major medical advances in general, a major unmet medical need has been identified within the field of rare disease, concerning diagnostic procedures and effective treatment strategies [15]. According to the German Health Ministry, in Germany alone approximately 4 million people are affected by rare diseases [16] and although this would seem to suggest that patient perspectives are particularly important within the field of rare diseases due to its heterogeneity. There is still a lack of research on the systematic inclusion of this field during health economic processes. As demonstrated by a review of 11 national strategies regarding patient engagement, the focus of political strategies rests with the involvement of patient organizations [17]. Admitting that traditional assessments fail to endorse medical technologies for rare diseases due to a lack of power, new endpoints have been explored with names such as Patient-Centered Outcome Measures [18].

Early benefit assessment for rare diseases in Germany

Since the Act on the Reorganization of the Pharmaceutical Market (AMNOG) within the statutory health insurance law issued in December 2010 came into effect in January 2011, all manufacturers need to provide evidence of the additional benefit of pharmaceutical products claimed over an appropriate comparator. The German Federal Joint Committee then decides whether and to what extent a drug can be granted an additional benefit and their decision forms the basis of price negotiations for the statutory health care setting [19]. Typically, the Institute for Quality and Efficiency in Health Care (Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen - IQWiG) is engaged to conduct early benefit assessment, and reports that patient perspectives play a key role in their judgments. Patient perspectives are generated using a standardized questionnaire regarding what is important to patients in terms of disease and treatment strategies [20].

However, in cases of drugs used solely for the treatment of rare diseases (orphan drugs), an additional benefit is presumed by the European drug approval, authorized in accordance with EC regulation number 141/2000 on orphan drugs [21]. In such cases, evidence must only be provided concerning the extent of the additional benefit to standard therapy for patients [22]. Here, the IQWiG is not involved in the benefit assessment, but in the estimation of patient numbers (target group, incidence, prevalence, and trends) as well as treatment costs. Only when the overall annual treatment costs of an orphan drug exceed the limit of 50 million euros for the statutory health insurance must it pass through the common early benefit assessment process [23]. Finally, decision making is taken over by the Federal Joint Committee, who describe patient involvement as the inclusion of patient representatives and patient organizations during the process but provide no further description of the procedure.

Aims of the study

To address this gap, this article examines how the integration of patient perspectives in the assessment of benefits proceeds. To this end, we examine the development of a methodologically grounded and direct approach to patient perspective integration, using the concept of patient reported outcomes, and analyzing the data submitted during early benefit assessments for rare diseases in Germany.


Data sources

Our empirical evidence comes from the database of the German Federal Joint Committee, withholding all procedures of early benefit assessment induced by §35a Volume V of the Social Code Book starting with the implementation of the AMNOG law [23]. All procedures with a starting date of between January 1, 2011 and March 1, 2019 were considered. In a second step, we filtered for procedures with an orphan drug status and only procedures marked “completed” were finally reviewed. Due to the exceeding of the 50-million-euro boundary or an extension of the area of application, newly developed active substances can be subject to multiple assessments.

The Federal Joint Committee makes available the following information on each procedure: the “dossier” submitted by the pharmaceutical company in accordance with the modular template, “benefit assessment,” “comments procedure,” and the “resolution” comprising “decisions” as well as the corresponding “rationales” [23]. Data were extracted from the dossier, in particular module 4, the benefit assessment, as well as the decision text developed by the Federal Joint Committee.

Strategy of analysis

To analyze the integration of patient perspectives within the data set, the PRO concept was used following the broad definition provided by Black (2003) [6] and Patrick et al. (2003) [5] linked with the nomenclature of Klose et al. (2016) [7]. Therefore, the following subcategories have been used during the underlying analysis:

  • Patient-reported outcomes clinical data (clinical PROs)

  • (Health-related) quality of life (HRQoL)

  • Patient preferences

  • Patient satisfaction.

We did not specifically analyze the presented data for adverse events as HRQoL registers adverse events as well and mortality captures the fatal adverse events. We particularly analyzed the data presented within the synopsis section of the dossiers, the benefit assessment, as well as the decision text. The respective text passages were screened particularly searching for terms such as “patient-reported outcomes,” “quality of life”, “patient preference” and “patient satisfaction”. The identified sections were than extracted and transferred to a separate Excel sheet. Quantitative and qualitative data were reviewed equally. To provide a first impression on the relevance of the reported patient perspective within the early benefit assessment of rare diseases, we also examined the extent of the requested, and later granted, additional benefit as well as the methodologies actually considered by the GBA for each PRO data category.


Characteristics of the data sample

Our final sample contained n = 81 value dossiers. The first dossier was submitted on September 15, 2011 and the last on September 15, 2018. The different disease groups are shown in Table 1. A total of 51% of the dossiers within the field of rare diseases addressed oncological indications, while metabolic diseases were the second most common, at 25%.

Table 1 Disease groups covered by the rare diseases benefit assessment procedures

Benefit assessments were predominantly (n = 71, 88%) conducted by the Federal Joint Committee itself. The IQWiG was commissioned with the rare diseases benefit assessment in 12% (n = 10) of cases, mostly when drugs were cross-passing the sales limit of 50 million euros (n = 7). In some cases, the manufacturer applied for an additional application area (n = 3). It should be noted that, whereas all agents cross passing the 50 million euro limit were commissioned to the IQWiG, applications for additional application areas were also assessed by the GBA itself.

Table 2 shows the benefit scores that were applied for by the pharmaceutical companies during those processes, as well as the GBA score granted by the Federal Joint Committee. Applied and granted benefit scores matched in 19% of cases. No additional benefit was determined in 2% of cases as part of a reassessment after the trespassing of the 50 million Euro limit conducted by the IQWiG.

Table 2 Benefit score of the orphan drug benefit assessment processes


Table 3 shows a summary of the analysis of patient-reported data during the early benefit assessment process for rare diseases.

Table 3 Submission and consideration of PROs as part of the early benefit assessment process of rare diseases

PROs are mostly presented in the form of self-reported clinical outcomes data (n = 39, 48%) followed by data on HRQoL (n = 61, 75%). Data on patient preferences were included twice (2%) and on patient satisfaction only once (1%). In only 15% (n = 12) of cases, no data on PROs were submitted. Therefore, data on PROs were presented to a relatively high extent. In 85% of cases, PROs were presented from at least one PRO subcategory. The Federal Joint Committee considered clinical PRO data in 37 (46%) cases, whereas HRQoL data were included in 59 (73%) cases. Patient satisfaction and patient preferences were not included within the decision text. Another example, which was often considered as part of the GBA decision text (n = 13), was the EORTC (Core Quality of Life Questionnaire). The splitting of the questionnaire items into clinical PROs and QoL PROs could also be observed here.

Table 4 shows the overall number of assessment processes for each year since the implementation of the AMNOG law. In 2017, 16 processes were initiated. Twelve processes presented data on Clinical PROs, 13 processes withheld HRQoL data, and 1 process presented data on preferences. Overall, the direct integration of patient perspectives in the form of PROs has gradually increased in its absolute number, with an increased number of induced processes since the implementation of the AMNOG law in 2011. Regarding the relative percentage of PROs in relation to the number of processes included within our analysis, no clear-cut trend is observable.

Table 4 Development of PRO data submissions for rare diseases over time

Analyzing the clinical PRO and HRQoL data, it was observable that the GBA split the surveys into symptom scales (clinical PRO data listed as part of the morbidity endpoints) and HRQoL scales. An example is the oncology specific EORTC QLQ-C30. In this context, the EQ-5D VAS scale has been categorized as part of the morbidity section. However, the EQ-5D Index has been appreciated as part of the HRQoL section. Moreover, further commonly acknowledged methodologies were the childhood health questionnaires and the Brief Pain Inventory (BPI) as well as the Functional Assessment of Chronic Illness Therapy - Fatigue (FACIT-F). On the other hand, commonly acknowledged HRQoL methodologies included disease-specific FACT-questionnaires, SF-questionnaires, and the Pediatric Quality of Life inventory (PedsQL).

The heatedly discussed categories “patient satisfaction” and “patient preferences” were rarely referred to, and when they were, it was in a qualitative manner [24,25,26]. Quantitative methods were not used. The dossier submitted for the agent Velmanase alfa (2018) offered “patient cases” in the form of short summaries, backing clinical PROs as well as the relevance of symptoms (preferences) and overall quality of life (not naming HRQoL in this case). There was no description of the detailed qualitative research strategy. Telotristatethyl (2017) provided “semi-structured telephone-interviews” on topics such as symptom description, preferences, and patient experiences. Some structural background data were provided, but again no description of the qualitative research strategy was included. In the case of Eftrenonacog alfa (2016), during a first phase, “focus groups” were cited as well as a “structured questioning” that also considered satisfaction. Results were presented in a qualitative manner but there was no solid description of the qualitative research strategy. Patient satisfaction and patient preferences were not included at all within the GBA decision body, thus providing no incentives for further data presentation. A detailed overview of the data can be found in Table 5.

Table 5 Clinical and patient-reported outcomes data considered by the German Federal Joint Committee for rare diseases


Summary of findings

The present study analyzes data on the direct involvement of the patient perspective, particularly in the form of PROs, which are submitted, evaluated, and considered during the early benefit assessment process for rare diseases in Germany. The results demonstrate that patient perspectives predominantly enter the process via clinical PROs and HRQoL. However, in comparison with clinical PRO and HRQoL, the categories “patient satisfaction” and “patient preferences” were rarely referred to, and if they were, it was in a qualitative manner that lacked a solid description of the methodological foundation within qualitative research. Nevertheless, we found that 16% of the orphan drug dossiers did not present any data on PROs.

Significance in the context of literature

To our knowledge, this study provides unique insights into the inclusion of patient perspectives within the early benefit analysis process, in particular using PROs as part of the early benefit assessment of orphan drugs.

Braithwaite et al. [27] highlight again the importance of PROs in the field of rare diseases since some of the methods used in this field of research permit smaller sample sizes. They also pinpoint the importance of primary outcome measures in general and that, in particular, traditional outcome measures have failed to demonstrate efficiency. While considerable progress has been made in the development of associated measures, it is still difficult to find tools for less common indications [27]. This may be one of the explanations for low acceptance and / or submission of PROs in the field of rare diseases.

control Changes within the political framework can affect pharmaceutical companies’ submission behavior. For example, before the introduction of the Patient Rights Law in 2013 [2], only three dossiers were submitted, while afterwards, the number of dossiers for orphan drugs increased to approximately 16 per year. However, the data cannot capture the possible impacts of changes in legislation since these events occur at a small rate and could be falsified by the overall orphan drug submission rate.

Furthermore, the methodological developments within the health economic environment in Germany can also influence the development of data submission and its appraisal. In 2013, the IQWiG discussed changes to its methodology for the very first time. Institutes and industrial representatives argued for the direct, transparent, and systematic integration of patient perspectives, in particular patient preferences and the definition of the precise integration processes [24]. In 2015, another discussion of the IQWiG general methods paper was published. However, the focus of the discussion concerning patient perspectives integration was predominantly in relation to the reintegration of patient satisfaction as optional data [25]. Moreover, in 2017, the last recorded methodological discussion was published and, in this context, the systematic direct integration of patient perspectives was again demanded in several parts of the IQWiG methods paper, e.g., the clear-cut acknowledgement of patient preferences [26]. The first pilot projects concerning the measurement and inclusion of patient preferences in health economic evaluation were published in 2013 (Analytic Hierarchy Process (AHP) [28, 29] and 2014 (Conjoint Analysis) [30, 31]. Although the named projects and discussions seem to lay the basis for the methodologically grounded inclusion of patient perspectives, the first inclusions of patient preferences were recorded in 2017 and 2018 in the field of rare diseases. However, since many of the above-named quantitative methods are not appropriate in the field of rare diseases due to the limitation of small sample sizes (see also [18]) (an exception is the AHP [32, 33]), further specified requirements for qualitative data presentation are required, as well as incentives for their adaptation. The same arguments hold for patient satisfaction, which is referred to as additional submittable data but in its patient-centeredness is relevant by definition.

Clinical patient-reported outcomes and health related quality of life

Furthermore, in terms of particular relevance, it has been argued that the documentation of clinical PROs - for example, as part of the phenotype “pain” - offers the chance to better align treatment options and outcomes [34].

Casamayor et al. [35] analyzed whether PROs in oncology matter in health technology assessments conducted in Germany, France, and the UK, and found that an improvement in such outcomes did not increase the chance of a positive health technology assessment (HTA) recommendation. The authors also demonstrated that PROs assessing Quality of Life (51/57, 89.4%) and pain measures (18/57, 31.6%) are the most common. PROs were not mentioned at all in 35.1% of cases [35]. Although our analysis examines HRQoL measures and morbidity-focused clinical PROs, the tendencies of both research papers seem to be similar. An early stage analysis of the first 25 dossiers in Germany regarded independently of the targeted indication demonstrated that in the beginning HRQoL outcomes were not considered during the early benefit process for different reasons [36, 37]. In our analysis, we found that this category was the most acceptable for the Joint Federal Committee. However, the general position of the Federal Joint Committee on the importance of quality of life data has changed significantly in the last 10 years. Initially rated as supporting or complementary information, quality of life data is today accounted equivalent to endpoints of mortality and morbidity [38]. Nevertheless, there are methodological questions regarding measurement and distinction that are not yet clear.

Patient preferences

The proportion of dossiers including data on patient preferences was quite low. Obradovic and Rauland [39] state that approximately 25% of all dossiers published between 2011 and 2014 referred to some extent to patient preferences. However, the database used seems to be more broadly designed. In the case of the present study, we included data from the as by the pharmaceutical company submitted studies but this also prompts further research questions regarding the differences between the integration of patient perspectives in the field of rare diseases and other indications. Of course, many quantitative measurement methods such as choice experiments/conjoint analyses are hardly feasible in the field of rare diseases. Furthermore, their specific aim is to compare different treatment methods (trade-off), which are often not provided in the field of rare diseases.

Although we controlled for incentives to render data on patient perspectives in a direct and systematic way considering PROs, it must also be stated that there are some factors outside the set framework that may also influence the presentation of data. For example, the benefit score and associated documents form the basis for price negotiation in Germany [40, 41].

Besides, the methodological foundation for patient preferences has also been developed in an international context and substantial literature has been published. The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) developed a good research practice checklist for conjoint-analysis in health.

The checklist included 10 items covering the research question, levels and attributes, task development, the design of the experiment, preference elicitation, design of instruments, data-collection, analyzing statistical data, results and conclusions as well as study presentation standards. Even though, not endorsing a specific methodological approach, the checklist can serve as a good foundation for further discussions of good research practice for the application of conjoint-analysis methods in health care studies [42]. Besides, further research efforts give in depth advice concerning specific elements of the research process, for example the experimental design [43]. Several studies review the usage of different methodologies raising patient preferences systematically for different indications such as for example diabetes [44, 45]. Further, CONSORT guidelines advise on the reporting of PRO data in general [46]. This study contributed to the existing literature by outlining the methodology of PRO data inclusion within the field of rare diseases in Germany.


In terms of limitations, data on clinical PROs could only be identified as such as long as they were highlighted as a self-reported measure or indicated to be a patient-reported measure. When no particular definition was provided, we assumed that the endpoint was physician-reported. We assume that almost all endpoints were specifically marked as patient-reported, as dossier providers have often argued that clinical PROs are particularly relevant to the patient and should therefore be specifically considered during the valuation and decision-making process. Furthermore, PROs are clearly defined as self-reported. However, in the case of clinical PROs, the reporting system was sometimes not indicated. In these cases, we searched for the primary classification of the symptoms scale.

In addition, pharmaceutical companies present their HRQoL data as a whole data subset. However, the Federal Joint Committee separates parts of the questionnaires selectively regarding mortality endpoints and HRQoL endpoints. Therefore, the data reveal a splitting of the datasets rendered by the pharmaceutical company. Endpoints were not shifted as this would not reflect the actual status quo of the data presented but would, rather, lead eventually to a presentation bias. However, it needs to be highlighted as a specific procedure presented by the German Federal Joint Committee and considered when selecting the appropriate data presentation technique.

Finally, it needs to be highlighted that some dossiers can fail due to formal reasons, for example not the appropriate comparator, a study population narrower than label etc. Therefore, the impact of PROs on the final decision is not always directly derivable.


The underlying evaluation demonstrates that although the political basis has been strengthened and the presented concepts have been broadly laid out as part of the health economic discussion in the context of benefit analysis and cost-benefit analysis, there remains a broad potential for the development of the practical framework regarding the systematic inclusion of patient perspectives, especially in referring to patient preferences and patient satisfaction, particularly considering the example of early benefit assessments for rare diseases in Germany. In this regard, it is interesting that patient preferences are presented in a qualitative manner. The broadly discussed and exemplified (by the IQWiG) quantitative methods have not been demonstrated in the field of rare diseases to date. While methodological standards for qualitative reporting have not yet been adopted, they must be appreciated with the same thoroughness as within quantitative research settings. An according clarification of the standard guidelines needs to be demanded. Moving even one step ahead, potentials of the integration of qualitative and quantitative research may be discussed, appreciated, and scientifically monitored in this specific context. Furthermore, the interim radiation of patient satisfaction has been commented on with vehement protest. In practice, however, it is only presented in 2% of cases in the field of rare diseases, even though this topic seems highly relevant due to the predominantly chronic and severe course of diseases. Neither of the PRO categories are enlisted within the GBA decision text. Acknowledged clinical PROs are often raised by the BPI-SF (pain scale) and FACIT-F (fatigue index). On the other hand, FACT-questionnaires, SF-questionnaires, and PedsQL are often GBA-appreciated HRQoL PROs. It is noteworthy that HRQoL questionnaires are in many cases split with regard to morbidity and HRQoL items, as datasets produced by one questionnaire are submitted cohesively. In this regard, the EQ-5D VAS is often appreciated as a morbidity endpoint by the GBA and therefore, in this context, it is categorized as a clinical PRO, whereas the EQ-5D-Index is categorized as a HRQoL. Another commonly accepted example is the oncology indication specific EORTC QLQ-C30. This may lead to irritation, hindering the preparation of PRO data inclusion by pharmaceutical companies. Therefore, potential implications should be clarified.

Furthermore, the extent of PRO data presentation withholds considerable potentials. It is questionable whether morbidity-oriented clinical PROs should only be included in every second dossier, when it is highly relevant to the patient and to treatment success. Patient satisfaction and patient preferences follow by the same token. Appreciating the central role of patient perspectives within early benefit assessments and the according legal framework, the GBA decision text should particularly appreciate the consideration of patient perspectives, flagging incentives for more extensive consideration. Considering the growing financial pressure on health care systems, strengthening direct patient perspective involvement by further integrating PROs holds an immense opportunity to align health care with actual patient needs and therefore to contribute to an effective and needs-oriented health care system development.

Availability of data and materials

The data that support the findings of this study are available from the authors upon reasonable request.



Arzneimittelmarktneuordnungsgesetz - Act on the Reorganization of the Pharmaceutical Market


Gemeinsamer Bundesauschuss – Federal Joint Committee


Health Related Quality of life


Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen – Institute for Quality and Efficiency in Health Care


Patient reported outcomes


  1. Regulation on the integration of male and female patients within the statuary health insurance (Patient Participation Regulation - Patientenbeteiligungsverordnung - PatBeteiligungsV). Federal Gazette. 2003; I (63):2753.

  2. Law on the improvement of rights of male and female patients (Patient Rights Law -Patientenrechtegesetz). Federal Gazette. 2013;I (9):277.

  3. Swan M. Emerging patient-driven health care models: an examination of health social networks, consumer personalized medicine and quantified self-tracking. Int J Environ Res Public Health. 2009 Feb;6(2):492–525.

    Article  PubMed  PubMed Central  Google Scholar 

  4. Facey K, Boivin A, Gracia J, Hansen HP, Lo Scalzo A, Mossman J, Single A. Patients’ perspectives in health technology assessment: a route to robust evidence and fair deliberation. Int J Technol Assess Health Care. 2010;26(3):334–40.

    Article  PubMed  Google Scholar 

  5. Patrick DL, Guyatt GH, Acquadro C. Patient reported outcomes. In: Cochrane handbook for systematic reviews of interventions. Cochrane Book Series. Wiley: Chichester; 2008. p. 531–545.

  6. Black N. Patient reported outcome measures could help transform healthcare. Br Med J. 2013;346:f167.

    Article  Google Scholar 

  7. Klose K, Kreimeier S, Tangermann U, Aumann I, Damm K. Patient- and person-reports on healthcare: preferences, outcomes, experiences, and satisfaction – an essay. Health Econ Rev. 2016;6:18.

    CAS  Article  Google Scholar 

  8. Acquadro C, Berzon R, Dubois D, Leidy NK, Marquis P, Revicki D, Rothman M. PRO Harmonization Group. Incorporating the patient's perspective into drug development and communication: an ad hoc task force report of the Patient-Reported Outcomes (PRO) Harmonization Group meeting at the Food and Drug Administration, February 16, 2001. Value Health. 2003;6(5):522–31.

    Article  PubMed  Google Scholar 

  9. Dirksen CD, Utens CM, Joore MA, van Barneveld TA, Boer B, Dreesens DH, et al. Integrating evidence on patient preferences in healthcare policy decisions: protocol of the patient-VIP study. IS. 2013;8:64.

    Article  PubMed  Google Scholar 

  10. Drummond MF, Sculpher MJ, Torrance GW, O’Brien BJ, Stoddart GL. Methods for the Economic Evaluation of Health Care Programmes. 3rd ed. Oxford: Oxford University Press; 2005.

    Google Scholar 

  11. Ruland CM. Improving Patient Outcomes by Including Patient Preferences in Nursing Care. Proc AMIA Symp. 1998;448–52. PMID: 9929259.

  12. Tai BB, Bae YH, Le QA. A Systematic Review of Health Economic Evaluation Studies Using the Patient's Perspective. Value Health. 2016;19(6):903–8.

    Article  PubMed  Google Scholar 

  13. Crow R, Gage H, Hampson S, Hart J, Kimber A, Storey L, Thomas H. The measurement of satisfaction with healthcare: implications for practice from a systematic review of the literature. HTA. 2002;6:1–244.

    CAS  PubMed  Google Scholar 

  14. Ware JE, Snyder MK, Wright W, Davies AR. Defining and measuring patient satisfaction with medical care. Eval Program Plann. 1983;6:247–63.

    Article  PubMed  Google Scholar 

  15. European Commission. 2017. Rare diseases: a major unmet medical need. Res Innov Projects Policy. doi: Accessed 7 Apr 2019.

  16. German Ministry of Health. 2019. Rare diseases. Accessed 7 Apr 2019.

    Google Scholar 

  17. Dharssi S, Wong-Rieger D, Harold M, Terry S. Review of 11 national policies for rare diseases in the context of key patient needs. Orphanet J Rare Dis. 2017;12(1):63.

    Article  PubMed  PubMed Central  Google Scholar 

  18. Morel T, Cano SJ. Measuring what matters to rare disease patients - reflections on the work by the IRDiRC taskforce on patient-centered outcome measures. Orphanet J Rare Dis. 2017;12(1):171.

    Article  PubMed  PubMed Central  Google Scholar 

  19. Act on the Reorganization of the Pharmaceutical Market within the Statutory Health Insurance (Arzneimittelmarktneuordnungsgesetz - AMNOG). Federal Gazette. 2010;I (67):2262.

  20. Institute for Quality and Efficiency in Health (Institut für Qualität und Wirtschaftlichkeit im gesundheitswesen – IQWiG). 2019. AMNOG since 2011. Accessed 7 Apr 2019.

  21. Council of the European Union. Regulation (EC) No 141/2000 of the European Parliament and Council of 16 December 1999 on orphan medicinal products. OJL. 1999;18:1.

  22. Federal Joint Committee (Gemeinsamer Bundesausschuss – GBA). Rules of procedure of the Federal Joint Committee [Verfahrensordnung des Gemeinsamen Bundesausschusses]. 2019. Accessed 7 April 2019.

  23. Federal Joint Committee (Gemeinsamer Bundesausschuss – GBA). Procedures of benefit assessment in accordance with §35a Volume V of the Social Code Book. 2019. Accessed 8 Apr 2019.

  24. Institute for Quality and Efficiency in Health (Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen – IQWiG). Documentation and acknowledgement of the official statements for the “Actualization of several sections of the general methods version 4.0 as well as new sections for the development of the general methods version 4.1.” Version 1.0 as of November 28, 2013. Accessed 8 Apr 2019.

  25. Institute for Quality and Efficiency in Health (Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen – IQWiG). Documentation and acknowledgement of the official statements for the draft of the general methods 4.2. Version 1.0 as of April 22, 2015. Accessed 8 Apr 2019

  26. Institute for Quality and Efficiency in Health (Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen – IQWiG). Documentation and acknowledgement of the official statements for the draft of the general methods 5.0. Version 1.0 as of July 10 2017. Accessed 8 Apr 2019.

  27. Braithwaite T, Calvert M, Gray A, Pesudovs K, Denniston AK. The use of patient-reported outcome research in modern ophthalmology: impact on clinical trials and routine clinical practice. Patient Relat Outcome Meas. 2019;10:9–24.

    Article  PubMed  PubMed Central  Google Scholar 

  28. Institute for Quality and Efficiency in Health (Institut für Qualität und Wirtschaftlichkeit im gesundheitswesen – IQWiG). Analytic Hierarchy Process (AHP) – Pilot project for the survey of patient preferences within the indication depression. Working paper. IQWiG-Reports – number 163. Publication date: 08.05.2013. Accessed 8 Apr 2019.

  29. Hummel MJ, Volz F, van Manen JG, Danner M, Dintsios CM, Ijzerman MJ, Gerber A. Using the analytic hierarchy process to elicit patient preferences: prioritizing multiple outcome measures of antidepressant drug treatment. Patient. 2012;5(4):225–37.

    Article  PubMed  Google Scholar 

  30. Institute for Quality and Efficiency in Health (Institut für Qualität und Wirtschaftlichkeit im gesundheitswesen – IQWiG). Choice based Conjoint-Analysis – Pilot project for the identification, weighting and prioritization of multiple attributes within the indication hepatitis C. IQWiG-Reports – Number 227. Working paper. Publication date: 23.07.2014. Version 1.1. Online: Accessed 8 Apr 2019.

  31. Mühlbacher AC, Bridges JF, Bethge S, Dintsios CM, Schwalm A, Gerber-Grote A, Nübling M. Preferences for antiviral therapy of chronic hepatitis C: a discrete choice experiment. Eur J Health Econ. 2017;18(2):155–65.

    Article  PubMed  Google Scholar 

  32. Babac A, Litzkendorf S, Schmidt K, Pauer F, Damm K, Frank M, von der Schulenburg JM G. Shaping an effective health information website on rare diseases using a group decision-making tool: inclusion of the perspectives of patients, their family members, and physicians. Interact J Med Res. 2017;6(2):e23.

    Article  PubMed  PubMed Central  Google Scholar 

  33. Pauer F, Schmidt K, Babac A, Damm K, Frank M, von der Schulenburg JM. Comparison of different approaches applied in Analytic Hierarchy Process - an example of information needs of patients with rare diseases. BMC Med Inform Decis Mak. 2016;16:117.

    Article  PubMed  PubMed Central  Google Scholar 

  34. Juckett DA, Davis FN, Gostine M, Reed P, Risko R. Patient-reported outcomes in a large community-based pain medicine practice: evaluation for use in phenotype modelling. BMC Med Informatics Dec Mak. 2015;15:41.

    Article  Google Scholar 

  35. Casamayor M, Palazzolo D, Gubbels L, Moïse P, Wijnands T, Mantuano M, Ivanescu C. Do patient reported outcomes (Pro) in oncology matter in health technology assessments (Hta)? doi:

    Article  Google Scholar 

  36. Ruof J, Schwartz FW, Schulenburg JM, Dintsios CM. Early benefit assessment (EBA) in Germany: analysing decisions 18 months after introducing the new AMNOG legislation. Eur J Health Econ. 2014;15(6):577–89.

    Article  PubMed  Google Scholar 

  37. Bender H, Dintsios CM. Health-Related Quality of Life in the Context of Early Benefit Assessment of Drugs According to § 35a of the German Social Code Book V: A Challenging Endpoint for all the Involved Stakeholders. Das Gesundheitswesen. 2018;80(02):132–43.

    CAS  Article  PubMed  Google Scholar 

  38. Greiner W, Witte J. AMNOG-Report 2018. medhochzwei Verlag GmbH. Heidelberg, 2018. ISBN 978-3-86216-478-3

  39. Obradovic M, Rauland M. Inclusion and consideration of patient preferences in Amnog Early Benefit Assessments. Value Health. 2014;17(7):A440. PMID: 27201180.

    CAS  Article  PubMed  Google Scholar 

  40. Theidel U, von der Schulenburg JM. Benefit assessment in Germany: implications for price discounts. Health Econ Rev. 2016;1:33.

    Article  Google Scholar 

  41. Schlander M, Dintsios CM, Gandjour A. Budgetary Impact and Cost Drivers of Drugs for Rare and Ultrarare Diseases. Value Health. 2018;21(5):525–31.

    Article  PubMed  Google Scholar 

  42. Bridges JF, Hauber AB, Marshall D, Lloyd A, Prosser LA, Regier DA, Johnson FR, Mauskopf J. Conjoint analysis applications in health--a checklist: a report of the ISPOR Good Research Practices for Conjoint Analysis Task Force. Value Health. 2011;14(4):403–13. Epub 2011 Apr 22.

    Article  PubMed  Google Scholar 

  43. Reed Johnson F, Lancsar E, Marshall D, Kilambi V, Mühlbacher A, Regier DA, Bresnahan BW, Kanninen B, Bridges JF. Constructing experimental designs for discrete-choice experiments: report of the ISPOR Conjoint Analysis Experimental Design Good Research Practices Task Force. Value Health. 2013;16(1):3–13.

    CAS  Article  PubMed  Google Scholar 

  44. Dintsios CM, Chernyak N, Grehl B, Icks A. Quantified patient preferences for lifestyle intervention programs for diabetes prevention-a protocol for a systematic review. Syst Rev. 2018;7(1):214.

    Article  PubMed  PubMed Central  Google Scholar 

  45. Joy SM, Little E, Maruthur NM, Purnell TS, Bridges JF. Patient preferences for the treatment of type 2 diabetes: a scoping review. Pharmacoeconomics. 2013;31(10):877–92.

    Article  PubMed  Google Scholar 

  46. Calvert M, Blazeby J, Altman DG, Revicki DA, Moher D, Brundage MD, CONSORT PRO Group. Reporting of patient-reported outcomes in randomized trials: the CONSORT PRO extension. JAMA. 2013;309(8):814–22.

    CAS  Article  PubMed  Google Scholar 

Download references


Not applicable.


No funding was attained for the research topic.

Author information

Authors and Affiliations



AB conducted the data analysis and drafted the manuscript. JMS and KD revised the document for important intellectual content. All authors have read and approved the final manuscript.

Corresponding author

Correspondence to Ana Babac.

Ethics declarations

Ethics approval and consent to participate

Not applicable.

Consent for publication

Not applicable.

Competing interests

The authors declare that they have no competing interests. AB is employed by the AOK Bayern.

Additional information

Publisher’s Note

Springer Nature remains neutral with regard to jurisdictional claims in published maps and institutional affiliations.

Rights and permissions

Open Access This article is distributed under the terms of the Creative Commons Attribution 4.0 International License (, which permits unrestricted use, distribution, and reproduction in any medium, provided you give appropriate credit to the original author(s) and the source, provide a link to the Creative Commons license, and indicate if changes were made.

Reprints and Permissions

About this article

Verify currency and authenticity via CrossMark

Cite this article

Babac, A., Damm, K. & Graf von der Schulenburg, JM. Patient-reported data informing early benefit assessment of rare diseases in Germany: A systematic review. Health Econ Rev 9, 34 (2019).

Download citation

  • Received:

  • Accepted:

  • Published:

  • DOI:


  • Patient perspective
  • patient-reported outcomes
  • health economic evaluation
  • early benefit assessment
  • Germany