Patient-reported data informing early benefit assessment of rare diseases in Germany: A systematic review

Background Since the implementation of the Regulation on Patient Integration (2003), the Act on the Reorganization of the Pharmaceutical Market (2011), and the Patient Rights Law (2013), the inclusion of patient perspectives has been further anchored in the German early benefit assessment process. During the assessment of rare disease interventions, patient perspectives are particularly important, as clinical studies are often designed acknowledging small samples and patients suffering from severe symptoms and the chronic course of the disease. Therefore, our research question is whether patient perspectives are considered as part of early benefit assessments for rare diseases. We also strive to examine how patient perspectives are methodologically elicited and presented. Methods Our empirical evidence comes from a systematic review of orphan drug value dossiers submitted to the German Federal Joint Committee as well as the corresponding evaluations conducted between January 1, 2011 and March 1, 2019 (n = 81). Data on patient perspective integration were extracted using the following patient-reported outcome subcategories: clinical patient-reported outcomes, health-related quality of life, patient preferences, and patient satisfaction. Results The analysis demonstrates the specific relevance of patient-reported outcomes raised as part of the medical data set and presented during the early benefit assessment process. They are predominantly presented in the form of health-related quality of life data (n = 75%) and clinical outcomes (n = 49%). Preferences (n = 2%) and satisfaction (n = 1%) are still rarely presented, although the heated methodological discussion in Germany would suggest otherwise. While various methodologies for the integration of clinical outcomes and quality of life data were found, presenting data on satisfaction and preferences still lacks methodological rigor. The German Federal Joint Committee has not yet integrated these data in their decision text. Clinical outcomes and quality of life have been included in 46% and 73% of the cases, respectively. Conclusions The underlying analysis demonstrates that there is still a relative high potential for the regular and systematic inclusion of patient perspectives within the early benefit assessment process for rare diseases. In particular, patient preferences and patient satisfaction are still rarely included suggesting the need for a clear-cut methodological foundation and incentives.


Introduction
The relevance of patient perspective integration in health care In Germany, the 2003 Patient Participation Regulation [1] as well as the 2013 Patient Rights Law [2] form the most important basis for the integration of patient perspectives within the health care system. The Patient Participation Regulation, which is linked to §140f of the German Social Insurance Code Book Volume V, regulates the mandatory involvement of patient organizations in health care decision making. Within the German Federal Joint Committee, patient organizations now have the right to advise and request, but not to vote. In 2013, patients' position was further strengthened in terms of involvement and rights [1,2].
In the extant literature, patient-oriented health care systems are created to extend traditional health care models using patient empowerment. Therefore, important attributes of patient-oriented health care systems are identified as: high-quality information generation and transparency, patient perspective integration through customization and collaboration, as well as the integration of patient choice and responsibility. To these, predictive and preventive instruments can also be added [3]. Consequently, empirical health economics have turned towards the collection of evidence regarding patient views. Patients should be effectively integrated during the health technology assessment process, beginning with evidence generation and value measurement and concluding with recommendations and communication of results, e.g., in the context of health policy [4].
Patient perspectives can be integrated in different ways. While summarizing data contributed by patients or their representatives is most common, this approach must be differentiated from the studies carried out from the patient perspective but contributing data, for example, collected by physicians or other health care specialists [5]. As a methodologically-grounded approach, patient-reported outcomes (PRO) stand for the reports directly originating from patients [5,6] without the involvement of a physician or other communicators [5]. For instance, Klose et al. [7] report that the terms of PRO and outcome sometimes diverge in their interpretation. Used as a medical term, an outcome indicates an end result or intervention consequence in terms of symptoms and functioning, as well as the health-related quality of life (HRQoL). However, as reported by the PRO Harmonization Group, the discussion expanded from only including HRQoL outcomes to considering any outcome based on data provided by the patient or the patient's proxy [8]. In this study, we follow the broader interpretation of the term PRO, as provided by the PRO Harmonization group.
Since the definition of subcategories also varies broadly, we follow the nomenclature of Klose et al. [7]. This meant that clinical PRO, HRQoL, patient satisfaction, patient experiences, and patient preferences are differentiated. Within our analysis, it seemed preferable to differentiate between the clinical PRO following the traditional medical interpretation and HRQoL due to differences in mortality, morbidity, and quality of life endpoints within the process of early benefit assessment [2]. Furthermore, patient experiences were not acknowledged separately, as they are predominantly reported in the context of patient satisfaction and patient preferences. Preferences describe whether one item is favored over another [9], meaning they withhold information regarding preferable treatment options from the affected individuals. "Preference" is often used as an umbrella term. As such, a preference measurement can result in either utility or value depending on the measurement approach [10]. There may be various reasons for the consideration of patient preferences in health care, such as improved therapy uptake or efficiency of health care interventions in practice, thus facilitating patient involvement and promoting shared decision-making in medicine. Medical decisions that are consistent with patient preferences may facilitate patient acceptance [9,11,12]. Therefore, patient satisfaction is also considered an important component of medical intervention assessments. However, it is a rather subjective assessment of the quality of care and is often used to incorporate the patient's perspective on the quality of care as part of medical evaluations [12]. Patient satisfaction has not yet been well defined but is generally considered to describe a subjective assessment of medical care by patients [13]. Further, the concept can contain various elements such as medical therapy, nonmedical aspects of treatment, as well as health care infrastructure [14].
In general, the PRO can be raised via qualitative, mixed, and quantitative methodologies. Within the field of clinical PROs and HRQoL generic and disease-specific instruments are differentiated by incorporating symptomspecific modules. Patient preferences can be analyzed using contingent evaluation approaches, self-explication approaches, analytic hierarchy process, conjoint analysis, standard gamble, time trade-off approaches, as well as rating scales. For the examination of patient satisfaction, various approaches exist, such as the Patient Satisfaction Questionnaire and European Project on Patient Evaluation of General Practice Care Questionnaire [7].

Patient perspectives in the field of rare diseases
The term "rare diseases" summarizes between 5000 and 8000 different diseases that are characterized by their severity, by their genetic origin and status as life threatening, or by the chronically debilitating course of the disease. Within the European Union, a disease is called "rare" when fewer than 5 out of 10,000 people are affected. Despite major medical advances in general, a major unmet medical need has been identified within the field of rare disease, concerning diagnostic procedures and effective treatment strategies [15]. According to the German Health Ministry, in Germany alone approximately 4 million people are affected by rare diseases [16] and although this would seem to suggest that patient perspectives are particularly important within the field of rare diseases due to its heterogeneity. There is still a lack of research on the systematic inclusion of this field during health economic processes. As demonstrated by a review of 11 national strategies regarding patient engagement, the focus of political strategies rests with the involvement of patient organizations [17]. Admitting that traditional assessments fail to endorse medical technologies for rare diseases due to a lack of power, new endpoints have been explored with names such as Patient-Centered Outcome Measures [18].

Early benefit assessment for rare diseases in Germany
Since the Act on the Reorganization of the Pharmaceutical Market (AMNOG) within the statutory health insurance law issued in December 2010 came into effect in January 2011, all manufacturers need to provide evidence of the additional benefit of pharmaceutical products claimed over an appropriate comparator. The German Federal Joint Committee then decides whether and to what extent a drug can be granted an additional benefit and their decision forms the basis of price negotiations for the statutory health care setting [19]. Typically, the Institute for Quality and Efficiency in Health Care (Institut für Qualität und Wirtschaftlichkeit im Gesundheitswesen -IQWiG) is engaged to conduct early benefit assessment, and reports that patient perspectives play a key role in their judgments. Patient perspectives are generated using a standardized questionnaire regarding what is important to patients in terms of disease and treatment strategies [20].
However, in cases of drugs used solely for the treatment of rare diseases (orphan drugs), an additional benefit is presumed by the European drug approval, authorized in accordance with EC regulation number 141/ 2000 on orphan drugs [21]. In such cases, evidence must only be provided concerning the extent of the additional benefit to standard therapy for patients [22]. Here, the IQWiG is not involved in the benefit assessment, but in the estimation of patient numbers (target group, incidence, prevalence, and trends) as well as treatment costs. Only when the overall annual treatment costs of an orphan drug exceed the limit of 50 million euros for the statutory health insurance must it pass through the common early benefit assessment process [23]. Finally, decision making is taken over by the Federal Joint Committee, who describe patient involvement as the inclusion of patient representatives and patient organizations during the process but provide no further description of the procedure.

Aims of the study
To address this gap, this article examines how the integration of patient perspectives in the assessment of benefits proceeds. To this end, we examine the development of a methodologically grounded and direct approach to patient perspective integration, using the concept of patient reported outcomes, and analyzing the data submitted during early benefit assessments for rare diseases in Germany.

Data sources
Our empirical evidence comes from the database of the German Federal Joint Committee, withholding all procedures of early benefit assessment induced by §35a Volume V of the Social Code Book starting with the implementation of the AMNOG law [23]. All procedures with a starting date of between January 1, 2011 and March 1, 2019 were considered. In a second step, we filtered for procedures with an orphan drug status and only procedures marked "completed" were finally reviewed. Due to the exceeding of the 50-million-euro boundary or an extension of the area of application, newly developed active substances can be subject to multiple assessments.
The Federal Joint Committee makes available the following information on each procedure: the "dossier" submitted by the pharmaceutical company in accordance with the modular template, "benefit assessment," "comments procedure," and the "resolution" comprising "decisions" as well as the corresponding "rationales" [23]. Data were extracted from the dossier, in particular module 4, the benefit assessment, as well as the decision text developed by the Federal Joint Committee.

Strategy of analysis
To analyze the integration of patient perspectives within the data set, the PRO concept was used following the broad definition provided by Black (2003) [6] and Patrick et al. (2003) [5] linked with the nomenclature of Klose et al. (2016) [7]. Therefore, the following subcategories have been used during the underlying analysis: Patient-reported outcomes clinical data (clinical PROs) (Health-related) quality of life (HRQoL) Patient preferences Patient satisfaction.
We did not specifically analyze the presented data for adverse events as HRQoL registers adverse events as well and mortality captures the fatal adverse events. We particularly analyzed the data presented within the synopsis section of the dossiers, the benefit assessment, as well as the decision text. The respective text passages were screened particularly searching for terms such as "patient-reported outcomes," "quality of life", "patient preference" and "patient satisfaction". The identified sections were than extracted and transferred to a separate Excel sheet. Quantitative and qualitative data were reviewed equally. To provide a first impression on the relevance of the reported patient perspective within the early benefit assessment of rare diseases, we also examined the extent of the requested, and later granted, additional benefit as well as the methodologies actually considered by the GBA for each PRO data category.

Characteristics of the data sample
Our final sample contained n = 81 value dossiers. The first dossier was submitted on September 15, 2011 and the last on September 15, 2018. The different disease groups are shown in Table 1. A total of 51% of the dossiers within the field of rare diseases addressed oncological indications, while metabolic diseases were the second most common, at 25%.
Benefit assessments were predominantly (n = 71, 88%) conducted by the Federal Joint Committee itself. The IQWiG was commissioned with the rare diseases benefit assessment in 12% (n = 10) of cases, mostly when drugs were cross-passing the sales limit of 50 million euros (n = 7). In some cases, the manufacturer applied for an additional application area (n = 3). It should be noted that, whereas all agents cross passing the 50 million euro limit were commissioned to the IQWiG, applications for additional application areas were also assessed by the GBA itself. Table 2 shows the benefit scores that were applied for by the pharmaceutical companies during those processes, as well as the GBA score granted by the Federal Joint Committee. Applied and granted benefit scores matched in 19% of cases. No additional benefit was determined in 2% of cases as part of a reassessment after the trespassing of the 50 million Euro limit conducted by the IQWiG. Table 3 shows a summary of the analysis of patientreported data during the early benefit assessment process for rare diseases.

Analysis
PROs are mostly presented in the form of selfreported clinical outcomes data (n = 39, 48%) followed by data on HRQoL (n = 61, 75%). Data on patient preferences were included twice (2%) and on patient satisfaction only once (1%). In only 15% (n = 12) of cases, no data on PROs were submitted. Therefore, data on PROs were presented to a relatively high extent. In 85% of cases, PROs were presented from at least one PRO subcategory. The Federal Joint Committee considered clinical PRO data in 37 (46%) cases, whereas HRQoL data were included in 59 (73%) cases. Patient satisfaction and patient preferences were not included within the decision text. Another example, which was often considered as part of the GBA decision text (n = 13), was the EORTC (Core Quality of Life Questionnaire). The splitting of the questionnaire items into clinical PROs and QoL PROs could also be observed here. Table 4 shows the overall number of assessment processes for each year since the implementation of the AMNOG law. In 2017, 16 processes were initiated. Twelve processes presented data on Clinical PROs, 13 processes withheld HRQoL data, and 1 process presented data on Diseases of the digestive system 4 5% Diseases of the respiratory tract 2 2% Diseases of the blood and the blood-forming tissues 2 2% Diseases of the musculoskeletal system 2 2% Diseases of the nervous system 3 4%

Oncological diseases 41 51%
Metabolic diseases 20 25% preferences. Overall, the direct integration of patient perspectives in the form of PROs has gradually increased in its absolute number, with an increased number of induced processes since the implementation of the AMNOG law in 2011. Regarding the relative percentage of PROs in relation to the number of processes included within our analysis, no clear-cut trend is observable. Analyzing the clinical PRO and HRQoL data, it was observable that the GBA split the surveys into symptom scales (clinical PRO data listed as part of the morbidity endpoints) and HRQoL scales. An example is the oncology specific EORTC QLQ-C30. In this context, the EQ-5D VAS scale has been categorized as part of the morbidity section. However, the EQ-5D Index has been appreciated as part of the HRQoL section. Moreover, further commonly acknowledged methodologies were the childhood health questionnaires and the Brief Pain Inventory (BPI) as well as the Functional Assessment of Chronic Illness Therapy -Fatigue (FACIT-F). On the other hand, commonly acknowledged HRQoL methodologies included disease-specific FACT-questionnaires, SF-questionnaires, and the Pediatric Quality of Life inventory (PedsQL).
The heatedly discussed categories "patient satisfaction" and "patient preferences" were rarely referred to, and when they were, it was in a qualitative manner [24][25][26]. Quantitative methods were not used. The dossier submitted for the agent Velmanase alfa (2018) offered "patient cases" in the form of short summaries, backing clinical PROs as well as the relevance of symptoms (preferences) and overall quality of life (not naming HRQoL in this case). There was no description of the detailed qualitative research strategy. Telotristatethyl (2017) provided "semi-structured telephone-interviews" on topics such as symptom description, preferences, and patient experiences. Some structural background data were provided, but again no description of the qualitative research strategy was included. In the case of Eftrenonacog alfa (2016), during a first phase, "focus groups" were cited as well as a "structured questioning" that also considered satisfaction. Results were presented in a qualitative manner but there was no solid description of the qualitative research strategy. Patient satisfaction and patient preferences were not included at all within the GBA decision body, thus providing no incentives for further data presentation. A detailed overview of the data can be found in Table 5.

Summary of findings
The present study analyzes data on the direct involvement of the patient perspective, particularly in the form of PROs, which are submitted, evaluated, and considered during the early benefit assessment process for rare diseases in Germany. The results demonstrate that patient perspectives predominantly enter the process via clinical PROs and HRQoL. However, in comparison with clinical PRO and HRQoL, the categories "patient satisfaction" and "patient preferences" were rarely referred to, and if they were, it was in a qualitative manner that lacked a solid description of the methodological foundation within qualitative research. Nevertheless, we found that 16% of the orphan drug dossiers did not present any data on PROs.

Significance in the context of literature
To our knowledge, this study provides unique insights into the inclusion of patient perspectives within the early benefit analysis process, in particular using PROs as part of the early benefit assessment of orphan drugs.
Braithwaite et al. [27] highlight again the importance of PROs in the field of rare diseases since some of the methods used in this field of research permit smaller sample sizes. They also pinpoint the importance of primary outcome measures in general and that, in particular, traditional outcome measures have failed to demonstrate efficiency. While considerable progress has been made in the development of associated measures, it is still difficult to find tools for less common indications [27]. This may be one of the explanations for low acceptance and / or submission of PROs in the field of rare diseases. control Changes within the political framework can affect pharmaceutical companies' submission behavior. For example, before the introduction of the Patient Rights Law in 2013 [2], only three dossiers were submitted, while afterwards, the number of dossiers for orphan drugs increased to approximately 16 per year. However, the data cannot capture the possible impacts of changes in legislation since these events occur at a small rate and could be falsified by the overall orphan drug submission rate. Furthermore, the methodological developments within the health economic environment in Germany can also influence the development of data submission and its appraisal. In 2013, the IQWiG discussed changes to its methodology for the very first time. Institutes and industrial representatives argued for the direct, transparent, and systematic integration of patient perspectives, in particular patient preferences and the definition of the precise integration processes [24]. In 2015, another discussion of the IQWiG general methods paper was published. However, the focus of the discussion concerning patient perspectives integration was predominantly in relation to the reintegration of patient satisfaction as optional data [25]. Moreover, in 2017, the last recorded methodological discussion was published and, in this context, the systematic direct integration of patient perspectives was again demanded in several parts of the IQWiG methods paper, e.g., the clear-cut acknowledgement of patient preferences [26]. The first pilot projects concerning the measurement and inclusion of patient preferences in health economic evaluation were published in 2013 (Analytic Hierarchy Process (AHP) [28,29] and 2014 (Conjoint Analysis) [30,31]. Although the named projects and discussions seem to lay the basis for the methodologically grounded inclusion of patient perspectives, the first inclusions of patient preferences were recorded in 2017 and 2018 in the field of rare diseases. However, since many of the above-named quantitative methods are not appropriate in the field of rare diseases due to the limitation of small sample sizes (see also [18]) (an exception is the AHP [32,33]), further specified requirements for qualitative data presentation are required, as well as incentives for their adaptation. The same arguments hold for patient satisfaction, which is referred to as additional submittable data but in its patient-centeredness is relevant by definition.
Clinical patient-reported outcomes and health related quality of life Furthermore, in terms of particular relevance, it has been argued that the documentation of clinical PROs -for example, as part of the phenotype "pain" -offers the chance to better align treatment options and outcomes [34].
Casamayor et al. [35] analyzed whether PROs in oncology matter in health technology assessments conducted in Germany, France, and the UK, and found that an improvement in such outcomes did not increase the chance of a positive health technology assessment (HTA) recommendation. The authors also demonstrated that PROs assessing Quality of Life (51/57, 89.4%) and pain measures (18/57, 31.6%) are the most common. PROs were not mentioned at all in 35.1% of cases [35]. Although our analysis examines HRQoL measures and morbidity-focused clinical PROs, the tendencies of both research papers seem to be similar. An early stage analysis of the first 25 dossiers in Germany regarded independently of the targeted indication demonstrated that in the beginning HRQoL outcomes were not considered during the early benefit process for different reasons [36,37]. In our analysis, we found that this category was the most acceptable for the Joint Federal Committee. However, the general position of the Federal Joint Committee on the importance of quality of life data has changed significantly in the last 10 years. Initially rated as supporting or complementary information, quality of life data is today accounted equivalent to endpoints of mortality and morbidity [38]. Nevertheless, there are methodological questions regarding measurement and distinction that are not yet clear.

Patient preferences
The proportion of dossiers including data on patient preferences was quite low. Obradovic and Rauland [39] state that approximately 25% of all dossiers published between 2011 and 2014 referred to some extent to patient preferences. However, the database used seems to be more broadly designed. In the case of the present study, we included data from the as by the pharmaceutical company submitted studies but this also prompts further research questions regarding the differences between the integration of patient perspectives in the field of rare diseases and other indications. Of course, many quantitative measurement methods such as choice experiments/conjoint analyses are hardly feasible in the field of rare diseases. Furthermore, their specific aim is to compare different treatment methods (trade-off), which are often not provided in the field of rare diseases.    Oncologic diseases n/a n/a n/a n/a n/a n/a     Although we controlled for incentives to render data on patient perspectives in a direct and systematic way considering PROs, it must also be stated that there are some factors outside the set framework that may also influence the presentation of data. For example, the benefit score and associated documents form the basis for price negotiation in Germany [40,41].
Besides, the methodological foundation for patient preferences has also been developed in an international context and substantial literature has been published. The International Society for Pharmacoeconomics and Outcomes Research (ISPOR) developed a good research practice checklist for conjoint-analysis in health.
The checklist included 10 items covering the research question, levels and attributes, task development, the design of the experiment, preference elicitation, design of instruments, data-collection, analyzing statistical data, results and conclusions as well as study presentation standards. Even though, not endorsing a specific methodological approach, the checklist can serve as a good foundation for further discussions of good research practice for the application of conjoint-analysis methods in health care studies [42]. Besides, further research efforts give in depth advice concerning specific elements of the research process, for example the experimental design [43]. Several studies review the usage of different methodologies raising patient preferences systematically for different indications such as for example diabetes [44,45]. Further, CONSORT guidelines advise on the reporting of PRO data in general [46]. This study contributed to the existing literature by outlining the methodology of PRO data inclusion within the field of rare diseases in Germany.

Limitations
In terms of limitations, data on clinical PROs could only be identified as such as long as they were highlighted as a self-reported measure or indicated to be a patientreported measure. When no particular definition was provided, we assumed that the endpoint was physicianreported. We assume that almost all endpoints were specifically marked as patient-reported, as dossier providers have often argued that clinical PROs are particularly relevant to the patient and should therefore be specifically considered during the valuation and decisionmaking process. Furthermore, PROs are clearly defined as self-reported. However, in the case of clinical PROs, the reporting system was sometimes not indicated. In these cases, we searched for the primary classification of the symptoms scale.
In addition, pharmaceutical companies present their HRQoL data as a whole data subset. However, the Federal Joint Committee separates parts of the questionnaires selectively regarding mortality endpoints and HRQoL endpoints. Therefore, the data reveal a splitting of the datasets rendered by the pharmaceutical company. Endpoints were not shifted as this would not reflect the actual status quo of the data presented but would, rather, lead eventually to a presentation bias. However, it needs to be highlighted as a specific procedure presented by the German Federal Joint Committee and considered when selecting the appropriate data presentation technique.
Finally, it needs to be highlighted that some dossiers can fail due to formal reasons, for example not the appropriate comparator, a study population narrower than label etc. Therefore, the impact of PROs on the final decision is not always directly derivable.

Conclusions
The underlying evaluation demonstrates that although the political basis has been strengthened and the presented concepts have been broadly laid out as part of the health economic discussion in the context of benefit analysis and cost-benefit analysis, there remains a broad potential for the development of the practical framework regarding the systematic inclusion of patient perspectives, especially in referring to patient preferences and patient satisfaction, particularly considering the example of early benefit assessments for rare diseases in Germany. In this regard, it is interesting that patient preferences are presented in a qualitative manner. The broadly discussed and exemplified (by the IQWiG) quantitative methods have not been demonstrated in the field of rare diseases to date. While methodological standards for qualitative reporting have not yet been adopted, they must be appreciated with the same thoroughness as within quantitative research settings. An according clarification of the standard guidelines needs to be demanded. Moving even one step ahead, potentials of the integration of qualitative and quantitative research may be discussed, appreciated, and scientifically monitored in this specific context. Furthermore, the interim radiation of patient satisfaction has been commented on with vehement protest. In practice, however, it is only presented in 2% of cases in the field of rare diseases, even though this topic seems highly relevant due to the predominantly chronic and severe course of diseases. Neither of the PRO categories are enlisted within the GBA decision text. Acknowledged clinical PROs are often raised by the BPI-SF (pain scale) and FACIT-F (fatigue index). On the other hand, FACT-questionnaires, SF-questionnaires, and PedsQL are often GBAappreciated HRQoL PROs. It is noteworthy that HRQoL questionnaires are in many cases split with regard to morbidity and HRQoL items, as datasets produced by one questionnaire are submitted cohesively. In this regard, the EQ-5D VAS is often appreciated as a morbidity endpoint by the GBA and therefore, in this context, it is categorized as a clinical PRO, whereas the EQ-5D-Index is categorized as a HRQoL. Another commonly accepted example is the oncology indication specific EORTC QLQ-C30. This may lead to irritation, hindering the preparation of PRO data inclusion by pharmaceutical companies. Therefore, potential implications should be clarified.
Furthermore, the extent of PRO data presentation withholds considerable potentials. It is questionable whether morbidity-oriented clinical PROs should only be included in every second dossier, when it is highly relevant to the patient and to treatment success. Patient satisfaction and patient preferences follow by the same token. Appreciating the central role of patient perspectives within early benefit assessments and the according legal framework, the GBA decision text should particularly appreciate the consideration of patient perspectives, flagging incentives for more extensive consideration. Considering the growing financial pressure on health care systems, strengthening direct patient perspective involvement by further integrating PROs holds an immense opportunity to align health care with actual patient needs and therefore to contribute to an effective and needs-oriented health care system development.