Although the use of rating scales in clinical research has a long history [1], the use of Patient Reported Outcome (PRO) instruments is a more recent trend [2]. In the 1970s, the Health Insurance Experiment [3] helped a shift in focus from more traditional clinical outcomes (i.e., mortality and morbidity) to the measurement of function (i.e., the ability of patients to perform activities of daily living) [4]. In the 1980s, phrases such as “quality of life” started to become buzz words [5] and many ubiquitous generic PRO instruments were born (e.g., Sickness Impact Profile, [6] Nottingham Health Profile, [7] and Short Form-36 [8]). The 1990s saw a proliferation of more targeted PRO instruments, including dimension-specific (e.g., mood [9]), disease-specific (eg, cancer [10]), site-specific (e.g., orthopedic [11]), and individualized [12]. Since the turn of the century, there has increasing interest in embedding PRO instruments in high stakes decision making including clinical research [13], clinical trials [14] and within health information systems [15]. In fact, it has been suggested that PRO instruments can transform healthcare into a more patient-centered model [16] [17].

shutterstock_763440268 Spotlight October 2018

However, although guidelines exist for best practice use in these high-stakes settings [14] [18], some suggest existing legacy PRO instruments may not be up to the job of delivering the required high quality data. Reasons for this have included: psychometric limitations [2], lack of instrument content based on patient-identified priorities [19], difficulty of integration into clinical settings [20], little evidence of effective impact [21], and absence of measurement standards [22] [23]. In fact, the use of legacy PRO instruments in high stakes decision making has been described as being unscientific and unethical [5].


Did you know…

In recently stated strategic priorities, the US Food and Drug Administration (FDA) expressed an interest in ‘bridging studies to adapt and update legacy PRO instruments for new indications and populations’, page 12) [24]. In follow up to that report, this month (October, 2018) the FDA is hosting a two-day public workshop to further explore methodology surrounding PROs including the potential to modify existing instruments [25]. In line with this, Modus Outcomes has had a long-standing interest in improving legacy PRO instruments to make them fit-for-purpose for high stakes decision-making settings (Table 1). By exploiting the diagnostic capabilities [26] [27] of Rasch Measurement Theory, we have recommended adaptation, extension and enhancement to a number of widely used PRO instruments, to ensure better coverage of the concepts of interest for specific contexts of use [28]. For example, in order to improve psychometric properties, clinical meaning and range of clinical concepts covered we have proposed:

  • identification of clinically comfortable concept subsets, explore rescored response and enhancement (using bolt-on items) [29] of the Movement Disorder Society-sponsored revision of the Unified Parkinson’s Disease Rating Scale (MDS-UPDRS) [30]
  • reconceptualization and enhancement (using bolt-on items) [31][32] of the ABILHAND-56 [33]
  • rescoring response options and conceptualization and enhancement (using bolt-on items) [34] of the 12-item Multiple Sclerosis Walking Scale (MSWS-12) [35]
  • reconceptualization [36][37][38] of the Multiple Sclerosis Impact Scale (MSIS-29) [39]
  • identification of supplemental items [40] [41] for the European Organization for Research and Treatment of Cancer (EORTC) QLQ-C30 [10]
  • identification of clinically comfortable concept subsets [42], equating self- and proxy- report [43] [44] of DEMQOL [45]
  • reconceptualization and response option rescoring [46] [47] for the 25-item National Eye Institute Visual Functioning Questionnaire (NEI VFQ25) [48]
Oct 2018 Table 1

Table 1 • Summary of research improving legacy PRO instruments

We at Modus Outcomes are committed to ensuring the fair assessment of the potential benefit of new therapies and will continue to conduct research to maximize the potential of existing PRO instruments. Careful leveraging of legacy tests and rating scales addressing psychometric, conceptual, and targeting issues will lead to lead to more fit-for-purpose, relevant, meaningful and sensitive measurement of important clinical concepts in future pharmaceutical clinical trials.


Further Reading

[1] Codman EA. The product of a hospital. Surgery, Gynaecology and Obstetrics. 1914;18:491-496.
[2] Cano S, Hobart J. The problem with health measurement. Patient preference and adherence. 2011;5:279-290
[3] Brook RH, Ware JEJ, Davies-Avery A, et al. Conceptualization and measurement of health for adults in the health insurance study: Vol. VIII,  overview. Santa Monica,  California: The Rand Corporation; October 1979. R-1987/8-HEW.
[4] Stewart A, Hays R, Ware J. The MOS short-form general health survey: reliability and validity in a patient population. Medical Cate. 1988;26:724-735.
[5] Hunt S. The problem of quality of life. Quality of Life Research. 1997;6:205-212.
[6] Bergner M, Bobbitt R, Pollard W, Martin D, Gilson B. The Sickness Impact Profile: validation of a health status measure. Medical Care. 1976;14:57-67.
[7] Hunt S, McEwen J, McKenna S. Measuring Health Status. J R Coll Gen Pract. 1985; 35(273): 185–188.
[8] Ware JEJ, Sherbourne DC. The MOS 36-Item Short-Form Health Survey (SF-36): I.  Conceptual framework and item selection. Medical Care. 1992;30(6):473-483.
[9] Zigmond AS, Snaith RP. The Hospital Anxiety and Depression Scale. Acta Psychiatrica Scandinavica. 1983;67:361-370.
[10] Aaronson N, Ahmedzai S, Bergman B, et al. The European Organization for Research and Treatment of Cancer QLQ-C30: a quality-of-life instrument for use in international clinical trials in oncology. J Natl Cancer Inst. 1993;85:365-376.
[11] Dawson J, Fitzpatrick R, Carr A. Questionnaire on the perceptions of patients about shoulder surgery. The Journal of Bone and Joint Surgery (British). 1996;78-B(4):593-600.
[12] O'Boyle CA, McGee HM, Hickey A, O'Malley KM, Joyce CRB. Individual quality of life in patients undergoing hip replacement. Lancet. 1992;339:1088-1091.
[13] Basch E. Patient-Reported Outcomes - Harnessing Patients' Voices to Improve Clinical Care. N Engl J Med. 2017;376(2):105-108.
[14] Calvert M, Kyte D, Mercieca-Bebber R, et al. Guidelines for Inclusion of Patient-Reported Outcomes in Clinical Trial Protocols: The SPIRIT-PRO Extension. Jama. 2018;319(5):483-494.
[15] Devlin N, Appleby J. Getting the most out of PROMs: putting health outcomes at the heart of NHS decision-making. London, UK: Kings Fund;2010.
[16] Black N. Patient-reported outcome measures could help transform healthcare. BMJ. 2013;346:f167.
[17] Organisation for EconomicCo-operation and Development (OECD). The Next Generation of Health Reforms: Ministerial Statement. 2017.
[18] Food and Drug Administration.Guidance for Industry - Patient-Reported Outcome Measures: Use in Medical Product Development to Support Labeling Claims. 2009.
[19] Wiering B, de Boer D, Delnoij D. Patient involvement in the development of patient-reported outcome measures: a scoping review. Health Expectations. 2017;20(1):11-23.
[20] Coulter A. Measuring what matters to patients. BMJ. 2017;356:j816.
[21] Browne JP, Cano SJ, Smith S. Using Patient-reported Outcome Measures to Improve Health Care: Time for a New Approach. Med Care. 2017;55(10):901-904.
[22] Teisberg E. The Quality Tower Of Babel. Health Affairs Blog. 2015.
[23] Cano SJ, Pendrill LR, Barbic SP, Fisher WP. Patient-centred outcome metrology for healthcare decision-making. Journal of Physics: Conference Series. 2017.
[24] Food and Drug Administration. Value and Use of Patient-Reported Outcomes (PROs) in Assessing Effects of Medical Devices CDRH Strategic Priorities 2016-2017.
[25] Food and Drug Administration. Patient-Focused Drug Development Guidance: Methods to Identify What is Important to Patients and Select, Develop or Modify Fit-for-Purpose Clinical Outcome Assessments. 2018.
[26] Andrich D, de Jong JHAL, Sheridan BE. Diagnostic opportunities with the Rasch model for ordered response categories. In: Rost J, Langeheine R, eds. Applications of latent trait and latent class models in the social sciences. Munster, Germany: Waxmann Verlag GmbH; 1997:59-70.
[27] Hobart J, Cano S. Improving the evaluation of therapeutic intervention in MS: the role of new psychometric methods. Monograph for the UK Health Technology Assessment Programme. 2009;13(12):1-200.
[28] Food and Drug Administration. Roadmap to Patient-focused Outcome Measurement in Clinical Trials. 2013.
[29] Regnault A, Boroojerdi B, Meunier J, Bani M, Cano S. A psychometric analysis of the MDS-UPDRS-II and -III in early Parkinson’s disease: Do the numbers add up? International congress of Parkison’s disease and movement disorders; 2018; Hong Kong.
[30] Goetz C, Tilley B, Shaftman S, et al. Movement Disorder Society-Sponsored Revision of the Unified Parkinson’s Disease Rating Scale (MDS-UPDRS): Scale Presentation and Clinimetric Testing Results. Movement Disorders. 2008;23(15):2129–2170.
[31] Cano S, Cleanthous S, Marquis P, et al. Measuring Upper Limb Function In Multiple Sclerosis: Enhancing The Abilhand’s Performance. ISPOR; 2015; Philadelphia, USA.
[32] Cleanthous S, Strzok S, Pompilus F, et al. Addressing the targeting range of the ABILHAND-56 in relapsing remitting multiple sclerosis: A mixed methods psychometric study. Multiple Sclerosis Journal: Experimental, Translational and Clinical. 2018;4(2):2055217318776990.
[33] Penta M, Thonnard J, Tesio L. ABILHAND: A Rasch-Built measure of manual ability. Arch Phys Med Rehabilitation. 1998;79:1038-1042.
[34] Strzok S, Cleanthous S, Pompilus F, et al. Development of a gait module to complement the 12-item Multiple Sclerosis Walking Scale: a mixed methods study. Mult Scler J Exp Transl Clin. 2018;15(3):2055.
[35] Hobart JC, Riazi A, Lamping DL, Fitzpatrick R, Thompson AJ. Measuring the impact of MS on walking ability: The 12-Item MS Walking Scale (MSWS-12). Neurology. 2003;60(1):31-36.
[36] Cano S, Cleanthous S, Kinter E, et al. Measuring the impact of multiple sclerosis: enhancing the MSIS-29's performance. Congress of European Committee for Treatment and Research in Multiple Sclerosis; 2015; Barcelona, Spain.
[37] Cano S, Cleanthous S, Marquis P, et al. Measuring the Impact of Multiple Sclerosis: Enhancing the Performance of the MSIS-29 and MSWS-12. ISPOR; 2015; Milan, Italy.
[38] Cleanthous S, Cano S, Kinter E, et al. Measuring the impact of multiple sclerosis: Enhancing the measurement performance of the Multiple Sclerosis Impact Scale (MSIS-29) using Rasch Measurement Theory (RMT). Mult Scler J Exp Transl Clin. 2017;15(3):2055.
[39] Hobart JC, Lamping DL, Fitzpatrick R, Riazi A, Thompson AJ. The Multiple Sclerosis Impact Scale (MSIS-29): a rigorous patient-based measure of the impact of MS (abstract). Annals of Neurology. 2000;48:448.
[40] Bell JA, Galaznik A, Pompilus F, et al. Patient Perspectives on Symptom Experience and Impact on Health-Related Quality of Life in Higher-Risk Myelodysplastic Syndromes, Chronic Myelomonocytic Leukemia and Low-Blast Acute Myeloid Leukemia. American Society of Hematology annual meeting; 2017; Atlanta, GA, USA.
[41] Bell JA, Pompilus F, Galaznik A, et al. Enhancing the Content of the EORTC QLQ-C30 to Maximize the Detection of Treatment Benefit in Hematological Stem Cell Disorders Using the EORTC Item Library. American Society of Hematology annual meeting; 2017; Atlanta, GA, USA.
[42] Smith SC, Hendriks AAJ, Chrysanthaki T, Cano S, Black NA. Improving the measurement of HRQL in dementia: a Rasch measurement theory approach. ISOQOL; 2015; Vancouver, Canada.
[43] Smith SC, Hendriks AAJ, Chrysanthaki T, Cano S, Black N. How can we interpret proxy reports of HRQL when it is no longer possible to obtain a self-report from people with dementia? ISOQOL; 2015; Vancouver, Canada.
[44] Hendriks AAJ, Smith SC, Chrysanthaki T, Cano SJ, Black N. DEMQOL and DEMQOL-Proxy: a Rasch analysis. Health Qual Life Outcomes. 2017;15(1):164.
[45] Smith S, Lamping D, Banerjee S, et al. Measurement of health-related quality of life for people with dementia: development of a new instrument (DEMQOL) and an evaluation of current methodology. Health Technology Assessment. 2005;9(10).
[46] Petrillo J, Ferreira A, Cano S, Bressler NM, Lamoureux E. Psychometric Performance Of The NEI VFQ-25: Rasch Analysis Of The NEI VFQ-25 As A Measure Of Patient-Reported Visual Function Across Four Retinal Diseases. ISPOR; 2013; New Orleans, LA, USA.
[47] Petrillo J, Bressler N, Lamoureux E, Ferreira A, Cano S. Development of a new Rasch-based scoring algorithm for the National Eye Institute Visual Functioning Questionnaire to improve its interpretability. Health and Quality of Life Outcomes. 2017;15:157.
[48] Mangione CM, Lee PP, Gutierrez PR, Spritzer K, Coleman AL. Development of the 25-item National Eye Institute visual function questionnaire. Arch Ophthalmol. 2001;119:1050-1058.