Abstracts
Abstract
Multisource feedback (MSF), often termed 360-degree feedback, is a formative performance assessment in which data about an individual’s observable workplace behaviors are collected through questionnaires from those interacting with the individual; data are aggregated for anonymity and confidentiality; the aggregated data, along with self-assessment if available, are provided to the individual; and the recipient meets with a trusted individual to review the data and develop an action plan. It is used along the continuum of medical education. This article provides an overview of MSF’s utility, its evidence base and cautions.
Résumé
La rétroaction multisource (RMS), ou rétroaction 360 degrés, est une évaluation formative du rendement dans laquelle des informations sur les comportements observables d’un individu dans son lieu de travail sont recueillies par le biais de questionnaires auprès de ceux avec qui il interagit. Après avoir été agrégées pour garantir l’anonymat et la confidentialité, ces données, et l’auto-évaluation s’il y a lieu, sont remises à la personne évaluée. Accompagnée d’une personne de confiance, elle les examinera et élaborera un plan d’action, qui sera utilisé tout au long du continuum de sa formation médicale. Cet article présente un aperçu de l’utilité et des fondements de la RMS, ainsi que quelques mises en garde.
Appendices
Bibliography
- Lockyer JM, Sargeant J, Introduction to MCC 360: a multi-source feedback initiative. Prepared for Medical Council of Canada, Ottawa, Canada, Nov 17, 2017. https://mcc.ca/media/MCC-360-MSF-Guidelines.pdf [Accessed Oct 12, 2021]
- Lockyer J, Multi Source Feedback, in ES Holmboe, RE Hawkins, SJ Durning (eds), A practical guide to the assessment of clinical competence, 2nd Ed. Elsevier. 2018. Paperback ISBN: 9780323447348; eBook ISBN: 9780323448949. https://www.elsevier.com/books/practical-guide-to-the-evaluation-of-clinical-competence/holmboe/978-0-323-44734-8
- Kane MT. Validating the Interpretations and Uses of Test Scores. J Educ Meas. 2013; 50: 1-73. https://doi.org/10.1111/jedm.12000
- Cook DA, Brydges R, Ginsburg S, Hatala R. A contemporary approach to validity arguments: a practical guide to Kane's framework. Med Educ. 2015;49(6):560-75. https://doi.org/10.1111/medu.12678.
- Moonen-van Loon JM, Overeem K, Donkers HH, van der Vleuten CP, Driessen EW. Composite reliability of a workplace-based assessment toolbox for postgraduate medical education. Adv Health Sci Educ Theory Pract. 2013;18(5):1087-102. https://doi.org/10.1007/s10459-013-9450-z.
- Donnon T, Al Ansari A, Al Alawi S, Violato C. The reliability, validity, and feasibility of multisource feedback physician assessment: a systematic review. Acad Med. 2014;89(3):511-6. https://doi.org/10.1097/ACM.0000000000000147.
- Stevens S, Read J, Baines R, Chatterjee A, Archer J. Validation of multisource feedback in assessing medical performance: a systematic review. J Contin Educ Health Prof. 2018;38(4):262-268. https://doi.org/10.1097/CEH.0000000000000219.
- Lockyer J. Multisource feedback: can it meet criteria for good assessment? J Contin Educ Health Prof. 2013;33(2):89-98. https://doi.org/10.1002/chp.21171.
- Norcini J, Anderson MB, Bollela V, et al. 2018 Consensus framework for good assessment. Med Teach. 2018;40(11):1102-1109. https://doi.org/10.1080/0142159X.2018.1500016.
- Francois J, Sisler J, Mowat S. Peer-assisted debriefing of multisource feedback: an exploratory qualitative study. BMC Med Educ. 2018;18(1):36. https://doi.org/10.1186/s12909-018-1137-y.
- Pooley M, Pizzuti C, Daly M. Optimizing multisource feedback implementation for Australasian physicians. J Contin Educ Health Prof. 2019;39(4):228-235. https://doi.org/10.1097/CEH.0000000000000267.
- Arabsky S, Castro N, Murray M, Bisca I, Eva KW. The influence of relationship-centered coaching on physician perceptions of peer review in the context of mandated regulatory practices. Acad Med. 2020; 95 (11S Association of American Medical Colleges Learn Serve Lead: Proceedings of the 59th Annual Research in Medical Education Presentations):S14-S19. https://doi.org/10.1097/ACM.0000000000003642.
- Hu J, Lee R, Mullin S, Schwaitzberg S, Harmon L, Gregory P, Elkin PL. How physicians change: multisource feedback driven intervention improves physician leadership and teamwork. Surgery. 2020;168(4):714-723. https://doi.org/10.1016/j.surg.2020.06.008.
- Sargeant J, Lockyer J, Mann K, et al. facilitated reflective performance feedback: developing an evidence- and theory-based model that builds relationship, explores reactions and content, and coaches for performance change (R2C2). Acad Med. 2015; 90(12):1698-706. Https://doi.org/10.1097/ACM.0000000000000809.
- van der Meulen MW, Boerebach BC, Smirnova A et al. Validation of the INCEPT: a multisource feedback tool for capturing different perspectives on physicians' professional performance. J Contin Educ Health Prof. 2017;37(1):9-18. https://doi.org/10.1097/CEH.0000000000000143.
- House JB, Franko LR, Haque F, Cranfod JA, Santen SA, Variation in assessment of first-year medical students' interprofessional competencies by rater profession, J. Interprof. Educ. Pract. 2021 (24); 100424. ISSN 2405-4526 https://doi.org/10.1016/j.xjep.2021.100424
- Lai MMY, Roberts N, Mohebbi M, Martin J. A randomised controlled trial of feedback to improve patient satisfaction and consultation skills in medical students. BMC Med Educ. 2020;20(1):277. https://doi.org/10.1186/s12909-020-02171-9.
- Emke AR, Cheng S, Chen L, Tian D, Dufault C. A Novel Approach to Assessing Professionalism in Preclinical Medical Students Using Multisource Feedback Through Paired Self- and Peer Evaluations. Teach Learn Med. 2017;29(4):402-410. https://doi.org/10.1080/10401334.2017.1306446. Epub 2017 May 12. PMID: 28498003.
- Prediger S, Fürstenberg S, Berberat PO, Kadmon M, Harendza S. Interprofessional assessment of medical students’ competences with an instrument suitable for physicians and nurses. BMC Med Educ. 2019;19(1):46. Https://doi.org/10.1186/s12909-019-1473-6. PMID: 30728006; PMCID: PMC6364398.
- Prediger S, Schick K, Fincke F, Fürstenberg S, Oubaid V, Kadmon M, Berberat PO, Harendza S. Validation of a competence-based assessment of medical students’ performance in the physician’s role. BMC Med Educ. 2020;20(1):6. Https://doi.org/10.1186/s12909-019-1919-x. PMID: 31910843; PMCID: PMC6947905.
- Lockyer JM, Violato C, Fidler H. The assessment of emergency physicians by a regulatory authority. Acad Emerg Med. 2006;13(12):1296-303. Https://doi.org/10.1197/j.aem.2006.07.030. Epub 2006 Nov 10. PMID: 17099191.
- Lockyer JM, Violato C, Fidler H, Alakija P. The assessment of pathologists/laboratory medicine physicians through a multisource feedback tool. Arch Pathol Lab Med. 2009;133(8):1301-8. Https://doi.org/10.5858/133.8.1301.
- Holmboe ES, Durning SJ, Hawkins RE, Practical guide to the evaluation of clinical competence 2nd ed, Philadelphia: Elsevier 2018. https://www.elsevier.com/books/practical-guide-to-the-evaluation-of-clinical-competence/holmboe/978-0-323-44734-8
- Gregory PJ, Robbins B, Schwaitzberg SD, Harmon L. Leadership development in a professional medical society using 360-degree survey feedback to assess emotional intelligence. Surg Endosc. 2017;31(9):3565-3573. https://doi.org/10.1007/s00464-016-5386-8.
- Rourke J, Bornstein S, Vardy C, Speed D, White T, Corbett P. Evaluation of and Feedback for Academic Medicine Leaders: Developing and Implementing the Memorial Method. Acad Med. 2017;92(11):1590-1594. https://doi.org/10.1097/ACM.0000000000001722.
- Desmedt M, Ulenaers D, Grosemans J, Hellings J, Bergs J. Clinical handover and handoff in healthcare: a systematic review of systematic reviews. Int J Qual Health Care. 2021;33(1):mzaa170. https://doi.org/10.1093/intqhc/mzaa170
- Bonds RL. SBAR Tool Implementation to advance communication, teamwork, and the perception of patient safety culture. Creative nursing, 2018; 24(2), 116–123. https://doi-org.ezproxy.lib.ucalgary.ca/10.1891/1078-4535.24.2.116
- Etherington C, Wu M, Cheng-Boivin O, Larrigan S, Boet S. Interprofessional communication in the operating room: a narrative review to advance research and practice. Communication interprofessionnelle en salle d’opération: un compte rendu narratif pour faire avancer la recherche et la pratique. Can J Anaesth. 2019;66(10):1251-1260. https://doi.org/10.1007/s12630-019-01413-9
- Chladek MS, Doughty C, Patel B et al. The standardisation of handoffs in a large academic paediatric emergency department using I-PASS. BMJ Open Qual. 2021;10(3):e001254. https://doi.org/10.1136/bmjoq-2020-001254.
- Narayanan A, Farmer EA, Greco MJ. Multisource feedback as part of the medical board of Australia's professional performance framework: outcomes from a preliminary study. BMC Med Educ. 2018;18(1):323. https://doi.org/10.1186/s12909-018-1432-7. Erratum in: BMC Med Educ. 2019 Mar 4;19(1):73.
- Medical Council of Canada, MCC 360, https://mcc.ca/assessments/mcc360/ [Accessed Feb 15, 2022].
- 360 PULSE Program: quality at work https://pulseprogram.com. [Accessed Feb 15, 2022].
- Saedon H, Salleh S, Balakrishnan A, Imray CH, Saedon M. The role of feedback in improving the effectiveness of workplace based assessments: a systematic review. BMC Med Educ. 2012; 12:25. https://doi.org/10.1186/1472-6920-12-25.
- Alofs L, Huiskes J, Heineman MJ, et al. User reception of a simple online multisource feedback tool for residents. Perspect Med Educ. 2015 (2):57-65. https://doi.org/10.1007/s40037-015-0173-0.
- Yama BA, Hodgins M, Boydell K, Schwartz SB. A qualitative exploration: questioning multisource feedback in residency education. BMC Med Educ. 2018;18(1):170. https://doi.org/10.1186/s12909-018-1270-7.
- G, Bandiera G. Multisource feedback and self-assessment of the communicator, collaborator, and professional CanMEDS roles for diagnostic radiology residents. Can Assoc Radiol J. 2014;65(4):379-84. https://doi.org/10.1016/j.carj.2014.04.003.
- Beaudoin PL, Labbé M, Fanous A, et al. Teaching communication skills to OTL-HNS residents: multisource feedback and simulated scenarios. J Otolaryngol Head Neck Surg. 2019;48(1):8. https://doi.org/10.1186/s40463-019-0329-8.
- Castonguay V, Lavoie P, Karazivan P, Morris J, Gagnon R. Perceptions of emergency medicine residents of multisource feedback: different, relevant, and useful information. Ann Emerg Med. 2019;74(5):660-669. https://doi.org/10.1016/j.annemergmed.2019.05.019.
- Buis CAM, Eckenhausen MAW, Ten Cate O. Processing multisource feedback during residency under the guidance of a non-medical coach. Int J Med Educ. 2018; 23;9:48-54. https://doi.org/10.5116/ijme.5a7f.169d.
- Wenghofer EF, Campbell C, Marlow B, Kam SM, Carter L, McCauley W. The effect of continuing professional development on public complaints: a case-control study. Med Educ. 2015;49(3):264-75. https://doi.org/10.1111/medu.12633.
- Chaulk D, Krueger C, Stang AS. A retrospective review of physician-related patient complaints from a tertiary pediatric hospital. Pediatr Qual Saf. 2019 13;4(1):e136. https://doi.org/10.1097/pq9.0000000000000136.
- Saha R, Kabanovski A, Klejman S, Margolin E, Buys YM. Patients' complaints involving ophthalmologists in the province of Ontario, Canada: a 5-year review. Can J Ophthalmol. 2020;55(3 Suppl 1):22-26. https://doi.org/10.1016/j.jcjo.2019.08.007.
- Brennan N, Bryce M, Pearson M, Wong G, Cooper C, Archer J. Towards an understanding of how appraisal of doctors produces its effects: a realist review. Med Educ. 2017 Oct;51(10):1002-1013. Https://doi.org/10.1111/medu.13348.
- Lockyer J, Sargeant J, Implementing multisource feedback, in D Boud and E Molloy (Eds), Feedback in Higher and Professional Education, London, Routledge, 2013. ISBN: 9781135107468. https://www.routledge.com/Feedback-in-Higher-and-Professional-Education-Understanding-it-and-doing/Boud-Molloy/p/book/9780415692298
- Moonen–van Loon JM, Overeem K, Govaerts MJ, Verhoeven BH, van der Vleuten CP, Driessen EW. The reliability of multisource feedback in competency-based assessment programs. Acad Med, 2015; 90 (8), 1093-1099. Https://doi.org/10.1097/ACM.0000000000000763.
- Al Khalifa K, Al Ansari A, Violato C, Donnon T. Multisource feedback to assess surgical practice: a systematic review. J surg educ, 2013; 70(4), 475–486. https://doi-org.ezproxy.lib.ucalgary.ca/10.1016/j.jsurg.2013.02.002
- Andrews JJ, Violato C, Al Ansari A, Donnon T, Pugliese G. Assessing psychologists in practice: Lessons from the health professions using multisource feedback. Professional psychology: research and practice. 2013;44(4):193.
- Sargeant JM, Holmboe ES, Feedback and coaching in clinical teaching and learning. ES Holmboe, RE Hawkins, SJ Durning (eds), A practical guide to the assessment of clinical competence, 2nd Ed. Elsevier. 2018. Paperback ISBN: 9780323447348; eBook ISBN: 9780323448949. https://www.elsevier.com/books/practical-guide-to-the-evaluation-of-clinical-competence/holmboe/978-0-323-44734-8
- Ashworth N, de Champlain AF, Kain N, A review of multi-source feedback focusing on psychometrics, pitfalls and some possible solutions, Springer Nature Social Sciences, 2021 I:24. https://link.springer.com/article/10.1007/s43545-020-00033-1