Abstracts
Abstract
Competency-based medical education (CBME) shifts us from static assessment of learning to developmental assessment for learning. However, implementation challenges associated with CBME remain a major hurdle, especially after training and into practice. The full benefit of developmental assessment for learning over time requires collaboration, cooperation, and trust among learners, regulators, and the public that transcends each individual phase. The authors introduce the concept of an “Education Passport” that provides evidence of readiness to travel across the boundaries between undergraduate medical education, graduate medical education, and the expanse of practice.
The Education Passport uses programmatic assessment, a process of collecting numerous low stakes assessments from multiple sources over time, judging these data using criterion-referencing, and enhancing this with coaching and competency committees to understand, process, and accelerate growth without end. Information in the Passport is housed on a cloud-based server controlled by the student/physician over the course of training and practice. These data are mapped to various educational frameworks such Entrustable Professional Activities or milestones for ease of longitudinal performance tracking. At each stage of education and practice the student/physician grants Passport access to all entities that can provide data on performance. Database managers use learning analytics to connect and display information over time that are then used by the student/physician, their assigned or chosen coaches, and review committees to maintain or improve performance. Global information is also collected and analyzed to improve the entire system of learning and care.
Developing a true continuum that embraces performance and growth will be a long-term adaptive challenge across many organizations and jurisdictions and will require coordination from regulatory and national agencies. An Education Passport could also serve as an organizing tool and will require research and high-value communication strategies to maximize public trust in the work.
Résumé
La formation médicale fondée sur les compétences (FMFC) nous fait passer d’une évaluation statique à une évaluation évolutive de l’apprentissage. Cependant, les défis qui accompagnent sa mise en œuvre demeurent un obstacle majeur, en particulier après la formation et dans la pratique. Pour tirer pleinement parti de l’évaluation évolutive de l’apprentissage au fil du temps, il faut une collaboration, une coopération et une confiance entre les apprenants, les organismes de réglementation et le public qui transcendent chaque phase individuelle. Les auteurs présentent le concept de « passeport éducatif » en guise de titre attestant que l’on est prêt à franchir les frontières entre la formation médicale de premier cycle, la formation postdoctorale et l’étendue de la pratique.
Dans le passeport éducatif, on utilise l’évaluation programmatique, un processus qui consiste à rassembler de nombreuses évaluations à faible enjeu provenant de sources multiples au fil du temps, dont les données sont évaluées à l’aide de critères de référence et améliorées par un encadrement et un examen par des comités de compétences afin de comprendre, de développer et d’accélérer la croissance de façon continue. Les informations contenues dans le passeport sont hébergées sur un serveur nuagique contrôlé par l’étudiant/médecin au cours de sa formation et de sa pratique. Ces données sont cartographies en fonction de divers cadres éducatifs comme les activités professionnelles confiables ou des jalons pour faciliter le suivi longitudinal des performances. À chaque étape de la formation et de la pratique, l’étudiant/médecin accorde l’accès au passeport à toutes les entités qui peuvent fournir des données sur ses performances. Les gestionnaires de la base de données utilisent l’analyse de l’apprentissage pour recouper et afficher les informations au fil du temps, informations qui sont ensuite utilisées par l’étudiant/médecin, les coachs qu’on lui a désignés ou qu’il a choisis, et les comités d’examen pour maintenir ou améliorer les performances. Des informations globales sont également recueillies et analysées pour améliorer l’ensemble du système d’apprentissage et de soins.
L’élaboration d’un véritable continuum qui englobe la performance et la croissance constituera un défi d’adaptation à long terme pour les organisations et les provinces, et nécessitera une coordination entre instances réglementaires à l’échelle du pays. Le passeport éducatif pourrait également servir d’outil d’organisation, mais il impliquera des recherches et des stratégies de communication importantes pour maximiser la confiance du public dans ce travail.
Download the article in PDF to read it.
Download
Appendices
Bibliography
- McGaghie WC, Miller GE, Sajid AW, Telder TV. Competency-based curriculum development on medical education: an introduction. Public Health Pap. 1978;(68):11-91.
- Frank JR, Snell LS, Cate OT, et al. Competency-based medical education: theory to practice. Med Teach. 2010;32(8):638-45. https://doi.org/10.3109/0142159X.2010.501190
- van der Vleuten CP, Schuwirth LW, Driessen EW, Govaerts MJ, Heeneman S. 12 Tips for programmatic assessment. Med Teach. Nov 2014:1-6. https://doi.org/10.3109/0142159X.2014.973388
- Frank JR, Danoff D. The CanMEDS initiative: implementing an outcomes-based framework of physician competencies. Med teach. 2007;29(7):642-647. https://doi.org/10.1080/01421590701746983
- Edgar L, Roberts S, Holmboe E. Milestones 2.0: a step forward. JGME. 2018;10(3):367. https://doi.org/10.4300/JGME-D-18-00372.1
- Englander R, Flynn T, Call S, et al. Toward defining the foundation of the md degree: core Entrustable Professional Activities for entering residency. Acad Med. Oct 2016;91(10):1352-1358. https://doi.org/10.1097/acm.0000000000001204
- Coalition for Physician Accountability. Initial summary report and preliminary recommendations of the undergraduate medical education to graduate medical education review committee (UGRC). https://physicianaccountability.org/wp-content/uploads/2021/04/UGRC-Initial-Summary-Report-and-Preliminary-Recommendations-1.pdf [Accessed May 15, 2021].
- Santen SA, Foster KW, Hemphill RR, Christner J, Mejicano G. Simplifying the interconnected alphabet soup of medical education. Acad Med. 2022;97(5)
- Gingerich A, Sebok-Syer SS, Larstone R, Watling CJ, Lingard L. Seeing but not believing: insights into the intractability of failure to fail. Med Educ. Dec 2020;54(12):1148-1158. https://doi.org/10.1111/medu.14271
- Papadakis MA, Teherani A, Banach MA, et al. Disciplinary action by medical boards and prior behavior in medical school. N Engl J Med. Dec 2005;353(25):2673-82. https://doi.org/10.1056/NEJMsa052596
- Papadakis MA, Arnold GK, Blank LL, Holmboe ES, Lipner RS. Performance during internal medicine residency training and subsequent disciplinary action by state licensing boards. Ann Intern Med. Jun 2008;148(11):869-76. https://doi.org/10.7326/0003-4819-148-11-200806030-00009
- Jonker G, Ochtman A, Marty AP, Kalkman CJ, Ten Cate O, Hoff RG. Would you trust your loved ones to this trainee? Certification decisions in postgraduate anaesthesia training. Br J Anaesth. Nov 2020;125(5):e408-e410. https://doi.org/10.1016/j.bja.2020.07.009
- Dweck CS. Mindset: The new psychology of success. Random House Digital, Inc; 2008.
- Lucey CR, Hauer KE, Boatright D, Fernandez A. Medical education's wicked problem: achieving equity in assessment for medical learners. Acad Med. Dec 2020;95(12S Addressing harmful bias and eliminating discrimination in health professions learning environments):S98-s108. https://doi.org/10.1097/acm.0000000000003717
- Low D, Pollack SW, Liao ZC, et al. Racial/ethnic disparities in clinical grading in medical school. Teach Learn Med. Oct-Dec 2019;31(5):487-496. https://doi.org/10.1080/10401334.2019.1597724
- Williams RG, Klamen DA, McGaghie WC. Cognitive, social and environmental sources of bias in clinical performance ratings. Teach Learn Med. 2003;15(4):270-92. https://doi.org/10.1207/S15328015TLM1504_11
- Warm EJ, Kinnear B, Pereira A, Hirsh DA. The residency match: escaping the prisoner's dilemma. J Grad Med Educ. Oct 2021;13(5):616-625. https://doi.org/10.4300/jgme-d-21-00477.1
- Mendu ML, Kachalia A, Eappen S. Revisiting US news & world report's hospital rankings-moving beyond mortality to metrics that improve care. J Gen Intern Med. 2021:209-210. vol. 1. https://doi.org/10.1007/s11606-020-06002-x
- Ried LD, Ried DB. Predicting the 2016 US News & world report rankings using a reputation and prestige model. Curr Pharm Teach Learn. Feb 2021;13(2):91-101. https://doi.org/10.1016/j.cptl.2020.09.011
- Newsweek. Best Hospitals 2021 - Canada. 2021. https://www.newsweek.com/best-hospitals-2021/canada [Accessed May 9, 2021].
- Fang DZ, Young CB, Golshan S, Moutier C, Zisook S. Burnout in premedical undergraduate students. Acad Psych. 2012;36(1):11-16.
- Ishak W, Nikravesh R, Lederer S, Perry R, Ogunyemi D, Bernstein C. Burnout in medical students: a systematic review. Clin Teach. Aug 2013;10(4):242-5. https://doi.org/10.1111/tct.12014
- Prins JT, Gazendam-Donofrio SM, Tubben BJ, van der Heijden FM, van de Wiel HB, Hoekstra-Weebers JE. Burnout in medical residents: a review. Med Educ. Aug 2007;41(8):788-800. https://doi.org/10.1111/j.1365-2923.2007.02797.x
- West CP, Dyrbye LN, Shanafelt TD. Physician burnout: contributors, consequences and solutions. J Intern Med. Jun 2018;283(6):516-529. https://doi.org/10.1111/joim.12752
- Bynum WEt, Artino AR, Jr., Uijtdehaage S, Webb AMB, Varpio L. Sentinel emotional events: the nature, triggers, and effects of shame experiences in medical residents. Acad Med. Jan 2019;94(1):85-93. https://doi.org/10.1097/acm.0000000000002479
- Bynum WEt, Varpio L, Lagoo J, Teunissen PW. 'I'm unworthy of being in this space': the origins of shame in medical students. Med Educ. Feb 2021;55(2):185-197. https://doi.org/10.1111/medu.14354
- Albuquerque J, Tulk S. Physician suicide. Cmaj. May 6 2019;191(18):E505. https://doi.org/10.1503/cmaj.181687
- Schneider EC, Sarnak DO, Squires D, Shah A, Doty MM. Mirror Mirror 2017. Commonwealth Fund New York; 2017.
- McDonald FS, Duhigg LM, Arnold GK, Hafer RM, Lipner RS. The American board of internal medicine maintenance of certification examination and state medical board disciplinary actions: a population cohort study. J Gen Intern Med. Aug 2018;33(8):1292-1298. https://doi.org/10.1007/s11606-018-4376-z
- Naji L, Singh B, Shah A, et al. Global prevalence of burnout among postgraduate medical trainees: a systematic review and meta-regression. CMAJ Open. Jan-Mar 2021;9(1):E189-e200. https://doi.org/10.9778/cmajo.20200068
- Dreyfus SE. The five-stage model of adult skill acquisition. Bulletin of science, technology & society. 2004;24(3):177-181. https://doi.org/10.1177/0270467604264992
- Dunning D. The Dunning–Kruger effect: On being ignorant of one's own ignorance. Advances exper soc psychol. Elsevier; 2011:247-296. https://doi.org/10.1016/B978-0-12-385522-0.00005-6
- Cutrer W, Pusic M, Gruppen LD, Hammoud MM, Santen SA. The Master Adaptive Learner. Elsevier Health Sciences; 2019.
- Cutrer WB, Miller B, Pusic MV, et al. Fostering the development of master adaptive learners: a conceptual model to guide skill acquisition in medical education. Acad Med. Jan 2017;92(1):70-75. https://doi.org/10.1097/acm.0000000000001323
- Andrews JS, Bale JF, Jr., Soep JB, et al. Education in pediatrics across the continuum (EPAC): first steps toward realizing the dream of competency-based education. Acad Med. Mar 2018;93(3):414-420. https://doi.org/10.1097/acm.0000000000002020
- Warm EJ, Held JD, Hellmann M, et al. entrusting observable practice activities and milestones over the 36 months of an internal medicine residency. Acad Med. Oct 2016;91(10):1398-1405. https://doi.org/10.1097/acm.0000000000001292
- Hauer KE, Chesluk B, Iobst W, et al. Reviewing residents' competence: a qualitative study of the role of clinical competency committees in performance assessment. Acad Med. Aug 2015;90(8):1084-92. https://doi.org/10.1097/acm.0000000000000736
- Hauer KE, Edgar L, Hogan SO, Kinnear B, Warm E. The science of effective group process: lessons for clinical competency committees. J Grad Med Educ. Apr 2021;13(2 Suppl):59-64. https://doi.org/10.4300/jgme-d-20-00827.1
- Thoma B, Warm E, Hamstra SJ, et al. Next steps in the implementation of learning analytics in medical education: consensus from an international cohort of medical educators. J Grad Med Educ. Jun 2020;12(3):303-311. https://doi.org/10.4300/jgme-d-19-00493.1
- Warm EJ, Mathis BR, Held JD, et al. Entrustment and mapping of observable practice activities for resident assessment. J Gen Intern Med. Aug 2014;29(8):1177-82. https://doi.org/10.1007/s11606-014-2801-5
- Schauer DP, Kinnear B, Kelleher M, Sall D, Schumacher DJ, Warm EJ. Developing the expected entrustment score: accounting for variation in resident assessment. J Gen Intern Med. Apr 4 2022; https://doi.org/10.1007/s11606-022-07492-7
- Nousiainen MT, Mironova P, Hynes M, et al. Eight-year outcomes of a competency-based residency training program in orthopedic surgery. Med teach. 2018;40(10):1042-1054. https://doi.org/10.1080/0142159X.2017.1421751
- Chan T, Sebok-Syer S, Thoma B, Wise A, Sherbino J, Pusic M. Learning analytics in medical education assessment: the past, the present, and the future. AEM Educ Train. Apr 2018;2(2):178-187. https://doi.org/10.1002/aet2.10087
- Thoma B, Caretta-Weyer H, Schumacher DJ, et al. Becoming a deliberately developmental organization: using competency based assessment data for organizational development. Med Teach. May 25 2021:1-9. https://doi.org/10.1080/0142159x.2021.1925100
- Ten Cate O, Graafmans L, Posthumus I, Welink L, van Dijk M. The EPA-based Utrecht undergraduate clinical curriculum: development and implementation. Med Teach. May 2018;40(5):506-513. https://doi.org/10.1080/0142159x.2018.1435856
- Choe JH, Knight CL, Stiling R, Corning K, Lock K, Steinberg KP. Shortening the miles to the milestones: connecting epa-based evaluations to ACGME milestone reports for internal medicine residency programs. Acad Med. Jul 2016;91(7):943-50. https://doi.org/10.1097/acm.0000000000001161
- Wagner LM, Dolansky MA, Englander R. Entrustable professional activities for quality and patient safety. Nurs Outlook. May - Jun 2018;66(3):237-243. https://doi.org/10.1016/j.outlook.2017.11.001
- Kelleher M, Kinnear B, Wong SEP, O'Toole J, Warm E. Linking workplace-based assessment to ACGME milestones: a comparison of mapping strategies in two specialties. Teach Learn Med. Apr-May 2020;32(2):194-203. https://doi.org/10.1080/10401334.2019.1653764
- Gerstle CR. Parallels in safety between aviation and healthcare. J Pediatr Surg. May 2018;53(5):875-878. https://doi.org/10.1016/j.jpedsurg.2018.02.002
- Borrelli RA. Reflections on developing an identity for the third generation nuclear engineer in the post-Fukushima society. Reflections on the Fukushima Daiichi Nuclear Accident. Springer, Cham; 2015:353-365. https://doi.org/10.1007/978-3-319-12090-4_19
- Jonker CM, Riemsdijk M, Vermeulen B. Shared mental models. Springer; 132-151. https://doi.org/10.1007/978-3-642-21268-0_8
- Warm EJ, Englander R, Pereira A, Barach P. Improving Learner Handovers in Medical Education. Acad Med. Jul 2017;92(7):927-931. https://doi.org/10.1097/acm.0000000000001457
- Eva KW, Regehr G. "I'll never play professional football" and other fallacies of self-assessment. J Contin Educ Health Prof. Winter 2008;28(1):14-9. https://doi.org/10.1002/chp.150
- Ikizler TA, Berns JS. Transformation of ABIM and What the Changes Mean to Nephrologists. Clin J Am Soc Nephrol.2018;13(1):164-166. https://doi.org/10.2215/CJN.05700517
- Teirstein PS. Boarded to death--why maintenance of certification is bad for doctors and patients. N Engl J Med. Jan 8 2015;372(2):106-8. https://doi.org/10.1056/NEJMp1407422
- Boyd VA, Whitehead CR, Thille P, Ginsburg S, Brydges R, Kuper A. Competency-based medical education: the discourse of infallibility. Med Educ. Jan 2018;52(1):45-57. https://doi.org/10.1111/medu.13467
- Murray KE, Lane JL, Carraccio C, et al. Crossing the gap: using competency-based assessment to determine whether learners are ready for the undergraduate-to-graduate transition. Acad Med. Mar 2019;94(3):338-345. https://doi.org/10.1097/acm.0000000000002535
- Heath JK, Wang T, Santhosh L, et al. Longitudinal milestone assessment extending through subspecialty training: the relationship between acgme internal medicine residency milestones and subsequent pulmonary and critical care fellowship milestones. Acad Med. May 18 2021; https://doi.org/10.1097/acm.0000000000004165
- Heifetz RA, Laurie DL. The work of leadership. Harvard business review. 1997;75:124-134.
- Heifetz R, Linsky M. Leadership on the line, with a new preface: Staying alive through the dangers of change. Harvard Business Press; 2017.
- Groupsmith. Technical Problems vs. Adaptive Challenges. 2021. https://www.sgaumc.org/files/files_library/technical_vs_adaptive_challenges.pdf [Accessed May 9, 2021].