Abstracts
Abstract
Background: Competency-based medical education (CBME) is an outcomes-based curricular paradigm focused on ensuring that graduates are competent to meet the needs of patients. Although resident engagement is key to CBME’s success, few studies have explored how trainees have experienced CBME implementation. We explored the experiences of residents in Canadian training programs that had implemented CBME.
Methods: We conducted semi-structured interviews with 16 residents in seven Canadian postgraduate training programs, exploring their experiences with CBME. Participants were equally divided between family medicine and specialty programs. Themes were identified using principles of constructivist grounded theory.
Results: Residents were receptive to the goals of CBME, but in practice, described several drawbacks primarily related to assessment and feedback. For many residents, the significant administrative burden and focus on assessment led to performance anxiety. At times, residents felt that assessments lacked meaning as supervisors focused on “checking-boxes” or provided overly broad, non-specific comments. Furthermore, they commonly expressed frustration with the perceived subjectivity and inconsistency of judgments on assessments, especially if assessments were used to delay progression to greater independence, contributing to attempts to "game the system". Faculty engagement and support improved resident experiences with CBME.
Conclusion: Although residents value the potential for CBME to improve the quality of education, assessment and feedback, the current operationalization of CBME may not be consistently achieving these objectives. The authors suggest several initiatives to improve how residents experience assessment and feedback processes in CBME.
Résumé
Contexte : La formation médicale axée sur les compétences (FMFC) est un paradigme d’apprentissage axé sur les résultats et visant à garantir que les diplômés aient les compétences nécessaires pour répondre aux besoins des patients. Bien que l’engagement des résidents soit la clé du succès de la FMFC, peu d’études ont exploré comment ils vivent son introduction. Nous nous sommes penchés sur l’expérience des résidents dans les programmes de formation canadiens qui ont mis en œuvre la FMFC.
Méthodes : Nous avons mené des entrevues semi-structurées avec 16 résidents de sept programmes de formation postdoctorale canadiens, afin de sonder leur expérience de la FMFC. Les participants provenaient de façon égale de la médecine familiale et de programmes de spécialité. Les thèmes ont été dégagés en appliquant les principes de la théorie enracinée constructiviste.
Résultats : Bien que réceptifs aux objectifs de la FMFC, les résidents décrivent des inconvénients de sa mise en pratique, notamment sur le plan de l’évaluation et de la rétroaction. Pour beaucoup d’entre eux, la focalisation sur l’évaluation et le fardeau administratif qui y est lié ont été une source d’anxiété de performance. Les résidents ont l’impression que les évaluations manquent parfois de pertinence, car les superviseurs, se sentant contraints de « cocher des cases », font des commentaires trop généraux et peu ciblés. De plus, un sentiment de frustration a été fréquemment exprimé face à la subjectivité et à l’incohérence perçues des jugements dans les évaluations, surtout lorsque ces dernières sont utilisées pour retarder le cheminement vers une plus grande indépendance, ce qui contribue à des tentatives de « déjouer le système ». L’implication et le soutien du corps professoral ont aidé à bonifier l’expérience des résidents.
Conclusion : Bien que les résidents apprécient le potentiel de la FMFC pour rehausser la qualité de la formation, de l’évaluation et de la rétroaction, son opérationnalisation actuelle ne permet pas d’atteindre ces objectifs de façon systématique. Les auteurs proposent quelques initiatives pour améliorer la façon dont les résidents vivent les processus d’évaluation et de rétroaction dans le cadre de la FMFC.
Download the article in PDF to read it.
Download
Appendices
Bibliography
- Frank JR, Snell LS, Cate O Ten, et al. Competency-based medical education: theory to practice. Med Teach. 2010;32:638–45. https://doi.org/10.3109/0142159X.2010.501190
- ten Cate O. Competency-based postgraduate medical education: past, present and future. GMS J Med Educ. 2017;17;34. https://doi.org/10.3205/zma001146
- Tannenbaum D, Kerr J, Konkin J, et al. Triple C competency-based curriculum. report of the working group on postgraduate curriculum review – part 1. Coll Fam Physicians Canada. 2011;1:101.
- Oandasan I, Wong E, Saucier D, Donoff M, Iglar K, Schipper S. Triple C: linking curriculum and assessment. Can Fam Physician. 2012;58:1165.
- The Royal College of Physicians and Surgeons of Canada. What is CBD? Available from: https://www.royalcollege.ca/rcsite/cbd/what-is-cbd-e [Accessed Jun 30, 2020].
- Ten Cate O, Scheele F, Ten Cate TJ. Viewpoint: competency-based postgraduate training: can we bridge the gap between theory and clinical practice? Acad Med. 2007;82:542–7. https://doi.org/10.1097/ACM.0b013e31805559c7
- Lockyer J, Carraccio C, Chan MK, et al. Core principles of assessment in competency-based medical education. Med Teach. 2017;39:609–16. https://doi.org/10.1080/0142159X.2017.1315082
- Carraccio C, Wolfsthal SD, Englander R, Ferentz K, Martin C. Shifting paradigms: from flexner to competencies. Acad Med. 2002;77:361–7. https://doi.org/10.1097/00001888-200205000-00003
- Van Melle E, Frank JR, Holmboe ES, Dagnone D, Stockley D, Sherbino J. A Core components framework for evaluating implementation of competency-based medical education programs. Acad Med. 2019;94:1002–9. https://doi.org/10.1097/ACM.0000000000002743
- Oandasan IF, Saucier D, (eds). Triple C Competency-based Curriculum Report – Part 2 Advancing Implementation Mississauga, ON: College of Family Physicians of Canada; 2013. https://portal.cfpc.ca/resourcesdocs/uploadedFiles/Education/_PDFs/TripleC_Report_pt2.pdf. [Accessed Sept 20, 2020].
- Cate O Ten, Carraccio C. Envisioning a true continuum of competency-based medical education, training, and practice. Acad Med. 2019;94:1283–8. https://doi.org/10.1097/ACM.0000000000002687
- Gofton W, Dudek N, Barton G, Bhanji F. Workplace-based assessment implementation guide: Formative tips for medical teaching practice. The Royal College of Physicians and Surgeons of Canada. 2017;1st ed.:1–12. Available at: 12. http://www.royalcollege.ca/rcsite/documents/cbd/wba-implementation-guide-tips-medical-teaching-practice-e.pdf
- Holmboe ES, Sherbino J, Long DM, Swing SR, Frank JR. The role of assessment in competency-based medical education. Med Teach. 2010;32:676–82. https://doi.org/10.3109/0142159X.2010.500704
- Frank JR, Mungroo R, Ahmad Y, Wang M, De Rossi S, Horsley T. Toward a definition of competency-based education in medicine: a systematic review of published definitions. Med Teach. 2010;32:631–7. https://doi.org/10.3109/0142159X.2010.500898
- Gruppen LD, ten Cate O, Lingard LA, Teunissen PW, Kogan JR. Enhanced requirements for assessment in a competency-based, time-variable medical education system. Acad Med. 2018;93:S17–21. https://doi.org/10.1097/ACM.0000000000002066
- ten Cate O. Entrustability of professional activities and competency-based training. Med Educ. 2005;39:1176–7. https://doi.org/10.1111/j.1365-2929.2005.02341.x
- Ellaway RH, Mackay MP, Lee S, et al. The impact of a national competency-based medical education initiative in family medicine. Acad Med. 2018;93:1850–7. https://doi.org/10.1097/acm.0000000000002387
- Hsu T, De Angelis F, Al-asaaed S, Basi SK, Tomiak A, Grenier D, et al. Ten ways to get a grip on designing and implementing a competency-based medical education training program. Can Med Educ J. 2021;12. https://doi.org/10.36834/cmej.70723
- ICE Blog. Introducing a core components framework for competency-based medical education. Available from: https://icenetblog.royalcollege.ca/2021/11/18/introducing-a-core-components-framework-for-cbme/ [Accessed on Dec 30, 2021].
- Lim J, Westerman ME, Stewart NH, Correa R, Eno C. Trainee Perspectives on the writing and implementation of milestones 2.0. J Grad Med Educ. 2021;13:8–10. https://doi.org/10.4300/JGME-D-20-00859.1
- Carraccio C, Englander R, Van Melle E, et al. Advancing competency-based medical education: A charter for clinician-educators. Acad Med. 2016;91(5):645-9. https://doi.org/10.1097/ACM.0000000000001048
- Harrison CJ, Könings KD, Schuwirth L, Wass V, van der Vleuten C. Barriers to the uptake and use of feedback in the context of summative assessment. Adv Heal Sci Educ. 2015;20:229–45. https://doi.org/10.1007/s10459-014-9524-6
- Iobst WF, Sherbino J, Cate O Ten, et al. Competency-based medical education in postgraduate medical education. Med Teach. 2010;32:651–6. https://doi.org/10.3109/0142159X.2010.500709
- Mann S, Hastings Truelove A, Beesley T, Howden S, Egan R. Resident perceptions of Competency-Based Medical Education. Can Med Educ J. 2020;11:e31. https://doi.org/10.36834/cmej.67958
- Upadhyaya S, Rashid M, Davila Cervantes A, Oswald A. Exploring resident perceptions of initial competency based medical education implementation. Can Med Educ J. 2021:e42–56. https://doi.org/10.36834/cmej.70943
- Branfield Day L, Miles A, Ginsburg S, Melvin L. Resident perceptions of assessment and feedback in competency-based medical education: a focus group study of one internal medicine residency program. Acad Med. 2020;95:1712–7. https://doi.org/10.1097/ACM.0000000000003315
- Marcotte L, Egan R, Soleas E, Dalgarno NJ, Norris M, Smith CA. Assessing the quality of feedback to general internal medicine residents in a competency-based environment. Can Med Educ J. 2019;10:e32–47. https://doi.org/10.36834/cmej.57323
- David V, Walsh M, Lockyer J, Mintz M. Entrustable professional activities: an analysis of faculty time, trainee perspectives and actionability. Can J Gen Intern Med. 2021;16:8–13. https://doi.org/10.22374/cjgim.v16i1.415
- Hall AK, Rich J, Dagnone JD, et al. It’s a marathon, not a sprint: rapid evaluation of competency-based medical education program implementation. Acad Med. 2020;95:786–93. https://doi.org/10.1097/ACM.0000000000003040
- Federation des medecins residents du Quebec. Year 3 of implementation of competence by design: negative impact still outweighs theoretical benefits observations on the day-to-day reality of CBD. 2020.
- Schultz K, Griffiths J. Implementing competency-based medical education in a postgraduate family medicine residency training program: a stepwise approach, facilitating factors, and processes or steps that would have been helpful. Acad Med. 2016;91:685–9. https://doi.org/10.1097/ACM.0000000000001066
- Hamza DM, Ross S, Oandasan I. Process and outcome evaluation of a CBME intervention guided by program theory. J Eval Clin Pract. 2020;26:1096–104. https://doi.org/10.1111/jep.13344
- Ross S, Poth CA, Donoff MG, et al. Involving users in the refinement of the competency-based achievement system: An innovative approach to competency-based assessment. Med Teach. 2012;34. https://doi.org/10.3109/0142159X.2012.644828
- Charmaz K, Belgrave LL. The SAGE handbook of interview research: the complexity of the craft. SAGE Handb Interview Res Complex Cr. 2nd ed. 2012;347–66. https://doi.org/10.4135/9781452218403.n25
- Watling C, Lingard L. Grounded theory in medical education research: AMEE Guide No. 70. Med Teach. 2012;34:850–61. https://doi.org/10.3109/0142159x.2012.704439
- Charmaz K. Constructing grounded theory: a practical guide through qualitative research. Sage Publications Ltd. London: SAGE Publications Ltd; 2006.
- Apramian T, Cristancho S, Watling C, Lingard L. (Re)Grounding grounded theory: a close reading of theory in four schools. Qual Res. 2017;17:359–76. https://doi.org/10.1177/1468794116672914
- Morse JM. The significance of saturation. Qual Health Res. 1995;5:147–9. https://doi.org/10.1177/104973239500500201
- Hennink MM, Kaiser BN, Marconi VC. Code saturation versus meaning saturation: how many interviews are enough? Qual Health Res. 2017;27:591–608. https://doi.org/10.1177/1049732316665344
- Martin L, Sibbald M, Brandt Vegas D, Russell D, Govaerts M. The impact of entrustment assessments on feedback and learning: trainee perspectives. Med Educ. 2020;54:328–36. http://dx.doi.org/10.1111/medu.14047
- Schut S, Driessen E, van Tartwijk J, van der Vleuten C, Heeneman S. Stakes in the eye of the beholder: an international study of learners’ perceptions within programmatic assessment. Med Educ. 2018;52:654–63. https://doi.org/10.1111/medu.13532
- Atkinson A, Watling CJ, Brand PLP. Feedback and coaching. Eur J Pediatr. 2021;1–6. https://doi.org/10.1007/s00431-021-04118-8
- Watling C, Ginsburg S. Assessment, feedback and the alchemy of learning. Med Educ. 2019;53:76–85. https://doi.org/10.1111/medu.13645
- MacNeil K, Cuncic C, Voyer S, Butler D, Hatala R. Necessary but not sufficient: identifying conditions for effective feedback during internal medicine residents’ clinical education. Adv Heal Sci Educ. 2020;25(3):641-654. https://doi.org/10.1111/medu.14154
- Harrison C, Wass V. The challenge of changing to an assessment for learning culture. Med Educ. 2016;50:704–6. https://doi.org/10.1111/medu.13058
- Ginsburg S, Watling CJ, Schumacher DJ, Gingerich A, Hatala R. Numbers encapsulate, words elaborate: toward the best use of comments for assessment and feedback on entrustment ratings. Acad Med. 2021;96:S81–6. https://doi.org/10.1097/ACM.0000000000004089
- Malone K, Supri S. A critical time for medical education: The perils of competence-based reform of the curriculum. Adv Heal Sci Educ. 2012;17:241–6. https://doi.org/10.1007/s10459-010-9247-2
- Ramani S, Krackov SK. Twelve tips for giving feedback effectively in the clinical environment. Med Teach. 2012;34:787–91. https://doi.org/10.3109/0142159x.2012.684916
- Boet S, Pigford AAE, Naik VN. Program director and resident perspectives of a competency-based medical education anesthesia residency program in Canada: a needs assessment. Korean J Med Educ. 2016;28:157–68. https://doi.org/10.1111/medu.12637
- Upadhyaya S, Rashid M, Davila Cervantes A, Oswald A. Exploring resident perceptions of initial competency based medical education implementation. Can Med Educ J. 2021;12(2):e42-e56. https://doi.org/10.36834/cmej.70943
- Acai A, Li SA, Sherbino J, Chan TM. Attending emergency physicians’ perceptions of a programmatic workplace-based assessment system: the McMaster Modular Assessment Program (McMAP). Teach Learn Med. 2019;31:434–44. https://doi.org/10.1080/10401334.2019.1574581
- Gaunt A, Patel A, Rusius V, Royle TJ, Markham DH, Pawlikowska T. ‘Playing the game’: How do surgical trainees seek feedback using workplace-based assessment? Med Educ. 2017;51:953–62. https://doi.org/10.1097/acm.0000000000001523
- Leung WC. Competency based medical training: review. BMJ. 2002;325 (7366):693–6. https://doi.org/10.1136/bmj.325.7366.693
- Pinsk M, Karpinski J, Carlisle E. Introduction of competence by design to Canadian nephrology postgraduate training. Can J Kidney Heal Dis. 2018;5. https://doi.org/10.1177/2054358118786972