Abstracts
Abstract
The Medical Council of Canada Qualifying Exam (MCCQE) Part II aims to protect societal interests through examining recently graduated physicians using clinical scenarios with standardized patients. This position paper debates the role of the MCCQE Part II in the national licensing of physicians in Canada by focusing on the consequential validity evidence of this exam and considering future directions through discussing contemporary developments in high stakes examinations. Specifically, this paper compares both MCCQE Part I and Part II in their ability to predict future practice patterns of physicians and generalizability across specialties. In weighing up the evidence this paper considers commonly used counterarguments as well as the financial implications of this exam for both the candidates and the MCC. Finally, it concludes by providing recommendations for future licensing of physicians in Canada. The available consequential validity evidence for MCCQE Part II is limited. Though still limited, MCCQE Part I has more robust evidence that it is a better predictor of future practice patterns compared to with Part II. Combined with a lack of evidence that national licensing examinations lead to graduation of substandard doctors or an improvement of care, and the shift away from assessment of learning towards assessment for learning, the maximum impact of the MCC on safeguarding public’s interests will lie in working closely with residency programs and specialty colleges to facilitate a robust assessment program of essential competencies and clinical skills during residency training and specialty certification.
Résumé
L’examen d’aptitude du Conseil médical du Canada (EACMC), partie II, vise à protéger le public en soumettant les nouveaux diplômés en médecine à une évaluation par le biais de scénarios cliniques avec des patients standardisés. Cet article porte un regard critique quant au rôle de l’EACMC, partie II, dans l’obtention du permis d’exercice en médecine compte tenu des preuves de la validité de l’examen du point de vue de ses conséquences sociales et proposee des orientations futures au regard des développements contemporains en matière d’examens à enjeux élevés. Plus spécifiquement, cet article compare la partie I et la partie II de l’EACMC quant à leur capacité à prédire les tendances dans la pratique future des médecins et à leur caractère généralisable aux diverses spécialités. En soupesant les preuves, l’auteure examine les contre-arguments couramment évoqués ainsi que les implications financières de l’examen pour les candidats et pour le CMC. En conclusion, notre travail formule des recommandations pour l’octroi futur de titres menant à l’obtention du permis d’exercer aux médecins au Canada. Les preuves quant à la validité des conséquences sociales de l’EACMC, partie II sont limitées. Bien qu’également limitées, les preuves de la validité de l’EACMC, partie I, sont plus solides que celles de l’EACMC, partie II, en ce qui a trait à sa valeur prédictive des tendances futures dans la pratique des médecins. En l’absence de données probantes indiquant que les examens mènent soit à l’octroi d’un titre menant à l’obtention d’un permis de pratique en médecine qui ne satisfont pas aux exigences soit à l’amélioration des soins, et compte tenu de l’abandon de l’évaluation de l’apprentissage au profit de l’évaluation pour l’apprentissage, la meilleure manière pour le CMC de favoriser la protection du public serait de travailler en étroite collaboration avec les programmes de résidence et les collèges de spécialité afin de faciliter la mise en place d’un solide programme d’évaluation des compétences essentielles et des compétences cliniques pendant la formation en résidence et à l’occasion de formation continue spécialisée.
Download the article in PDF to read it.
Download
Appendices
Bibliography
- Kennedy B. The Part II examination: political exercise or national standard? Cmaj. 1995;152(8):1183-1184. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1337789/pdf/cmaj00068-0013.pdf
- Lougheed T. Is it time to rethink the MCCQE Part II? Can Med Educ J. 2016;7(1):e87-88. https://doi.org/10.36834/cmej.36630
- Benusic M. Should the MCCQE II exams go ahead? In: CMAJ Blogs; 2019. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1337789/pdf/cmaj00068-0013.pdf
- Bowmer MI. Response to: Is it time to rethink the MCCQE Part II? Can Med Educ J. 2016;7(1):e89-91. https://doi.org/10.36834/cmej.36722
- Topps M. Why the MCC qualifying examination part II still matters. In: CMAJ Blogs; 2019. https://cmajblogs.com/why-the-mcc-qualifying-examination-part-ii-still-matters/
- Sir Thomas Roddick: his work in medicine and public life. JAMA. 1940;115(9):802-803. https://doi.org/10.1001/jama.1940.02810350146035
- Reda J. The new LMCC. Cmaj. 1992;146(1):10-11. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC1488208/pdf/cmaj00290-0012a.pdf
- Tamblyn R, Abrahamowicz M, Dauphinee D, et al. Physician scores on a national clinical skills examination as predictors of complaints to medical regulatory authorities. Jama. 2007;298(9):993-1001. https://doi.org/10.1001/jama.298.9.993
- Norcini J, Anderson MB, Bollela V, et al. 2018 Consensus framework for good assessment. Med teach. 2018;40(11):1102-1109. https://doi.org/10.1080/0142159X.2018.1500016
- Cook DA, Brydges R, Ginsburg S, Hatala R. A contemporary approach to validity arguments: a practical guide to Kane's framework. Med Educ. 2015;49(6):560-575 https://doi.org/10.1111/medu.12678
- Kane M. Validating score interpretations and uses. Language Testing. 2012;29(1):3-17. https://doi.org/10.1177/0265532211417210
- Bachman LF. Fundamental considerations in language testing. Oxford, England: Oxford University Press; 1990.
- Boulet JR. Establishing the validity of licensing examination scores. J Grad Med Educ. 2019;11(5):527-529. https://doi.org/10.4300/JGME-D-19-00611.1
- Burdick WP, Boulet JR, LeBlanc KE. Can we increase the value and decrease the cost of clinical skills assessment? Acad Med. 2018;93(5):690-692. https://doi.org/10.1097/ACM.0000000000001867
- Tamblyn RA M, Bartlett G, Winslade N, et al. The Quebec-Ontario follow-up study of the association between scores achieved on the MCCQE Part II examination and performance in clinical practice. Online: Medical Council of Canada; 2009. https://h5a9c8a9.stackpathcdn.com/media/Tamblyn_Score-Association_MCCQE-Part-II_Clinical-Practice-Performance_2009.pdf
- De Champlain AQ, Tian F, Ashworth N, Kain N, Wiebe D. Do national licensing examination scores predict patient complaints as well as physician Opioid and Benzodiazepine prescribing patterns? Online: Medical Council of Canada; 2018. https://h5a9c8a9.stackpathcdn.com/media/IAMRA-2018Poster-A.DeChamplain.pdf
- Wenghofer E, Klass D, Abrahamowicz M, et al. Doctor scores on national qualifying examinations predict quality of care in future practice. Med Educ. 2009;43(12):1166-1173. https://doi.org/10.1111/j.1365-2923.2009.03534.x
- Bordage G, Meguerditchian AN, Tamblyn R. Practice indicators of suboptimal care and avoidable adverse events: a content analysis of a national qualifying examination. Acad Med. 2013;88(10):1493-1498. https://doi.org/10.1097/acm.0b013e3182a356af
- McKendry RJ, Busing N, Dauphinee DW, Brailovsky CA, Boulais AP. Does the site of postgraduate family medicine training predict performance on summative examinations? A comparison of urban and remote programs. Cmaj. 2000;163(6):708-711. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC80166/pdf/20000919s00013p708.pdf
- Dore KL, Reiter HI, Kreuger S, Norman GR. CASPer, an online pre-interview screen for personal/professional characteristics: prediction of national licensure scores. Adv Health Sci Educ Theory Pract. 2017;22(2):327-336. https://doi.org/10.1007/s10459-016-9739-9
- Eva KW, Reiter HI, Rosenfeld J, Trinh K, Wood TJ, Norman GR. Association between a medical school admission process using the multiple mini-interview and national licensing examination scores. Jama. 2012;308(21):2233-2240. https://doi.org/10.1001/jama.2012.36914
- Kulatunga-Moruzi C, Norman GR. Validity of admissions measures in predicting performance outcomes: the contribution of cognitive and non-cognitive dimensions. Teach Learn Med. 2002;14(1):34-42. https://doi.org/10.1207/S15328015TLM1401_9
- Medical Council of Canada. Annual Report 2019-2020. Ottawa; 2020. https://h5a9c8a9.stackpathcdn.com/media/MCC-Annual-Report-2019-2020.pdf
- Medical Council of Canada. MCCQE Part II Annual Technical Report. Ottawa; 2019. https://h5a9c8a9.stackpathcdn.com/media/MCCQE-Part-II-Annual-Technical-Report-2019.pdf
- Norcini JJ, Boulet JR, Opalek A, Dauphinee WD. The relationship between licensing examination performance and the outcomes of care by international medical school graduates. Acad Med. 2014;89(8):1157-1162. https://doi.org/10.1097/acm.0000000000000310
- Tatem GB, Gardner-Gray J, Standifer B, Alexander K. While you don't see color, i see bias: identifying barriers in access to graduate medical education training. ATS Sch. 2021;2(4):544-555. https://doi.org/10.34197/ats-scholar.2020-0134PS
- Low D, Pollack SW, Liao ZC, et al. Racial/Ethnic Disparities in Clinical Grading in Medical School. Teach Learn Med. 2019;31(5):487-496. https://doi.org/10.1080/10401334.2019.1597724
- MacFarlane MM. When a Canadian is not a Canadian: marginalization of IMGs in the CaRMS match. Can Med Educ J. 2021;12(4):132-140. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC8463232/pdf/CMEJ-12-132.pdf
- Weggemans MM, van Dijk B, van Dooijeweert B, Veenendaal AG, Ten Cate O. The postgraduate medical education pathway: an international comparison. GMS J Med Educ. 2017;34(5):Doc63. https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5704606/pdf/JME-34-63.pdf
- Harden RM. Five myths and the case against a European or national licensing examination. Med Teach. 2009;31(3):217-220. https://doi.org/10.1080/01421590902741155
- Archer J, Lynn N, Coombes L, et al. The impact of large scale licensing examinations in highly developed countries: a systematic review. BMC Med Educ. 2016;16(1):212. https://doi.org/10.1186/s12909-016-0729-7
- Babla K, Crampton P, Kronfli M. National licensing examinations: what are they good for? Clin Teach. 2020;17(3):323-325. https://doi.org/10.1111/tct.13083
- Schuwirth L. National licensing examinations, not without dilemmas. Med Educ. 2016;50(1):15-17. https://doi.org/10.1111/medu.12891
- Lockyer J, Carraccio C, Chan MK, et al. Core principles of assessment in competency-based medical education. Med Teach. 2017;39(6):609-616. https://doi.org/10.1080/0142159x.2017.1315082
- Karay Y, Schauber SK. A validity argument for progress testing: Examining the relation between growth trajectories obtained by progress tests and national licensing examinations using a latent growth curve approach. Med Teach. 2018;40(11):1123-1129. https://doi.org/10.1080/0142159x.2018.1472370