Abstracts
Abstract
Two studies were conducted to assess meta-cognitive and individual difference influences on students’ choice of writing tests in paper-and-pencil or computer-administered format. In Study 1, university students chose the test format for an accounting exam (paper-and-pencil or computer). In Study 2, students disclosed their reasons for their choice of test format, predicted their scores on the first test and provided confidence ratings for their predictions. The results of both studies show that the reasons for choosing a computer vs. a paper-and-pencil test format differ, and that both choice and performance can be explained to some extent by individual difference and meta-cognitive factors.
Keywords:
- computer-based testing,
- paper-and-pencil testing,
- meta-cognitive influences,
- test format
Résumé
Deux enquêtes ont été menées afin de mesurer l’apport des différences métacognitives et des différences individuelles dans une tâche où un étudiant doit choisir de compléter un examen en format papier ou administré par ordinateur. Dans la première enquête, des étudiants universitaires ont choisi le format d’un examen de comptabilité (format papier ou administré par ordinateur). Dans la deuxième enquête, on a demandé aux étudiants d’expliquer leur choix, de prédire les résultats de leurs examens, et de fournir leur niveau de confiance quant à l’exactitude de leurs prédictions. Les résultats des deux enquêtes ont montré que le choix d’un exam en format papier est associé à un raisonnement différent que celui associé au choix d’un examen administré par ordinateur. De plus, les résultats ont montré que le choix du format de l’examen et la performance aux examens peuvent être, en partie, expliqués par des facteurs individuels et métacognitifs.
Download the article in PDF to read it.
Download
Appendices
References
- Anderson, N. (2003). Applicant and recruiter reactions to new technology in selection: A critical review and agenda for future research. International Journal of Selection and Assessment, 11, 121-136.
- Baird, W. E., & Silvern, S. B. (1992). Computer learning and appropriate testing: A first step in validity assessment. Journal of Research on Computing in Education, 25, 18-28.
- Bocij, P., & Greasley, A. (1999). Can computer-based testing achieve quality and efficiency in assessment? International Journal of Educational Technology, 1, 1-16.
- Bugbee, A. C., Jr. (1996). The equivalence of paper-and-pencil and computer-based testing. Journal of Research on Computing in Education, 28, 282-290.
- Bugbee, A. C., Jr., & Bernt, F. M. (1990). Testing by computer: Findings in six years of use 1982-1988. Journal of Research on Computing in Education, 23, 87-101.
- Cacioppo, J. T., Petty, R. E., & Kao, C. F. (1984). The efficient assessment of need for cognition. Journal of Personality Assessment, 48, 306-307.
- Chalhoub-Deville, M., & Deville, C. (1999). Computer adaptive testing in second language contexts. Annual Review of Applied Linguistics, 19, 273-299
- Chapelle, C. A. (2001). Computer application in second language acquisition. Cambridge, UK: Cambridge University Press.
- Elias, S. M., & Loomis, R. J. (2002). Utilizing need for cognition and perceived self-efficacy to predict academic performance. Journal of Applied Social Psychology, 32, 1687-1702.
- Flavell, J. H. (1979). Metacognition and cognitive monitoring: A new area of cognitive-developmental inquiry. American Psychologist, 34, 906-911.
- Gershon, R. C., & Bergstrom, B. (1991, April). Individual differences in computer adaptive testing: Anxiety, computer literacy, and satisfaction. Paper presented at the Annual Meeting of the National Council on Measurement in Education, San Francisco, CA.
- Glowacki, M. L., McFadden, A. C., & Price, B. J. (1995, November). Developing computerized tests for classroom teachers: A pilot study. Paper presented at the Annual Meeting of the Mid-South Educational Research Asssociation, Biloxi, MS. (ERIC Document Reproduction Service No. ED391471). Retrieved from Education Resources Information Center [ERIC] Web site: http://www.eric.ed.gov
- Green, B. F., Bock, R. D., Humphreys, L. G., Linn, R. L., & Reckase, M. D. (1984). Technical guidelines for assessing computerized adaptive tests. Journal of Educational Measurement, 21, 347-360.
- Hacker, D. J. (1998). Definitions and empirical foundations. In D. J. Hacker, J. Dunlosky, & A. C. Graesser (Eds.), Metacognition in educational theory and practice (pp. 1-24). Mahwah, NJ: Lawrence Erlbaum Associates.
- Jarvis, W. B. G., & Petty, R. E. (1996). The need to evaluate. Journal of Personality and Social Psychology, 70, 172-194.
- Lee, M. (1995, November 13). Technology (A special report): In the classroom – Examined Options. True or false: Standardized tests can be easily put on computers. Wall Street Journal (Eastern ed.), p. R26.
- Leone, C., Modglin, K., & Wallace. H. M. (1999). The need for closure and the need for structure: Interrelationships, correlates, and outcomes. Journal of Psychology, 133, 553-562.
- Maki, R. H., & Berry, S. L. (1984). Metacomprehension of text material. Journal of Experimental Psychology: Learning, Memory, and Cognition, 10, 663-679.
- Merritt, J. (2003, December 29). Why the folks at ETS flunked the course: A tech-savvy rival service will soon be giving B-school applicants their GMATs. Business Week, p. 48.
- Neuberg, S. L., & Newsome, J. T. (1993). Personal need for structure: Individual differences in the desire for simple structure. Journal of Personality and Social Psychology, 65, 113-131.
- Oakes, D. R. (1999). Institutes moving to computerized exams. National Underwriter, 103, 42.
- Parshall, C. G., & Kromrey, J. D. (1993, April). Computer testing versus paper-and-pencil testing: An analysis of examinee characteristics associated with mode effect. Paper presented at the Annual Meeting of the American Educational Research Association, Atlanta, GA. (ERIC Document Reproduction Service No. ED363272). Retrieved from Education Resources Information Center [ERIC] Web site: http://www.eric.ed.gov
- Peterson, B. K., & Reider, B. P. (2002). Perceptions of computer-based testing: A focus on the CFM examination. Journal of Accounting Education, 20, 265-284.
- Ployhart, R. E., Weekley, J. A., Holtz, B. C., & Kemp, C. (2003). Web-based and paper-and-pencil testing of applicants in a proctored setting: Are personality, biodata, and situational judgment tests comparable? Personnel Psychology,56, 733-752.
- Potosky, D., & Bobko, P. (2004). Selection testing via the internet: Practical considerations and exploratory empirical findings. Personnel Psychology, 57, 1003-1034.
- Rabinowitz, S., & Brandt, T. (2001). Computer-based assessment: Can it deliver on its promise? Knowledge brief. San Francisco, CA: WestEd. (ERIC Document Reproduction Service No. ED462447). Retrieved from Educational Resources Information Center [ERIC] Web site: http://www.eric.ed.gov
- Russell, M. (1999). Testing on computers: A follow-up study comparing performance on computer and on paper. Education Policy Analysis Archives, 7, 472-522.
- Russell, M., & Haney, W. (1997). Testing writing on computers: An experiment comparing student performance on tests conducted via computer and via paper-and-pencil. Educational Policy Analysis Archives, 5(3). Retrieved from the publication’s Web site: http://epaa.asu.edu/epaa/v5n3.html
- Smith, B., & Caputi, P. (2004). The development of the attitude towards computerized assessment scale. Journal of Educational Computing Research, 31, 407-422.
- Wainer, H., Dorans, N. J., Eignor, D., Flaugher, R., Green, B. F., Mislevy, R. J., et al. (2000). Computer adaptive testing: A primer (2nd ed.). Mahwah, NJ: Lawrence Erlbaum Associates.
- Wise, S. L., & Plake, B. S. (1990). Computer-based testing in higher education. Measurement and Evaluation in Counseling and Development, 23, 3-11.
- Wood, S. (1984). Computer use in testing and assessment. Journal of Counseling and Development, 63, 177-179.
- Zenisky, A. L., & Sireci, S. G. (2002). Technological innovations in large-scale assessment. Applied Measurement in Education, 15, 337-362.