Abstracts
Abstract
This study aims to find evidence that validates an unprecedented scale assessing metacognitive knowledge about digital reading, developed for PISA 2018. More precisely, the study investigates whether the scale measures metacognitive knowledge about reading as opposed to ICT literacy. Data from the 37 OECD countries were used. The results show strong correlation between this metacognitive knowledge scale, reading performance, and two scales which have already been validated for measuring metacognition about reading. The scale is also significantly correlated with students’ interest, self-concept, and self-efficacy in reading. On the other hand, no correlation was shown between their use of ICT, and their interest and perceived self-efficacy in their use. The results provide evidence that the scale assesses metacognitive knowledge about digital reading rather than digital literacy. The results also demonstrate a strong correlation between the scale and reading proficiency.
Keywords:
- metacognitive awareness,
- digital reading,
- validation process,
- large-scale assessments
Résumé
Cette étude vise à valider une échelle de mesure des connaissances métacognitives de la lecture numérique inédite, développée pour le PISA 2018 et, plus particulièrement, à établir si cette échelle mesure des connaissances relevant de la lecture plutôt que de la littératie numérique. Les données des 37 pays de l’OCDE ont été utilisées. Les résultats montrent que l’échelle de connaissances métacognitives présente une forte corrélation avec les performances en lecture et avec deux échelles ayant subi un processus de validation de la mesure de la métacognition en lecture. Cette échelle est aussi significativement liée à l’intérêt des élèves, à leur concept de soi et à leur sentiment d’efficacité en lecture, mais n’est pas liée avec leur utilisation des TIC, leur intérêt ou leur sentiment d’efficacité perçue dans ce domaine. Les résultats appuient le fait que l’échelle mesure des connaissances relevant de la lecture plutôt que de la littératie numérique et mettent en évidence une corrélation robuste entre les connaissances métacognitives et les performances en lecture.
Mots-clés :
- connaissances métacognitives,
- lecture numérique,
- processus de validation,
- enquêtes à large échelle
Resumo
Este estudo tem como objetivo identificar os indícios de validade de u m a escala inédita de medida de conhecimentos metacognitivos da leitura digital, desenvolvida para o PISA 2018 e, mais especificamente, verificar se essa escala mede conhecimentos mais relacionados com a leitura do que com a literacia digital. Foram usados dados de 37 países da OCDE. As correlações mostram que a escala de conhecimentos metacognitivos está fortemente correlacionada com o desempenho em leitura e com duas escalas de medição de metacognição de leitura validadas. Está também significativamente relacionada com o interesse dos alunos, o seu autoconceito e o seu sentimento de eficácia em leitura, mas não está relacionada com a utilização das TIC, o seu interesse ou o seu sentimento de eficácia percebido neste domínio. Os resultados validam o facto de que a escala mede conhecimentos mais relacionados com a leitura do que com a literacia digital e demonstra uma correlação robusta dos conhecimentos metacognitivos com os desempenhos em leitura.
Palavras chaves:
- conhecimento metacognitivo,
- leitura digital,
- processo de validação,
- pesquisas em larga escala
Appendices
List of References
- Afflerbach, P. (2000). Verbal reports and protocol analysis. In M. L. Kamil, P. B. Mosenthal, P. D. Pearson, & R. Barr (Eds.), Handbook of reading research (volume 3) (pp. 163-181). Longman.
- Afflerbach, P., & Cho, B. Y. (2010). Determining and describing reading strategies. Internet and traditional forms of reading. In H. S. Waters & Schneider, W. (Eds.), Metacognition, use, and instruction (pp. 201-225). Guilford Press.
- Allen, B. A., & Armour-Thomas, E. (1993). Construct validation of metacognition. The Journal of Psychology: Interdisciplinary and Applied, 127(2), 203-211. https://doi.org/10.1080/00223980.1993.9915555
- Anderson, N. J. (2003). Scrolling, clicking, and reading English: Online reading strategies in a second/foreign language. The Reading Matrix, 3(3), 1-33.
- Artelt, C., & Schneider, W. (2015). Cross-country generalizability of the role of metacognitive knowledge in students’ strategy use and reading competence. Teachers College Record, 117(1), 1-32.
- Azevedo, R., Behnagh, R., Duffy, M., Harley, J., & Trevors, G. (2013). Metacognition and self-regulated learning in student-centered leaning environments. In D. Jonassen & S. Land (Ed.), Theoretical foundations of learning environments (pp. 171-197). Lawrence Erlbaum.
- Baker, L., & Brown, A. L. (1984). Metacognitive skills and reading. In P. D. Pearson (ed.), Handbook of reading research (pp. 353-394). Longman.
- Baker, L., & Wigfield, A. (1999). Dimensions of children’s motivation for reading and their relations to reading activity and reading achievement. Reading Research Quarterly, 34, 452-477.
- Bawden, E. (2008) Origin and concepts of digital literacy. In C. Lankshear & M. Knobel (eds.). Digital literacies: Concepts, policies and practices. (p. 17-33). Peter Lang Publishing.
- Brozo, W., Sulkunen, S., Shiel, G., Garbe, C. Pandian, A. & Valtin, R. (2014). Reading, gender and engagement. Journal of adolescent literacy, 57(7), 584-593.
- Burin, D. I., Gonzalez, F. M., Barreyro, J. P., & Injoque-Ricle, I. (2020). Metacognitive regulation contributes to digital text comprehension in E-learning. Metacognition and learning, 15, 391-410.
- Campbell, D. T., & Fiske, D. W. (1959). Convergent and discriminant validation by the multitrait-multimethod matrix. Psychological bulletin, 56(2), 81-105.
- Coiro, J., & Dobler, E. (2006). Exploring the online reading comprehension strategies used by sixth-grade skilled readers to search for and locate information on the Internet. Reading Research Quarterly Journal, 42(2), 214-250.
- Coiro, J., Knobel, M., Lankshear, C., & Leu, D. (2014), Central Issues in New Literacies and New Literacies Research. In J. Coiro, M. Knobel, C. Lankshear & D. Leu (Eds.), Handbook of Research on New Literacies (pp. 1-22). Lawrence Erlbaum Associates.
- Denton, C.A., Wolters, C. A., York, M., Swanson, E., Kulesz, P., & Francis, D. J. (2015). Adolescents’ use of reading comprehension strategies: Differences related to reading proficiency, grade level, and gender. Learning and Individual Differences, 37, 81-95. https://doi.org/10.1016/j.lindif.2014.11.016
- Fischer, G. H., & Molenaar, I. W. (Eds.). 1995. Rasch models: Foundations, recent developments, and applications. Springer.
- Flavell, J. H. (1976). Metacognitive aspects of problem solving. In L. B. Resnick (Ed.), The nature of intelligence (pp. 231-235). Lawrence Erlbaum Associates.
- Flavell, J. H., Miller, P. H., & Miller, S. A. (2002). Cognitive development (4th ed.). Pearson Education.
- Guthrie, J. T., & Alvermann, D. E. (Eds.) (1999). Engaged reading: Processes, practices, and policy implications. Teachers College Press.
- Guthrie, J. T., Klauda, S. L., & Ho, A. (2013). Modeling the relationships among reading instruction, motivation, engagement, and achievement for adolescents. Reading Research Quarterly, 48(1), 9-26.
- He, J., & van de Vijver, F. J. R. (2013). Methodological issues in cross-cultural studies in educational psychology. In G. A. D. Liem & A. B. I. Bernardo (eds.), Advancing cross-cultural perspectives on educational psychology: A festschrift for Dennis McInerney (pp. 39-56). Information Age Publishing.
- Horner, S. L., & Shewry, C. S. (2002). Becoming an engaged, self-regulated reader. Theory Into Practice, 41, 102-109.
- Kintsch, W., & Kintsch, E. (2005). Comprehension. In S. G. Paris & S. A. Stahl (Eds.), Children’s reading: Comprehension and assessment (pp. 71-92). Lawrence Erlbaum.
- Lafontaine, D., Dupont, V., Jaegers, D., & Schillings, P. (2019). Self-concept in reading: Factor structure, cross-cultural invariance and relationships with reading achievement in an international context (PIRLS 2011). Studies in Educational Evaluation, 60, 78-89. https ://doi.org/10.1016/j.stueduc.2018.11.005
- Leu, D., Forzani, E., Rhoads, C., Maykel, C., Kennedy, C. & Timbrell, N. (2015). The new literacies of online research and comprehension: Rethinking the reading achievement gap. Reading Research Quarterly, 50(1), 37-59. https://doi.org/10.1002/rrq.85
- Li, J. (2020). Development and validation of second language online reading strategy inventory. Computers and education, 145, 103733. https://doi.org/10.1016/j.compedu.2019.103733
- Lihong, M., Haifeng, L., & Leifeng, X. (2021). Perceived teacher support, self-concept, enjoyment and achievement in reading: a multilevel mediation model based on PISA 2018. Learning and individual differences, 85, 101947. https://doi.org/10.1016/j.lindif.2020.101947
- Loye, N. (2018). What if validation was more than a sequence of technical procedures? Mesure et évaluation en éducation, 41(1), 97-125.
- Mc Elvany, N., & Schwabe, F. (2019). Gender gap in reading digitally? Examining the role of motivation and self-concept. Journal for educational research online, 11, https://doi.org/10.25656/01:16791
- McNamara, D. & J. Magliano (2009), Toward a Comprehensive Model of Comprehension. The Psychology of Learning and Motivation, 51, 297-384.
- Martin, A. (2006). A European framework for digital literacy. Nordic Journal of Digital Literacy, 1(2), 151-161. https://doi.org/10.18261/ISSN1891-943X-2006-02-06
- Messick, S. (1990). Validity of test interpretation and use. Educational Testing Service.
- Mokhtari, K., Dimitrov, & Reichard, C. A. (2018). Revising the Metacognitive awareness of reading strategies (MARSI) and testing for factorial invariance. Studies in Second Language Learning and Teaching, 8(2), 219-246.
- Mokhtari, K., & Reichard, C. A. (2002). Assessing students’ metacognitive awareness of reading strategies. Journal of Educational Psychology, 94, 249-259.
- Ohtani, K., & Hisakawa, T. (2018). Beyond intelligence: a meta-analytic review of the relationship among metacognition, intelligence and academic performance. Metacognition learning, 13, 179-212. https://doi.org/10.1007/s11409-018-9183-8
- OECD (2012). PISA 2009 technical report. OECD Publishing. https://doi.org/10.1787/9789264167872-en
- OECD (2019). PISA 2018 Assessment and Analytical Framework. OECD Publishing. https://doi.org/10.1787/b25efab8-en
- OECD (2020). PISA 2018 technical background. OECD Publishing. https://doi.org/10.1787/1b045c06-en.
- OECD (2021). 21st-Century Readers: Developing Literacy Skills in a Digital World. OECD Publishing. https://doi.org/10.1787/a83d84cb-en
- Paris, S. G., Lipson, M. Y., & Wixson, K. K. (1983). Becoming a strategic reader. Contemporary Educational Psychology, 8, 293-316.
- Paris, S. G., & Winograd, P. (1990). How metacognition can promote academic learning and instruction. In B. F. Jones & L. Idol (Eds.), Dimensions of thinking and cognitive instruction (pp. 15-51). Lawrence Erlbaum Associates.
- Rouet, J. (2006). The Skills of Document Use: From Text Comprehension to Web-based Learning. Lawrence Erlbaum Associates.
- Rouet, J., & Britt, M. (2011). Relevance Processes in Multiple Document Comprehension. In M. McCrudden, J. Magliano & G. Schraw (Eds.), Text Relevance and Learning from Text Information Age (pp. 19-52). Information age publishing.
- Rouet, J., & Coutelet, B. (2008). The Acquisition of Document Search Strategies in GradeSchool Students. Applied Cognitive Psychology, 22, 389-406. https://doi.org/10.1002/acp.1415
- Samuelsen, M. S., & Braten, I. (2007). Examining the validity of self-reports on scales measuring students’ strategic processing. British Journal of Educational Psychology, 77(2), 351-378. https://doi.org/10.1348/000709906X106147
- Schellings, G., Van Hout, B. H. A. M., Veenman, M. V. J., & Meijer, J. (2013). Assessing metacognitive activities: the in-depth comparison of a task-specific questionnaire with think-aloud protocols. European Journal of Psychology of Education, 28, 963-990. https://doi.org/10.1007/s10212-012-0149-y.
- Snow, C. E., Burns, M. S., & Griffin, P. (1998). Preventing reading difficulties in young children. National Academy Press. http://www.ierinstitute.org/IERI_Monograph_Volume_02_Chapter_01.pdf
- Snow, C., & the RAND Corporation (2002). Reading for Understanding: Toward an R and D Program in Reading Comprehension. RAND Reading Study Group.
- Von Davier, M., Gonzalez, E., & Mislevy, R. (2009). What are plausible values and why are they useful? IERI monograph series, 2(1), 9-36.
- Wu, L., Valcke, M., & Vankeer, H. (2012). Validation of a Chinese version of the Metacognitive awareness of reading strategies. Eurasian Journal of Educational Research, 48, 117-134.
- Zhou, J., He, J., & Lafontaine, D. (2020). Cross-cultural comparability and validity of metacognitive knowledge in reading in PISA 2009: a comparison of two scoring methods. Assessment in Education: Principles, Policy & Practice. 27(6). 635-654. https://doi.org/10.1080/0969594X.2020.1828820
10.7202/1055898ar