The purpose of reading, according to Elder and Paul (2006:3) is to ‘figure out what an author has to say on a subject’. This ‘figuring out’, or reading comprehension as we know it in schools, has become a proxy for reading ability and the manner in which reading ability is assessed and measured (see Tarjinder’s excellent post here). As such, the teaching of reading comprehension as a skill became ubiquitous throughout the last few decades (see here) and the way of assessing reading comprehension is, of course, the reading comprehension test.
But consider whether a reading comprehension test actually assesses reading comprehension. The nature of any comprehension test requires the assessed pupil to read a text and then answer questions related to that text based on a series of assessment criteria. This is not, however, how it seems we extract meaning from a text. According to Pressley and Afflerbach (1995), comprehension is a moment by moment process during the act of reading. The mind is assailed by every word in the moment of reading and is thus charged with determining which words to repress, emphasise, correlate and organise as it constructs the right weight, influence and force for each expression, all under the influence of establishing purpose and meaning from a non-present communicator (Thorndike, 1917). This results in meaning, the product of comprehension, never being stable. A change in one variable, knowledge for example, will alter the construction of meaning with the product of comprehension changing moment by moment (Pearson, 2001).
Rather than the extraction of information from text, which is what comprehension tests require, readers actually interact with information in the text in order to make sense of it (Palincsar and Brown, 1989), drawing on information from several sources concurrently to construct meaning. As a reader progresses through a text, they attend to the information and combine it with vocabulary knowledge, language conventions, semantic knowledge and general and disciplinary knowledge. The reader is continually updating the new information and relating it to prior content knowledge and integrating it with knowledge gleaned from the text, drawing connections and making inferences from within and without the text (Beck et al. 1997).
A comprehension test cannot assess this understanding and it can certainly not assess the primary goal of deep reading: ‘understanding the meaning of a text such that this knowledge can be used later’ (Allen and McNamara, 2020, p262).
For an assessment to be valid, it must measure what it purports to measure otherwise it is worthless (Cohen et al. 2015). So, if a reading comprehension test does not measure reading comprehension then it does not appear to have very much value…if any.
This has significant implications for the development of reading comprehension in schools and the assessment of it.
Firstly, the teaching of comprehension as a skills-based construct has no place in reading education despite the improvements in comprehension outcomes associated with this approach (see here). Teaching comprehension skills is a test-based and focused construct driven by the assessment. It puts the cart before the horse. The skills taught and learned will certainly improve test outcomes and, by implication, statutory assessment results but are unlikely to improve reading comprehension beyond the benefits of reading practice. Take, for example, the skills-based reading programme ‘Success For All’, which reports significant gains in reading test outcomes. Pupils are taught to take the role of ‘Text Detective’ when questions requiring inference are addressed and encouraged to ‘read between the lines’ to find the answer. But as Willingham (2015) posits (great blog from Clare Sealy), authors require readers to infer what they assume the reader already knows and therefore omit. How can a reader infer what they don’t know? This is possibly why skills-based comprehension had such robust research initially: because researchers used comprehension test data and that is exactly what skills-based comprehension teaching improved. It may also be why ‘Success For All’, according to Jonathon Kozol (2006), cannot sustain its results, with students, one year after leaving the programme, in significant reading deficit, and three years after leaving, unable to read. So incensed was Kozol (2006) that he described it as, ‘…an apartheid course of study for an apartheid schooling system.’
The next implication for the development of reading comprehension within schools is an understanding of the role that prior knowledge plays in the extraction of meaning from text. This is nothing new - although it may seem like it sometimes. Bartlett, as long ago as 1932, described “schemata” as organisations of past experience and Ausubel (1963:18) claimed that ‘the most important single factor influencing learning is what the learner already knows.’ Anderson et al.’s (1977) study on the interpretation of ambiguous passages in text concluded that reader’s interpretations of messages from text were heavily influenced by high level schemata.
Kintsch’s (1998) ‘Construction-Integration Model’ of reading comprehension is particularly helpful (Rayner et al. 2012) as it portrays comprehension as having three distinct layers. At the first level readers identify the words and phrases that make up the text. At the second level the reader uses information from the text, allied to their own knowledge to form a basic idea about notions proposed by the text. At the third level, however, the reader elaborates ideas and events from the text by integrating them with related prior knowledge to create a fully fleshed out mental representation of the text. And it is this deep, rich, coherent understanding, argues Kintsch (1998), that readers are seeking as it is associated with learning from a text rather than merely reading a text – not something a comprehension test can easily measure, if at all
Kintsch’s (1998) model dovetails with the concept of top down and bottom up reading comprehension strategies (Carrell, 1989; Young and Oxford, 1997). Readers who utilise a bottom up strategy (usually novice readers) understand texts at a local level through word decoding and sentence parsing thus, creating meaning from the read words. Readers who utilise a top down strategy bring a global perspective to their reading (Mills et al., 2006), utilising wide contextual and domain knowledge and deep understanding of complex semantic structures and patterns to create meaning with the read words (Afflerbach, 1986; Shapiro, 2004).
‘Reading comprehension and knowledge have a reciprocal relationship in which knowledge supports comprehension and comprehension builds new knowledge…’ (Cervetti and Wright, 2020); or as Pearson (2006) put it, ‘knowledge begets comprehension begets knowledge.’
There are, nevertheless, different kinds of knowledge, which in turn, it appears, have differing impacts on reading comprehension. Topic knowledge (closely related to the topic of a text) appears to have a consistently positive impact on questions that require bridging inferences and connecting ideas and information within a text (Pearson et al.,1979), whereas domain knowledge (related to a disciplinary area, eg. biology) appears to benefit the identification of themes and gist within a text and the answering of questions that require background knowledge (Haenggi and Perfetti, 1992). Cultural knowledge has been shown to support comprehension of narrative texts with student’s showing far greater understanding of texts with themes and characters consistent with their own socio-cultural experiences (Bell and Clark, 1998; Kelley et al., 2015) suggesting that where this is not the case, explicit investigation of cultural knowledge may be useful and necessary. More general, or world knowledge, has a greater positive effect on the comprehension of expository texts rather than narrative texts but still appears to be important for understanding of narrative (Best et al., 2008).
It is perhaps unsurprising that making inferences places a greater demand on prior knowledge compared to recall. Knowledge allows readers to selectively focus on the most important ideas in a text and form stronger connections among individual elements (Cervetti and Wright, 2020). This helps readers connect a series of discrete pieces of information into meaningful parts and chains, such as sequences, cause-effect relationships and plots. The integration of text with prior knowledge enables the reader to employ bridging inferences and in-text inferences thus, enabling the reader to learn from the text (Kintsch, 1986).
Furthermore, strong background knowledge supports vocabulary development (Puilido, 2007). The link between vocabulary and content knowledge, particularly in expository texts has a significant research base with strong background knowledge supporting high rates of incidental vocabulary learning while reading (Puilido, 2003, 2004, 2007). It has been suggested that high levels of knowledge are a proxy for vocabulary (Moje et al., 2020). (More on vocabulary in the next TRA post).
Knowledge and reading ability do appear to make complementary contributions to reading comprehension. High levels of reading ability allied with excellent prior knowledge result in highly effective comprehension (Cervetti and Wright, 2020); however, particularly in younger readers, good topic and domain knowledge can compensate for less effective reading ability (Recht and Leslie, 1988; Taylor, 1979). Younger readers, nevertheless, often require explicit instruction (from a teacher) in making meaningful connections between knowledge and inferencing (Rapp et al., 2007). Furthermore, high levels of prior knowledge can benefit students when reading texts with less cohesion which require greater inference during reading (McNamara et al., 1996).
So, if prior knowledge is a key element of extracting deep meaning from a text how can schools and teachers focus on this rather than on comprehension as a ‘skill’?
Studies appear to suggest that the activation of prior knowledge for students is the answer; but only once there is knowledge to activate. Where students have merely been questioned about their prior knowledge in an attempt to activate it, comprehension of text has not improved (Peek et al., 1982) and activating inaccurate prior knowledge may negatively impact comprehension (Van Loon et al., 2013). More effective, is where students have received explicit topic and domain knowledge instruction prior to attempting to read a text (Elbro and Buch-Iversen, 2013).
This building of specific prior knowledge to support reading comprehension appears to be key (Guthrie et al., 2004). Cervetti and Wright (2020) suggest that it is the building and leveraging of conceptual, disciplinary, cultural and world knowledge that are the most powerful elements to support reading comprehension and thus developing students’ engagement with texts. A conclusion seems to be appearing from the research that if students are to extract deep, rich and coherent meaning from any text, then they must first have sufficient prior knowledge, and that this knowledge needs to be explicitly taught.
This emphasis on knowledge as a fundament of comprehension implies that ‘close reading’ of text is thus not per se a necessary comprehension activity and perhaps even that knowledge can compensate for text complexity.
This is not borne out by the research which suggests that exposure to complex syntactic structures and language features is an essential element of developing deep reading comprehension (Just and Carpenter, 1992). Hiebert (2017) suggested that where teachers use texts simplified for readability (usually in areas of higher social deprivation), reading comprehension of more complex texts is compromised. For deep learning to occur. Hiebert concluded, students should be exposed to the complex language structures of texts above instructional level. Furthermore, studies revealed that students often read for less than four minutes per English class (Fisher, 2009) with the majority of the time taken up with watching films or listening to lectures and teacher-led comprehension was observed as little as 20% of the time (Vaughn et al., 2013). For deep comprehension to develop, students need to read a great deal, read complex texts, have the comprehension scaffolded by an expert (teacher) and have regular opportunities to discuss the text (Litman et al., 2017).
The research appears to suggest the following ‘best bets’ for schools:
• Reading comprehension is not ‘the next stage in reading’. The artificial divide between ‘learning to read’ and ‘reading to learn’ is a misapplication of Chall’s (1983) stages of reading development. Comprehension develops in tandem with reading acquisition and is aligned with listening comprehension and developing schema.
• Reading comprehension is not a skill and should only be taught as such as an exam technique.
• Prior knowledge is fundamental to reading comprehension and crucial for inference. Knowledge comes in many forms. For a pupil to be able to understand a text they must be furnished with sufficient knowledge to be able to read it with a top down, global perspective.
• This relevant knowledge must be built explicitly…
• …by teachers.
• Knowledge of complex language and syntactic structures is essential to reading comprehension. Regular exposure to these structures is essential through experience with high-quality texts above instructional level. Students will require expert, explicit scaffolding to negotiate these text…
• …from teachers.
• Reading comprehension is also a factor of vocabulary. Vocabulary is a function of greater knowledge. Vocabulary grows more quickly through explicit teaching. Students’ vocabulary appears to grow more quickly in classes where the environment is rich in language and teachers have a wide vocabulary. This is particularly important in early schooling and in areas of higher social and economic need.
A final thought on statutory assessment:
The conclusion that comprehension tests are an invalid assessment of reading comprehension begs the question: how can we measure it? It would seem that we can’t. There is just too much going in in the brain. So why bother? To hold schools accountable for ensuring that children can read. So why not measure what we can measure. We can measure words read per minute and we can measure reading accuracy. Not very satisfactory? Not according to Rasinski et al. (2015); it seems that words-read-per-minute is a very accurate proxy for academic outcome.
And if we can’t measure reading comprehension…at least we now know how to teach it.