Visible Learning: The Sequel

When the original Visible Learning® was published in 2008, it instantly became a publishing sensation. Interest in the book was unparalleled; it sold out in days and was described by the TES as revealing “teaching’s Holy Grail.” Now John Hattie returns to this ground-breaking work. The research underlying this book is now informed by more than 2,100 meta-analyses (more than double that of the original), drawn from more than 130,000 studies, and has involved more than 400 million students from all around the world.

But this is more than just a new edition. This book is a sequel that highlights the major story, taking in the big picture to reflect on the implementation in schools of Visible Learning, how it has been understood – and at times misunderstood – and what future directions research should take.

Visible Learning: The Sequel reiterates the author’s desire to move beyond claiming what works to what works best by asking crucial questions such as: Why is the current grammar of schooling so embedded in so many classrooms, and can we improve it? Why is the learning curve for teachers after the first few years so flat? How can we develop teacher mind-frames to focus more on learning and listening? How can we incorporate research evidence as part of the discussions within schools?

Areas covered include:

The evidence base and reactions to Visible Learning

The Visible Learning model

The intentional alignment of learning and teaching strategies

The influence of home, students, teachers, classrooms, schools, learning, and curriculum on achievement

The impact of technology

Building upon the success of the original, this highly anticipated sequel expands Hattie’s model of teaching and learning based on evidence of impact and is essential reading for anyone involved in the field of education either as a researcher, teacher, student, school leader, teacher trainer, or policy maker.

John Hattie is Emeritus Laureate Professor at the Graduate School of Education, University of Melbourne, Australia. He is one of the world’s best-known and most widely read education experts, and his Visible Learning series of books have been translated into 29 different languages and have sold over 2 million copies.

Praise received for John Hattie’s previous work:

“Reveals teaching’s ‘holy grail”

Times Educational Supplement

“John Hattie has made a supersized contribution to the improvement of learning. Visible Learning came crashing on the scene in 2009 and got the attention of all of us. What I especially like is how Hattie and team continue to be internally critical and externally open to considering all possibilities. Their new series in search of ‘the Gold Standard’ is typical of Hattie’s commitment and skill in processing criticism and modelling continuous learning and improvement. A million books! Congratulations for a monumental achievement and the promise of more, and more”

Michael Fullan, OC. Professor Emeritus, OISE/University of Toronto

“I’m not sure John knows his own impact - I can’t think of anyone in education on this planet with a higher effect size. We teachers needed a sieve to sort the research that helps teachers make a difference, from the research that doesn’t. John provided it and now we all know how to increase our own impact. Thank you John!”

Geoff Petty, Author of ‘Teaching Today’ and ‘Evidence-Based Teaching’

“Education research suffers from an embarrassment of riches—there’s too much of it! John Hattie has been a resolute leader in summarizing this wealth of information into practical terms that are digestible yet reliably capture the essence. Better still, they translate readily to classroom practice! Hattie’s success and influence are richly deserved.”

Dan Willingham, Professor of Psychology, University of West Virginia, author of ‘Why Don’t Students Like School?’

“Everything changed when Visible Learning revealed the key messages from research into influences on learning. Thousands of studies brought together confirmed what we must focus on … educators across the world continue to have a reference point for what matters in guiding thinking and practice.”

Shirley Clarke, Author of ‘Unlocking Formative Assessment’ and ‘Thinking Classrooms’

“John Hattie’s contributions to generations of students, teachers, and educational leaders around the world are profound and lasting. He has that rare combination of courage and humility, fearlessly challenging conventional wisdom while continually improving and expanding his work. He makes our profession and the world of education better, and children on every continent benefit from his scholarship, wisdom, and practical guidance.”

Douglas Reeves, Creative Leadership.net

“Hattie’s work enables us to see the effects of what teachers do in the classroom. His work on lesson preparation is one of those must-reads for all teachers. In teaching we are often looking for those light-bulb moments of illumination. Hattie has mapped the hidden wiring.”

Huw Thomas, Former Headteacher, college lecturer and Diocesan Director of Education

“In a world where educators are often tempted and encouraged to utilize the latest fads to support student learning, John Hattie’s Visible Learning books serve as exceptional resources. Every practicing teacher, school administrator, and education researcher should have these books on their shelves.”

Eric M. Anderman, Professor of Educational Psychology, The Ohio State University

“John Hattie has a dream that one day every child learns not by chance, but by design. He has a dream that one day the expertise which is all around us gets together and changes the system. He has a dream that one day teachers ask themselves the powerful question “What works best?” instead of only “What works”. And he has a dream that one day teachers always seek maximum impact. And – most important – he lives his dream. He has started the fire with his passion. He sparked the learning. And he sparked the teaching. Congratulations John and: Know your impact!”

Klaus Zierer, Professor of Education, University of Augsburg, Germany, and Associate Research Fellow of the ESRC-funded Centre on Skills, Knowledge and Organisational Performance (SKOPE) at the University of Oxford

Visible Learning: The Sequel

A Synthesis of Over 2,100 Meta-Analyses Relating to Achievement

John Hattie

Designed cover image: Stephen Crumly

First edition published 2023

by Routledge

4 Park Square, Milton Park, Abingdon, Oxon, OX14 4RN

and by Routledge

605 Third Avenue, New York, NY 10158

Routledge is an imprint of the Taylor & Francis Group, an informa business

© 2023 John Hattie

The right of John Hattie to be identified as author of this work has been asserted in accordance with sections 77 and 78 of the Copyright, Designs and Patents Act 1988.

All rights reserved. No part of this book may be reprinted or reproduced or utilised in any form or by any electronic, mechanical, or other means, now known or hereafter invented, including photocopying and recording, or in any information storage or retrieval system, without permission in writing from the publishers.

Visible Learning® is a trademark of Corwin Press, Inc. All rights reserved.

Trademark notice: Product or corporate names may be trademarks or registered trademarks, and are used only for identification and explanation without intent to infringe.

British Library Cataloguing-in-Publication Data

A catalogue record for this book is available from the British Library

Library of Congress Cataloging-in-Publication Data

Names: Hattie, John, author.

Title: Visible learning, the sequel : a synthesis of over 2,100 meta-analyses

relating to achievement / John Hattie.

Description: First edition. | Abingdon, Oxon; New York, NY: Routledge,

2023. | Includes bibliographical references and index.

Identifiers: LCCN 2022050430 (print) | LCCN 2022050431 (ebook) |

ISBN 9781032462035 (paperback) | ISBN 9781032462028 (hardback) |

ISBN 9781003380542 (ebook other)

Subjects: LCSH: Learning—Longitudinal studies. | Teaching—Longitudinal

studies. | Effective teaching—Longitudinal studies. | Teacher effectiveness—

Longitudinal studies. | Academic achievement—Longitudinal studies.

Classification: LCC LB1060 .H389 2023 (print) | LCC LB1060 (ebook) |

DDC 370.15/23—dc23/eng/20221228

LC record available at https://lccn.loc.gov/2022050430

LC ebook record available at https://lccn.loc.gov/2022050431

ISBN: 978-1-032-46202-8 (hbk)

ISBN: 978-1-032-46203-5 (pbk)

ISBN: 978-1-003-38054-2 (ebk)

DOI: 10.4324/9781003380542

Typeset in Bembo and Helvetica Neue

by Apex CoVantage, LLC

Dedicated to Michael Scriven, evaluator, critic, and generous friend

Acknowledgments

I dedicate this book to my long-time friend, Michael Scriven. As noted throughout the book, he has been most influential in my thinking, and his immense breadth of knowledge, his generosity in sharing, and his skills to think aloud in such profound ways have improved me greatly. Janet and I met with him in San Francisco recently (he is 94), and he continues to inspire. When I review my greatest contributions to academia, it is that I have been his boss for 20 years and I have been his referee. He is the grandfather of evaluation; had Gilbert Ryle (see Chapter 3) as his supervisor; has written on car mechanics, the scientific method, computers and computing, applied logic, philosophy of religion and history, reasoning, creativity and critical thinking, causality and explanation, psychology, psychiatry and psychotherapy, parapsychology, and more. He is a University Blue in rowing; his first book was on the gas turbine in car design; he trained as a physicist and became a philosopher and educator; he was the world’s first professor of evaluation (Janet is the second); he invented formative and summative evaluation, goal-free evaluation, and meta-evaluation; and he established the Faster Forward Fund. His works on evaluation are foundational, and he has never stopped writing and talking about evaluation as a trans-discipline. He is generous, he shared his deep knowledge of knives with my sons, and we have had great times together. Thank you, Michael and Mary-Ann.

Over the past 12 years, so many have contributed to this book – when they were implementing Visible Learning+™ (VL+) in their schools, reading the books and articles about VL by me and others, and critiquing and blogging about VL. The opposite of love is not hate but indifference, and I thank you all for caring enough to read, critique, and use the ideas.

When writing VL1, I gathered all articles on each of the 150 influences, drove to our beach house in Tairua, and read and wrote. It took 20+ years of doing this to work out the big messages, and the beautiful balcony overlooking the sea was my reward for this venture. For this book, technology took over, and no trees were spared printing out the meta-analyses or studies, and (now that we have moved from New Zealand to Australia) our beach house in Anglesea is less cluttered with papers and misfiled articles. Core to this venture has been the support, advocacy, and critique of Janet. Typically, one thanks one’s wife when writing a book (if for nothing else for keeping out of my way, ignoring my solitary pursuit, and grumping that I should spend more time with her and family). But this is not the case for me. Janet is my partner and my best critic, and many the night we have debated an influence, its part in the bigger picture, and the core role of evaluative thinking. She has raised the quality of the messages, the flow, the tone, and the big ideas such that the book moved from being a compendium of 350 influences to building on a big story (and she devised the barometer in VL1 and thermometer in this sequel). After 40 years together, we have learned how to fight about ideas, reconcile, and move forward not only to create a family but to advance our work and careers.

The family has put up with VL for so long, and three are now teachers. I am so proud they became teachers, the best role in the world, and may they long enjoy having an impact on their students. The others are behavioral analysts, evaluators, and project managers. During writing VL2, we became grandparents and now have Joel, Kat, Ella, and Florence; Kyle, Jess, Emma, Danielle, and Charlie; Kieran, Aleesha, and Riley (John); as well as Edna and Patterson (who loved when I became stuck at home during the COVID pandemic, as then there were many walks and doggie treats).

Shirley Clarke read and critiqued every page of VL2, wrote the core of the reflections for each chapter, and is the connoisseur of feedback. Others who critiqued various chapters include Bill Tumner, James Chapman, Steve Graham, Jennifer Buckingham, Korbinian Kiendl, Dan and Lawrence Ingvarson, Lyn Yates, Tim Shanahan, Julie and Ray Smith, Sam Rodgers, John Almarode, Doug Fisher, Nancy Frey, Geoff Petty, Doug Reeves, Greg Ashman, Amy Berry, Peter Blatchford, Lyn Sharratt, Wolfgang Beryl, Klaus Zierer, Dylan Wiliam, and Jim Knight. We did not always agree, but this is the beauty of critics. So many others contributed to the ideas, including all the VL consultants from Corwin, Osiris, Bazalt, Cognition – you are core to the implementation, and I trust you see your influence throughout these chapters. The Corwin team have been partners for five years. I thank Michael Soules, Chris Devling, Sonja Hollins-Alexander, Vania Tiatto, for leading this work.

Bruce Roberts has been my Routledge editor from when we first met in Budapest, through VL1, this sequel, and many other ventures. Your commitment to quality, your gentle but persuasive hassling to endure, the great meals and fun times help make it all worthwhile – and his team in production have added so much.

And to those of you who have used the ideas in your school and class, may you continue to critique, evaluate, and know thy impact.

CHAPTER 1 The challenge

DOI: 10.4324/9781003380542-1

So much has changed since VL1 was published in 2008 (Hattie, 2008). The internet is now even more ubiquitous, iPads were released, climate change is now firmly on the agenda, same-sex marriage has been legalized in many countries, we experienced the global financial crisis, the UK voted for Brexit, there was the Arab Spring, and there have been 26 civil and international wars. In addition, we have seen the rise of populist leaders, the MeToo movement, the gig economy, Angry Birds, selfie sticks, the discovery of the Higgs boson, the first images of a black hole, the rise of electric cars, Fifty Shades of Grey, and the COVID pandemic has caused significant disruptions to schools.

Over this decade, my research career as a psychometrician and in education psychology went into the background as Visible Learning® took over. I have become a grandfather (five granddaughters, one grandson); after 25 years together, I was married to Janet by Elvis Presley (thanks, Peter DeWitt), moved from New Zealand to Australia (following Janet who was head-hunted as professor of evaluation), saw my boys become men; I retired and am enjoying policy roles in government agencies. Visible Learning truly changed my ways of thinking and writing. Since VL1, I have published 75+ articles and 40+ books (Figure 1.1), and presented at 400+ conferences or events worldwide relating to VL1 (Appendix A). Appendix B lists the chapters in the Visible Learning: Guide to Student Achievement – Schools (Hattie & Anderman, 2022), and these chapters provide more depth about many of the influences throughout this sequel (see also Hattie & Anderman, 2013, for further chapters). A mistake of many critics is to overly focus on the one book in 2009, as I have expanded, clarified, and explored many of the ideas in VL1 in these other sources.

Figure 1.1 Books relating to VL.

Improvements to the VL1 model also are a function of working with exceptional implementers, who took the model and turned it into school and classroom practice. The VL+ model has been implemented by this team in over 10,000 schools worldwide. Further, many systems, school leaders, and teachers have informed us about enhancing this process. This allowed greater refinement of the VL1 model, more teasing out of the priorities, a closer consideration of implementation models and scaling up success, and a more detailed collection of evidence of the impact of the model. I also took on a more policy role when the Australian Federal Minister invited me to chair the board of the Australian Institute of Schools and Leaders (www.aitsl.edu.au/), and this has meant meeting all state and territory ministers and their director generals regularly (now up to my 61st in the 9 years as chair – we change our leaders a lot).

The pressure was to write a second edition of Visible Learning, but I resisted. I knew that even my closest colleagues would not be happy when the effect of an influence changed even a little, that any changes would see a new ranking list inconsistent with previous versions, and when the focus would go back to the individual influences. The world of researchers has also moved forward, as there are many more meta-analyses and now other meta-analyses of meta-analyses, and the debates about the interpretation of effect sizes have become more sophisticated. I have learned a lot from my critics. You can spend your career as an academic and not many may notice, so it is an honor to have such world-famous critics.

Instead, this is a sequel, with more attention to the big underlying story. This story covers much more than in VL1, particularly as an additional 1,300 meta-analyses have been added (the production of meta-analyses did not stop in 2009). The growth is shown in Figure 1.2, with about 800 in VL1 and now 2,100 in this sequel.

Figure 1.2 Number of meta-analyses per year.

The details matter, but I have relegated the details and references to all meta-analyses to an online resource. Metax™ (www.visiblelearningmetax.com) is a free website with all the meta-analyses, references, glossary, and FAQ, and it is updated regularly to include the more recent meta-analyses. In this sequel, there is no list of all metas or league tables but an emphasis on the major Visible Learning story or big messages. Too many read and critiqued VL1 as if each influence stood alone despite my stating emphatically that it was the story and overlap that mattered. In VL1, I argued the aim was to build a story about the power of teachers and feedback, and construct a model of learning and understanding. The sequel highlights the story and then provides a set of supporting accounts of this story.

The challenges

The same series of challenges identified in VL1 are still topical: how to move beyond claiming what works to what works best; why the current grammar of schooling that serves many but far from all is so embedded in so many classrooms, and how to improve it; why the learning curve for teachers after the first few years is pretty flat; how to move beyond the inputs (the lessons, the intentions) to the moment-by-moment decisions that teachers make as they teach; how to focus more on learning; and most critical, how to incorporate research evidence as part of the discussions within schools. There remains a need for more focus on the use of successful models of implementation within systems, schools, and classrooms; the attention to the increasing turnover of leaders and its impact; the lack of building evaluation into programs from the outset; and the rush to find and fix failure and ignore the stunning successes we already have and how to upscale this success.

Among the most significant improvements over the last ten years is that research evidence is now a regular topic for discussion in schools – this is not claiming it is privileged nor is it the determiner. But more regularly, teachers and especially school leaders seek the view of the research literature to add to the debates and decisions. The spread of more accessible resources has helped this process – such as Educational Endowment Foundation, Evidence for Learning, What Works Clearing House, Best Evidence Encyclopedia, and others. Gleeson et al. (2022) surveyed 492 teachers from 414 schools about their use of research evidence. About two-thirds reported using research over the past 12 months (91% for school leaders and 61% for teachers), particularly to “design and plan a new initiative” and to “mobilize support for an important issue or decision.” This is the opposite of what Kirschner et al. (2022) call the pedagogy delusion: a set of beliefs and assumptions about what should happen in a classroom that is characterized by a rejection of evidence, an acceptance of the romantic and philosophical, a celebration of the superficial in the form of fads and myths, an assertion that pedagogy is an end in itself, and the creation of an often toxic culture for teachers of unsustainable workload.

The focus now needs to be on optimizing the implementation and mobilization of this research. Our work has shown that this means attending to the mind frames or ways that educators think about evidence, the implementation, and the impact of evidence, and this is more dominant in this sequel than in VL1.

The misconceptions

I wrote five versions of VL1. The fourth was the best, as it was resplendent with detailed stats, variances, conditionals, moderators, and 500 pages of beauty. My best critic, Janet, read it and asked, “Which two people in the world did you write this for?” Ouch. I expunged that version, and the final and fifth version was written. This time, she said, you have increased the audience from two to about ten – then Janet invented the barometer dials to provide more flow-through and common themes, and at the last minute, I added the appendix listing all influences in rank order – the league table. The league table led to the most common misunderstanding of the claims – too many aimed to tick off the influences at the top and disparage those at the bottom of the rankings. Others proclaimed that the influences were not unique, which showed they had not read a page of the book – as it was all about the overlap. More specifically, the aim was to explain the underlying story of those influences above and below the average of all influences or hinge-point of 0.40. The message is in the patterns, not the details; it is the interpretation of evidence not the evidence, the boldness of the big messages not the humdrum of decimal points.

In VL1, I noted the wars as to what counts as evidence for causation are raging as never before – some advocate only randomized control trials (RCT). RCTs are trials in which subjects are allocated to an experimental or a control group according to a strictly random procedure. There are now many RCTs in education despite many arguing that they are not possible or reasonable (Styles & Torgerso, 2018). I prefer not to use a method as a gold standard and raise the bar preferring Michael Scriven’s (2007) claim that a higher gold standard relates to studies capable of establishing conclusions or “beyond reasonable doubt.” Maybe this is too high a bar, but at least the evidence should be “clear and consistent” and more than a “preponderance of evidence,” beyond probable cause,” and certainly “exceed reasonable suspicion.” I claimed that a major aim of VL1 is to weave a story from the studies and their effect sizes that have convincing power, coherence, and generalizability. Building this story remains the mission of this sequel.

I noted in VL1 what the book was not about, which has led to much confusion. I should have been more explicit that it was not about the intricacies of life in a classroom but more about the influences on students learning, doing, and being in classrooms – at the class, teacher, school, and systems levels. I have tried synthesizing the classroom observation research but had difficulty deciding on a standard effect measure. So instead, we developed an app (VisibleClassroom) that provides immediate transcripts of teacher talk with automated feedback, and this has been used in 17,000 hours of class experiences (see Chapter 8). We have built an experimental classroom such that we have replicated Graham Nuthall’s (2007) intensive research using modern technology and have much data on the “hidden lives of learners” (Specjal, 2022). A major synthesis of classroom observation research of the lives of students in classes is sorely needed.

I noted that VL1 was not about more generic factors that educators have little control over, such as social class, poverty, family resources, health in families, and nutrition. Educators cannot change these societal and family issues. These issues, however, can have substantial impacts in ameliorating their influences within the school. A major purpose of schooling is to provide ways for young people to move out of these conditions. Critics jumped on this claim to argue that I did not care about these factors and that issues such as poverty did not stop at the school gate. Again, these comments reflect their lack of opening the book, which discusses the impact of many of these issues within the school gates.

Another misunderstanding was the claim in VL1 that it was not a book about criticism of meta-studies. I argued that these matters were dealt with elsewhere. This led to many saying that I did not care about quality and I included garbage, and again, they did not read much further past this comment. Slavin (2018) was very upset I did not use his “best-evidence” method, but this fails to acknowledge that I synthesized metas, not original studies, and as many have argued (Glass, 2019), asking about the impact of quality is a key focus of meta-analyses, and such studies should not necessarily be thrown out. Throughout VL1, I commented on specific studies (e.g., see the section on Learning styles, p.195–7) and queried some weird results in some studies (e.g., whole language), but I could have made more of the quality issue. Chapter 2 in this sequel deals with many of these issues, and a robustness factor is introduced for the meta-analyses.

Others have claimed I have a fixation on achievement and that there are many other critical attributes of schooling. Again, this ignores the claim upfront that “of course, there are many outcomes of schooling, such as attitudes, physical outcomes, belongingness, respect, citizenship, and the love of learning. This book focuses on student achievement, and that is a limitation of this review” (p. 6). It is pleasing to see syntheses of meta-analyses of motivation (Jansen et al., 2022) and learning strategies (Hattie & Donoghue, 2016), and I look forward to others synthesizing influences on some of the other important outcomes of schooling. I make no apologies for focusing on achievement and wonder what schools would look like if achievement were not an important outcome of schooling.

Others have questioned my utopian view of education or have been critical of my disparaging of teachers or making teachers the fall guys for criticism. By making teachers central to the most powerful influences on students, the criticism is I am blaming them for school failures. It is exactly the opposite: I credit the wonderful successes we have in schools to the expertise of teachers. Another critic wrote that I was inconsistent in praising teachers and then pointing out their limitations – which shows this critic’s lack of understanding of what variance means. The VL data is clear – excellence is all around us, and there is much of it in our schools. The VL model is asking us to have the courage to reliably identify this excellence, create a coalition of this success, and invite others with lower impacts on their students to join this coalition. Often the biggest barrier is a lack of courage.

Looking back to help move forward

The research in VL is based on what has been – and the current system has oodles of excellence. Looking back at what has been or rear-mirror phenomena can allow us more safely to move forward. VL aims to use this past research to identify the common attributes of this excellence. The Turning Point elaborates that this excellence is based on educators’ evaluative thinking expertise, which is the essence of our profession (Rickards et al., 2021). This form of thinking is explored more throughout this sequel. What keeps me going is the many excellent leaders and teachers I meet when I travel the world. Perhaps I could be criticized for being more Pollyanna than Scrooge, but I have evidence to see the world of schools positively.

In VL1, I missed some crucial parts of the model. For example, I noted the power of teacher expectations but did not highlight the even more powerful influence of student expectations; I overly focused on teachers giving feedback but did not highlight the importance of whether or how this feedback is heard, understood, and actioned by students; did not put enough emphasis on the quality of implementation of teaching methods or school interventions; did not explore with sufficient depth the essence of the particular nature of thinking that underlines the effective educator; and missed the importance of teaching students multiple learning strategies and the skills to work in teams. I struggled to make sense of the remarkable variation in the effectiveness of many teaching methods, the low effect of teacher subject matter knowledge, and the low effects of many deeper teaching methods. Indeed, I receive many emails from those advocating methods like problem-based learning proclaiming I am wrong (they mean the research is wrong) and those failing to understand that they may be brilliant, but for every teacher well above, there is another well below (this is what an average means). I have aimed to attend to these conundrums in this sequel.

Teaching and learning

The central message has not changed and is simple – how teachers, leaders, parents, and students think matters most. Their mind frames, ways of thinking, interpreting, and evaluating are core to the success of teaching. It is their thinking that leads to their choice of interventions, devising and explaining the learning intentions and success criteria, knowing when a student is successful in attaining those intentions or not, having sufficient understanding of the students understanding that they bring to the task, and knowing sufficient about the content to provide meaningful and challenging experiences in various progressive pathways to success in learning. The VL model involves a teacher who knows how to implement a range of teaching strategies to provide the student when they seem not to understand, to give direction and redirection in terms of the content being understood and misunderstood and, thus, maximize the power of feedback received by the student. It is having the skill to get out the way when learning is progressing toward the success criteria. This is one of the major principles of Montessori methods, and getting out of the way relates to the notion of teachers gradually reducing responsibility and teaching students to become their own teachers (to know what to do when they do not know what to do). This claim is explored in the updated model of Visible Learning (Chapter 3).

One factor noted in the VL1 model was passion – the joy, the thrill, and the infectious nature of the teacher to cause students to experience learning. Specifically, I noted that passion is among the most prized outcomes of schooling. While rarely explored in any of the studies reviewed in this book, it infuses many of the influences that make the difference in the outcomes. It requires more than content knowledge, acts of skilled teaching, or engaged students to make the difference (although these help). It requires a love of the content, a caring ethical stance to wish to imbue others with a liking or even love of the discipline being taught, and a demonstration that the teacher is not only teaching but learning – typically about the students’ processes and outcomes of learning. This claim has been rarely noted, there are still too few studies on the power of passion, but it remains a visible feature of many classrooms, especially among the students.

In VL1, there were many references to learning, and in this sequel, I have been more specific, devoting a chapter to how educators can influence students’ learning strategies (Chapter 12) and how there needs to be greater alignment of teaching and learning methods. As part of the Science of Learning Project (www.slrc.org.au/), Greg Donoghue and I led a team to complete a meta-synthesis on learning strategies, and their effect on achievement led to a more integrated model of learning (Donoghue & Hattie, 2021; Hattie & Donoghue, 2016). We needed to build this model as we found some strategies were effective at some points in the learning journey but not at other points, and this led to the discovery that learning differs qualitatively at the knowing-that and the knowing-how phases of learning (the surface, deep, and transfer from VL1) – a notion well explored in the more qualitative models of learning (e.g., Marton, 2014). When we aligned the various teaching methods with this model, most teaching methods were silent or random about how they related to this learning journey, and an important plea is more attention to the intentional alignment of cognitive and learning skills.

The mantra from VL1 remains: Visible Learning involves teachers seeing learning through the eyes of students and students learning to become their own teachers. It is more than how we teach and much more about the impact of our teaching. It is about the expertise of educators, how they engage in evaluative thinking, how they work together to critique their interpretations of student learning to move to their next teaching acts and decisions, their openness to learn and seek and receive and use feedback, and their collective efficacy to ensure all (and this means all) students gain at least a year’s growth for a year’s input (and what a year’s growth looks like in this class/school/district). Note that it is growth to achievement (and not merely achievement). Teachers also need to engage students in the thrill of learning, develop skills and confidence to contribute to teams and peers, and develop each student’s respect for self and others.

A decade of impact

The model developed throughout this sequel is deeply informed by the implementation of VL in schools around the world. This deep implementation started soon after the book was published. I was invited to speak at schools, and the after-school presentations did not affect much at all. At the time of publication, I led a team designing, developing, and implementing an elementary and high school assessment model in New Zealand schools (see www.easTTle.org). I had advised the Ministry that development was complete, and they needed to take back maintaining the tool. This meant a number of the team would become redundant. They asked to move into implementing VL in schools, and I set some conditions. It had to be scalable (not rest on the skills of one or a few people), it had to collect and show evidence of impact on the learning lives of students (much more than the usual professional learning criterion of whether the teachers were satisfied and learned much), and it had not to involve me presenting (my skill set is research and not PL). Debra Masters led the team, and soon, all three conditions were met and demand increased.

Initially, the program development and delivery were housed within Cognition in New Zealand. Cognition is a Trust company that had a track record of delivery (in New Zealand and overseas), and I knew the quality of the Cognition staff (I had been on the board for some years). After some years of success scaling up the delivery, Cognition changed direction, and the home of VL+ moved to Corwin in the US (www.visiblelearning.com/). A new team led by Julie Smith, a major refresh of materials, and a greater reach. It has been ten years since implementation commenced, and Clinton and Clarke (2020) completed an evaluation of the extensive database, case studies, and qualitative evidence. The title of their report was “A Decade of Impact.” This implementation and evaluation have deeply informed my current thinking about the model.

The focus of VL+

VL+ is a series of professional activities that promote individual learning and school- or system-based reform. These activities include professional learning events, coaching, evaluation and assessment, and a range of resources. Over the past decades, the major theme has morphed as new research emerged. It started with the power of feedback and then moved to know thy impact, with the focus on teaching the skills of evaluating the implementation of high-probability impact strategies within the teachers’ or leaders’ own school. The continued plea to see success in terms of student learning more than teacher satisfaction led to deeper dialogue about intentions and impact, using Martin Luther King’s plea to focus first on the dream (Hattie & Zierer, 2018, pp. 166–168). The synthesis of learning showed the importance of tailoring teaching to include learning strategies and knowing the right time and context to use particular strategies (the Kenny Rogers idea, “know when to hold ’em, know when to play ’em,” see Chapter 12).

Where VL+ was well implemented, it became apparent there was a strong undercurrent of collective efficacy across the school. Successful innovation can lead to enhanced teacher collaboration and satisfaction (Blömeke et al., 2021). We continued to monitor the implementation to detect when the emphasis was shifting back to primarily changing the staffroom and not the classroom. Hence, we began asking for collective efficacy to deliver at least a year’s growth for a year’s impact. This led to a focus on knowing what this year’s growth looks like, knowing what all teachers meant by impact, and seeing the evidence of this notion of impact in the students learning. The hardest lesson from the decade of working in schools was less the understanding and use of research but that many schools lacked a concept of deep implementation. Schools introduced VL+, and it was hoped that it worked, so we spent much time developing the DIIE model. More recently, the research has aimed to identify educators’ specific evaluative thinking skills that underlined the deeper implementation and consequential impact on students.

The DIIE model

Our experience in schools and the implementation research has led to the development of the DIIE model (diagnose, intervene, implement, and evaluate, see Figure 1.3) to ensure the deep application of VL. After agreement about what impact means, there needs to be an excellent diagnosis, choice of high-probability interventions relative to this diagnosis, choice of an implementation strategy (involving monitoring quality, fidelity, acceptability, and dosage of implementation), and evaluation throughout and also of the impact on students. Too often, interventions are chosen before diagnosing the issues for which the intervention is meant to address; too often, interventions are not introduced and fail because they were not sufficiently implemented; too often, adaptation leads to adding tips and tricks to the current model so that the intervention does not get implemented; and too often, evaluation is not undertaken in schools as they have already moved to the next intervention. DIIE aimed to provide more rigor and steps for leaders to monitor and evaluate the impact of interventions in a more formative manner.

Figure 1.3 The DIIE model and 5D model.

We added a step relating to scaling up in the 5D model (Hamilton et al., 2022). This model was derived from a review of 50 implementation methodologies from various disciplines (such as computing, engineering, business, medicine), 23 implementation processes, meta and systematic review on implementation, and our experiences across more than 50 countries. The model is expanded in Table 1.1, and the relentless focus on the quality, fidelity, and adaptations when implementing remains one of the most crucial and difficult processes when working in systems, schools, and classes. One of the most difficult processes is de-implementation, and we have found it difficult to get educators to stop doing many of the less effective or inefficient practices (Hamilton et al., 2022).

Table 1.1 A framework of the 5D implementation model

D1 Discover
D2 Design
D3 Deliver
D4 Double-back
D5 Double-up

Agree to one education chal-enge that's worth progressing above all else
Systematically search and agree on high-probability interventions to start and to stop
Implement and de-implement agreed designs
Monitor and evaluate your delivery chain and agree on priority actions
Maintain and grow your impact

1.1 Establish a backbone organization

1.2 Decide on the education challenge

1.3 Explain the education challenge

1.4 Agree what better looks like

2.1 Explore options in design space

2.2 Build program logic model(s)

2.3 Stress test logic model(s)

2.4 Agree what to stop

2.5 Establish monitoring and evaluation plan

3.1 Lock the delivery approach and plan

3.2 Undertake delivery

3.3 Collect monitoring and evaluation data

4.1 Monitor your evaluation

4.2 Monitor your delivery

4.3 Evaluate your delivery and agree on next steps

4.4 Evaluate your evaluation

5.1 Consider sustain-ability

5.2 Consider scaling

The decade of evaluation findings

Clinton and Clarke (2020) evaluated all the data from the decade of implementation of VL in over 10,000 schools worldwide. The evaluation included data from the School Capability Assessment, the Visible Learning Matrix, the mind frames survey, the feedback survey, and a meta-synthesis of 47 case study schools.

Half the schools that chose the VL+ implementation had priority goals to improve teaching. The other half aimed to have greater focus more on student learning, such as developing more effective learners; growing student independence as a learner; teaching students how to set their own learning goals and assess their own learning about those goals, how to engage in collaborative learning with their peers, how to know the difference between good behavior and learning behaviors, how to connect new information with known information, and how to engage in deep thinking.

We soon discovered that many schools were not ready to implement the VL improvement model. Some wanted tips and tricks to add to their current repertoire, some wanted to tick boxes that they had engaged with evidence-based programs, some wanted to know how to implement (a few of) the top influences in the VL league table, some had not won over the teachers as to the need for improvement, and many had not already engaged in diagnosing their current improvement needs. We employed a common refrain: what question is Visible Learning the answer to? We developed a readiness scale focused on motivation, capacity in relation to the intervention, the availability of resources (especially time), and a more general capacity for change within a school. This readiness survey is administered along with the School Capability Assessment (SCA). The SCA invites leaders and teachers to evaluate their impact across 16 dimensions, including teachers’ and students’ conception of learning, student progress through to higher achievement, and the school climate.

Across the schools, the average level of readiness to implement was 74%, but there is much variability (from 20% to 100%). This determined the starting levels of engagement, and in some cases, where the score was very low, we requested the school spend more time and attention working with staff (e.g., around collecting and interpreting the SCA evidence, clarifying the intent of engagement with VL, reading the case studies from other schools, visiting or meeting with leaders from VL+ schools) so that they could be readier and more aware of what the model entailed – and thus, increasing the probability of the model having the desired impact on students.

The SCA and readiness review then leads to choices about specific interventions and the building of the within-school(s) program logic. Within the logic model, we pay much attention to identifying the shorter-term (within six weeks), medium-term (within one year), and longer-term (one to three years) desired outcomes. The leadership component of the SCA relates to setting strategic direction and school planning about engagement with VL+, identifying the critical aspects of student outcomes that were the priority focus, and identifying elements of the leaders’ own knowledge and skills. This early work aims to develop the learning intentions and success criteria for the VL+ implementation, setting priorities, skills, coaching to interpret and triangulate the current school test data, teacher judgments, artifacts of students’ work, and student voice about their learning.

The major reasons teachers wanted to engage in the intervention model were to enhance their mindsets, work more collaboratively with colleagues (e.g., thinking aloud about dilemmas, difficult situations or students, or teaching curricula topics), create more engaging and safe class climates where errors were seen as opportunities to learn, include all students in the dialogue of learning, work with students so that they understand their learning intentions and success criteria, improve the impact of feedback and building a feedback culture in every class, move from surface knowing also to include deeper understanding and transfer, move from collecting data to demonstrably interpreting evidence to lead to next teaching decisions, and apply interventions to identify, establish, foster and esteem a language of learning across the school (Hattie K., 2021).

As part of the diagnosis, we also interview samples of students. Typically, students want to learn the language of learning to articulate how they could learn more effectively and efficiently, and how they would know they had learned and attained the success criteria. In addition, they wanted more where-to-next feedback, opportunities to use the feedback provided, greater opportunity to listen to how others successfully learned, and more engaging tasks (engagement for students meant it was worthwhile doing the activities as they could then see their progress in learning, and not that they were interesting, fun tasks, or relevant to their future).

The School Capability Assessment was used not only at the outset but also after a year or so into the model – as a core part of the evaluation. The effect size changes included the following: is there a shared language of learning in this school (0.83), are students taught to be assessment capable (0.79), do educators clearly communicate valued learner characteristics to students (0.62), are educators familiar with different modes of effective feedback (0.58), does the school plan incorporate a focus on developing assessment-capable learners (0.57), do educators have a clear picture of the type of learners they are aspiring to have in their school (0.48), are their opportunities for students to give teachers feedback (0.47), and do teachers make the learning intentions and success criteria clear to students (0.42). After implementing the system-wide improvement model, 100% of school leaders argued that there were greater impacts on learning from involvement in the program, and 70–80% noted increased use of information in class and school decisions, improvements in the language of learning, increased engagement by students, and positive impact on teacher practice, and 84% reported enhanced impact on student achievement.

The effects on achievement were harder to capture but still critical to evaluating the model. Overall, there were 10% more students at or above the respective national (or regional) norms one year after implementation of the model, and the effect size gain from standardized tests of reading and math were greater than gains from normative samples (using the test manual norms). Blewdin and Baldwin (2015), for example, evaluated the implementation of a 2-year VL+ implementation across 32 schools (3,172 students) in Queensland. The achievement tests (math and reading) results showed that students in years 1 and 6 achieved higher scores and progressed at significantly greater rates than non–Visible Learning students.

An independent evaluation of the implementation of the VL+ model in 31 schools in Stockholm (Frej et al., 2017) concluded that VL+ was “well placed to contribute to a sustainable development of a school, act as a coherent perspective, which brings together different development efforts, and can create a common language use in the school around teaching and learning.” The report noted other effects, such as students dedicating themselves to a new use of language around learning, which increased the quality of student-led development talks and generally improved the conversation about learning between students and teachers. School leaders saw VL+ as the binding element, the piece that made the jigsaw come together. It thus became easier to engage teaching staff in joint development projects, led to a higher degree of consensus in the school around teaching and learning, and helped break a culture of isolated islands; developed a new common language use around teaching and learning, a higher degree of professionalism, and increased interest among teachers to discuss and observe each other’s teaching.

There is now a deeply rooted and all-embracing approach at the school that everyone can learn, that all students can progress in their learning; students have also taken on a new language use regarding their own learning, which has significantly increased the quality of student-led development talks; students were better at assessing where they were in their own learning and generally improved the conversations about learning between pupils and staff.

Leeson (2017) evaluated the implementation in 150+ schools across the Northern Territory of Australia. The school leaders rated the baseline measures on the School Capability Assessment at the outset (about their own schools, and following is a sample of ten schools – each school is on one line) as mainly not established (red), existent only in pockets (orange), evident within the school (green), or embedded in the school (blue). The top line represents the four major VL strands. By the end of the three years, most schools had moved from not established to evident or embedded in the school (Figures 1.4a, 1.4b, and 1.4c).

Figure 1.4a Baseline School Capability Assessment.

Figure 1.4b End of year 1

Figure 1.4c End of year 2. Extract from the School Capability matrix for 11 schools across three years of VL+ intervention.

The changes in some of the major measures over these three years are depicted in Figure 1.5. The first line in each bar is the baseline, and the next three indicate how the mean across all the teachers and school leaders improved over the three years. The evaluation team’s overarching finding was that there is sufficient evidence to indicate that the VL model continues to influence progress in school leader and teacher capabilities across the four strands in the SCA.

Figure 1.5 Baseline, fi rst, second, and third year means for each capability measure.

The greatest changes from observations of classrooms pre- to post-implementation were for clarifying questions about the learning intentions (39% to 55%), students working in groups and having significant conversations among each other (19% to 32%), students could state that they were learning and talking to other students and the teacher about aspects of the learning (31% to 63%), student supporting their peers’ learning (37% to 72%), students seeking feedback and seeing errors as opportunities (39% to 81%), and students being aware of their learning steps (30% to 52%). Across the measures of achievement, the reading and mathematics data shows statistically significant gains between time 1 and time 2, and effect size gains above 0.4 that can be attributed to the professional learning model across years 1–6 students. Reading effect size gains above 0.4 are also evident for years 7–10 students. The chief executive wrote to us that the “roll out of Visible Learning across the Government schooling Sector in the NT is having a measurable impact in our NAPLAN (national tests) data” (personal communication).

In another study, a structural model was developed to establish how components of the model impact student achievement (Leeson, 2017). By modeling these relationships, it is possible to predict the VL model’s causal effect on the achievement. These gains were particularly apparent for the low- and medium-performing students. Similarly, but to a slightly lesser degree, improvements in the mind frames survey constructs were also related to a gain in student achievement (Leeson, 2016). Figure 1.6 shows the model that provided the best fit to the empirical data collected for each of the model’s components (Leeson, 2017).

Figure 1.6 Modeling evaluation tools to assess their impact on gains made in student achievement data.

Source: Leeson (2017)

These results show that the gains made by schools in their School Capability Assessments were strongly related to the subsequent gains made by their students in terms of their achievement results (r = 0.78). The strands representing the visible learner, inspired and passionate teachers, and feedback were particularly predictive of student achievement. Furthermore, the development of the leaders’ aspirations (i.e., vision and values) (r = 0.85) and their strategic tools and actions (r = 0.79) had a strong direct relationship with student achievement gains over the year.

A major focus of the VL+ model is to enhance the ten teacher mind frames. About 30,000 teachers completed the mind frames survey (estimate of reliability alpha = 0.75). Figure 1.7 provides the means from time 1 and time 2, and the effect size changes from time 1 to time 2. The greatest improvements relate to “I see assessment about me,” “I focus on learning,” and “I seek and receive feedback,” and the least change is on “I see myself as a change agent.” Changing teachers’ mind frames, evaluative thinking, and expertise to improve student learning (the core of “I am a change agent”) is the most challenging part of the model. How come we are in a profession that struggles to acknowledge its expertise?

Figure 1.7 Mind frames means and changes from time 1 to time 2.

Mind frames

A major message from VL1 was to encourage a move away from debates concerning the structures of schools and classes, and how to teach – to the thinking or mind frames that are the core determinants of success in the learning lives of students. The argument is that what matters most is how educators think. This thinking is the precursor to choosing high-impact strategies, ensuring the fidelity of implementation, and evaluating if there is an important impact on students. Accordingly, we identified ten mind frames for teachers, school leaders, parents, and students. Our more recent work has elaborated the major underpinning of these mind frames – in terms of evaluative thinking (see Chapter 3). The ten mind frames for teachers (Chapter 9) and school leaders (Chapter 7) cover impact, change and challenge, and learning:

Impact

I am an evaluator of my impact on student learning.

I see assessment as informing my impact and next steps.

I collaborate with my peers and my students about my conceptions of progress and my impact.

Change and challenge

4. I am a change agent and believe all students can improve.

5. I strive for challenge and not for the goal of doing my best.

Learning focus

6. I give and help students understand feedback, and I interpret and act on feedback to me.

7. I engage as much in dialogue as monologue.

8. I explicitly inform students what successful impact looks like from the outset.

9. I build relationships and trust so that learning can occur in a place where it is safe to make mistakes and learn from others.

10. I focus on the learning and the language of learning.

We have also developed mind frames for students (Chapter 5), parents (see Chapter 6), and school culture and climate (Chapter 7).

Concluding comments

This sequel is more about the big messages and the core notions underlying the evidence from the 2,100+ meta-analyses. The model in Chapter 3 uses this evidence from the research studies, augmented with experiences from implementing the model via VL+ in 10,000+ schools to build a set of propositions. Within subsequent chapters, there are short summaries of the major messages from the many influences, and most should be read in conjunction with elaborations in VL1. For details, it is essential to access Metax™ to see the detailed evidence and references to the various meta-analyses.

CHAPTER 2 Meta-analysis and criticisms of Visible Learning

DOI: 10.4324/9781003380542-2

The clue for 78-across in the November 26, 2019, New York Times Crossword is “Methods of studying pooled data,” and the answer is “meta-analyses.” It is a milestone when a word is assumed sufficiently common to be permitted in the NYT Sunday Crossword. An approximate indication of the spread of meta-analysis is the number of citations in Google Scholar: 3.5 million for “meta-analysis,” 2.3 million for “meta-analysis and achievement,” 4.3 million for “medicine,” 3.7 million for “environment,” 2.7 million for “psychology,” and 2.4 million for “economics.”

Meta-analysis, invented by Gene Glass in the 1970s, has been widely used and robustly critiqued. It is a method and, like all methods, can be implemented poorly or well. In the early years, meta-analyses tended to be more summaries of many studies; then they became quite statistically sophisticated (Hedges’s g, heterogeneity measures, random models, etc.). More recent meta-analyses defend higher-order interpretations, but there are still too few that evaluate competing explanations. There have been over 2,100 meta-analyses in education relating to achievement outcomes, many more relating to other school outcomes (attitude, motivation, learning strategies), and many more in medicine, and the method has proliferated across many other domains. Meta-analysis is a systematic way of research reviewing and is much less biased than many traditional reviewing methods.

Imagine you are interested in reciprocal teaching (RT) as you are excited about teaching the skills of summarizing, question generating, clarifying, and predicting. You do a Google and Google Scholar search and soon become overwhelmed with the many resources – some claiming the method is of high value and some the opposite. What to believe? So you hunt for a review of these many studies. A traditional review would take many studies, summarize the major findings, and outline where there are moderators (i.e., factors that influence the impact of reciprocal teaching, such as the age of students, curricula domain). The criticism of this traditional method is that there can be bias in the choice of studies – if the author is initially predisposed toward seeing RT as worthwhile, they may leave out negative articles or critique any not-so-positive articles as having poor design and so on. Along came Gene Glass (1976), who explored a way of coding each article for the size of the effect of RT and coding the moderators so there could be a more systematic way of seeing the size of the effect in different circumstances (e.g., which moderators matter). So from the outset, the two major statistics of interest are the mean value of impact and the variability around this mean. Meta-analysis is still fundamentally a method of reviewing. Reviewing is the foundation of our business – we so often start by researching what has been.

This example identifies three significant benefits of meta-analysis. It is less biased in the choice of studies in the review (although all reviews should aim to be transparent about how the studies are selected), determines an overall measure of impact (size of effect), and asks about the moderators. Many critics focus on the flaw of the average, and they are right. The average only makes sense if (a) there is no heterogeneity of effects (which is most unlikely) and (b) there are no known moderators. In education, we have searched for these moderators from day one, and even if we cannot find them, we must continue to search. A major contribution of meta-analysis is the search and methodology around discovering the importance of various moderators (or not).

Yes, the many meta-analyses can vary in quality – some are based on fewer than five studies (these are not included in Metax™), some include minimal moderators (indeed, across all metas, there is no great depth of moderators), some include incorrect estimates of effect sizes (e.g., many of the learning styles metas, see VL1), and some have limited discussion and directions for future work. Too rarely are competing influences compared, the costs (money and time) mentioned, and the degree and fidelity of implementation included, and still many argue that the findings are critical when the overall average is statistically significant from zero rather than compared to the magnitude of other influences.

VL is not based on conducting a meta-analysis of primary studies but on meta-synthesizing many meta-analyses. When VL1 was published, this was still somewhat novel, but now, there are many more syntheses, and certainly, this is the case in medicine (where the intervention and outcomes can be more straightforward). In medicine, they use the term “umbrella reviews” for these syntheses of synthesis (Papatheodorou, 2019). In education, the terms are more likely second-order meta-analyses, meta-meta-analysis, and my preferred synthesis of meta-analyses. There are an increasing number of these syntheses: English language teaching (Alsowat, 2020), creativity (Da Costa et al., 2015), preservice teacher preparation (*Dunst et al., 2020), socioeconomic background (Harwell et al., 2017; Korous et al., 2020), digital technology (Higgins et al., 2012; Tamim et al., 2011; Young, 2017), flipped learning (Kapur et al., 2022), parental involvement (Kim, 2022), math and science teaching (Knogler et al., 2022), distance and online learning (Martin et al., 2022), multimedia design (Noetel et al., 2022), and higher education (Schneider & Preckel, 2017).

* All references commencing with a * are found at www.visiblelearningmetax.com

Among the advantages of such meta-syntheses is that the larger number of studies enables a closer focus on possible causes of heterogeneities, such as the presence of outlier studies, better overviews of possible moderator effects, and greater allowance for deeper investigations of power and bias issues. For example, van Aert et al. (2019) studied the possible biases due to the small sample size, unreliable measures, and the power of studies. They used 83 metas published in Psychological Bulletin and 499 systematic reviews from the Cochrane Database of Systematic Review. They found minimal evidence of overestimation and weak evidence for publication bias in psychology and medicine (but see Kvarven et al., 2020). But like all reviews, the value of meta-synthesis is the quality of interpretation; the building, defense, or critique of models to maximize interpretations; and the new questions that can be asked.

The evidence in this book is based on 2,103 meta-analyses. The reason for 2,100+ in the title is that in most cases, only the average effect per meta-analyses is included, but where there is more than one set of influences within a study, these are also added (210 meta-analyses include more than one effect; so the total sample is 2,313). The meta-analyses are segmented into domains and sub-domains, although there may be other ways to cut and slice these influences. Table 2.1 lists the domains, sub-domains, and relevant chapter in this book where each is discussed in greater detail. Within each sub-domain, further classifications of the influences are outlined within each chapter.

Table 2.1 Chapters, domains, and sub-domains throughout this sequel

Domain
Sub-domain
Chapter
Domain
Sub-domain
Chapter

Student
Prior knowledge and background Beliefs, attitudes, and dispositions Motivational approaches Physical influences
5
Curriculum
Reading, writing, and spelling Drama, arts, music Mathematics and science Other curricula
10

Home
Family resources
6
Teaching with intent
Learning intentions
11

Family structure Home environment

Success criteria Feedback

School and society
Finance and accountability
7
Learning strategies
Skill, will, and thrill
12

Types of schools

Surface, deep, and transfer

Preschool interventions School compositional effects

Teaching strategies
Knowing-that strategies Knowing-how and knowing-with strategies
13

Leadership

Technology
Social media
14

Classroom
Class composition effects Classroom management Classroom climate
8

Distance learning

Apps, gaming, and simulations Consolidation uses of technology

Teacher
Teacher attributes
9
Whole-school and out-of-school
Whole-school programs
15

Teacher education

Out-of-school programs

Table 2.2 provides a big-picture overview of each of the domains, showing the larger effects, which are under the control of the teacher (d = 0.44: curriculum choices, teaching strategies, student learning, technology, teacher), compared to those less in control (d = 0.21: student, home, school, whole-school, and out-of-school).

Table 2.2 Summary statistics for each domain

Domain
No. metas
No. of studies
Estimated total no.
No. of effects
Effect size
Weighted ES
SE

Student
373
26,245
67,186,805
104,174
0.24
0.23
0.06

Home
117
6,676
24,192,643
16,696
0.15
0.15
0.08

School
146
7,446
10,510,357
26,150
0.19
0.20
0.06

Classroom
120
4,752
10,686,418
18,689
0.21
0.22
0.06

Teacher
81
3,837
7,104,805
8,310
0.53
0.55
0.05

Curriculum
377
17,228
20,639,762
52,289
0.50
0.50
0.08

Student learning
278
15,821
3,726,064
30,694
0.55
0.53
0.09

Teaching strategies
423
29,867
11,758,883
56,751
0.51
0.51
0.09

Technology
350
18,905
7,443,108
32,917
0.36
0.34
0.09

School and out-of-school strategies
48
1,612
43,887,942
6,406
0.25
0.24
0.05

Total
2,313
132,389
207,136,787
353,076
0.42
0.42
0.07

Surprisingly, only half the 2,103 metas included information about sample size from the various studies in a meta-analysis. The total N from these 988 metas is 200,863,503 students. A projection of total N, thus, could be 200,863,503 / 988 × 2,103 = 427,546,505. But assume 40% of the students come from overlap in the studies; this reduces the estimate to 256,527,903. Another, and more defensible estimate of sample size, is to impute the average number of students per study from the 500 metas with the smallest sample data (thus, the number of studies × 92 students). This led to an overall estimate of about 207 million. Whatever, it is a large sample size.

More important is the robustness index (described later, and a rating of 1 is assigned if the N is not provided), as this is a more defensible indicator of the confidence in the findings.

The distribution of effects from all the meta-analyses (number of effects) and the summary effects are depicted in Figure 2.1. There are hardly any effects that negatively influence achievement, and most of these are explainable (as most are negative emotions, such as boredom, procrastination, anxiety, attendance, bullying, illness, depression). There are, however, some school effects that are negative, and these are worth noting (e.g., mobility, grade retention, screen time, students feeling disliked, labeling). Otherwise, the vast majority of influences positively impact student achievement. Echoing a major finding in VL1, merely claiming that you can enhance achievement is trivial – almost everyone can. The fascination remains the underlying story of what best discriminates those influences above (in the red zone) and below (in the yellow zone) the average (of 0.42). No surprise that the summary distribution removes some of the larger and smaller effects.

Figure 2.1 Distribution of the number of effects and summary effects.

The barometers from VL1 are not included in this sequel (they would take up too much space given there are now 350+ influences), so Janet invented a new diagram to encapsulate the big-picture story (Figure 2.2). Within the thermometer or temperature gauge, the mean of the influence is reported with the confidence interval of this mean indicated by the ends of the circle. Underneath the thermometer is the summary data of the number of metas, number of effects, the estimated number of people, and the robustness index in the left-most circle (see Figure 2.2). The metaphor is that the temperature is highest to the left (moving to red hot), the impact of the sun optimal and energy highest, and the lower effects leave us somewhat cold (or blue and needs a lot of dressing up). Thus, a teacher’s use of the influence Creativity Programs with a 0.51 effect size is an example of a red-energy practice that can potentially accelerate student learning within her or his classroom. The white circle shows the lower and upper bounds of the mean (+−2 SE around the mean); that is, the confidence bounds around the mean effect size. For those influences where there is but one meta-analysis or where there is insufficient information to estimate a standard error (SE), the average SE = 0.08 is used. An educator can have high levels of robustness (see p. 41) in this estimate as indicated by the robustness of 5, on a scale of 1 (low) to 5 (highest).

Figure 2.2 The thermometer/temperature gauge depicting the summary information, the average effect size, the confi dence interval around this mean, and the robustness index.

Criticisms of meta-analysis

The important point is that meta-analysis is but a method, and like all methods, it has generated a lot of heat. One blogger compiled a list of the criticisms of VL1, including the following:

“Potentially misleading” (Snook et al., 2009)

“Statistical malpractice disguised as statistical razzle-dazzle” (Berk, 2011)

“Bullying by numbers” (Wrigley, 2015)

“Pseudo-science and a house of cards” (Bergeron & Rivard, 2017)

“More akin to pig farming than science” (Biesta, 2017)

“A cult … a tragedy for Australian School Leadership” (Eacott, 2017)

“What monsters wait under the bed of the meta-meta-analysis conducted by Hattie?” (McKnight & Whitburn, 2020)

These criticisms are addressed in two ways. First, addressing the criticism of meta-analysis, or synthesis of meta-analyses, and second, criticisms specific to Visible Learning.

There have been major contributions to improving the method since Glass first outlined the method. This section summarizes some of these major criticisms, and there are many resources to elaborate on each. Wiliam (2016) has mounted the most robust critique but starts by noting he is not anti-meta-analyses and that the use of effect sizes is a “huge improvement on previous ways of reporting the outcomes of educational experiments” (p. 82), but he is anti-inappropriate use and interpretations. I agree but disagree with his two conclusions: that meta-analysis is “simply not a suitable technique for summarizing the relative effects of different approaches to improving students learning,” and he instead recommends “best evidence reviews of research” (which ironically often include many meta-analyses) that need to “take into account the quality of studies the relevance of the findings of the particular age group being taught, and the contexts in which the teachers and students operate” (p. 98). Wiliam makes five major claims.

The intensity and duration of the intervention

Interpretations should consider the length of the intervention, the size of groups affected by the intervention, and the intensity of the intervention. This has long been acknowledged and is a major reason Glass was adamant about the importance of using moderators within a meta-analysis. One of the strengths of meta-analysis is the quality, richness, and information from such moderators. More often, shorter than a longer duration of education interventions have higher effects, probably because the implementation of the innovative can capture the enthusiasm of the teacher and students, and there can be heightened attention to whether it is making a difference or not. There are, however, too few metas that code for the nature and intensity of interventions, the fidelity and dosage of implementation, and this is particularly highlighted later in our meta-synthesis of flipped learning interventions (Chapter 13). So Wiliam is correct to ask for more effective coding of richer moderators, especially relating to implementation.

The file-drawer problem

Building on the argument that everyone loves a winner, many critics suggest that researchers are disinclined to publish studies with unfavorable outcomes. Thus, reviews based on only published studies could overinflate the true effect size. So the file-drawer problem relates to how many studies are buried in file drawers showing no effect for any published effect to be reduced to zero. Rosenthal (1979) introduced a method for estimating a fail-safe number of unpublished articles with an average null effect that would be needed to counter the reported effect size. With his method, it is possible to understand whether this number is convincing to threaten the overall conclusion drawn by the researcher. To show the relations, Table 2.3 presents the number of meta-analyses, number of studies, effect size, and number of file-drawer studies for the meta-analyses with d > 0.2 (the value used to estimate the fail-safe estimate based on Orwin, 1983). There are 39 meta-analyses with less than 25 studies (ave. = 15 studies), each with an average of 114,404 students, 26 effect sizes, and an average of 0.21. There only needs to be 8 studies per meta-analysis in a file drawer with a < 0.2 or negative effects to reduce the current average to nonsignificantly different from zero – and finding these 8 studies seems very plausible. Once the number of studies rises above 100 studies, the level of plausibility of finding goes down (although caution is still needed if the effects are overestimates of the true value due to low statistical power or publication biases). The robustness measure (introduced on p. 41, includes the file-drawer estimate) and should be used to indicate the power and the dependability of the results from the meta-analyses. There can be confidence in the findings when the robustness measure is 3 or greater, and below this, there needs to be caution, as more studies are needed to provide more stability in the estimates.

Table 2.3 The number of studies, meta-analyses, studies, effect size, fail-safe number of publications, and robustness measure

No. studies
No. metas
No. studies
Est no. people
No. effects
Effect size
Fail safe
Robust ness

<5
39
15
114,404
26
0.21
8
1

26–100
91
64
622,515
162
0.38
271
2

101–1,000
196
363
378,497
1,081
0.42
3,777
4

>1,000
30
1,839
2,559,527
4,191
0.45
167,931
5

The age dependence of effect size

Wiliam noted that the variability of older students tends to be greater than that for younger students and for students with and without special needs. This is particularly a problem when more generic measures (such as literacy and numeracy) are compared across wide ranges of ages. In such cases, it would be expected to have greater gains (effect sizes) for younger than older students. But this does not mean you cannot get higher effects with older students. Contrary to Wiliam’s claims, age effects were discussed in VL1 and again throughout this sequel. Further, across the meta-analyses, there is similarity in the average effect sizes by age (Table 2.4). The message is to check for age or background moderators, and where they are present, then note and discuss them.

Table 2.4 Effect sizes moderated by age of participants

Age
Mean
SD
N

ECE
0.40
0.25
117

Elementary
0.36
0.34
130

High
0.32
0.33
73

K-12
0.36
0.35
56

Tertiary
0.42
0.39
168

Adult
0.40
0.46
16

All ages
0.43
0.37
1,752

The sensitivity of instruction of outcome measure used

The claim is that to make combining the results from different studies meaningful, it is necessary to assume that the intervention has the same impact on all the different outcome measures (e.g., teacher-assigned grades, others state test scores, researcher-made, etc.) and that the measures used are sensitive to the intervention instruction. Wiliam argued that this could be resolved if studies were coded for the relation of the assessment to the instruction, such as immediate (classroom test), close, proximal, distal, or remote (standardized national assessment). I agree, and this is a problem with not coding for this moderator. However, a more serious issue is the narrowness or wideness of the outcome measure – with higher effects more likely for narrow (e.g., vocabulary) than wide (e.g., creativity) measures, and this is commented in various places in this sequel.

The generalizability of the studies

Wiliam argues as follows:

the research that actually gets done is often a strange and quirky selection of the research that might get done and generalizing … (and) meta-analysis is simply incapable of yielding meaningful findings that leaders can use to direct the activities of the teachers they led.

(pp. 96–97)

He then uses some extracts from the VL1 league table to argue his case. For example, for self-report grades (d = 1.44), he argues that it is “clearly absurd” that 9–11-year-old students would make 4.6 years of progress and that it “strains the bounds of credibility,” for kindergarten students also to make this gain. But I am no fan of converting effect sizes to age gains, mainly because it leads to such absurd claims. A d = 1.44 merely means that there is a gain beyond their prescore or compared to similar to students not exposed to the intervention. But his claim does show the care needed to generalize interpretations of influences.

Other criticisms of meta-analysis

Unsurprisingly, there is a healthy critique of the meta-analysis method beyond the Dylan five. These include the following:

Reductio ad absurdum

As I noted in VL1, my colleague Alison Jones remarked how fascinating it was that I could understand classrooms to the second decimal point. Her comment is a sobering reminder of the importance of knowing the cultural context of the classroom, being aware of what the student and teachers bring to the class from cultural and sociological perspectives, and not reducing findings to simple homilies.

Not combining apples and oranges

This claim is that it is unreasonable to bundle together different types of data. But we make decisions about fruit, and the moderators remain critical to see how apples differ from oranges. This combining is common to all forms of literature reviews, and the essence of discovery is finding the underlying themes of comparison and difference.

Driving the car backwards

Meta-analysis is a rear-view mirror activity: it accounts for what was, not what could be or should be. Yes, but I would not want to drive forward without attending to what has happened and continuing to check as I venture down new roads. The essence of research is searching again – hence, re-search. Some research areas have produced low effects despite inordinate advocacy, cost, and studies, and ignoring the rear mirror may condemn us to continue false advocacy at a large cost. By looking back and accepting the evidence, we are much better placed to ask the right next questions.

Costs

The criticism is that the effect sizes do not take into account the costs of the intervention. This is indeed the case and if only there were more cost-benefit analyses. I am not a fan of many of the transformations to cost – for example, the Education Endowment Fund marks feedback, mastery learning, and teaching phonics all at a very low cost (up to $4,000 per year per class). But the professional learning to enhance these skills and implement them in classes can be very expensive. This cost model falsely assumes it is simple for teachers to use these methods.

Other criticisms

We also hunted for all criticisms of VL and detail these in “Real Gold vs. Fool’s Gold” paper (Hattie & Hamilton, 2019), which should be read in parallel with this chapter (Table 2.5).

Table 2.5 List of criticisms in the “Real Gold” paper

A.
Issues of meta-analysis
C.
Interpretation of effect sizes

1.
Weighting
19.
The use of the hinge point of d = 0.40 seems arbitrary

2.
Sample size
20.
The hinge point does not control for moderators and mediators

3.
Sampling
21.
The average effect size can be moderated by age

4.
Reductionism
22.
There are so few moderators discovered in VL such that we can use the average

5.
Quality and aggregation
23.
It is wrong to focus on influences with high effect sizes and leave out the low ones

6.
The file-drawer problem
24.
Correlation does not imply causation

7.
Fruit salad
25.
The non-inclusion on qualitative studies. The following is an example of this criticism:

8.
Important studies are ignored
26.
Meta-analyses are not sensitive to instruction

9.
The average effects keep changing
27.
Meta-analyses do not control for costs

10.
The effect sizes in Visible Learning change over time
28.
Others are now recanting their own use of effect sizes

11.
Rear view of the world
29.
Visible Learning ignores debates about what is worth learning

12.
A narrative review is better
30.
Visible Learning is only about achievement, and this is not all that school is about

B.
Effect sizes
31.
Visible Learning ignores socioeco-nomic effects

13.
Mixing different effect sizes is bad science
32.
There are many risks involved when interpreting meta-analyses.

14.
Common language effect size (CLE) is wrong
D.
The Visible Learning model

15.
Half of the statistics in Visible Learning are wrong
33.
There are alternative interpretations based on the Visible Learning data

16.
Effect sizes are not used by mathematicians
34.
It is but a model

17.
Effect sizes should not be used in education, as they came out of med
35.
The main messages of Visible Learning defy widespread teacher experience

18.
The variability of effects is ignored
36.
VL reports the opposite conclusion to that of the actual authors of the studies it reports on

37.
The influences are not separate

38.
Narrowness for breadth of outcome (i.e., VL just focuses on student assessment outcomes)

Specific reactions to criticism of VL

The challenges of ranking

PISA ranks schools, QS ranks universities, the English Premier League ranks football/soccer teams, and the Dow Jones ranks companies. One of the common criticisms of VL centers on the appendix ranking of the various influences on student achievement, and the argument is that this ranking creates the perception that each influence is unique and that simply doing more of the things at the top of this list and perhaps stopping things ranked at the bottom, great things will happen.

These views are mistaken. Each influence is far from unique, and the reason it took 15+ years to write Visible Learning was the time required to understand the relation among the many influences and to develop a model that best discriminates between the influences above and others below the hinge point. Some of the lower influences need deep understanding why they are low – so that they can be improved. Lower effects should never be automatically dismissed as not worthy (e.g., see discussion on problem-based learning, individualization, teacher subject matter knowledge).

Beyond the school gates

A major barrage of criticism comes from those who argue that the Visible Learning research does not place sufficient emphasis on out-of-school variables (see Snook et al., 2009). This criticism is a little unfair, as VL1 and the research database reviews the home domain, including family dynamics, structure, and resources. But it is the case that the analysis and the interpretation of the data look far more closely at the within-school influences. There are two reasons for this. The first is that the intended audience for the research is teachers, leaders, and education policy makers, so it makes sense to focus much more on things they have some control over and can do something about. Neither teachers nor school leaders can quickly or easily change the home environment of students, but if they work together, they can significantly improve the school experience for learners from all home environments (Hattie & Hattie, 2022).

Of course, students do not leave their home backgrounds or cultural influences at the school gate; so awareness of what students bring from the home is an important factor and needs to be considered by schools in making a difference in the learning lives of all students. It is a truism that improving the school experience alone is enough to make a difference to the learning outcomes of all students, and the partnerships with home are critical (and this is investigated in Chapter 7).

The use of d = 0.40 hinge point

Hinges are great: they enable us to swing wardrobe doors and the arms of our eyeglasses, and move our knees and elbows. In VL1, both the analysis and the effect size barometer graphics presented d = 0.40 as a hinge point. VL aimed to discriminate between those above and below this hinge point. Yes, in earlier writings and presentations, I probably disparaged the effects < 0.40, and some remain worth disparaging, but as noted in the last section, some of these low effects need understanding to improve.

We must not get too oversold on using d = 0.40 in all circumstances – the interpretation can differ in light of how narrow (e.g., vocabulary) or wide (e.g., comprehension) the outcome, the cost of the intervention, the challenge of learning the intervention, and many other factors. When implementing the VL model, it is worth developing local knowledge about what works best in the context and not overly relying on 0.40 blindly. For example, when working in schools, we more often use the mean of the class or year group on a particular assessment to discriminate between above and below average, and then aim to understand the variance of student outcomes. The 0.40 merely is the average of all 2,100+ meta-analyses, and it serves as a worthwhile hinge than the usual zero (which allows nearly all to claim that their favorite strategy or influence can enhance achievement).

Quality of studies and mixing of the methods for calculating effect sizes

The argument is that the studies in the various meta-analyses mix RCTs, pre-post, and group comparison methods, and that they used a variety of testing instruments ranging from standardized student achievement tests, IQ tests, self-perception survey data, quantified observations, teacher-made tests, and correlational data (all correct). Finally, many of the studies are quasi-experimental and have no real control group, and others are much weaker correlational studies (correct again).

It is suggested that by including all these types of data, rather than just the gold standard RCT studies, then the quality of the research has been significantly compromised. If only perfect studies were included, there would be insufficient data to draw conclusions. Indeed, in the What Works Clearinghouse, which only allows RCTs and similarly high-quality designs, the median number of studies in each of the 500 reviews is 2 (Lortie-Lortie-Forgues et al., 2021; Stockard & Wood, 2017). It is impossible to conclude much based on two studies other than more research is needed.

So we have a choice to make. We either limit ourselves to collecting the perfect studies or harvest the lot but take great care over how we interpret the data and the conclusions we draw from it. In the case of Visible Learning, the latter approach was taken. It is possible then to ask whether the quality of the study of meta-analysis makes a difference to the overall conclusions.

There were two exceptions to this take-all approach. First, there were poorly executed meta-analyses such that they were impossible to defend, and these were discarded. Second, there were about 30 where the effects were so high to be unbelievable. Any means greater than d = 2.00 were only included if the sample size was large, the study was well conducted, and there were other studies in that category to provide comparisons (there were n = 2 metas with d > 2.0, one in flipped learning and one in reading).

There were no meaningful correlations among the estimate of the effect size and the number of studies (r = 0.01), total or estimated number of participants (r = −0.06), or the number of effects (r = −0.01). These sample size characteristics were unrelated and give some comfort to including metas regardless of the sample size.

Quality of meta-analyses

There is a healthy debate about quality measures of original studies used in meta-analyses (Luchini et al., 2021) but far fewer discussions about the quality of meta-analyses. For meta-analysis and meta-synthesis, the more common debate is about publication bias; that is, the selective publication of statistically significant results, which is reviewed by comparing the effects in published and unpublished outputs (e.g., conferences, theses). Across the 2,312 metas, the effects were lower in the grayer sources (conferences and reports) and highest in books (Table 2.6). There was no difference in the average mean for dissertations and journal articles.

Table 2.6 Source of publication by effect size

Source
Mean
SD
N

Conference
0.34
0.32
108

Report
0.36
0.30
71

Dissertation
0.41
0.36
285

Journal
0.42
0.37
1,772

Book
0.52
0.37
76

Another way to check bias is to ask if there are differences relating to the quality of the journal. Mathur and VanderWeele (2021) compared the publication source (they classified them into lower- and higher-tiered journals) across 63 meta-analyses for such bias and concluded that “that publication bias is perhaps milder than expected in meta-analyses” (p. 14). Similarly, I coded the 1,397 journals from the meta-analyses in terms of various measures of journal quality (www.scimagojr.com/journalrank.php). There is a hint that meta-analyses with lower effects and larger samples of studies are published in higher-quality journals (see Table 2.7).

Table 2.7 Correlations of the number of studies, students, effects, and effect size with two quality indicators of the journals in which the meta-analyses were published

No. studies
Est. total no.
No. effects
Effect size

SJR - average number of weighted citations in the journal over the previous three years
0.22
0.05
0.04
−0.11

h-index - no articles receiving at least h citations
0.19
0.02
0.04
−0.15

To tease this hint out some more, the journals can be ranked according to the overall journal rankings and divided into four equal groups. Q1 comprises the journals with the highest values, and Q4 the lowest (https://en.wikipedia.org/wiki/SCImago_Journal_Rank). It is obvious that the highest effect sizes are more likely to come from the lowest-rated journals and the lowest from the highest-rated journals. This is clear evidence of a publication bias, although the overall effect of the majority of the journals are in the highest-quality journals (which is comforting), with an average close to the overall 0.40 (Table 2.8).

Table 2.8 Effect sizes moderated by the quality of the journal

SJR best quartile
Mean
SD
N

Q1–highest rated
0.38
0.36
1,147

Q2
0.43
0.38
166

Q3
0.52
0.30
55

Q4–lowest rated
0.66
0.46
23

Others have recommended weighting the effect sizes by sample size (or the inverse of the standard deviation) to thence place more importance on studies with larger samples. Each summary influence mean was weighted by the number of studies. Clearly, this gives more weight to the larger studies. Across the 358 summaries, the average difference between the weighted and unweighted means was 0.00 (and 0.04 for absolute differences). There were 7 summaries where the differences exceeded plus or minus 0.20, and these will be noted in the summaries section across the various chapters: task complexity (−0.22), mainstreaming (−0.20), alternative assessment methods (−0.20), cognitive task analysis (0.20), success criteria (0.23), rehearsal and memorization (0.25), and deliberate practice (0.30). Where the effect is positive, then the weighted mean (which is probably more correct) is higher when the larger sample of studies is considered. When it is negative, then the larger studies led to a lower estimate.

Country of the first author

There are over 4,800 authors who have contributed to the meta-analyses in this sequel. Each of the meta-analyses was tagged to the country of the first author. The first authors of the meta-analyses came from 52 countries. Almost all (87%) of the metas come from high-income countries (Table 2.9). The majority come from the US (58%), Turkey (5.8%), the Netherlands (4.7%), China (4.2%), Canada (3.8%), Australia (3.6%), Germany (3.5%), Korea (3.2%), the UK (2.9%), Taiwan (1.2%); and these account for 90% of all metas (Table 2.10, 2.11). The highest mean effect sizes come from Africa, Southern Asia, Central and Eastern Europe, and in particular, Indonesia (0.75) and Turkey (0.73), and the lowest from the UK and Canada. It is likely that the comparison groups (classroom as business as normal) are much different (more regimented, teacher dominated, focused on content) in the lower-income than in higher-income countries, but this needs further exploration.

Table 2.9 Number of countries, number of effects, means, and SD for metas moderated by income resources of the country of first author

Income
No. countries
No. effects
Mean
SD

High income
36
2,020
0.43
0.15

Upper-middle income
10
248
0.55
0.22

Lower-middle income
6
43
0.75
0.15

Table 2.10 Number of countries, number of effects, means, and SD for metas moderated by areas of the world of the country of first author

Where
No. countries
No. effects
Mean
SD

Latin America and Caribbean
5
10
0.29
0.24

North America
2
1,424
0.34
0.06

Europe and Central Asia
24
477
0.46
0.15

Middle East and North Africa
5
27
0.48
0.20

East Asia and Pacific
13
362
0.60
0.17

South Asia
1
7
0.62
0.00

Sub-Saharan Africa
2
4
0.75
0.06

Table 2.11 Number of countries, number of effects, means, and SD for metas moderated by country of first author

Country
ES
N
Country
ES
N
Country
ES
N

Brazil
0.04
3
Germany
0.37
80
Romania
0.61
2

Chile
0.10
4
Belgium
0.37
9
Ireland
0.62
1

Israel
0.23
9
USA
0.38
1,332
India
0.62
7

Italy
0.23
6
Puerto Rico
0.41
1
Morocco
0.62
1

Mexico
0.26
1
Poland
0.41
4
Columbia
0.64
1

Sweden
0.27
2
Total
0.42
2,312
Estonia
0.65
1

Denmark
0.29
12
New Zealand
0.43
20
Lithuania
0.65
1

Dubai
0.30
1
Luxembourg
0.44
3
Greece
0.66
2

UK
0.30
68
Spain
0.47
20
Saudi Arabi
0.67
1

Canada
0.30
87
Cyprus
0.49
3
Thailand
0.68
3

Finland
0.31
4
Portugal
0.51
4
Serbia
0.68
2

Norway
0.36
9
Saudi Arabia
0.52
7
Iran
0.70
8

France
0.36
1
Taiwan
0.54
28
Macau
0.70
2

Hong Kong
0.36
13
Japan
0.56
13
South Africa
0.71
1

Slovenia
0.37
1
Korea
0.57
73
Turkey
0.73
135

Australia
0.37
84
Malaysia
0.57
6
Indonesia
0.75
20

Switzerland
0.37
4
China
0.58
96
Nigeria
0.79
3

Netherlands
0.37
107
Singapore
0.61
1
Philippines
1.02
5

There can be confidence that the research conducted in the more well-resourced countries has a high level of transference across other well-resourced countries.

But we need to be careful about transposing this to developing country contexts, like Sub-Saharan Africa or South Asia – now dubbed Global South countries. Hamilton and Hattie (2022) have reviewed close to 1,000 meta-analyses and random controlled studies conducted within Global South countries. The messages are critically different in enhancing the learning outcomes of these schools. The two ecosystems are quite different. In the developed world, education policy is focused mainly on the quality of provision, equity of outcomes, and enhancement of the learning lives of students. In the developing world, the challenges are all too often around access and infrastructure, and how they focus more on investing in schools than in learning.

Correlations and causation

Care is always needed when making causal statements, and while correlations can play a major part in making such claims (e.g., via structural models), the well-known adage “a correlation is not a cause” is still a great starting point. The term “effect size” does not help, as the notion of effect falsely can imply a causal effect. Campbell and Stanley (1966) noted five conditions for making causal claims. There needs to be an association between the influence and the effect, a time order, ruling out no-spurious relations (e.g., eliminating third variables that may explain the causal relation), an explainable mechanism among the variables, and internal and external threats to the validity of the claims. Thus, researchers need to investigate counter-factuals to their causal claims and reduce “the number of plausible rival hypotheses available to account for the data. The fewer such plausible rival hypotheses remaining, the greater the degree of ‘confirmation’” (Campbell & Stanley, 1966, p. 206).

Throughout this sequel, there are many examples of correlates of achievement (e.g., sex of student, preterm birth weight, personality characteristics), and I use r rather than d when discussing such correlates. But there are also many effects based on attempts to claim causality (introducing a teaching method compared to other methods or pre-post gains). There is always care needed, however, to ensure the mentioned five conditions are met, and too often, the quality of the moderators is not sufficiently of high quality to ensure plausible rival hypotheses are considered.

The core set of causal claims is the explanatory story in Chapter 3 and Chapter 4 to explain the mechanisms among the variables. I subject these claims to refutation every time a new meta-analysis is published, inviting others to provide evidence for counter-factuals and inviting all to subject these claims to refutation. I have aimed to be bold in the claims, as Popper (1992, p. 280) said as follows:

of boldly proposing theories; of trying our best to show that these are erroneous; and of accepting them tentatively if our critical efforts are unsuccessful. From the point of view here developed, all laws, all theories, remain essentially tentative, or conjectural, or hypothetical, even when we feel unable to doubt them any longer.

Type of effect sizes

A related criticism is mixing studies based on correlations, pre-post, and experimental-comparison designs. This is a reasonable criticism, although at the level of meta-synthesis, this can be harder to disentangle. Those undertaking first-order meta-analyses may need to include the nature of the design as a key moderator, as this would untangle this issue. It is the case that most meta-analyses are clear when they use a correlation compared to mean gains, although the latter can be a mix of post-pre, postgroup comparison of two groups, and others. In a pre-post design, the effect is more a measure of the gains beyond business as normal, but then so is comparing the experimental to the comparison of business as normal groups. There certainly needs to be more coding of the nature of what is business as normal, as this would help untangle the meaning of a gain. One overview method is to ask if there is a difference in the metas based on correlations (0.42, n = 315) or mean differences (0.40, n = 1,443) as the effect sizes; there is not.

All metas were coded using a method recommended by Wolfgang Berwyl (personal communication). He argued that there are at least five major groupings of influences, from those directly aiming to be causal interventions on the teaching of students to causal no-intervention studies, which schools often have little influence over.

Causal – intervention – teaching. Educators can implement such an influence in a very short or medium term (a few minutes to several months to very few years). To do so, they intentionally intervene in the school (e.g., via staff development, etc.) or change the teaching. In doing so, there is at least an implicit causal model. Examples include most teaching methods, feedback, and new programs.

Causal – intervention. This relates to where an influence can be argued to relate to a change in achievement outcome. Examples include increased motivation.

Correlation. This study relates one variable to an achievement outcome; there are no necessarily causal claims, and examples include anxiety, emotional intelligence, and many emotional variables.

Causal – intervention – context. This relates to interventions in the structure of school and teaching, such as class size and grouping methods.

Causal – no intervention. This relates to an influence that has an effect on learning performance without there being a (relevant) retroactive effect. Examples include physical characteristics of students (diseases), socioeconomic status, school types (e.g., charter schools), or also characteristics of teachers (e.g., credibility).

In some cases, there can be a mixture of studies correlating to achievement and others aiming to enhance achievement. For example, correlating degrees of fitness of BMI to achievement compared to a program to deliberately engage students in exercise and sport to improve their learning; or well-being can be a correlate or the focus of a deliberate program to modify well-being and evaluate the effects on achievement; or there are many studies correlated learning strategies with achievement and others deliberately implementing programs to enhance learning strategies to see the effects on achievement. For most influences, the classification is clear. The effects are highest for the more causal influences and lowest for the correlates, and those that relate to the structures of schools (causal – intervention – context) and causal no-intervention studies (Table 2.12).

Table 2.12 Effect sizes moderated by nature of the causal to correlation influence

Type of influence
Mean
SD
N

Causal - intervention - teaching
0.51
0.32
1,175

Causal - intervention
0.53
0.32
164

Correlation
0.30
0.44
319

Causal - intervention - context
0.30
0.30
369

Causal - no intervention
0.26
0.41
285

Fixed and random models

Over the past 15 years, more meta-analyses have switched from estimating the values via random rather than fixed models (Hedges & Vevea, 1998). The random model assumes each study is from a universe of possible effects and, thus, considers the width of the confidence interval to estimate the overall mean. If instead the variability of the universe effect size is zero or if there is good reason to assume the universe effect size is common to all studies, then we used a fixed model. Despite the difference in assumptions (and my expectation that random would be much higher), this is not the case (fixed = 0.40, n = 1,365; random = 0.44, n = 947).

Curriculum areas

Among the most common criticisms I receive via emails are about the uniqueness of subject areas – surely math, languages, performance arts (and you could almost name every curriculum area) are different. The effects are higher in the social sciences (humanities, social sciences, language), but the other curricula domains are similar (Table 2.13). Yes, this should be an investigated moderator, but as noted earlier, the narrowness or wideness of the outcome variable (whatever the curricula domain) is probably more critical. The overall VL messages are argued to apply, regardless of discipline, but note it is the proportion of knowing-that (the content) and knowing-how (the relations and conceptual ideas) that is more critical within any curricula domain; so curricula are not omitted but just not a major moderator to the importance of the teaching and learning processes.

Table 2.13 Effect sizes moderated by curriculum domains

Subjects
Mean
SD
N

Hum and social sciences
0.64
0.62
7

Language
0.60
0.29
34

Creative arts
0.55
0.27
29

Science
0.55
0.37
174

Writing
0.53
0.29
26

Reading
0.48
0.38
238

Math
0.41
0.40
205

All
0.38
0.35
1,599

Age

Almost as common is the claim that surely, the effects are different for 5-, 15- and 25-year-olds; we all know elementary is different from high and tertiary. They seem not to be; so again, the messages cross age boundaries (Table 2.14). It is noted, however, that care is needed in extrapolating to adult populations, as they were not part of the selection for meta-analyses in this book, and adults in learning to speak English classes, adults in professional learning, and adults in psychology studies are most different. It is hard to get representative samples of adults.

Table 2.14 Effect sizes moderated by age of students

Age
Mean
SD
N

ECE
0.40
0.25
117

Elementary
0.36
0.34
130

High
0.32
0.33
73

K-12
0.36
0.35
56

Tertiary
0.43
0.40
171

All ages
0.43
0.37
1,765

Special-needs and gifted students

The claim is that these students are different from nonspecial-needs students. Table 2.15 shows that those meta-analyses that use students with learning difficulties have higher effects; otherwise, there is little difference relating to the classification of students. Again, these learning difficulties meta-analysis tends to focus on specific content on knowing-that skills and knowledge, which is more likely the reason the effects are higher.

Table 2.15 Effect sizes moderated by classification of students

Mean
SD
N

All
0.41
0.35
2,053

English Language Learners
0.49
0.37
33

Gifted
0.45
0.26
20

Learning difficulties
0.63
0.33
84

Special needs
0.45
0.48
122

Overlapping data

When there is more than one meta about an influence, the various metas are likely to include many articles in common, which could bias the average effect. For example, in our analysis of 28 metas on flipped learning, there are 765 references (3 did not provide a list of studies), and 471 of the studies (62%) were unique to any one meta-analysis (see also Hew et al., 2021). This was a surprise, as I expected lower levels of unique articles. The plea is for meta-analysts who study a similar topic to provide the extent of overlap of studies with other meta-analyses. Having unique studies is valuable not to merely replicate the earlier findings (if the studies are the same, then surely the mean and other estimates should also be the same, although they could be coded for different moderators).

Robustness index

As I have updated the Metax™ database, sometimes the average within an influence changes – sometimes a lot, mostly not by much. This has led to some questioning the stability of the findings, and I often get asked to stop adding more, as it becomes confusing when average effects change. The interest in producing more meta-analyses is not abating, and this is as it should be, so stopping and freezing the data in VL1 does not make sense. I continually find, read, and add new meta-analyses to the database, and as seen in some parts of this book, these additions can modify the conclusions from VL1. It is not surprising that there are likely to be changes in the average effect since the earlier meta-analyses were based on very few studies.

The robustness index is a weighted average of five major indicators of stability: the number of meta-analyses, the number of studies, the estimated number of students, the number of effects within an influence, and the fail-safe number of studies. Each of these five measures was divided into quintiles, and the average quintile is the robustness index for an influence (e.g., in the discussion of the thermometer earlier, the average across the quintiles of the five measures for creativity programs is R = 5, indicating it had high robustness, as the sampling was most extensive). For each influence, a robustness index is provided, and there can be more confidence in these robustness indices from 3 to 5, and those influences with robustness indices of 1 and 2 may likely wobble as more studies are added to this influence over time (Table 2.16).

Table 2.16 The five quintiles of robustness and the data used to calculate them

No. meta
N
%
No. studies
N
%
No. people
N
%
No. effects
N
%
Fail safe
N
%
Robustness
N
%

1
1
65
18%
1-50
65
18%
0
69
19%
1-72
65
18%
0
85
24%
1
44
12%

2
2
50
14%
51-118
50
14%
1-6,639
73
21%
73-176
50
14%
1-193
57
16%
2
79
22%

3
3-4
85
24%
119-239
85
24%
6,640-23,979
72
20%
177-464
85
24%
194-972
72
20%
3
93
26%

4
5-9
79
22%
240-531
79
22%
23,980-166,799
71
20%
465-1,260
79
22%
973-4,063
71
20%
4
78
22%

5
10+
77
22%
532+
77
22%
166,800+
71
20%
1,261 +
77
22%
4,064
71
20%
5
62
17%

Conclusion

As outlined in the previous two sections, most of the criticism about Visible Learning centers on the research methodology rather than the interpretation, explanations, or prescriptions from the data. A major reason for releasing Metax™ was to save other researchers the decades it took to create the database and the open invite to come up with alternative explanations.

Meta-analysis is but a method, and the critics are correct if the effect of VL is merely the ranking, and seeing each influence as (quasi-)independent. The influences are more probability statements, and the quality and fidelity of implementing any interventions matter most. Gene Glass’s contribution was his relentless focus on not only devising a measure of magnitude but also searching for moderators, as so often the answers lie in the variability. My own supervisor, Rod McDonald (2014), drilled into us that the answers lie in the residuals. Marton and Pang (2006) have powerfully shown how error variance is a source of major insights for teachers and researchers, and a teacher’s sensitivity to the varied reactions of students to their teaching is essential for effective impact. My aim in this sequel is to outline the model of VL I have been building and testing (in schools and with every new meta-analysis) over the past 40 years.

Of course, the quality issues when conducting a meta-analysis matter a lot, and the care to interpret effect sizes is paramount. Kraft (2020) developed worthwhile guidelines: highlight that results from correlational studies presented as effect sizes are not causal effects; know that the magnitude of effect sizes depends on what, when, and how outcomes are measured; expect outcomes to be more closely aligned with and, soon after the intervention, to have larger effect sizes; know that subjective decisions about research design and analyses influence effect sizes; know that costs and scalability matter for evaluating the policy relevance of effect sizes. And I would add, be aware of the narrowness or breadth of the outcome measure.

The current synthesis of meta-analyses adds an important perspective on classrooms. It needs to sit aside other methods for understanding the dynamics of classrooms and the optimal interventions to enhance learning (such as observation studies as noted in Chapter 15, qualitative studies, and so much more). The major claim is not that meta-analysis and Visible Learning lead directly to specific policies about what to do. At best, the interpretations from meta-analyses offer probabilities about likely high-impact interventions. There should be comfort in choosing high-probability interventions, but the mantra “know thy impact” aims to put more focus on the quality of the implementation of these high-probability interventions.

The key is still the quality of interpretation from the findings of the meta-analyses and the (sometimes unexpected) consequences of the major underlying explanations. This is why know thy impact is so important as it begs the use of high-probability interventions for specific questions, specific students, at specific times in the learning cycle, and then the quest for educators to evaluate the fidelity of any implementation on their skills to influence student learning. The fidelity of the implementation in the local context and the skills of teachers to adapt and modify in light of their evaluations of the implementation are what lead to maximizing the impact on the learning lives of their students. Hence, know thy impact.

CHAPTER 3 The model

DOI: 10.4324/9781003380542-3

The Visible Learning model is based on five premises: the why, how, what, the doing, and the evaluating:

Being clear about the purpose – e.g., teachers see learning through the eyes of the students, student become their own teachers, and teachers become their own students.

It is about how educators and students think, hence the importance of mind frames.

The intentional alignment among decisions about the knowing-that, knowing-how, knowing-with, aspects of the curricula, cognitive task analysis, optimal teaching interventions, and assessment or evaluation strategies.

The quality of implementation based on discover, design, deliver, double-back, and double-up.

Underlying all aspects of this model is evaluative thinking.

These five premises address the why, how, what, the doing, and the evaluating. This model is firmly based on the expertise of educators. It aims for every student to gain at least a year’s growth in learning for a year’s input, to students wanting to reinvest in their learning of the curricula, knowing how to learn alone and in teams, and learning to become their own teachers. It acknowledges the need for opportunities to learn, access to great teachers by design not chance, supportive families and societies, resources to invest in instructional time and resources, and an inviting culture and climate. Finally, it pertains to all students no matter where they start, what they bring, and how others support them.

This chapter outlines these five premises, and the next chapter delves deeper into the all-so-critical implementation phases.

Figure 3.1 Five premises of the Visible Learning model.

Purposes

The model starts with the purposes of schooling (Table 3.1). The purposes are often contested and can differ for educators (system leaders, school leaders, teachers, resource support), families, and students (Hattie & Larsen, 2020). The major claim here is that schooling aims to enable students to become their own teachers and want to invest in school learning. This does not mean a narrow focus on curricula or test scores but means inviting students to invest in learning sufficient foundation skills to be able to make decisions about their passions, achievement focus, and worthwhile activities that enable them to respect themselves and others, work alone and with others, and learn to become their own teachers. It means exposing students to what they may not know and venture into curricula domains to entice them to invest their energies and gain at least a year’s growth for a year’s input. Of course, there can be much debate and contention about what a year’s growth should look like, but this is among the most worthwhile educational debates. Through progress, learning, and motivation, we lead to achievement.

Table 3.1 The purposes for teachers, students, school/leaders, and families

Teachers
Students

See their impact through the eyes of students
Become their own teachers Want to reinvest in school learning

Teach students to become their own teachers

Become students of their own teaching

School/leaders
Families

Lead school-system-wide improvement
Become their child’s first learner

processes so that all educators maximize their impact on students

The major question for educators is to know thy impact, which begs questions about what impact means. At a minimum, this impact refers to educating students to know that, know how, and know with. This emphasis on the right balance of surface, deep, and transfer knowing and thinking requires critical judgments, appropriate teaching, optimal learning strategies, and appropriate assessment. Educators need to be explicit about what they want their students to know, do, and care about.

At the end of our Mind frames book, Klaus Zierer and I invoked our personal beliefs about the purposes of education (Hattie & Zierer, 2018; and since further explored in Hattie & Larsen, 2020).

People spark revolutions – through their visions, beliefs, and dreams. We have a dream of developing passion for learning and developing an education system that values inviting all students to come and learn, to belong, and to reinvest in their own learning. We have a dream for an education system that develops respect for self and respect for others; to have sensitivity to collaborate with others in the pursuit of learning – no matter white, Black, Hispanic, local, refugee, struggling, bright, or other.

We have a dream for an education system peopled by those who want to improve, who want to esteem excellence, who want to know the progress, and who know what to do when they do not know what to do. We have a dream for teachers and school leaders to be respected and valued, both in societal and financial terms. Who see themselves and whom we also see as the major change agents in student learning – thus demonstrating the law and the faith we have in schools to truly make a difference in our society. Surely, the greatest civilizing institution in our democracy is our schools.

We have a dream for students to be taught to have the confidence to want to take on challenges; to say, “this is hard and I want to try,” not “this is hard, I cannot do it.” We have a dream to get rid of those things that do not work; to get rid of so much stuff we try to implant in kids’ heads, to find the balance of surface, to deep, to transfer in a fun and joyful way that means our students want to belong, come back, and learn more.

We want all students to learn precious and productive knowledge – to critique, reorganize, disrupt, and celebrate this knowledge; to learn what they would not learn if they did not come to school. The purpose of education should not be to help students reach their potential, as again, this lowers the aspirations for many and defeats the purpose of schooling. The prime purpose of education is to help students exceed what they think is their potential. To see in students something they may not see in themselves and to imbue them with our passion for learning.

We noted that there is excellence around us – have we the courage to dependably identify these schools, leaders, and teachers that are making major changes to the learning lives of their students; build and scale up coalitions of success around these educators and then support collaborative communities of such excellence within and between schools? There are many Visible Learning schools and others experiencing this success every day – and they are our living dreams.

Mind frames

The model starts less with what we do but how we think about what we do. It is more about how educators, families, and students think about purpose, priorities, context, safe, high trust, inviting places, curriculum, what it means to be successful, and much more. We spend far too much time on what we do and far too little on the thinking that precedes, decides, and evaluates what we do.

The mind frames speak to the why of teaching and learning, the intentional alignment model to the how, and the decisions about curricula content to the what. But unfortunately, the mind frames are too rarely discussed, contested, or critiqued. This is especially important as the mind frames are the basis of decision-making, and these decisions go to the heart of the ethical nature of teaching. Every teacher I have met has strong theories about their teaching and impact, and the aim here is to acknowledge, identify, allow critique, and enhance these ways of thinking.

The mind frames act as filters when interpreting evidence and guiding actions. These include beliefs about roles, responsibilities, efficacy, motivations, models of learning and students’ capabilities, and assessment beliefs. These core beliefs relate to developing learning skills, well-being, social change, and mastery; to personally being committed and responsible for the success of students to value, enjoy, and take an interest in subject matter, having high expectations about one’s impact, for earning trust and showing students they care about them, and for the effectiveness of their teaching. These beliefs are founded on confidence to manage classrooms, taking responsibility for developing students’ intrinsic motivation and self-efficacy, believing all students can be taught to learn and be motivated to learn, and seeing assessment as feedback to them about their impact, how to improve, and a positive part of teaching and learning. In addition, there is a strong focus on decision-making, inclusion, social and emotional competence (including developing self-efficacy, self-regulation, self-concept, and self-awareness), and communication, collegiality, and competence.

Hattie and Zierer (2018) introduced the notion of mind frames by referring to Sinek (2009) and the Golden Circle, where he outlined the what, how, and why. He argued that average leaders start and finish their thinking with the what and often, thus, lose sight of their actual goal or the why. For successful leaders, the main question is why something should be done. This leads them to question how to do something and, finally, what to do. In schools, the why relates to students, teachers, families, and school leaders’ vision, passion, and belief that they can and do enhance the learning lives of their students/themselves. This is the core of why educators do what they do. It is not getting through the curriculum, passing the tests, filling the time with engaging real-world activities, completing worksheets, or filling the student’s empty bank of knowledge. Instead, it is turning students into investing in knowing-that, knowing-how, and knowing-with; experiencing the joy of the aha learning moment; and learning to become their own teachers. Our thinking is based on the passion and enthusiasm we have to impact students’ lives positively (indeed, having an impact is among the major reasons most become educators). This passion and enthusiasm manifest in the following ten mind frames for teachers and school leaders, students, families, and collective equity, belonging, and identities (Table 3.2).

Table 3.2 The mind frames for teachers, leaders, students, families, and culture and climate.

Teachers and school leaders
Equity, belonging, and identities

1. I am an evaluator of my impact on student learning.
1. We acknowledge and honor the diverse cultures of our students.

2. I see assessment as informing my impact and next steps.
2. We address and interrupt the systemic inequities that have impacted our students and their families.

3. I collaborate with my peers and my students about my conceptions of progress and my impact.
3. We evaluate our impact over our intent when disrupting implicit biases, and seek to eliminate biases from our practice.

4. I am a change agent and believe all students can improve.
4. We incorporate evidence-based practices that culturally fortify and mitigate inequities in the learning community.

5. I strive for challenge and not for the goal of doing my best.
5. We are collectively responsible for the well-being of students and removing barriers to students' learning, including barriers related to identities.

6. I give and help students understand feedback, and I interpret and act on feedback to me.
6. We foster a culture that builds and honors students' identities, particularly as a learner.

7. I engage as much in dialogue as monologue.
7. We seek opportunities to support the identities of all our students.

8. I explicitly inform students what successful impact looks like from the outset.
8. We collaborate with our colleagues and students to ensure that our school is a highly welcoming and a hate-free zone.

9. I build relationships and trust so that learning can occur in a place where it is safe to make mistakes and learn from others.
9. We aim to make all students feel like they belong and feel invited into all classes.

10. I focus on the learning and the language of learning.
10. We ensure that our school is a highly welcoming and safe zone.

Students
Parents, caregivers, and families

1. I am confident that I can learn.
1. I have appropriately high expectations.

2. I set, implement, and monitor an appropriate mix of achieving and deep learning goals.
2. I make reasonable demands and am highly responsive to my child.

3. I strive to improve and enjoy my learning.
3. I am not alone.

4. I strive to master and acquire surface and deep learning.
4. I develop my child's skill, will, and sense of thrill.

5. I work to contribute to a positive learning culture.
5. I love learning.

6. I have multiple learning strategies and know when best to use them.
6. I know the power of feedback and success thrives on errors.

7. I have the confidence and skills to learn from and contribute to group learning.
7. I am a parent not a teacher.

8. I can hear, understand, and action feedback.
8. I expose my child to language, language, language.

9. I can evaluate my learning.
9. I appreciate that my child is not perfect, nor am I.

10. I am my own teacher.
10. I am an evaluator of my impact.

It is more about how we think about what we do that matters most, how we understand their impact, and how our search for feedback improves the positive impact on students. In this way, mind frames can become more visible. Thus, expert educators, students, and families have answers to the question of what they are doing and the questions of how and why they are doing what they are doing. Successful teachers, for example, are passionate not just about the subject they teach but also about the impact of their teaching on students – passion and purpose matter.

These four sources (teachers, students, families, and leaders) all work in a social context, and this context can be an enabler or barrier to successful learning. For example, the climate of a school can be inviting, a place of vicious inequalities, a safe haven, or a place where students are compelled to be. Educators create this climate; thus, we developed ten mind frames about equalities, belonging, and identity (see Chapter 7).

Intentional alignment

The enactment part of the model relates to the intentional alignment of seven major decisions relating to the proportion of knowing-that, knowing-how, and knowing-with (surface, deep, transfer) in the lessons. It is bringing together in alignment the notions of success, the complexity of what is to be encountered, the high trust and safety of the learning environment to take on the challenges, the methods of teaching, the optimal strategies of learning, how these all align with the activities, and the assessment methods. If students do not experience this alignment, they can readily pick and choose which parts they wish to invest in, can be confused with the intent of the teaching and the methods, and prioritize their working memory and motivation on parts not aligned with the teacher’s intent.

Biggs et al. (2023) noted that “traditionally, teachers planned their teaching by asking such questions as What topics or content do I teach? What teaching methods do I use? How do I assess to see if the students have learned what I have taught them?” They saw this as a process of transmitting content to the students. Hence, the methods tend to be expository, such as didactic talking and demonstrations followed by asking students to complete some activities. The in-class questioning checks on how well the teacher’s talk was received, and the assessments asked students to report what they had heard. Instead, intentional alignment is based on what learning outcomes I intend my students to achieve after my teaching and to what standard. What proportion of knowing-that, knowing-how, and knowing-with? How do I design and implement learning activities to help them achieve those outcomes? Finally, how do I assess them to see how well they have achieved those outcomes? In this sense, it has more in common with backward design by asking not what the teacher will teach but what the outcome of that teaching is intended to be. The most critical part of this model is aligning the various parts relative to the complexity level of the ideas to be learned in the success criteria. Ensuring, for example, that if the focus of success is knowing-that (learning ideas), as opposed to knowing-how (learning relations among ideas) or knowing-with (transferring ideas to a new situation), then the teaching methods, teaching of learning strategies, choice of activities and assessment should be so aligned.

There are seven major parts of the intentional alignment model:

Determining the learning intentions and success criteria, typically based on a curriculum, an understanding of progressions (where the students have been progressing, where they are now, and where they need to be), and the motivations and dispositions students bring to the class.

Analyzing the cognitive task of the knowing-that, knowing-how, and knowing-with foci of the lessons (i.e., content, deeper understanding, and transfer skills and knowing). Understanding the cognitive complexity involved in the learning, ensuring students have the strategies for learning, understanding their progress, and a concept of what success looks like.

Setting the climate and culture of the class to ensure that students and teachers see errors or misconceptions as opportunities to learn. There needs to be safety to work, discover, and explore with peers, and an inviting climate for all, acknowledging the diversity of what each and all students bring to the class environment.

Aligning teaching methods with the cognitive complexity of the various components of the tasks, the required confidence to take on the challenges of this complexity, and the efficient and effective improvement of students to move toward the success criteria.

Ensuring students have appropriate and effective learning strategies to engage in the complexity of the tasks.

Choosing activities that align with the content’s complexity levels, the deeper conceptual and relational thinking.

Applying assessment methods that focus on the content (knowing-that), the relational (knowing-how), and transfer (knowing-with) aspects of the success criteria that inform the teachers and students of their progress, success, and gaps to be then addressed.

Throughout each step, there needs to be an evaluation strategy for monitoring the implementation and impact on students during and at the end of the lesson(s) to understand and realize this impact (effectiveness and efficiency) of teaching methods to move students toward the success criteria.

The alignment aspect “refers to what the teacher does, which is to set up a learning environment that supports the learning activities appropriate to achieving the desired learning outcomes” (Biggs, 2003, p. 1). Steps 2–7 need to be aligned with the experiences and understandings of the student so that they are clear and have the confidence to be successful in capitalizing on the teaching, have the optimal strategies of learning, are motivated by the teacher to invest in the activities, and see the assessments as informing them of their progress and their realization of attaining the success criteria.

To achieve mastery in any field, a person needs to master not only facts, content, and current theories/explanations, but also the ways of thinking, the use of judgments, and the act of building relationships among these facts. Others have made similar claims about declarative (knowledge about facts and things) and procedural knowing (how to perform certain cognitive activities, such as reasoning, decision-making, and problem-solving). For example, Ryle (1945) distinguished between knowing-that, or the knowledge that something is the case, and knowing-how, or the knowledge you have when you know how to do something, such as how to ride a bike, bake a cake, or make an evaluation interpretation. When designing lessons, they most often require knowing-that/declarative knowledge and knowing-how/procedural knowledge. Broudy (2017) added that transfer was more akin to knowing-with, as students come with the knowledge of new problems or situations.

Perhaps the most well-known model in education is Bloom’s taxonomy. The first version had six levels: know, comprehend, apply, analyze, synthesize, and evaluate (Bloom et al., 1956). However, there is little evidence supporting their claims that these increase cognitive complexity (Hattie & Brown, 2004; Hattie & Purdie, 1998), as the levels mix ways of thinking with the processing required in completing tasks or answering questions, and there is surprisingly little evidence of its effect on improving lessons or impact on students. A new edition of Bloom’s taxonomy (Anderson & Krathwohl, 2001) recognized the limitations of the original version, changed the order of the cognitive processes (remember, understand, apply, analyze, evaluate, and create), and introduced a new dimension of knowledge types (i.e., factual, conceptual, procedural, and metacognitive). These latter four knowledge types form a hierarchy of increasing cognitive complexity.

Similarly, Webb’s (1997) Depth of Knowledge model relates to the depth of content understanding, the scope of a learning activity, and the skills required to complete a task.

There are four levels:

Recall relates to reproducing a fact, reproducing a principle, or performing a routine procedure.

Skills or concepts relate to using information, selecting appropriate procedures for a task, or organizing and displaying interpreted information.

Strategic thinking involves reasoning or developing plans to approach a problem, employing some decision-making and justification, and solving abstract, complex, or nonroutine problems.

Extended thinking involves performing investigations or applying concepts and skills to the real world that require time to research, problem-solving, and processing multiple conditions of the problem or task.

Another cognitive complexity taxonomy is the Structure of Observed Learning Outcomes (SOLO) Taxonomy (Biggs & Collis, 1982). This taxonomy describes four levels of complexity: unistructural, multistructural, relational, and extended abstract. The first two are considered knowing-that or surface, and the latter, knowing-how or deep processing. The knowing-that levels demonstrate an understanding of ideas or facts. Unistructural responses and questions require the knowledge or use of only one piece of given information, fact, perspective, or idea, obtained directly from the problem. With an increase in quantity, multistructural responses or items require knowledge or use of more than one piece of given information, facts, perspectives, or ideas, each used separately or in two or more distinct steps, with no integration of the ideas.

In contrast, the two deep processes (knowing-how) constitute a change of quality of thinking that is cognitively more challenging than surface ideas. In addition, relational responses or questions require the integration or development of an organizing pattern based on the multistructural ideas. Finally, the highest level of the SOLO taxonomy, extended abstract (more akin to knowing-with), requires going beyond the given information, knowledge, or ideas to deduce a more general rule or proof that applies to all cases. In this latter case, it is necessary to think beyond the given and bring in related, prior knowledge, ideas, or information to create an answer, prediction, interpretation, judgment, or hypothesis that extends the given to a broader range of situations.

Dinsmore and Alexander (2012), reviewing the research on surface and deep processing, note the implicit assumption that this processing is invariant over who is doing the processing, where the processing is taking place (i.e., the context), when the processing occurs, and to what end they are employed (the target of learning). Hence, a key question is the relation between processing levels and learning outcomes for whom, at what point in development, in what situations, and for what end. Hence, the notion that surface is bad and deep is good is naive and misleading. They recommend close attention to the precision in deciding what is surface and deep, and that they can occur contiguously in some instances (cf., Dinsmore & Alexander, 2016). Hence the importance of decisions about the who, where, when, and what – which makes the notions of intentional alignment all the more important.

Clinton et al. (2021) combined the attributes across these three taxonomies to provide a deeper understanding of the complexity of thinking and knowing that can be planned as part of a lesson (Table 3.3). Knowing-that includes both learning factual knowing and conceptual knowledge and skills; knowing-how includes procedural, strategic, and relational, as well as transfer to similar or new contexts; and knowing-with is using the knowledge and understanding to transfer to near or far new contexts.

Table 3.3 Three levels of cognitive complexity and related activities

Name
Definition
Activities

Knowing-that (surface)
Factual
Recall and reproduction

Conceptual
Basic application

Knowing-how (deep)
Procedural and relational
Connecting, applying, and relating together surface knowledge from different areas (e.g., theory and context)

Knowing-with (transfer)
Transfer to similar or new contexts
Using knowledge gained in one context in another

Quality of implementation

Implementing these seven steps involves five steps (Table 3.4): discover, design, deliver, double-back, and double-up. In Building to Impact, we developed this 5D model by analyzing 50 implementation methodologies and processes (from business, computing, engineering, and medicine). We identified 23 common implementation processes across the 50 methodologies, drew on existing meta-analysis and systematic reviews on implementation, and combined this with our hands-on experience of supporting system, district, and school-level implementation initiatives across more than 50 countries (Hamilton et al., 2022). The methodology we developed is called 5D, and this is used to help frame knowing about the quality of implementation: discovering goals and what students walk into the class knowing, designing high-probability interventions, delivering the implementation of the interventions, doubling back and monitoring and evaluating your impact, and either doubling up to scale up the intervention or stop it (see Figure 1.3, and Table 1.1 earlier).

Table 3.4 Strategies for implementing the intentional alignment model

Teachers
Students
Families
School/leaders

D1: Discover- identifying goals that are worth pursuing above all else, building a theory of the present, and agreeing what success looks like.
Knowledge of their students and their prior learning Learning intentions/success criteria Levels of challenge Expectations and not labeling Multiple progressions
Bringing values, expectations, and learnings from their home, culture, and society Bringing skills, wills, and thrills for learning
Children are born into a world of expectations Parents as first learners
Press for both progress and achievement, well-being, and togetherness Know it is within, not between schools, that matters Create time, resources, expertise to collectively impact on students

D2: Design- systematically examining the different options in design space, selecting/designing a high-probability intervention, stress testing before launch. and developing your monitoring and evaluation plan. It's also about identifying things you will simultaneously stop to make.
Intentional alignment Backward design Lesson planning Surface, deep, transfer Cognitive task analysis Teaching of strategies Curriculum Methods Evaluation plan Technology
Expecting Understanding LI, and knowing SC Understanding feedback Unlearning, disequilibrium
Learn the language of learning Become good listeners Have high expectations Create a climate where failure is seen as positive
Language of learning Know the role of money Climate of invitation High expectations Collective efficacy

D3: Deliver- putting your agreed interventions into action and collecting monitoring and evaluation data.
Intent to teach Culture of class Evaluative thinking Interpreting evidence Differential teaching methods Giving and seeking feedback Listening Passion
Using multiple learning strategies Investing and driving Taking errors as opportunities Working with others

Shared narrative Implementation science

D4: Double-back- monitoring and evaluating your delivery chain and deciding where to go next.
Reteaching Know thy impact Work-critique others' impact Consolidating Progress to achievement Progressions
Taking feedback Using student assessment capabilities Assimilating and accommodating

Trust but verify Frequent evaluating and monitoring

D5: Double-up-either implementing and maintaining enhanced versions of an intervention in the same local setting and/or working to embed it across multiple schools (i.e., this is all about sustainability).
Investment in enhancing expertise Skill in de-implementation
Wanting to invest in more and deeper learning
Want to invest in more years of schooling
Building of the collective to upscale Skill in de-implementation

Evaluative thinking

Underlying the purpose, mind frames, intentional alignment, and implementation is a way of thinking by the teachers and leaders, and a way of thinking we wish to engender in our students. This is called evaluative thinking. Evaluative thinking invokes critical thinking and problem-solving, whereby the educator and student seek evaluative data and evidence to construct arguments that lead to decisions and value judgments (Clinton & Hattie, 2014). Evaluative thinking involves reasoning and critical thinking in valuing evidence, leading to where-to-next recommendations; involves addressing the fidelity of implementation of intent to teach, continually checking for unintended consequences; allows for adaptations to maximize the value of the outcomes; involves investigating potential biases and confounds that may lead to false conclusions; and involves skills and confidence in working to understand others’ points of view; and the purpose of evaluative thinking is to maximize the educator impact on students’ love of knowing and understanding, and wish to reinvest in learning the content. Such evaluative thinking is the essence of expertise and the foundation of the profession (Rickards & Hattie, 2021).

We have used the notions of evaluative thinking to explicate better the nature of this thinking underpinning the mind frames, the implementation process, and the why of what we do, value, and assess. We started our work investigating National Board Certified teachers (NBCs) in the US (Hattie & Clinton, 2008), heavily drawing on Berliner’s (2008) study of experts vs. novices; drawing on similar literatures of expertise in other disciplines (e.g., Dreyfus & Dreyfus, 2005; Kahneman, 2011); and developing a model of evaluative thinking.

Our work comparing NBCs who passed and did not pass a series of assessments (controlling for experience and school context) found that NBCs were higher on measures of their use of knowledge, the depth of their representations of knowledge, their expressed passion, their problem-solving skills, their degree of challenge in their lessons, their moving beyond the surface to deeper representations of the content, and their skillfulness in monitoring and providing feedback to their students (Smith et al., 2008). Using student work samples, 74% of those obtained from the students of NBCs (compared to 29% from non-NBCs) demonstrated higher understanding through more relational and more abstract student work. We concluded that the National Board process was as follows:

identifying and certifying teachers producing students who differ in profound and important ways from those taught by less proficient teachers. These students appear to exhibit an understanding of concepts targeted in instruction that is more integrated, more coherent, and at a higher level of abstraction than understanding achieved by other students.

(p. 113)

Berliner (2008) has undertaken the most extensive study of expert teachers and derived the following propositions:

Expert teachers excel mainly in their own domain and in particular contexts.

Expert teachers develop automaticity for the repetitive operations that are needed to accomplish their goals.

Expert teachers are more opportunistic and flexible in their teaching than are novices.

Expert teachers are more sensitive to the task demands and social situations surrounding them when solving problems.

Expert teachers represent problems in qualitatively different ways than do novices.

Expert teachers have faster and more accurate pattern recognition capabilities.

Expert teachers perceive more meaningful patterns in the domain in which they are experienced.

Expert teachers may begin to solve problems slower, but they bring richer and more personal sources of information to bear on the problems they are trying to solve.

All these attributes of expertise relate to how educators think, and there is a strong theme about making evaluative decisions. Evaluative thinking is a particular kind of critical thinking and problem-solving whereby one marshals evaluative data and evidence to construct arguments that allow one to arrive at contextualized value judgments in a transparent fashion (Vo, 2013). We argued there were at least five major evaluative thinking skills (Rickards et al., 2021):

Evaluative thinking involves invoking reasoning and critical thinking in valuing evidence, leading to where-to-next recommendations.

Evaluative thinking involves addressing the fidelity of implementation, continually checking for unintended consequences, and allowing for adaptations to maximize the value of the outcomes.

The focus of evaluative thinking is specific to the major decisions each profession aims to make: for teachers, it is maximizing impact on student learning.

Evaluative thinking involves investigating potential biases and confounding factors that may lead to false conclusions.

Evaluative thinking relates to understanding others’ points of view, leading to judgments of value or worth.

Thus, evaluative thinking is a way of reasoning and critical thinking about the impact of what we plan and do to improve student learning. Expert teachers have higher levels of evaluative thinking, have greater pattern recognition, are more adept at checking biases and constraints, and are more likely to seek alternative actions when the first does not work sufficiently. However, evaluative thinkers are slower to come to problem representations and conclusions, try to see the world through others’ eyes, check back with the problem statement more regularly, and are proficient in evaluating their impact on students and learning from their students’ impact on them.

The major message for maximizing impact on student learning relates to how teachers, students, families, and leaders think. For educators, it is how they think when preparing lessons; how they think when discovering the skill, will, and thrill that each student brings to the lesson; how they think when making moves to make classrooms inviting and high-trust places to want to come to learn; how they think in the moment-by-moment decisions when implementing these lessons; how they think when interpreting from their noticing, from students engagement in the lesson, and from the artifacts of the student’s work; and how they know when and how to listen to how students are thinking, how they know the learning strategies they are using, and how students cope with errors and not-knowing. Most critically, it is how educators evaluate their impact to ensure their next learning steps continue a positive learning trajectory. It is educators thinking about the choice of interventions, devising and explaining the learning intentions and success criteria, knowing when a student is successful in attaining those intentions or not, having sufficient knowledge of the students understanding that they bring to the task, knowing when to get out of the way when student learning is progressing toward the success criteria, and knowing sufficiently about the content to provide meaningful and challenging experiences in various progressive pathways to succeed in learning.

Engaging in this thinking requires more than knowledge of content, more than knowing many teaching methods, more than social skills to develop good relations (not just between teacher and student but also between students), but a passion to meaningfully improve the learning lives of students. Often, this is exhibited as a passion, and visitors to a class and students in a class can observe and feel the joy, the thrill, and the infectious nature of the teacher when they cause students to experience learning. This is the same passion we wish students to have for learning (as they often exhibit for their social activities, video games, and sports). This passion requires a love of the content; requires an ethical, caring stance to wish to imbue others with a liking or even love of the discipline being taught; and requires having and communicating high expectations that all students can exceed what they think is their potential.

Reflections

Develop agreement on the major purposes of teachers, students, parents, and leaders in your school or system.

For each of the ten mind frames, discuss exemplars of practice and questions to detect evidence of these ways of thinking.

Provide an example of conscious planning for units of study to move from surface, to deep, to transfer. Can we see those transitions in individual students’ learning, helping us know when to consolidate or extend?

With colleagues, create a lesson plan based on Figure 3.2, clearly articulating the knowing-that and knowing-how/with aspects of the lesson. Start with the final assessments that will inform you of where, with whom, and about what was successful in your teaching.

Figure 3.2 Intentional alignment model highlighting the proportion of knowing-that, knowing-how, and knowing-with – in planning, delivery, and evaluation.

Ask a colleague to confidentially interview your students about what it means to be a learner in the class, who is the best learner and why, and what do you do when you make a mistake in class. Elaborate on how you and your students view the climate of your class for taking on challenging learning.

The beginnings of cognitive task analysis ask for clear statements about the required knowledge and skills of the lesson (knowing-that), the core relations among ideas (knowing-how), and the desired transfer of knowledge and deeper learning to new situations and tasks. Take a topic and work through a task yourself to help ascertain these key skills; then devise at least three activities (one for each level) and teach them to your students. Evaluate the consequential impact on their learning and how you would modify the lesson.

What strategies or techniques do we use to check the impact of our teaching? For example, could we find ways of asking students which aspects of our teaching helped them learn a particular piece of knowledge (that or how to), and what else we could have done?

How can we continually find out what children already know from yesterday’s lesson, from last week, so that we can teach accordingly? Are we using quick prior knowledge discussion questions, hinge questions, and on-the-move feedback strategies to detect prior knowledge and understanding?

Evaluative thinking teachers are nosy, continuously inquiring what worked and what did not, fussing about the fidelity of their teaching to see if it has the right impact, checking their biases about what is happening in the learning of the students, and trying to stand in their student’s shoes to understand their enabler, barriers, and experiences of learning. To what degree does this describe you?

Study the expert teachers findings by Berliner on pp. 55–56 and discuss their meanings, implications, and possible strategies to enable us to share and see expert teachers in action.

CHAPTER 4 Implementing the intentional alignment VL model

DOI: 10.4324/9781003380542-4

Starting with the five premises outlined in Chapter 2, this chapter introduces the major arguments for implementing the Visible Learning model of intentional alignment from the perspective of the teachers, students, leaders, and parents. It outlines the major issues for teachers, students, leaders, and families for the first three Ds from the 5D model, and then overviews the important considerations for D4 and D5 – scaling back and scaling up.

Teachers

D1: Discover and diagnose

The discovery phase primarily establishes the optimal education challenge for each child, the class, the school, and the system. Before rushing to the box of tools or the education solutions, it is ensuring that there are great discovery skills (including listening to the students, considering test scores, and questioning); there is a deep knowledge of how students learn and develop, and how emotions are involved in learning; there are multiple ways to progress through curricula; and most of all, there is excellent critical analysis of what the students know, understand, and care about. Know thy student.

Knowledge of their students and their prior learning

Learning intentions/success criteria

Levels of challenge

Expectations and not labeling

Multiple progressions

David Ausubel (1968) claimed that the most important single factor influencing learning is what the learner already knows. Ascertain this and teach them accordingly. He showed how students absorb new information by tying it to current understanding and prior knowledge. These schemata (or coat hangers) are the frameworks in which students confront new information and ideas. This is like Piaget’s notion of disequilibrium or when we experience a state of imbalance due to encountering information that requires us to develop a new schema or modify our current understanding. Ausubel developed the notion of the advance organizer, which is the precursor to learning intentions and success criteria, and they serve as conceptual coat hangers for learning new ideas. This means teachers need to know what each student brings to the class in terms of their skills (prior knowledge, learning strategies, working memory, executive processing, culture), will (dispositions, such as willingness to be challenged, confidence in confusion, skills working with others), and thrill (motivation to learn what is being taught). Then the teacher must adapt and respond to this background information in devising their lessons, and in any class, this background readiness can be most variable. The aim for the teachers is less about relating ideas to the real world, the future, or careers, but more about relating the ideas in the lesson to the current knowledge and interests of students. This does not necessarily mean pandering to feed their interests, as the aim may be to entice students to now have new interests. Appealing to interest, however, can have negative effects, as students tend to become overconfident and allocate less time to consolidating new knowledge and understanding, so getting the balance right (boredom is not an option) is critical. (Senko et al., 2021). However, when teachers clearly understand their students’ prior knowledge and their strengths and gaps, this has a powerful effect on subsequent achievement.

A central claim in the intent to teach notion presupposes that teachers make predictions or estimates of the future about what the teacher expects the actions that leads to the success criteria for all the students. Thus, learning intentions are forward-looking motives; they describe future states of affairs (Anscombe, 2000). A learning intention is what pupils should know, understand, or be able to do by the end of the lesson or series of lessons, and they are the focus of what we want students to learn (not merely do). Success criteria should describe how the teacher and students will know they have successfully achieved the learning intention.

There must be appropriately challenging expectations embedded in the success criteria and resistance to labeling students, which often assumes and leads to lower expectations. That is, challenging relative to what each student cannot do (yet) and focusing on enabling students to engage in the challenge. The Goldilocks principle of challenge is optimal: not too hard, not too easy, not too boring. Unfortunately, some students are reluctant to take on challenges (fearing failure, becoming an embarrassment in front of peers and the teacher, or not having the skills to address the challenge). Hence, it is important to specifically know, plan for, and develop confidence to take on appropriate challenges (self-efficacy) and have high levels of trust that failure is the best friend of learning in this class.

Note, the term “success criteria” is plural, as there rarely is but one aim in a lesson. Too often, when we have interviewed students about their understanding of success, they comment in terms of learning content (the knowing-that, despite what the teachers have proclaimed, written on whiteboards, etc.). This is because the tasks or assignments too often privilege knowing the content, the feedback is primarily content-related, and the language and questions of the teacher are dominated by content. The aim is not to bipolarize success criteria as knowing-that or knowing-how but to have the right proportions of knowing-that and knowing-how.

Moving from the new success criteria to what the students know, can, and care about is rarely a linear or one-way street. The fundamental notion of differentiation allows for different times and multiple pathways to succeed. Tomlinson (2014) argued that differentiated instruction is student-aware teaching, whereby teachers vigilantly monitor student proximity to the lesson goals throughout a learning cycle such that the teacher adapts teaching plans to allow multiple ways and times for students to reach success criteria and enhance student efficacy and ownership of learning. This involves skills to anticipate roadblocks, cul-de-sacs, and students going down the pathways in different directions. Hence, it is important to realize there are multiple progressions to success. All these require a deep understanding of the curricula; a knowledge of how to advance learning in multiple ways, from knowing-that, to knowing-how, to knowing-with; and a knowledge of how to detect where students are on their roads to success and where optimally to focus next.

D2: Design

So often we speak about teaching, assessments, learning, curricula, feedback as if they are separate activities and can be modified independently. Students experience lessons as a point-in-time event, often involving instruction (teacher talking), activity (student doing), reporting, and the wrap-up phase. If the work is completed on time, if the students are seen to be doing the work, and if the teacher is satisfied that the topic has been covered, then there is a repeat experience in the next lesson(s). There is barely a modicum of research on the impact of the plethora of lesson plans, but there are resources aplenty on the internet, and this seems a major evidence gap that desperately needs addressing. Yes, it will be difficult, as context matters, prior learning matters, the relation among the resources matters; but wise heads need to prevail to make an evidence base of evaluations of resources a primary focus to assist teachers to optimize the impact of the learning (see www.edreports.org/ and https://beyou.edu.au/resources/programs-directory as two exemplar starting points).

Intentional alignment

Backward design

Lesson planning

Surface, deep, transfer

Cognitive task analysis

Teaching of strategies

Curriculum

Methods

Evaluation plan

Technology

The major finding is to ensure intentional alignment of all the lesson(s) parts. The starting point is identifying the content and understanding. This derives typically from the curricula – although it will be noted that there can be very low alignment of curricula from different jurisdictions and even less in the many scope and sequence or learning progression claims (Porter et al., 2011). There are at least three decisions to be made – about the necessary content that will then be used in the deeper understanding when relating ideas, and some attention to how this transfers to near and far contexts. Using the backward by design principles, the key idea is to consider up front that which we wish students to truly understand what they are asked to learn (not take in and recall, not the tasks) and what the students should be able to know, do, care about, and understand at the end of a series of lessons (Wiggins & McTighe, 2005).

The backward design approach has teachers considering the course’s learning goals first. These learning goals embody the knowledge and skills instructors want their students to have learned when they finish the series of lessons. Once the success criteria have been established, the second stage involves considering the assessment, and the intent is to be clear to the teacher (and students) what is valued in terms of the content, the deeper relations between ideas and conceptual understandings, and the desired skills to transfer to other tasks and contexts. The third stage is designing the teaching methods, activities, and resources to move students from where they start to the desired and transparent success criteria. This is the essence of teaching with intent.

Thus, there are at least two major considerations in lesson design. There is a need to intentionally align the activities, assessments, feedback, and teaching based on the knowing-that, knowing-how, and knowing-with (surface, deep, and transfer) learning with the success criteria. When teachers have developed the learning sequence in such a way that students first develop relevant and needed surface knowledge (the content or ideas) and then go on to develop deeper understandings (relate and extend these ideas), this combination of deeper learning following surface learning can significantly impact on improving student learning outcomes. For example, problem-based learning is ineffective when students don’t have enough relevant and necessary prior knowledge to be able to make their own connections and solve the problems (see Chapter 12). However, when students have appropriate surface knowledge, problem-based learning has been shown to be very effective in consolidating and developing deeper understandings.

Note that care is needed not to be too literal in the sequencing from surface to deep and transfer. As Kapur (2016) has repeatedly shown, starting with problem-solving tasks can expose the lack of knowledge and the needed content, which becomes the teaching focus. If only growth in learning were always linear, but it is more staccato, and we need to generate and treasure the aha moments in teaching and learning (Berkely & Hattie, in review).

The model involves teaching not only the precious knowledge, the deeper conceptual ideas, the near and far transfer to other tasks, but also the optimal learning strategies, being aware of the complexity of the cognitive demands and creating an environment of high trust for errors, not knowing, and misunderstandings to be seen by students as welcome and legitimate parts of the lesson. The art of teaching is knowing what levels of challenge each student is prepared to make, creating teaching acts to deal with anxiety and promote confidence to take on more challenging tasks, reducing the inequities in the class by using student peers as part of the teaching (per the jigsaw method), and having the evaluative thinking skills to make decisions about optimal teaching methods and knowing when best to implement them.

The second major consideration is the intentional alignment of goals, success, feedback, learning strategies, and assessments. Lesson design is premised on a deep understanding of the following:

The curriculum, learning progressions, and knowledge of what success looks like (in the time available) leading to the learning intentions.

The cognitive task analysis of the knowing-that, knowing-how, and knowing-with foci of the LI and SC.

The knowledge of where students are and where they have come from, and the making of informed predictions of their learning trajectories.

The provision of teaching interventions and feedback appropriate to the various dimensions of the cognitive complexity to reduce the learning gap from where the students commence to the desired success criteria.

An evaluation strategy for monitoring the implementation and impact on students during and at the end of the lesson(s) to understand and realize the impact (effectiveness and efficiency) of teaching methods to move students toward the success criteria. Feedback to the teacher rules.

These five processes are fundamental to all teaching strategies: the intentions of the lesson(s), the criteria of success of the lesson(s), and the assessment and feedback to the teacher and to the student during and at the end of the lesson(s). A major need is for the LI, SC, and implementation evaluation to be intentionally aligned. Biggs (1999) argued that this meant three core notions underpin intentional alignment: identifying the desired outcomes, deliberately aligning the planned learning activities and the desired success criteria, and determining the assessment tasks will tell us if the actual outcomes match those that are intended or desired.

Being intentional leads to the deliberate choice of learning intentions, success criteria, evaluative methods, and evaluative feedback about the effectiveness and efficiency of teaching methods and their implementation. It entails being intentional about the learning process, which (in Chapter 12) can be considered to include developing knowledge, deeper relational and conceptual understanding, and transfer skills. It means being intentional about the optimal learning strategies to be taught and used in the lessons, being intentional about the choice of teaching methods, and acknowledging that there are very few optimal teaching methods at all three parts of the learning cycle. This requires teachers to have skills in differentiating teaching methods depending on where their student(s) are in the learning cycle. Being intentional means stopping using the teaching method when it is not having the intended impact; not making students do activities for the sake of doing them; and seeing the impact of teaching through students being taught to think aloud and knowing and evaluating their progress toward the success criteria.

There is also much emphasis on teachers understanding their impact of their teaching but also listening to students as to whether they share these understandings. Students are not passive recipients of content, tasks, and tests; they are core players in the learning process. The model includes informing the students of the learning intentions and criteria of success of the learning task and checking whether they have similar conceptions to the teacher. For example, do students know what it means to succeed at the assigned tasks? Students who can articulate or are taught these success criteria are more likely to be strategic in their choice of learning strategies, more likely to enjoy the thrill of success in learning, and more likely to reinvest in attaining the lesson goals. Further, there can be so much unintended learning and success despite the avowed and agreed success criteria; and being aware, welcoming, and promoting such positive bonuses are part of evaluative thinking by the teacher and student.

The overall effect across the 21 meta-analyses from success criteria is 0.67, with the greatest effects relating to providing students with success criteria, having intentions to implement goals, mastery learning, worked examples, setting standards for self-judgments, and setting the appropriate difficulty of goals. All these learning strategies allow students to see the whole or the gestalt of what is targeted to learn before starting the series of lessons. It thus provides a coat hanger on which surface-level knowledge can be organized. When a teacher provides students with a concept map, for example, the effect on student learning is very low; but in contrast, when teachers work together with students to develop a concept map, the effect is much higher (although it is expected that the teacher has prethoughts about what the concept map should look like). It is the working with students to develop the main ideas and to show the relations among these ideas to allow students to see higher-order notions that influences learning. Thus, when students begin learning the ideas, they can start to know how they relate to each other, how they are meant to form higher-order notions, and how they can begin to have some control or self-regulation on the relation among the ideas.

D3: Deliver

The key elements of the intent to teach implementation model are implementing the parts from the diagnosis stage (excellent diagnosis, clarity about the nature of success from the lessons of knowing-that, knowing-how, and knowing-with), using backward design, and preparing to attend to the learning strategies (knowing which strategies students are using). During the implementation, teachers need to evaluate each students’ progress to success and then implement optimal teaching methods to maximize all that is occurring such that all students know about, have the strategies to succeed, and are engaged in and feel invited to invest in their learning to the goals of the lesson. This needs to occur in a climate of safety and fairness, and with high levels of teacher- and student-to-student relationships so that errors, mistakes, and misunderstandings can be raised and dealt with. The core of these relationships is showing students you can and have listened to them, giving and seeking feedback, and inviting students to share your passion for learning the content, learning the relationship among ideas, and transferring the learning to new situations.

Intent to teach

Culture of class

Evaluative thinking

Interpreting evidence

Differential teaching methods

Giving and seeking feedback

Listening

Passion

Intent to teach requires high levels of intentional alignment of the curriculum content, learning intentions, success criteria, assessment, and feedback, and a deep understanding of the nature of the cogni-tive engagement of the lessons. When this is misaligned, students tend to believe learning is about knowing lots and assessment is about recall. This leads to a major mismatch among the lessons’ surface and deeper foci and outcomes. The argument is not that surface or deep is more critical than the other, but the right proportion or balance of knowing-that and knowing-how across the lessons should be ensured. Certainly, it is unreasonable to expect students to relate, extend, develop deeper conceptual understanding, and transfer unless they have ideas to relate, extend, solve problems, and so on. Identifying the thinking, learning strategies, and ways of knowing are key aspects of planning and teaching strategies. At all times, teachers need the evaluative mind frame when making choices about teaching methods, monitoring the fidelity of these methods on the student’s progression to success, and seeking feedback from students about the efficiency and effectiveness of the teaching choices.

In many cases, it seems surprising that there are so few teaching methods that capture all three levels of knowing (that, how, with) into one method, and more development of methods (like jigsaw, paideia) may be most beneficial (see Chapter 12). The overemphasis on content and coverage is the death of depth and enjoyment in learning. “The greatest enemy of understanding is coverage – I can’t repeat that often enough” (Gardner, 1993, p. 24). Near the start of a series of lessons, there may be more attention to content, but over the lesson series, there needs to be a balance of surface and depth. Some countries have minimal curricula specifications (New Zealand had 69 pages for all years, all subjects), allowing for more optimal proportions of surface and depth. Other countries are the opposite (the Australian curriculum is over 2,500 pages), and when high-stakes assessment is implemented, then depth is close to dead, and the experiences for too many students are deadly.

During the implementation phase, there need to be much attention to learning strategies. These strategies include relating and learning with and from others; hence, the value of developing collective student efficacy. To acquire surface learning, it is worth knowing how to summarize, outline, and relate the learning to prior achievement; then consolidating this learning by engaging in deliberate practice, rehearsing over time, and learning how to seek and receive feedback to modify this effort. To acquire deep understanding requires the strategies of planning and evaluation and learning to monitor the use of one’s learning strategies; and then consolidating deep understanding calls on the strategy of self-talk, self-evaluation, and self-questioning, and help from peers. Such consolidation requires the learner to think aloud, learn the language of thinking, know how to seek help, self-question, and work through the consequences of the next steps in learning. To transfer learning to new situations involves knowing how to detect similarities and differences between the old and the new problem or situations. This should not imply that learning always follows or even best follows a linear model. *Sinha and Kapur (2021) have argued that delving into problem-solving to identify the misunderstood or missing content may be a valuable strategy.

Thus, intent to teach means we should care less about how we teach and care more about the impact of our teaching, and we should adopt the Singapore mantra of “teach less, learn more.” This is not eclecticism but a deliberate, purposeful, and considered decision to use a method with a clear eye to what students bring to the class, use the success criteria, and intentionally align the teaching methods to the assessments, success criteria, and learning processes.

Intent to teach is not dependent on the medium of instruction – in-class, online, distance, or via technology. Technology has become another medium of instruction, often supporting the teaching methods already in place (which is why the impact has not been so great; see Chapter 14). The power of technology, well seen during the COVID pandemic times, is the communicative power of social media to bring learners together, allow them to hear their and other students’ learning processing, engage in critique with peers, deliberate consolidate learning after first exposure by teachers, permit students opportunities to learn to think evaluatively and become their own teachers, and seek, hear, and activate feedback. Similarly, the high use of teachers of social media can be capitalized to privilege expertise, to spread professional learning, to create communities, and perhaps also to enhance more effective diagnosis on which to then based lesson design.

Students

D1: Discover and diagnose

When babies enter this world, they enter a world of expectations. When children cross the school gate, they bring their expectations influenced by their families and confront even more adult expectations. Their expectations from the home, preschool, playground, and community can be positive, nurturing, aspiring (or not). In their early years, children build theories of their world, develop a language about their world, and learn with others (or not). These expectations can be moderated, for good or bad, by their race, socioeconomic status, gender, identity, achievements, learning, and so much more. These expectations influence the child’s early conceptions of their skills, dispositions (will), and decisions to invest in learning tasks (their motivations or thrill).

They bring values, expectation, and learnings from their home, culture, and society

They bring skills, wills, and thrills for learning

No wonder schools need to be places of hope, high expectations, and opportunities for learning for all students. No matter where the students commence on the distribution of achievement, from whatever the home and culture, our role is to raise the expectations and success for all. All students are learners, are teachable, and can be taught to love learning. All students come to school with high levels of motivational resources, and our role is to turn these motivations toward school learning. For many students, one of our major aims of schooling should be to mess up the predictability of who succeeds and fails – history, postcodes, and prior ability should not be determiners of success. Indeed, the essence of equity involves removing the predictability of success or failures that currently correlates with any social, cultural, or desired education factor (National Equity Project, 2018).

The fundamental attributes of skill that each student brings to class include prior achievement, cultural practices and home influences, working memory, executive functioning skills (of inhibition, shifting, and updating), and levels of students’ intention to accelerate strategies of learning and curiosity, and to learn how to impose structures and meaning on their world. The will relates to building confidence to take on challenges, appraisal skills to use problem-based coping strategies, and confidence that they can master a task and that it is worth the investment. Finally, the thrill attributes are the motivation to learn the surface and deeper aspect of what is being taught; to develop engagement skills, concentration, conscientiousness, and a desire to improve; and at all times, to learn to become their own teacher. Notwithstanding, it is the role of educators to make this possible, exciting, and worthwhile for the student.

D2: Design

All learners are novices to some extent, but that does not mean that they come to class with a blank slate. They certainly come into lessons with clear expectations about their probabilities of success, beliefs about whether the teacher can enhance their learning and cares about them, and knowledge of their skills and confidence in learning with and from others and with the teacher. As recipients of the teaching, they need to be heard about their strategies of learning they are employing, their interpretation of feedback, and their understanding of their progress and notions of success. Given the fundamental premise of Visible Learning that we aim for stu dents becoming their own teachers, then involving them in the design of interventions, developing a monitoring and evaluation plan, and learning from their reactions to the interventions are core.

High Expectations

Understanding LI and knowing SC

Understanding feedback

Unlearning, disequilibrium

These student expectations are powerful – for good or bad. Students enter classes with expectations about the success of teachers to engage them, educate them, and evaluate their progress. These expectations are formed from their early years relative to their achievement success. Those who put up their hands, answer questions correctly, complete the work on time and neatly, and comply in class are seen as the classroom winners. By age 8, if the child senses they are not part of this group, their expectations of success can diminish appreciably. Then the Matthew effect can doom some and advantage others – as those who comply and have sufficient language and numeracy skills can grow and play the classroom game and thence thrive, whereas others are left behind. This can be turned around by a teacher provided they do not overfavor compliance, are willing and able to see talents in learning, and show respect for their students particularly by communicating their high expectations for improvement. Such respect tells students that the teacher can and will listen and demonstrate that they can and have listened. All children have remarkable reserves of motivation, of curiosity, and of the joy of discovery, but great teachers use these resources to enable investment in school learning, create opportunities, exhibit openness in expectations, and can show their intent to teach, leading to high levels of impact (Reeve et al., 2023).

This proficiency to listen includes listening to how students understand the intentions and success criteria of the lesson, ensuring that these are intentionally aligned with the teachers’ concepts; listening to how they are choosing and using various strategies of learning and, if not productive, then teaching them alternative strategies; appreciating that the teacher’s feedback needs to be heard, understood, and actionable; and sharing the evaluation of progress toward success with the student.

D3: Deliver

Students take risks every time they encounter new knowledge and new understandings. There are risks to their previous thinking, and such disequilibrium can be challenging, invoke anxiety, and lead to self-doubt. Moreover, there are risks that showing misunderstanding or no understanding, making errors, going down false paths can be embarrassing in front of teachers and particularly peers. Hence, not engaging, not being challenged, employing more emotional than problem-solving coping strategies, and not attending can be skills of survival. Disequilibrium, however, is the perfect opportunity to advance learning provided teachers optimize this opportunity.

Using multiple learning strategies

Investing and driving

Taking errors as opportunities

Working with others

Popper argued that knowledge grows not through the building up of certain information but through a continual correction, alteration, and rejection of earlier knowledge, a process he termed “conjectures and refutations.” This is the opposite of the bucket of the mind notion (akin to Freire’s banking) and instead emphasizes learning from mistakes, questioning and finding gaps, and checking that relations among ideas build and not refute our ideas. As Piaget highlighted the importance of disequilibrium as the greatest precursor to learning, Popper saw learning as not merely tolerating dissenting ideas but respecting, playing, and questioning ideas. This all needs an open, permissive classroom that tolerates many views and ideas, and Bailey (2000, p. 162) summed up the core learning attributes of a successful student from a Popperian stance:

Stay well informed of matters of fact, significance, and relevance.

Adopt a critical stance toward the information presented.

Give reasons for holding a certain view or behaving in a certain way.

Respect evidence corroborating or falsifying a theory.

Be open to the possibility of changing a belief in light of evidence.

Judge a range of important issues fairly and impartially.

Challenge and seek reasons for changing or continuing a policy.

Value a diversity of ideas, beliefs, and values.

Treat fellow members as of equal value.

Put aside self-interest, if appropriate.

These are attributes of successful students, teachers, and leaders. These demand a safe and welcoming classroom of student questioning, seeing errors and failures as core to learning, encouraging the playfulness of learning and trying new ideas and building relations among these ideas, and then testing, adapting, and eliminating solutions to problems.

Families

D1: Discover and diagnose

In our parenting book, we note that within and across different societies, there are different notions of what it means to be a good person, what it means to raise children, what and who are members of a family, and what it means to be successful (Hattie & Hattie, 2022). But across all, childhood is the now for the child. Our children will create their future, not you. They will reinvent the world, critique, and refine what we now know and the world they (and we) live in. Parents (or educators) cannot predict their future and will not know the range and nature of many jobs and careers 20 years hence.

Children are born into a world of expectations

Parents as first learners

Children are born into a world of expectations and live in the now more than any other time in their lives. Their childhood should be adventurous, be a time to develop many learning and coping strategies, enjoy the dignity of risks to develop autonomy, and flourish in a socially enriching experience. Families can have powerful influences on how students learn, the risks they are confident and willing to take, and the joys of investing in various pursuits. Parents begin developing a child’s sense of competence, respect for self and others, and degree of autonomy to act and be responsible for those actions. Parents communicate their low or high expectations; show they can listen and be responsive; show whether they love challenges and learning; provide and receive feedback; create opportunities to develop language, language, and language; and evaluate their impact.

One of our major arguments is that the parent is not the first teacher, as this confuses the role and skills of school teachers, but that parents are first learners: children mimic how their parents react to failure, their degree of confidence to take on new and challenging tasks, their joys in curiosity and discovery, their skills working with others, their listening and feedback proficiencies, and their understanding of what their children bring to a task or new learning.

D2: Design

In most democracies, families (and all voters) have a say in approving curricula, and the many wars and reactions to curriculum changes can generate a lot of debate, as they should. However, families have little to no say in the design phases of school learning, but they can play a large part in embracing the language of learning in the home to support this learning. This language relates to the parents’ or caregivers’ sense of expectations, listening and responsive skills, confidence to tackle challenging problems, and climate where failure is seen as positive. Homes are about loving, learning, listening, and being responsive to all family members, encouraging friends and the skills of friendship, and developing the skills of self-determination.

Parents can establish routines about homework, create a space for learning (when I am in this space, I am in learning mode), see teachers as partners in home learning and vice versa, and ask for clarifications (through their child) as to three questions: What am I learning today? Why am I learning it? And how will I know that I have learned it? Particularly in the early years, parents can set expectations and values about reading volume, reading widely, reading aloud; be supportive about tackling the challenges in math and writing often; guide not tell; show the value of wait time and deliberate practice; and foster well-being.

Learn the language of learning

Become good listeners

Have high expectations

Create a climate where failure is seen as positive

School leaders

D1: Discover and diagnose

There are additional burdens on school and system leaders. They are expected to have an impact not only on students in the school but also on the teachers and on teachers’ proficiencies to influence the learning lives of students. Their most significant power is choosing the narrative for the school, setting the expectations, and building a collective of educators to deliver on the purposes of the school. It is establishing a climate of sustained mission across the school to promote progress to achievement; to feel safe and inviting to come to school; to create a sense of belonging, respect, and citizenship, and a love of learning; to know how and to have the confidence to learn with others (as teachers and students); and to support multiple ways to strive for multiple concepts of excellence. Leaders set the tone, set the climate, and press to maximize progress to achievement, well-being, impact, and togetherness in the school.

Press for both progress and achievement, well-being, and togetherness

Know it is within, not between schools, that matters

Create time, resources, expertise to collectively impact on students

It is realizing that the greatest source of variance in improving the lives of students is more within their school than between schools, acknowledging the presence of vicious inequalities in society and deliberately ensuring this is not replicated within the school, setting the standard for quality interpretations and subsequent imple mentation of effective interventions while evaluating the impact on teachers and students, and becoming the critics and conscience of the school. This means the school leader needs to create time for educators to work, plan, evaluate, and critique their impact; to know when and how to bring expertise into the school to upskill and upscale the educators; to inform and keep parents aware of the language of learning in the school and for their children; and to be mindful of the barriers and enablers of learning. It is not charisma and charm but competence to reliability identify concerns, deal with difficult problems, engender trust, and develop a collective for all in the schools to ensure every child gains more than a year’s growth for a year’s input (and making this goal concrete).

This is personified in the skills of leaders to enable the teachers (and all support staff) to work collaboratively to plan and work together, critique their expectations, evaluate their impact on students, and decide where best to go next in light of their impact. It is not merely teacher collective efficacy; it is collective teacher efficacy about their impact. This means school leaders are the lead evaluators, the exemplars of interpretative judgments, the appreciators, and the leaders of evaluative thinking.

D2: Design

A major influence of school leaders is to develop a language of learning across the school based on the school’s mission for developing growth, achievement, well-being, and inclusion – for all. This sets the tone for developing the collective efficacy of all adults in the school, working with students to know, evaluate, and maximize their impact on this learning. The point of improvement is not adjusting the curricula, tinkering with the structural aspects, introducing ability grouping and enrichment classes for some, worrying about the socioeconomic resources and deficits of the parents, screaming for more money, and building brilliant web pages with happy children – but the collective power of the adults in the school to positively impact on student learning and outcomes. This costs time and money, as focusing on diagnosis, building teams to enhance impact, and privileging expertise and courage can be resource-intensive.

Language of learning

Role of money

Climate of invitation

High expectations

Collective efficacy

Ensuring there is intentional alignment in the lesson designs, ensuring teachers critique each other’s impact, knowing that all adults (and students) have high expectations that all can make a profound difference to each student, creating high-trust environments (in the staffroom) to monitor progress collectively, conducting diagnoses to focus professional learning, and ensuring every class is inviting for students to come to school. Distributing resources on instructional ways of thinking, doing, and caring is much more powerful than spending the majority of resources on structures, creating ability groupings, having smaller class sizes, creating common or new ways of teaching (without attention to the impact of the teaching), and ignoring the language of learning. The core question is this: Does the school have a shared language about what it means to be a great learner (Hattie K., 2021)? It is less about the structure and the composition of the students within the school or class and more about the climate, the culture, the development of a sense of belonging, the language of learning, and the school as an inviting place for students to want to come to.

Leaders (and system leavers) have a particular role in negating deficit thinking. Deficit thinking pathologizes the student and their background as the problem and relieves educators of self-scrutiny about their thinking and impact, misuses the power imbalance of educators to students when they then choose curricula, reduces the difficulty and complexity of the work, and decides who participates in this discourse in the system, school, and classroom. This leads to dominant group-think at the expense of those they then marginalize and, thus, begins a vicious process that reinforces these students’ deficiency, limitations, and shortcomings. This is compounded when interventions then are designed to remediate the students’ deficits and by the need to identify these students in deficit terms in order to attract funding. Instead, the location needs to start with the educator’s sense of expectations for all students and implement programs or enhance instruction that improves the learning and outcomes for all students. All students can learn, but we as educators just may not be good enough to teach them well.

In most of today’s schools, variance is the norm. This variance needs to be thought of as an asset leading to creating an open-to-learn class climate, where there is a focus on every student making at least a year’s growth for a year’s input – no matter where they start in their learning journey and regardless of their background. It is the leader’s role to ensure this occurs. Each student needs a learning intervention, an acceleration plan, a deliberate invitation to come and learn, and the opportunity to make errors and move beyond the edge of what they already know, can do, and care about. The climate in the school and class needs to be seen by students as fair, be led by teachers who believe can collectively enhance the students’ learning, be led by teachers who have high expectations of each student’s success, have clarity of purpose, and show excellent listening skills. Creating a sense of cohesion and positive relations between teacher and student and among students leads to high trust such that students can fail quickly, learn from failure, and feel comfortable not knowing and, thus, receptive to the impact of the teacher. The reason for establishing these relations is to move the onus of being the teacher to the student, to permit students the safety of working with their peers in thinking aloud, to develop their confidence and skills to contribute to group work, and to develop a sense of belonging in the place of learning.

D3: Deliver

As noted earlier, the greatest power of leaders is choosing the narrative for the school or system. The VL narrative is about understanding the impact on the learning lives of students, ensuring high-trust cul tures for students to feel invited to come and learn, and teaching students to become their own teachers.

Shared narrative

Implementation science

Teachers, students, leaders, and families

D4: Double-back

While there is an emphasis on evaluative thinking in the previous D1–D3 phases, it is during this double-back phase that it becomes paramount. All actors (teachers, students, families, leaders) need to systematically and deliberately monitor and evaluate the impact of all the factors on the learning lives of the students. It helps if the plan to do this is articulated and agreed up front in the planning and diagnoses phase, and often, it entails repeating the evaluation of diagnoses at the end of the implementation. The purpose is not to rank and provide numbers to students or teachers but to learn from the interpretations about the next optimal teaching and learning foci. Sometimes it leads, simply, to reteaching using different teaching methods; sometimes to focus on specific parts of the lesson; sometimes on addressing content that was (falsely) assumed to be learned the first time (recall Nuthall’s, 2006, finding that all of us typically need three to five exposures to content); sometimes to providing more consolidation of content to thence use these ideas in deeper relational and transfer activities; and sometimes to attend to the lack of confidence to tackle challenges that students failed to engage with.

Assessment is one form of evaluative feedback to teachers and students. It needs to be triangulated with teacher noticing, interpreting from artifacts of student work, and listening to students interpret what they know, can do, and care about. Much emphasis is placed on the interpretative power of these sources of evidence, making the interpretations explicit and transparent to both teachers and students, and the core notion of assessment-capable students and Visible Learning teachers is their skills to know when to seek evidence, how to interpret this evidence, and when to seek critique of these interpretations so that they can decide on the next optimal teaching and learning act. Too often, assessment stops at the numbers, and if there are no where-to-next claims from assessment for the teacher or student, it probably was not worth doing.

At the school level, collectively interpreting, critiquing, and making consequential decisions based on evidence of progress to achievement need to be a school-wide task. The purpose of collective efficacy needs to be about querying, assisting, and clarifying interpretations of evidence, leading to more impactful teaching. Teacher collective efficacy requires leaders to create an authorizing environment for all teachers to improve, a realization that all students belong to all teachers, a movement from cooperative to collaboration, an investment by leaders in their and the teachers’ success, an understanding and exhibition of high expectations for all students and staff, agreements on the whys (moral purpose, sense of impact), a space where we is more dominant than I, and a safe environment where risk taking and working together are paramount.

Accountability models of trust but verify can have positive impacts. This does not mean accountability systems based on standardized tests but mean systems based on asking school leaders to put the evidence on the table that all students in the school feel invited to come; can learn; have developed the skills, will, and thrills of learning; and have gained at least a year’s growth for a year’s input – and what that means in action. Systems would need to provide resources to do this – and a common lack is assessment systems to measure growth, develop a basket of goods approach that has more than achievement in the basket, and have systems where school leaders can learn from the presence of excellence around them to enhance their own school (Hattie, 2018). This means that the development of evaluative thinking is the core skill of school leaders.

D5: Double-up

Teachers, students, families, and leaders need to focus more on discovering success and upscaling rather than dwelling on finding failure and trying to fix it. Scaling requires great diagnosing; identifying what worked well or not; identifying errors, failures, misunderstandings; but also understanding the causes of success, hearing the thinking that leads to successes, and capitalizing on the deep skill of so many educators to critique. Kotter (2012) identified core elements of successful scaling up, including establishing a great enough sense of urgency, building a powerful guiding coalition, promoting and communicating a vision, removing obstacles to the new vision, systematically planning for and creating short-term wins, not declaring victory too soon, and making the improvements as the new normal.

Elmore (1996) argued that it is possible to scale success if we develop strong normative structures for practice (e.g., focus on the impact of teaching), make expertise the new norm of schooling, develop an organizational structure that intensifies and focuses rather than dissipates and scatters, create intentional processes for the reproduction of successes, create structures that promote learning of new practices and incentive systems that support them, and develop theories of change about how educators maximize their impact. I would add developing strong models of implementation that adjust and allow for the diversity now built into most schools and classes, learning how to privilege collective thinking within and across schools, investing in human capital and expertise, and engaging the larger community in the urgency, diagnosis, and respect of expertise (Cohen & Mehta, 2017).

The lack of attention to scaling success is not due to resistance to change. Maybe we have too much change, and one of the major barriers to implementing our own Visible Learning+™ program is when the principal changes – showing the power of leaders and the lack of embedding improvements in the school’s culture. As Tyack and Cuban (1995) so aptly noted, instead, we prefer to tinker to utopia. We tweak the curriculum, create new tests, favor particular teaching methods, create dichotomies (DI vs. almost everything), fiddle with the composition of students (lower class sizes, ability groups), invent new names for schools (charter trust, academies), maintain incentives to privilege experience over expertise, and scream for autonomy to do it my way. It can also be very difficult to stop initiatives in schools when they are not effective (Hamilton et al., 2023).

As noted earlier, there is a great lack of contingencies and systems for school leaders to work across schools to improve their schools and, thus, the system. The notion of collective efficacy of leaders is hardly explored but possibly exciting and transformational. If only there were sufficient studies for a meta-analysis of successful scaling in education.

The key to success is having a model of implementation, continually evaluating the fidelity, dosage, acceptability, quality, and adaptations to the effectiveness and efficacy of the implementation, with a laser gaze on the impact on the learning lives of students (progress to achievement, keenness to invest and reinvest in this learning, and enhancement of the knowing-that, knowing-how, and knowing-with, while also developing skills in learning alone and with others).

Reflections

Elaborate on your model of implementation. For example, how do you determine the focus, check the fidelity of implementation, monitor the adaptations to the local environment, adjudge the quality of the intervention, evaluate its impact, and seek and deliver improvements?

What interventions should you stop in your school due to the lack of impact on students?

If Nuthall (2007) is correct that 40–50% of the lesson is known by all students, then how could you more effectively diagnose this prior knowledge and, instead of reteaching it, use it as coat hangers for more advanced knowledge and understanding?

Describe your expectations for each student – how well will they perform at the end of this year? Next, ask your students the same questions. Now, strategize how you can raise your and the students’ expectations and aim to deliver on these enhanced expectations.

Create a lesson with two success criteria, two sets of activities, two assignment tasks, two different teaching methods, and two checks on mastery – one at the knowing-that, and the other at the knowing-how/with levels of complexity. Where are the students more successful in this lesson, with the dual but transparent focus on the two levels of complexity?

To what degree does your class/school privilege growth to achievement, not seeing high achievers (especially those who are cruising) as the best learners and privileging where there is evidence of accelerated progress – as well as mastery and achievement?

Next time you administer a test, provide the feedback and grades, then ask students to provide bullet points of their interpretations of their performance (in light of your comments and others) and what they could best do next to improve. Use this information as an indicator of your success at providing effective feedback, and where students cannot interpret correctly or have no where-to-next, teach accordingly.

CHAPTER 5 Students

DOI: 10.4324/9781003380542-5

Of all the factors in the VL database, the greatest source of achievement variance relates to the students. There are 373 meta-analyses relating to aspects of the student, based on over 25,000 studies involving over 67 million students and about 104,000 effects (Table 5.1). The average effect is 0.24. The major student influences come from the skills students bring to the class, their prior knowledge and background variables, their will or beliefs, their attitudes and dispositions, and their sense of thrill or motivation. Care is needed in these sections, particularly within the will influences, as many are (meaningfully) negative (e.g., boredom, anxiety) – so the devil is in the detail.

Table 5.1 Summary statistics for the student attributes

Student
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Skills: prior knowledge and background
50
6,825
1,867,089
21,977
0.65
0.05
3

Will: beliefs, attitudes, and dispositions
145
8,620
8,742,770
60,207
0.16
0.07
3

Thrill: motivational approach/ orientation
63
4,705
28,755,174
9,895
0.30
0.06
4

Physical: physica influences
I 115
6,095
27,821,772
12,095
−0.15
0.07
3

Total/average
373
26,245
67,186,805
104,174
0.24
0.06
3

The skills

Skills: prior knowledge and background

Table 5.2 presents the 12 summary categories relating to what the student brings to the classroom in terms of their skills. The average effect is 0.70. The major attributes of successful students include their beliefs and expectations about their achievement, their prior achievement and ability, their proficiency to restructure and impose a structure on information (field independence, Piagetian levels), or in more modern jargon, their skill to self-regulate.

Table 5.2 Summary statistics for the skill-related student variables

Skill
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Prior ability and intelligence
5
1,128
572,240
1,703
0.96
0.02
5

Prior achievement
11
1,862
785,667
6,120
0.73
0.07
5

Relations of high school to university
7
2,155
164,054
7,101
0.55
0.03
5

Relation of high school to career performance
2
147
36,756
147
0.37
0.02
3

Piagetian levels
1
51
6,000
65
1.28
0.08
2

Executive functioning
2
314
73,552
314
0.62
0.05
3

Working memory strength
4
437
58,287
4,107
0.63
0.08
4

Early years noncognitive skills
1
78
7,179
78
0.20
0.08
2

Cross-laterality
1
26
3,578
26
−0.03
0.00
1

Field independence
2
46
4,423
157
0.94
0.05
2

Self-reported grades
6
218
30,347
427
0.96
0.11
3

Relating creativity to achievement
5
269
110,239
1,618
0.40
0.09
4

Critical thinking
3
94
14,768
114
0.84
0.08
3

Total/average
50
6,825
1,867,089
21,977
0.70
0.26
5

The skills

Prior ability, prior achievement, relation to later learning

There has been a long debate about intelligence and its heritability. It is the interaction between hereditary and environment that is most powerful – and it is as follows:

likely that there are as many genetically well-endowed IQ people who live in poverty, have little access to schooling, and have been given few opportunities to learn as there are genetically less-endowed IQ people ruling nations and living in excess.

(Fletcher & Hattie, 2011)

In many senses, our job as educators is to mess up the predictability of prior learning, to see skills and talents in the child they may not see in themselves, and to help children exceed what they think are their expectations.

While it is not acceptable to talk about intelligence, in many circles, it remains that not all children start equally endowed with the potential to become Einstein or Madame Curie. This does not mean that every child cannot learn, that every child cannot make excellent contributions to society in many ways, or that intelligence needs to be a barrier. But it may mean that if we knew both current achievement and intellectual potential, we may be better placed to understand the potential yet unrealized. By introducing some independent estimate rather than achievement, many students (poor, refugee, minority) might have greater potential for accelerating learning and not be as weighed down with false expectations or captured in a future determined by their prior lack of opportunities to learn and develop. It seems a pity that the history of IQ has such a checkered and, at times, a negative impact.

The relation between ability and achievement should not be surprising, but as noted, it is far from perfect. *Hattie and Hansford (1982) reported an average correlation of r = 0.51 between measures of intelligence and achievement, which is high, but there is much opportunity for achievement to be enhanced for all students – independent of their intelligence. Furthermore, prior achievement predicts success from preschool to the first years of schooling, between high school and college or university grades, between college and adult success, and between grades in school and later job performance. This all points to the power of understanding a student’s prior achievement and endorses Ausubel’s (1968) principle of first finding out what students know in order to teach accordingly.

Piagetian stages

Piaget’s stages are now relegated to history, but his contributions are still powerful (Piaget & Inhelder, 1969). He argued that there were four major stages of children’s thinking, moving from cognitive egocentrism to decentered thinking. During the sensorimotor stage (0–2), the child exhibits a completely egocentric approach to the world, is unable to separate thoughts from action, and is unable to recognize that the perspective of the object would differ depending on their position relative to the object. The child then moves to the pre-operational stage (2–7), where object permanence is firmly established and symbolic thoughts develop. To move to the next stage, referred to as the concrete operational stage (7–12), children need to be able to perform what Piaget termed “operations”; these are internalized actions that the individual can use to manipulate, transform, and then return an object to its original state. The concrete operational stage is marked by the child beginning to apply logic to steps and stages. The final stage from 11+, the formal operation stage, is characterized by abstract and hypothetical thought.

While there is a large body of literature that supports the underlying principles of Piaget’s theory (Rindermann & Ackermann, 2021), there are also a number of weaknesses, such as the assumption that children exist at only one stage at a time, and the impact of cultural context is often ignored.

Executive functioning

Bolton and Hattie (2018) demonstrated that the changes outlined by Piaget are primarily related to changes in executive processing. “Executive processing” is an umbrella term for a set of higher-order control processes regulating several different cognitive functions for attaining specific goals. These include working memory, cognitive flexibility, attention control, planning, concept formation, and feedback processing – each varying in complexity (Karbach & Unger, 2014). There is a large evidence base illustrating that the development of executive functioning is a major predictor of scholastic performance (Swanson & Alloway, 2012). Several longitudinal studies indicate that executive functioning contributes to academic achievement rather than vice versa (Best et al., 2011; Bull et al., 2008). In the following chapters, the development and teaching of executive functioning will be most evident as powerful aspects of learning.

There are three major dimensions of executive functioning (Miyake & Friedman, 2012):

Inhibition of dominant or proponent responses and skills not to be distracted.

Shifting between tasks or mental sets and relating to the ability to move back and forth between multiple different tasks or ideas.

Updating and monitoring of working memory representations. We can hold a limited number of items of information in short-term memory at any one time, irrespective of ability (usually about four to six bits of information).

These executive functioning skills are much lower among children with intellectual disabilities and, thus, may require closer deliberate teaching (Spaniol & Danielsson, 2022). Therefore, it should not be surprising that *Jordan and Brownlee (1981) found that the link between the development of these executive skills and achievement is very high (r = 0.54, d = 1.28). This is especially the case for mathematics (d = 0.73), which is still high, but somewhat a lower effect in reading (d = 0.40). Thus, knowing how students think and how their stages of development may constrain this thinking may be most important to how teachers choose materials and tasks, how the concept of difficulty and challenge can be realized in different tasks, and what appropriate cognitive load can be asked of the student (Das et al., 1994; Sweller, 2008). This points to the critical admonition by Shayer (2003, p. 481):

If you cannot assess the range of mental levels of the children in your class, and simultaneously the level of cognitive demand of each of the lesson activities, how can you plan and then execute – in response to the minute by minute response of the pupils – tactics with results in all engaging fruitfully?

Note that this is not matching the teaching material to the current stage but instead aiming to maximize the rate of cognitive development of every child to the next stage. This approach is embedded in the Cognitive Acceleration project (Shayer & Adey, 2002), based on three premises:

The mind develops in response to the challenge or disequilibrium, so the intervention must provide some cognitive conflict.

The child has a growing ability to take control of their own thinking processes, so the intervention must encourage students to be meta-cognitive and develop their executive functioning.

Cognitive development is a social process promoted by high-quality discussion among peers scaffolded by a teacher; thus, the intervention must encourage social construction as much as individual thinking and creativity.

The effect sizes of cognitive acceleration are appreciable (d = 1.09), and it is noted that this method requires detailed cognitive task analysis (see also Chapter 11). They argue that in order to find suitable lessons, they had to engage in the following steps:

Analyze the hierarchical steps or cognitive complexity in the subject matter (e.g., science and mathematics).

Find contexts within these steps that contain at least two or three elements of learning the important concepts.

Plan lessons so that every student, regardless of their previous achievement, would have some possible achievement during the lesson activity.

Plan lessons so that all students experience enough cognitive conflict to be sufficiently challenged.

Working memory strength

Most of us can only hold between four and six notions in working memory at one time – so imagine two students, one using working memory to find the answer to 9 × 6, and another who has overlearned that 9 × 6 = 54. The former uses all their working memory on a calculation, leaving little working brainpower to use this information for solving a problem, whereas the second student can use all their working memory to use 54 in a problem-solving task. Teaching students to consolidate or overlearn critical knowledge is, thus, essential if they are then able to relate, extend, and transfer deeper understanding. This means helping students understand what is critical, making sure any overlearning leads (not merely aimed at short-term gains, like higher test scores) to deeper learning opportunities, and deliberately teaching consolidation strategies (see Chapter 11).

Some have argued that working memory is a general capability (e.g., Baddeley, 2017), whereas others claim it is more domain-specific (Ericsson & Kintsch, 1995). Based on 197 studies, *Peng et al. (2015) found a correlation between working memory and reading of 0.29 (d = 0.61), with similar effects for phonological coding (0.34), decoding (0.28), vocabulary (0.26), and comprehension (0.31) – and these effects were unrelated to age or bilingual backgrounds. *Carletti et al. (2009) showed that the magnitude of the working memory effect was higher for tasks with a verbal domain, requiring both maintenance and manipulation of information (0.75) compared to visual tasks (0.36). They also demonstrated that the working memory deficits of students with poor comprehension relates mainly to those tasks requiring storage and processing of information, while inhibiting off-goal information and/or updating memory content information. Be wary of cognitive overload.

There are commercial online packages to improve working memory, and they can be most successful, but the transfer to other tasks is close to zero (also see Foer, 2012). For example, CogMed is a computer set of adaptive routines that are intended to be used 30–40 minutes a day for 25 days. A recent meta-analysis (by the commercial owners, Owen et al., 2010) found average effect sizes (across 43 studies) exceeding 0.70, but in a separate meta-analysis of 21 studies on the longer-term effects of CogMed, there was zero evidence of transfer to subjects, such as mathematics or reading (*Melby-Lervag & Hulme, 2013). As well, any gains were not maintained at follow-up (about nine months later), and there was no evidence to support the claim that working memory training produces generalized gains in the other skills that have been investigated (verbal ability, word decoding, or arithmetic) even when assessment takes place immediately after training. For the most robust studies, the effect of transfer is zero. It may be better to reduce working memory demands in the classroom and teach consolidation skills (Lovell, 2020).

Early years noncognitive skills

*Smithers et al. (2017) asked whether some early-years noncognitive skills, such as attention, self-regulation, and perseverance in childhood, influence later outcomes. Intervention studies that focused on these skills (at about 4 years of age) had a d = 0.2–0.4 on short-term academic achievement, psychosocial outcomes, language, and cognitive outcomes – although they reduced in effect over longer times. There seems to be a need to address these noncognitive skills regardless of age, as detailed later.

Cross-laterality

Does handedness make a difference to achievement? Corballis et al. (2008) have argued that dominance is more critical than which hand you use to write. They found that ambidextrous individuals perform more poorly than left- or right-handers, especially on subscales measuring arithmetic, memory, and reasoning, and there is little if any difference between left- and right-handers (Fletcher & Hattie, 2011). *Ferraro et al. (2017) failed to find any relationship between the preference for different sides of the body when performing tasks that involved any combination of hand, eye, foot, or ear, and the performance in reading, spelling, arithmetic, language, or intelligence tests.

Field independence

Field-independent learners are more able to be more analytic, be independent, and identify parts and ideas with less interference from the context, whereas field-dependent learners tend to be more interpersonal, rely heavily on external influences, and seek cues from their environment or the context. Field-independents are more likely to see the whole and then parse for details, and field-dependents are more likely to see the detail and induce the patterns. The former is indeed more successful in learning, although *Garlinger and Frank (1986) showed that matching field-dependent students with field-dependent teachers (and similar for independent students and teachers) has no effect on the students’ achievement. These notions are less used today, as more attention is being placed on the specific learning strategies and self-regulation, and not generalized to one of only two options (see Chapter 11).

Self-reported grades

Students, particularly after age 8–9, have reasonably accurate understandings of their levels of achievement. *Kuncel et al. (2005) found that high school students accurately understood their achievement levels across all subjects (r = 0.80+). This was the case for all but minority students who received lower grades than non-minority students and were more likely to be less accurate in their self-estimates or self-understanding of their achievement. Overall, however, students were very knowledgeable about their chances of success.

This shows a remarkably high level of predictability about achievement in the classroom (and should question the necessity of many tests when students already have much of the information the tests supposedly provide). Further, the student’s own expectations of success (which are sometimes set lower than what could be possible) might become barriers for some students, as they only perform to whatever expectations they already have of their current potential.

Further, many have shown that students’ judgments of their learning, such as how they have learned something or their confidence that they would recall and use the learned knowledge, are often inaccurate (Dunlosky & Lipko, 2007). If there is false overconfidence, this can impair later learning (e.g., not studying topics they believe they have mastered), and students with poor meta-cognitive accuracy often make ineffective decisions about what to relearn and tend to over-calibrate on topics of high interest or that are novel. This led Senko et al. (2021) to recommend that triggering based on seductive interests or high-situational interest (e.g., jokes, anecdotes, imagery, or the novelty of the task) can lead to overconfidence, poorer learning outcomes, and less effective self-regulation. This should not mean to remove interest and make the class boring but to enhance the personal interest of desiring to master the content and, thus, make it interesting and valuable.

We have developed a major set of arguments around this influence by reworking the notion into developing students’ assessment capabilities. How to teach students to know their current level of understanding accurately, know where they’re going, and have the confidence to take on the challenges of learning from current to desired states of learning. They also need to know how to select tools to guide their learning, how to seek feedback and recognize that errors are opportunities to learn, how to monitor their progress and adjust their learning, and how to recognize their learning and teach others (Frey et al., 2018).

Relating creativity to achievement

Koestler (1964) summed up the essence of creativity – the bringing together two or more seemingly unrelated ideas. Some have argued that there needs to be a minimum threshold before creativity and achievement are highly related (but see Hattie & Rogers, 1986; *Kim, 2005), and the relation can be seen when creativity tests are administered in untimed, game-like conditions (but see Hattie, 1977). Across many tests of creativity, the usual dimensions include fluency, flexibility, originality, and elaboration of ideas. Such problem-solving strategies have high relations to achievement outcomes (*Ma, 2009, reported d = 0.86), and *Gadga et al. (2017) found higher relations when measured with creativity tests (r = 0.30) than with self-report measures (r = 0.12).

A key is whether teachers not only teach these problem-solving and creativity skills but also communicate a welcoming for creative thinking. For example, if they are strict about meeting specific correct answers, if their assessments overly focus on the content and facts, and if they do not model creative thinking in their teaching, this may send messages that creative thinking is not welcomed.

Critical thinking

Critical thinking is skilled interpretation and evaluation of observations and communications, information, and argumentation – that meets standards of clarity, relevance, reasonableness, and fairness (Fisher & Scriven, 1997, p. 21). It is particularly important for students to apply critical thinking principles and practices to their own ideas and communications (especially when using the internet). These skills are the essence of self-regulation, which will feature a lot throughout this book. *Fong et al. (2017) found an overall relationship of r = 0.24 (or d = 0.49) between critical thinking and community college student achievement. We return to specific critical thinking learning strategies in Chapter 12.

The will

The will refers to the dispositions that students bring to the class and are often referred to as social and emotional learning (see Mahoney et al., 2020, for an overview). While these influences have been long debated, the COVID pandemic has magnified their importance to learning. They are not separate from learning but are a core part of making classrooms inviting and developing the students’ sense of enjoying learning, and need to be considered in lesson design. A major debate is whether these are precursors to learning, part of the learning process, or appropriate outcomes of learning, and whether they can be enhanced in more general courses or need to be embedded within the content domains taught in schools. The message in this section shows how critical many dispositions are to learning within curricula domains and lessons, and that we need more attention to the joy and happiness from being engaged in learning.

The will includes four major categories of these dispositions (Table 5.3). There is overlap in these categories, there is more to the will than emotions, and the focus remains on identifying these attributes of the will that have the greatest impact on learning and achievement. Much of the COVID pandemic issues about social and emotional learning seem to presuppose they are separate attributes, they are new (they have always been a focus of teachers), and educators may now need to become counselors (as opposed to employing specialists to work with teachers to educate these students). However, there are social and emotional aspects of children beyond what we expect educators to deal with, and many programs assist teachers with the more extreme issues (e.g., trauma-informed methods, response to intervention models). The four categories are as follows:

Perseverance and confidence (growth vs. fixed mindsets, mindfulness, grit, self-efficacy, positive self-concept, self-control)

Student personality, including emotional intelligence and well-being

Emotions

Positive activating dispositions (hope, curiosity, happiness, enjoyment)

Positive deactivating dispositions (relief, relaxation)

Negative activating – anxiety, depression, stress, boredom, anger, frustration, aggression/violence

Negative deactivating – boredom, hopelessness

Cognitive dispositions (stereotype threat, morning vs. evening)

Table 5.3 Summary statistics for the will-related student variables

Will
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Growth vs. fixed mindsets
5
325
867,175
565
0.19
0.03
4

Mindfulness
7
228
20,208
244
0.26
0.13
3

Grit
3
267
356,004
782
0.35
0.02
4

Concentration-persistence-engagement
10
550
354,511
1,202
0.41
0.12
5

Attendance
2
257
73,147
287
−0.52
0.03
3

Self-efficacy
12
640
1,313,310
648
0.64
0.03
5

Positive self-concept
13
846
634,598
3,557
0.51
0.03
5

Self-control
4
126
90,405
157
0.66
0.06
3

Student personality
20
1,929
2,384,177
4,545
0.18
0.02
5

Perfectionism
2
104
15,068
534
−0.03
0.02
3

Emotions
2
61
19,941
61
0.61
0.08
2

Emotional intelligence
11
567
137,458
1,845
0.50
0.11
5

Well-being
5
379
395,547
39,282
0.24
0.08
5

Enjoyment
3
93
36,225
102
0.50
0.10
3

Hope
2
56
13,101
34
0.20
0.08
2

Curiosity
2
15
3,330
18
0.74
0.17
2

Happiness
2
51
4,694
51
0.54
0.13
2

Relaxation
1
20
1,841
36
0.16
0.09
1

Anxiety
22
1,551
1,698,986
4,983
−0.40
0.04
5

Depression
3
96
59,008
113
−0.30
0.08
3

Anger
1
25
11,153
25
−0.65
0.04
2

Frustration
1
9
1,418
9
−0.04
0.06
1

Aggression/ violence
3
158
140,148
810
0.03
0.01
4

Boredom
4
89
51,128
139
−0.46
0.13
3

Morning vs. evening
2
50
31,229
50
0.18
0.00
2

Procrastination
3
128
28,961
128
−0.41
0.04
3

Total/average
143
8,363
8,669,623
59,920
0.18
0.06
3

Perseverance and confidence: growth vs. fixed mindsets

The most popular model is growth vs. fixed mindsets (Dweck, 2006). A fixed mindset invokes the belief that one’s abilities are set in stone and genetically inherited, and a growth mindset invokes the belief that one’s intelligence, skills, and qualities can be developed through effort, input, and a range of learning strategies. Dweck has claimed that growth mindsets can inspire taking on challenging goals and shape views about effort, but she has never claimed in her academic writings that there is a state of mind called “growth mindset” – it is not an attribute of a person; it is a way of thinking in particular circumstances. The key question is, “When is it appropriate for thinking in a growth manner over a fixed manner?” Dweck’s answer has been when we do not know an answer, make an error, experience failure, or are anxious or confused. In these situations, having a growth mindset (believing that one can change and that ability is not fixed) is a most appropriate coping strategy and can move the student forward, and not lead to resistance, overreaction, or fear of flight into a fixed mindset (see Dweck & Yeager, 2019; Hattie, 2017). The current meta-analysis of programs to enhance growth mindset have very low effects partly because the programs fail to appreciate the importance of the optimal conditions to observe and enhance growth thinking; too often, these programs involved generalized training in “I can” or “not yet” catch-cries, which have limited effect in situations of not knowing, and they are not specific to the conditions when growth mindsets are most needed.

Perseverance and confidence: mindfulness

Mindfulness derives from the Buddhist tradition of becoming attentive and nonjudgmental, and having conscious awareness of one’s immediate experiences while remaining curious, open, and accepting. The purpose is “awareness, guarding, keeping the mind on course, and actively encouraging active states of mind, … and a discrimination as to what is helpful. It is like the salt that gives experiences their taste” (Shaw, 2020, p. 48, 185). Like condiments, these programs often appeal to students, parents, and teachers as nice extras.

These generalized mindfulness programs are becoming increasingly popular in schools, usually in the cause of enhanced mental health and well-being. But the effects of mindfulness programs are very low. *Zainal (2020) reported d = 0.19 for verbal fluency, and *Sedleimer et al. (2012), d = 0.15 for learning and memory. *Carsley et al. (2018) also noted that the effects were lower when the program was facilitated by trained teachers compared to outside facilitators (0.19 vs. 0.28). While mindfulness programs may have important effects on reducing stress and emotional problems, and increasing attention, it is not a major influence when converting these effects into enhanced learning and achievement.

A related notion is “with-it-ness,” a term coined by Kounin (1970) about teachers’ skills in knowing what is going on in their classrooms or, more colloquially, having eyes in the back of their heads. Marzano et al. (2000) noted that the attributes of teachers that had the greatest influence on ensuring well-managed classrooms and reducing disruption came from with-it-ness (d = 1.42) by the teacher. Thus, the teacher’s with-it-ness may be the most important aspect of mindfulness, particularly in the impact on student learning. This with-it-ness notion can be applied to students, particularly when it relates to increased focus and attention to the task.

Perseverance and confidence: grit

Grit has recently become popular mainly due to the writings of Duckworth (2016). She defines grit as working strenuously toward challenges, maintaining effort and interest over the years despite failure, adversity, and plateaus in progress. Stokas (2015) traced the genealogy of grit, and noted a major risk with the claim that students’ poor performance can be attributed to a lack of ‘grittiness’. This attribution may ignore many critical supports and teacher influences that may be lacking (see also Ris, 2015). Stokas used the boxer and cowboy as epitomes of grit, but I note that criminals also need and have high levels of grit. Grit represents perseverance and passion for long-term goals, consistency, and conscientiousness to work hard even in the face of setbacks. Each of these (particularly conscientiousness) must develop at the right time for the right outcomes. Many students persist on tasks using poor strategies, have strong beliefs they are not learners, and experience teachers who have no joy or belief in the importance of teaching them when to be and when not to be gritty.

One group of researchers (Morell et al., 2020) found that perseverance of effort (keeping working toward goals) predicted student grades more than consistency of effort (pursuit of a goal with passion). *Crede et al. (2017) also demonstrated that perseverance and conscientiousness are much better predictors of performance than any generalized measure of grit. They concluded that “overall conscientiousness is so strong as to not only limit the incremental value of grit scores for the prediction of performance over and above conscientiousness but also suggest that grit may be redundant” (p. 30). The key is for teachers to structure tasks to entice students to persist on tasks and skills, engender a joy in persisting, and ensure they are gritty using the optimal learning strategies.

Perseverance and confidence: concentration, persistence, engagement

Engagement is often divided into three: behavioral (participation in activities, attentiveness, attendance), emotional (affective attitudes, belonging), and cognitive (use of meta-cognitive strategies) (Fredricks et al., 2016). The correlations are not that dissimilar among these three: *Lei et al. (2018) reported behavioral engagement of r = 0.35, emotional engagement of r = 0.22, and cognitive engagement of r = 0.25; and *Chang et al. (2016) reported similarly: behavioral of r = 0.30, emotional of r = 0.23, and cognitive engagement of r = 0.24.

The dilemma is that this three-dimension notion could privilege doing the work. As Nuthall (2007) noted, “Our research shows that students can be busiest and most involved with the material they already know. In most of the classrooms we have studied, each student already knows about 40–50% of what the teacher is teaching” (p. 24). Further, Nuthall argued that teachers’ thinking focuses on keeping students busily engaged in activities that produce some tangible product. This leads to ritualized routines, whereby teachers and students talk about resources and tasks, and how long an activity should take. When he showed students videos of their classroom behaviors and asked what they were thinking, the answer typically was in terms of how to finish quickly and how to get the answers with the least possible effort. This was the case for students, regardless of prior ability. When he asked teachers how they knew their students were learning, they argued that it was when students are actively engaged with learning activities. “They monitor the look in their students’ eyes, their enthusiasm, their puzzlement, the questions they ask. In most teachers’ minds, the criteria for successful learning are the same as the criteria for successful classroom management” (p. 916). Performance is more a function of student interest, motivation, and understanding of the purposes and beliefs of the teacher’s activities. However, knowing that a student is engaged does not mean they are learning.

An alternative model was developed by Berry (2023), starting from her interviews with teachers about what they understand by student engagement. The dominant response was that if students were doing work, they were engaged. The sad fact is that in a lot of doing, sometimes there is very little learning. For example, Van Hees (2011) videotaped classes of 5–6-year-old students learning to read over many weeks, and by far the most common activity in these reading classes was cutting out and pasting. How much learning to read is there in such doing? Yes, the students were engaged, reported they were happy and liked reading, but the evidence of reading progress was minimal.

Berry also showed that teachers were more able to estimate the behavioral engagement in the tasks (the doing) but not necessarily the students’ emotional or cognitive motivations for investing (cf., Lee & Reeve, 2012). They were more accurate at identifying the negative emotions of their students (e.g., distress, frustration) and compliance but not the more positive motivation dimensions, such as confidence, enthusiasm, or preference for challenging tasks. Further, teachers often confuse engagement with motivation, assuming that if students are compliant or even engaged in the classroom tasks, then ergo, students are motivated and learning – although they may be engaged so as to demonstrate compliance and avoid censure. Teachers, thus, see their role as monitoring, diagnosing, and intervening for student disengagement, maximizing participation in the tasks, completing the work, paying attention, and responding to the teacher’s questions.

Berry (2023) noted the following:

to a lesser extent, teachers may attempt to promote a level of investment in the lesson by appealing to student interests and encouraging students to share their ideas or opinions within the lesson, but the main focus remains at the level of participating in the lesson events as designed by the teacher.

Berry found support for Kennedy’s (2016) claim that the task of ensuring all students are highly engaged in learning at all times might be seen as too difficult by the teacher, leading the teacher to settle for a more achievable and more observable goal of compliant participation in classroom events.

Berry (2023) built a six-factor model of engagement: disrupting, avoiding, withdrawing, participating, investing, and driving (Figure 5.1). The aim is to move students’ engagement from participating to driving and to be more cognizant of the various types of disengagement, all having a negative impact on the progress to achievement. Investing and striving are more likely to follow successful learning; thus, learning begets engagement more than engagement begets learning.

Figure 5.1 Berry’s model of engagement for disrupting to driving.

*Kumar (2007) found three major dimensions of teachers’ actions that led to higher engagement: clarity, opportunity to learn, and feedback. Clarity included giving directions and explanations relating to lesson context (r = 0.73), reinforcing and encouraging the efforts of students to maintain involvement (r = 0.66), teaching methods appropriate for the objectives of the learner and the environment (r = 0.63), demonstrating an ability to conduct lessons using a variety of teaching methods (r = 0.66), and questioning clarity (r = 0.49). The opportunity included the following: provides learners with opportunities for participation (r = 0.62), uses instructional materials that provide students with appropriate opportunities to participate (r = 0.62), manages disruptive behaviors among students (r = 0.55), and promotes good interpersonal relationships (r = 0.49). Feedback included the following: provides feedback to students throughout the lesson (r = 0.56).

*Quinn (2017) demonstrated that better-quality teacher-student relations were associated with higher levels of psychological engagement, academic grades, and attendance, and lower levels of disruptive behaviors, suspension, and dropping out. He also showed that when relationships declined over the years of schooling, there was a commensurate decline in psychological engagement, attendance, and behavioral compliance.

Perseverance and confidence: self-efficacy

The earlier analysis of growth mindsets, mindfulness, and grit should not lead to viewing them as not critical. On the contrary, a common denominator of all three is the sense of confidence that one can successfully engage, persist, and meet the success criteria of the task. Therefore, when these three influences are cast in terms of developing self-efficacy, it is much easier for the teachers to develop programs and techniques to engender confidence to tackle challenging learning tasks and much easier to monitor their impact on students’ confidence to tackle and persevere toward the success criteria.

Self-efficacy has one of the largest effects on learning of all will attributes. Self-efficacy is having the confidence that one will triumph, one can do the task, one knows how to ask for help, or one knows what to do when one does not know what to do. Self-efficacy relates to judgments of our capabilities to organize and execute courses of action required to attain success criteria (Bandura, 1986). It leads to a willingness to exert effort and persist toward the goal, and is about whether a student believes they can accomplish the task. In many senses, it is the inverse of anxiety, which very much involves our belief that we cannot do the task, our fear of failure, and our questioning why we should invest effort and suffer negative consequences. Self-efficacy relates to our skills for performing the task and our confidence in the likelihood of completing the task to a sufficient level.

Perseverance and confidence: positive self-concept

Self-concept and self-efficacy are related, and some argue that self-concept and generalized self-efficacy measure similar constructs. Self-efficacy relates more to confidence to engage, enact, and be successful in a task. In contrast, self-concept is broader and refers to cognitive appraisals, expressed in terms such as prescriptions, expectations, and/or descriptions that we attribute to ourselves (Hattie, 1992).

Self-concept can be seen as a rope: that is, self-concept consists of many fibers or dimensions that intertwine and overlap rather than having any one concept of self overpowering all (Hattie, 2008b). The first premise of the rope model relates to the maxim by Wittgenstein (1958) that the strength in the rope “lies not in one fiber running throughout its length, but in the overlapping of many fibers” (Section 67). The second premise is that various strands of self-concept serve as primary motives that lead to the invoking of situation-specific orientations of self, such as self-efficacy, anxiety, performance, or learning orientations. In turn, these situation-specific orientations lead us to choose various self-strategies to serve the self-motivations and, thus, bring meaning, predictability to our sense of self-concept and self-esteem. Hence, the rope model works as a series of inter-weaving threads to form fibers and thence strands to make the rope – or the sense of continuity we have of ourselves. Primary to this rope model is that self-concept relates more to how we select and interpret the information we receive and present.

The argument here is that a sense of confidence to succeed in learning is a most powerful precursor and outcome of schooling. It is particularly powerful in the face of adversity – when things do not go right or errors are made. Having high levels of confidence (the specific wow factor in growth and grit claims, i.e., can do, want to do) can assist in getting through many roadblocks. Such high self-concept leads to students invoking learning rather than performance strategies, accepting rather than discounting feedback, benchmarking to challenging rather than easy goals, comparing themselves to subject criteria rather than to other students, possessing high rather than low efficacy in learning, and invoking self-regulation and personal control rather than learned helplessness in the academic situation. The key dispositional factors that relate to achievement are the willingness to invest in learning, gain a reputation as a learner, and show openness to experiences.

Perseverance and confidence: self-control

Self-control relates to the capacity to alter or override dominant response tendencies and to regulate behavior, thoughts, and emotions (Bandura, 1986). The claim is that those students with high self-control are better able to control their thoughts, regulate their emotions, and expend effort more successfully when they choose. Conversely, those with low self-control are more impulsive, more distracted, and less likely to inhibit distractions. Self-control requires effort or willpower to seek longer-term goals, resistance to temptations to switch to other tasks, persistence, emotion regulation, choice-making, and optimization of the resources a student has for the learning task (Baumeister et al., 2007). Many of the actions are similar to the attributes of the meta-cognition claims (see Chapter 11). It is not, however, providing students with control over their learning, which has very low effects, as students are novices and often do not know the next best learning actions. Instead, they need an expert and know when and how to call upon experts to help make learning steps beyond the students’ current capabilities (see Vygotsky’s 2012, zone of proximal development).

Student personality

*O’Connor and Paunonen (2007) provide three major reasons why personality variables could have an effect on achievement. First, behavioral tendencies reflected in personality traits can affect certain habits that influence academic achievement (e.g., perseverance, conscientiousness, talkativeness). Second, whereas cognitive ability reflects what an individual can do, personality traits reflect what an individual will do. Third, as students get older, personality and cognitive proficiency can combine to predict subsequent performance better.

The most well-researched model of personality is the Big Five (Table 5.4): agreeableness, conscientiousness, extraversion, openness to experience, and neuroticism (McCrae & Costa, 1997). Across ten metas that have correlated agreeableness or extraversion with achievement, the overall relations are quite small. The largest, however, conscientiousness is quite marked – and from the discussion earlier, it was noted that this is a major (if not the most important) part of grit. Conscientious students are thought to be more motivated to perform well academically and are typically more organized, hard-working, diligent, self-disciplined, and achievement-oriented.

Table 5.4 Correlations between the Big Five personality variables and achievement

Poropat (2011)
Poropat (2014)
Vedel
O’Connor
Ibrahim
Trapman
Cuadrado
Boyd
Gatka
McBee
Average

Agreeableness
0.08
0.07
0.09
0.06
0.10
0.08
0.10

0.08
0.08

Conscientiousness
0.43
0.19
0.28
0.24
0.26
0.22
0.20

0.23
0.26

Extraversion
0.10
−0.01
0.00
−0.05
0.00
0.01
−0.01
−0.05

−0.03
0.00

Openness
0.37
0.10
0.07
0.06
0.06
0.08
0.05

0.02
0.08
0.10

Neuroticism
−0.15
−0.01
−0.01
−0.03
−0.02
−0.04
0.01

−0.03
−0.04

The core message is that teachers must consider developing conscientiousness by attending to the components, such as persistence, organization, carefulness, dependability, and rigor. For example, structuring lessons to ensure that students know the success criteria and know what the students bring to the lessons in prior knowledge; teaching them how to apply optimal learning strategies, engage in error detection, and then enjoy delving into the deeper relational and conceptual understandings built on precious knowledge about the topic. In such lessons, there can be higher levels of industriousness or perseverance; the student can develop a sense of order and responsibility for their learning (key to developing self-regulation), and have pride in the success and the rigor of their attending to this success (without overly stressing about being perfect all the time).

Personality factors, such as openness and developing conscientiousness to invest in learning, are a major theme of the synthesis of meta-analyses outlined in this sequel. Therefore, teachers aim to enhance the willingness to gain self-confidence and a reputation of being a learner, and such skills can be taught. Such reputation enhancement that “I am a good learner” is particularly powerful during the early adolescent years, and it is during these years that the decision to continue in education or not is often made (see Carroll et al., 2009).

Perfectionism

Perfectionism involves students setting excessively high and often unrealistic standards of performance and compulsive and unremitting efforts to attain these standards, and measuring self-worth largely or entirely in terms of accomplishment, resulting in self-criticism when standards are not attained (Frost & Marten, 1990). There are two major dimensions, and the relations to achievement are opposite for the two. Perfectionistic strivings that capture personal standards and a self-oriented striving for perfection have positive relations (r = 0.25), and perfectionistic concerns that capture concern over mistakes, feelings of a discrepancy between one’s expectations and performance, and negative reactions to imperfection (r = −0.10 to −0.20). Thus, aiming to foster success without a rigid pursuit of perfection, seeing mistakes as opportunities to attain the strivings, and seeking help, not internalizing anxieties, need to be fostered. Thus, aiming high with successful failing.

Emotions

Emotions are most involved in the achievement process – both as precursors and outcomes for learning. They can be motivators (joy of learning) or inhibitors (anxiety, fear of failure). Pekrun et al. (2017) have developed a most comprehensive model of emotions and achievement. They begins by noting that it is the student’s appraisals that matter most – the ways they interpret, react, and explain what is happening are more critical than the actual incidences that led to these appraisals (as the same trigger can lead to different appraisals) – so creating safe environments where we can hear how students are appraising situations is a first step to reducing the negative and maximizing the positive appraisals. These appraisals very much influence the expectations, motivation, use of strategies, task choices, and investment the student is prepared to make the learning.

Students begin with their appraisal of the value of a given outcome – if it is valued, then they form expectations about whether the situation will produce an outcome (e.g., “Can his teacher help me learn”), an action-outcome expectation of belief about the consequences other own actions (“Can I do this alone, will feedback and help be available), or an action-control expectations of their belief whether they have the confidence to undertake the action (self-efficacy). The appraised value can relate to the intrinsic value of the enjoyment the student derives from performing an activity or attitude to the task, attainment value or the personal importance of succeeding in a task, utility value or the perceived usefulness of engagement and achievement, and cost of the perceived negative consequences of engaging in a task (anxiety, fear of failure, effort required, and opportunity costs).

Pekrun et al. (2017) group emotions into positive and negative – and activating and deactivating:

Positive activating – enjoyment, hope, pride, curiosity

Positive deactivating – relief, relaxation

Negative activating – anger, anxiety, frustration, shame

Negative deactivating – boredom, hopelessness

Emotions can be triggered by cognitive issues, such as surprise about a new task, or curiosity, confusion, delight, and frustration when a problem is encountered. In addition, some content presented in the lesson and learning materials can induce emotions (sadness, empathy about characters in a novel, disgust with harming animals, etc.), and classes can be places to invoke social emotions, such as sympathy, compassion, admiration, contempt, envy, anger, or social anxiety. A major claim in this sequel is that schools and teachers (and researchers) may need to be more explicit that such emotions and dispositions to learning should be key performance indicators of the outcomes of schooling – along with progress and achievement.

Emotions: emotional intelligence

Emotional intelligence refers to skills in regulation, management, and expression of emotion-related information, as well as self-control and self-motivation. A very broad grouping but the intent is for students to thus manage their emotions and develop cognitive coping strategies to reduce stress, communicate effectively, engage in social sensitivity with others, and reduce conflict. That there is a common dimension (the underlying claim for saying there is an intelligence of emotions) is disputed, but as can be seen from other sections of this chapter, there are particular components that seem important in developing students’ achievement.

*MacCann et al. (2019) found the relations of emotional intelligence higher in the humanities than sciences (r = 0.38 vs. 0.21) and argued this was because the humanities often demand relevance of emotional knowledge to understand much literature, history, civics, drama, and others.

Content knowledge of emotion words, as well as the causes and consequences of emotions, appear highly relevant for understanding character motivations in literature as well as other academic subject matter relating to people and how they shape societies, countries and history.

(p. 21)

But then some of us can become quite emotional when there is low variance, when a factor analysis model converges, and when making discoveries about the distributions of residuals.

Emotions: well-being

“Well-being” is another very broad term that refers to how students feel and think about their lives, especially the presence of pleasant and positive affect (e.g., happiness) and the absence of unpleasant affect (e.g., depression). *Chu et al. (2010) found r = 0.17 between perceived social support and overall well-being; and similar for support from family (0.19) and teacher/school (0.21); and lesser when support was from friends (0.10). The correlation between well-being and various outcomes was also similar: achievement (r = 0.11), self-concept (r = 0.27), social adjustment (r = 0.21), coping skills (r = 0.10), and conduct (r = 0.14). *Bücker et al. (2018) reported an overall relationship with academic achievement of 0.16, and this relation was similar in language (0.12) and STEM (0.13), and across various measurement type (grades/GPE, 0.19; self-reported grades, 0.15; class achievement tests, 0.15). Maybe generalized well-being programs are less effective than attending to more specific programs intended to support mental health and the promotion of positive well-being, functioning well, and flourishing.

Positive activating: enjoyment

There is an argument that enjoyment can reduce performance on tasks that require sustained attention (Meinhardt & Pekrun, 2003) and can lead to tackling easier rather than more challenging tasks. Further, when not successfully completing tasks, focusing on enjoyment can lead to more disengagement (Carver & Scheier, 2001). Others claim that enjoyment can lead to greater use of planning and monitoring (King & Areepattamannil, 2014). *Camacho-Morles (2020) found an overall correlation between enjoyment and achievement of r = 0.27, higher in secondary than primary students (0.34 vs. 0.21), and similar in math and science (0.29 vs. 0.21).

Positive activating: hope

Hope has been a political slogan, has been a weak form of implementation, and typically is defined as an optimistic state of mind and expectations of positive outcomes. *Yarcheski and Mahon (2016) found r = −0.02 and *Gallagher et al. (2017), r = 0.13, between students’ feelings of hope and academic achievement. Not surprisingly, hope was positively related to self-worth (r = 0.56), positive affect (r = 0.47), coping (r = 0.35), optimism/satisfaction (r = 0.64), and goal-directed thinking (r = 0.43); and negatively to depression (r = −0.47) and negative affect (r = −0.32). This pattern was similar in elementary, middle school, high school, and tertiary contexts (*Gallagher et al., 2017). Much more than hope seems necessary to enhance one’s achievement.

Positive activating: curiosity

Curiosity relates to the urge to explain the unexpected, resolve uncertainty, or know more. Engel (2011) documented the low rate of curiosity in classrooms primarily because of the limited number of student questions and restrictive time to follow up and find something new. It is often claimed to be a valued attribute in students and correlates highly with creativity outcomes (r = 0.41, *Schutte & Maloufe, 2019). Perhaps we need more why questions and, thus, learn to capitalize on students’ curiosity.

Positive activating: happiness

There are substantive relations between happiness and achievement, but the direction of this relationship is far from clear. When we are happy, we are more open to experiences and more likely to take on challenges, and as Fredrickson (2001) argued, we are more ideally situated to “broaden and build.” As was argued for engagement, experiencing success in learning is the greatest predictor of engagement, whereas too many argue that tasks need to be engaging and the students need to be happy to entice students to learn.

*Lyubomirsky et al. (2005) proposed that the success of happy people rests on two main factors.

First, because happy people experience frequent positive moods, they are more likely to work actively toward new goals while experiencing those moods. Second, happy people are in possession of past skills and resources, which they have built over time during previous pleasant moods.

(p. 804)

The average effect (d = 0.54) indicates that chronically happy people and those in pleasant moods are more likely to be creative and efficient problem-solvers. The longitudinal studies in their meta-analysis showed that happiness is more likely to precede important outcomes and indicators of thriving, including fulfilling and productive work (mean r = 0.24), and satisfying relationships (r = 0.21). But there can be costs to happiness. When happy students fail, this can be quite a negative downer, leading to not engaging with similarly challenging problems; they can rely more on heuristic shortcuts; they can be judged as shallow, complacent, or not trying hard enough; and happiness in the context of lack of engagement and effort is often judged negatively. When happiness follows learning, we need to embrace this relation, but when it precedes learning, we need to be attentive to potential costs. When students are neutral on happiness, this is not necessarily a barrier or enabler, but the opposite of happiness is no friend of learning.

Positive deactivating: relaxation

There are many relaxing methods, such as progressive muscle relaxation, deep breathing exercises, and visualization. *Moon et al. (1985) found an effect of d = 0.16 on achievement between students who received some relaxation techniques and controls that did not. They recommended that “a future meta-analysis might examine the relationship between relaxation and anxiety within educational contexts” (p. 7), but I could not locate one 35+ years later.

Negative activating: anxiety

Spielberger (1972, p. 1) described anxiety as the outcome of a “chain reaction consisting of a stressor, a perception of threat, a state reaction, cognitive reappraisal, and coping.” The meta-analysis research in education often focuses on two prominent forms of anxiety: test anxiety (d = −0.40) and mathematics anxiety (d = −0.52). The subject of mathematics, in particular, promotes expressions of anxiety that take many forms, such as tension and dislike (attitudinal features); worry, helplessness, and mental disorganization (cognitive features); and fear (emotional feature). The consequences of anxiety include avoidance of courses and an inability to achieve in the subject. In addition, teaching mathematics without being aware and having an intent to teach confidence skills can reinforce anxiety.

*Seipp (1991) reported an effect size of −0.43 (r = −0.21) between anxiety and performance, and noted it was similar for males and females. *Barroso et al. (2020) found no difference in the relations between anxiety and achievement among males and females across ethnicities or contents. The correlations were higher in grades 1 and 2 (r = −0.40) compared to grades 3–5 (r = 0.00) and increased in magnitude in high school (r = −0.12). They also noted similarities across the domains of math content (number systems, −0.09; basic number knowledge, −0.16; whole-number calculations, −23; word problem-solving, −0.27; fractions, decimals, and percentages, −0.23; and statistic knowledge, − 0.23) and across the dimensions of anxiety (worry, −0.37; emotionality, −0.35; both worry and emotionality, −0.24; evaluation anxiety, −0.21; and math learning anxiety, −0.28). *Hembree (1988) noted that students with high or low (as opposed to middle) self-concept tended to be more test-anxious, and there were direct relationships to students’ fears of negative evaluation, defensiveness, and dislike of tests.

We can think of anxiety as the reciprocal of confidence, and the strategies to boost confidence can mitigate the negative effects of anxiety. Anxiety can be induced by the subject and by the teacher, but also, it is the teacher who can instill confidence in the subject by listening to what levels of anxiety or confidence they bring to the class, being clear about the current levels of skills, having the appropriately challenging success criteria, and being transparent to the students about this diagnosis, success criteria, and progress.

Negative activating: depression

*Buehner (2017) found a close to zero correlation (r = −0.08) between achievement at baseline and subsequent depression, and this relation was not moderated by age, time between measurement, or form of assessment (self-report, −0.01; report from others, 0.07). *Huang et al. (2014) reported r = −0.15 between achievement and subsequent depression, and the correlation decreased as the time between assessments increased. *Chan et al. (2008) also investigated the longitudinal relation between achievement and later depression (r = −0.25); and also cross-sectional (−0.20), and depression vs. no depression (−0.28).

Negative activating: anger

Anger in learning can occur when students face difficulties or blocks moving toward the desired goal and when students see the demands or task as unfair. Such anger decreases intrinsic motivation. The overall effect is a substantive, d = −0.82.

Negative activating: frustration

Frustration can be seen as a blend of anger and disappointment (*Camacho-Morles, 2020). Some have argued that frustration can be favorable as it may elicit increased effort toward task completion (Carver & Scheier, 2001). This is reflected in d = −0.04, suggesting a mixture of positive and negative effects.

Negative activating: aggression and violence

The United Nations Development Goals call for providing safe, nonviolent, inclusive, and effective learning environments for all. Violence can be against students, by students toward others, and to which children are exposed. Violence includes physical, sexual, emotional, neglect, bullying, and community violence. *Frey et al. (2018) found negative effects (d = −0.10) for all forms of violence, and such violence led to a two-fold increase in the likelihood of school dropout. *Savage et al. (2017) found similar negative relations (d = −0.29) for males and females, children and adolescents, even when controlling for parent education and SES.

Negative deactivating: boredom

Boredom in class is a negative and deactivating emotion, it is the antithesis of high self-efficacy, and its relation to achievement is markedly negative (−0.46). *Tze et al. (2015) found similar high negative correlations in terms of the effects of boredom on academic motivation (r = −0.40), study strategies/behaviors (−0.35), and achievement (−0.16), and this was similar across age groups, and when students are in-class (−0.35) or while studying (−0.25). *Trimble et al. (2015) also found that boredom on an initial task significantly negatively predicted performance and persistence on a subsequent task, even when controlling for initial task performance and time spent on the initial task. The effects of boredom on subsequent tasks were stronger for higher-performing students.

Boredom indicates that the task lacks value and meaning for the student. This should convey information to the teacher to increase the meaning value to the student, increase the confidence to engage in the task successfully, regulate cognitive resources by being more explicit about the why and success criteria, or switch tasks (Westgate & Steidle, 2020).

Cognitive dispositions: morning vs. evening

Students often have preferences to learn in the morning (larks) or evening (owls). However, the two meta-analyses showed that morning investment was more successful than evening (e.g., *Preckel et al., 2011; r = 0.16 vs. −0.14), regardless of preference. Perhaps this difference is related to sleep and tiredness, but the effects are sufficiently small not to be of major concern. Fletcher and Hattie (2011) also showed that larks performed better in the afternoon and owls in the morning, despite the students’ avowed preferences.

Cognitive dispositions: procrastination

Five years ago, my editor, Bruce, reminded me that VL1 was about to be 10 years old and it was time for an update. This was a big task, and I was brilliant in coming up with reasons not to do this work. But I said, even my co-workers get upset when an influence changes nary a second-decimal point and think something must be wrong, But I said all I wanted to say in 2008, and it was time for someone else to update; but I need to wait to have substantially more metas to add to be worth it; but a second edition will just add more studies and be boring; but I would prefer to delve deeper into the defense of the big picture; but the underlying story has not substantially changed; but I have written many other updates in other sources. I am the master of procrastination. So are many students, as about one-third of their day is engaged in such dilly-dallying (Pychyl et al., 2000).

*Steel (2007) reported correlations of procrastination with finding tasks aversive was 0.40; conscientiousness, −0.62; impulsiveness, 0.41; self-control, −0.58; self-efficacy, −0.38; depression, 0.28; perfectionism, −0.03; fear of failure, 0.18; intelligence, 0.03; and subsequent achievement, −0.19 (−0.16 with GPA, −0.17 with final exams, −0.21 with assignments). *Akpur (2020) found much larger negative effects (r = −0.61) from 22 studies specifically on achievement and, thus, concluded that procrastination is generally detrimental and never helpful. He recommended that teachers need to set realistic goals, break down tasks into certain small and manageable steps, give convenient rewards, and eliminate or remove the distractions.

The thrill: motivation

One of my favorite authors is the philosopher R.S. Peters – I met him as a graduate student, and at that time, he was dominant in education and, thus, permeated many of my courses. He noted that to probe one’s motives is almost an occupational malaise among psychologists, and he was derisive of those who considered motives as reasons for claiming something happened (Peters, 1960). Instead, he claimed that we humans are moving anyway, so any reason why we are pushed or pulled misses this continuing striving. It is not why a person is pushed or pulled to do x or y but why we chose x over y (why choose to complete the task or why not be distracted). Once we make this choice, this leads to directedness to some outcome and, thus, can invoke persistence or focus/mindfulness on a task or activity. This directedness, claims Peters (rather than any initiating conditions), is the crucial aspect when we talk about assigning a motive (Table 5.5). So what is the striving when a student walks into a classroom and begins or sustains focus during a lesson; why do they engage in learning rather than turning off or become naughty, distracted, or attentive?

Table 5.5 Summary statistics for the thrill-related student variables

Thrill
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Motivation
23
1,810
26,210,109
6,274
0.39
0.06
5

Surface motivation and approach
4
447
132,854
452
0.00
0.08
4

Deep motivation and approach
3
165
15,186
170
0.58
0.08
2

Achieving motivation and approach
4
252
23,194
257
0.44
0.02
3

Mastery goals
8
655
133,634
898
0.20
0.09
4

Performance goals
9
625
56,513
450
0.04
0.03
4

Attitude toward content domains
12
751
2,183,685
1,394
0.48
0.11
5

Total/average
63
4,705
28,755,174
9,895
0.30
0.04
4

Students come into classes with rich motivational resources – they just may not be consistent with the teacher’s goals (Reeve et al., 2023). To make claims such as “they are not motivated,” “they don’t do what I ask them to do,” or “I’ll reward (or punish them) if they (do not) complete these tasks” shows a remarkable misunderstanding of the fundamental motivation question: Why should they do this rather than that? It is less about teachers adopting controlling motivating messages and more about understanding current motivations (the student’s needs, wants, and interests) and then building their instruction to work from these resources, and maybe even to enhance or change the nature of how students see their wants, needs, and interests. The other implication is that motivation does not always precede achievement (students engage in learning because of prior interest) but more often follows achievement (students get turned on to a topic as they begin to accrue knowledge, understanding, and skills). As is known from video games, the opportunity to learn and master can be a powerful reward. Our motivations when choosing x over y is also related to the student’s emotions – particularly, their sense of efficacy, task value, and expectations. As noted earlier, student engagement is tied to this notion of driving and striving, and is much more than participating and doing what we are told.

A lot of current literature focuses on striving to goals usually set by teachers as if the goals are necessarily all agreed upon, shared, and desired by students. However, these goals can invoke strategies and motivations toward mastery, performance, and/or social goals:

Mastery-approach goals focus on the development of competence and task mastery.

Mastery-avoidance goals focus on avoiding tasks (ties in with procrastination).

Performance-approach goals focus on attaining competence relative to others.

Performance-avoidance goals focus on avoiding incompetence relative to others.

Social-approach goals focus on performing well in front of friends or family.

Social-avoidance goals focus on avoiding conflict in relationships and fearing rejection by others.

The premise is that motivation is derived from the perception that one’s actions change the prospect of goal attainment (Ballard et al., 2022). Motivation can increase as the distance to the goal decreases, and the goal is seen as challenging but attainable.

Goals that are too easy are poor motivators because the chance of goal attainment is already so high that further efforts do little to raise these chances. Goals that are too difficult fail to inspire because the chances of success are so low that effort expenditure will not make the goal any more achievable.

(p. 23)

The typical claims are that students with mastery goals are more intrinsically motivated to learn, believe competence develops over time through practice and effort, choose tasks that maximize the opportunities for their learning, seek feedback that helps them master, and so on. Students with performance goals are more likely to be extrinsically motivated, believe competence is a function of stable factors (ability is fixed), avoid tasks that make them look incompetent, and seek feedback that praises them. The evidence, however, for those motivated for mastery (0.13) and or performance (0.03) are both very low and seriously questions the value of this distinction.

Alternative models to the mastery, performance, and social model are outlined in a special issue of Contemporary Education Review. Five of the leading motivational researchers were invited to outline their model of motivation:

Attribution theory (Graham, 2020), whereby the student seeks why reasons for engaging in a task, such as their ability, effort, or affiliation, which lead to three causal dimensions: locus (within or outside the person), stability (enduring over time or not), and controllability (subject to volitional influence).

Task expectancy (Eccles & Wigfield, 2020), whereby motivation is a function of the student’s self-concepts of ability, task, and personal values, goals, and weighed by perceived costs (effort, opportunity, emotional) and benefits to the individual.

Self-efficacy (Schunk & DiBenedetto, 2020), whereby motivation is a function of agency, monitoring progress toward a goal, and the individual’s sense of perceived capabilities to learn and perform actions.

Sociocognitive (Urdan & Kaplan, 2020), whereby the student’s perceptions, beliefs, and goals (such as achievement, task, learning, performance, appearance, avoidance, and approach) guide motivation decisions.

Self-determination (Ryan & Deci, 2020) distinguishes between intrinsic motivation (activities done for their own sake) and extrinsic motivation concerns (activities done for instrumental reasons), and students are motivated toward a sense of competence, autonomy, and relatedness.

We (Hattie et al., 2020) merged these models around four major dimensions: self, task, goals, and costs and benefits. The self or person dimension includes self aspects, such as expectations of success, self-efficacy/confidence, or the person’s perceived capabilities to learn and perform actions at designed levels. The tasks can be valued or perceived as important or useful to engage and learn, particularly when people perceive their goals to be aligned with the important outcomes. The costs and benefits of doing this rather than that, such as effort costs (the perception of how much effort would need to be exerted to complete a task and whether it is worth doing so), opportunity costs (the extent to which doing one task takes away from one’s ability or time to do other valued tasks), and emotional costs (the emotional or psychological costs of pursuing the task, particularly anticipated anxiety, and the emotional and social costs of failure). The benefits include external rewards, intrinsic satisfaction, identification of core values; compliance (and thus, often reduced or absent negative consequences), autonomy or agency, and a sense of competence. There is a need for more competitive comparisons to bring more unity to these models, and particularly asking the question as to developing models that maximize the information relative to outcomes such as achievement.

This topic of how teachers motivate students to engage in precious knowledge and understanding may be the greatest educational challenge of our times. The Jenkin’s curve (Jenkins, 2015) is powerful and concerning: 95% of 5-year-old children want to come to school and are motivated to learn what teachers want them to learn, which dips to 37% by grade 9 (and slightly increases after this). Thus, by the end of elementary school, about 60% of students do not see their classrooms as motivating and worth coming to learn – they come for friendships, and many would rather be elsewhere. These less-engaged students are not only the naughty kids but also the avoiding and withdrawing students (Berry, 2023) – and they all have similar profiles of low learning gains. This makes a large cohort of students not motivated to engage in schoolwork.

*Jansen et al. (2021) synthesized 125 meta-analyses relating motivation to school achievement based on 488 effects and 25 million students. The mean overall effect relating student motivation to achievement was d = 0.42 (r = 0.21). The effects from focusing on internal motivation were twice than those relating to external motivation (0.53 vs. 0.26); similar in verbal (0.54) and STEM domains (0.48), and girls had higher levels of motivation in the verbal domain than boys (0.34) but very little differences in the STEM subjects (0.06). Their most important findings were that the socioemotional variables, including self-beliefs (e.g., academic self-concept), emotions (like anxiety), and social variables (sense of belonging in school, students’ relatedness with their peers or their environment, and perceived parent support) had higher relations to academic motivation. This indicates how critical students’ interpretations of their experiences in school are to their motivation rather than the experiences (like grades and test scores) themselves. We ignore student self-beliefs at our peril.

They argued that to develop positive relationships with their students, teachers need skills of listening, empathy, caring, and positive regard for others. What matters is students’ perceptions of the impact of teachers’ instructional practices or techniques (including things such as their knowledge of the subject and enthusiasm for it; their planning, organizing of their classes; and the kinds of feedback they provide students) – all can impact students’ academic motivation and whether teachers foreground the importance of learning and developing competencies. This begs attention to what Anderman and Anderman (2020) noted: support for student’s motivation starts with first understanding why teachers are motivated, and the argument here is that such motivation needs to be premised on maximizing their impact on the learning lives of students much more than covering the curriculum or ensuring high levels of management and engagement (striving and driving).

Dörnyei (2001) noted that motivation is highest when students are competent, have sufficient autonomy, can set worthwhile goals, get feedback, and are affirmed by others. He also challenged educators to seriously consider student demotivation caused by, for example, public humiliation, devastating test results, or conflicts with teachers or peers. For many, demotivation has more impact than motivation. Such demotivation can directly affect commitment to learning goals, turn off the wish for and power of feedback, and decrease involvement. It can take less effort by a teacher to demotivate students compared to the often-greater effort required to motivate them – to turn students onto learning.

Similarly, *Schwinger et al. (2021) noted that in the expectation of failure, students can use self-handicapping as a successful strategy to avoid threats to their self-esteem (e.g., I failed because I didn’t sleep well last night). Whitely and Frieze (1985) found that unsuccessful students (particularly adolescents) made stronger attributions to the task, whereas successful students make more attributions to their ability and effort (cf., *Findley & Cooper, 1983; *Kalechstein & Nowicki, 1997; *Ross, 1988).

Surface, deep, and strategic motivation

Biggs (1985) developed three forms of motivation. First, surface motives (perhaps akin to performance motivation) is when the student decides to undertake the task for utilitarian reasons, to reproduce knowledge, to attain minimum outcomes, to learn the content sufficiently, or to invoke more rote memory for short-term gains, such as passing the test or completing the work. Second, deep motivation (akin to mastery) occurs when the aim actualizes interaction and competence in the subject, where the student is motivated to learn more and deeply about the subject, when the student aims to understand what is to be learned through interrelating ideas and reading and studying widely. However, it is not possible in many learning situations to be deep all the time (time does not permit, and teachers often value surface (knowing lots) despite what they say).

The third form, strategic motivation, invokes the proficiency of the student to read the task or assessment as to the surface or deep requirements so that they then adapt to the task at a surface or deep level, sometimes intermixing when the task demands. Thus, what matters is the appraisal students make to understand the task’s demands (e.g., whether they believe it invokes mastery or performance, surface or deep; and what it means to be good at this task). Does the task require surface-level content maximization (despite what the teachers say) or deeper, more relational and problem-solving skills? This strategic or achieving motivation is at the core of self-regulation, and it is the skill that leads to students being motivated to master, perform, or be social in their goals (Table 5.6).

Table 5.6 Summary statistics from the various motivation strategies

Motivation strategy
No. metas.
No. studies
No. effects
d

Surface
4
447
452
−0.14

Deep
3
165
170
0.57

Achieving
4
252
257
0.38

Mastery
7
433
502
0.13

Performance
9
529
478
0.03

This means teachers need to understand how students consider what the success criteria are, ensure the tasks or tests related to the lessons are intentionally aligned (i.e., the tasks are appropriately a mix of surface and deep tasks proportional to the surface and depth of the assessments/grades and reflected in the success criteria and teaching). The fundamental premise of intentional alignment (Chapters 3, 4, & 11) is that the teaching methods and assessments are aligned in supporting appropriate student learning – from the student’s perspective. Later in this book, we will see the power of developing and sharing success criteria, scoring rubrics, being clear about what the task requirements are, developing cognitive task analysis to ensure the tasks are aligned with the intentions of the success criteria, encouraging teachers to continually seek feedback about their impact on students, continually asking students how they interpret feedback about where best next to move in their learning. The relations between deep and achieving motivations and achievement are much higher than surface motivations.

Attitude toward content domains

Attitude to school involves many dimensions, such as positive or negative feelings, the tendency to engage in or avoid school activities, a belief that one is good at schoolwork, and a belief that school is useful. The correlations are, overall, substantive (d = 0.27 to 0.87) such that teachers should be concerned about developing positive attitudes before and as a consequence of studying school curricula. This relates to teachers developing a sense of challenge, developing confidence, and reducing stereotypes (such as boys are better at math and girls at reading).

The physical

Many physical attributes of students can impact their learning at school (Table 5.7). Of course, many are out of control of the student, home, teacher, or school; and care is needed to recognize that some of these factors (e.g., illness) are more related to the opportunity to learn than a function of the proficiencies of the student. In all cases, early diagnosis and excellent interventions can often make a difference in the impact of these influences.

Table 5.7 Summary statistics for the physical-related student variables

Physical
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Preterm/low birth weight vs. full-term
13
352
1,259,247
985
−0.59
0.06
5

Breastfeeding
1
12
1,104
12
−0.04
0.00
1

Illness
6
913
125,334
982
−0.51
0.02
4

Physical syndromes
15
302
2,042,506
450
−0.42
0.09
4

Maltreated children
1
32
2,945
72
−0.63
0.08
1

Autism
3
76
5,169
186
0.08
0.14
3

Lack of sleep
4
107
75,483
107
0.02
0.03
3

Sugar
1
23
1,791
93
−0.13
0.08
2

Fitness
4
279
182,106
1,939
0.16
0.03
4

Exercise
4
279
182,106
1,939
0.16
0.03
-

Gender (male-female)
34
2,884
23,957,726
5,478
0.00
0.04
5

Positive ethnicity self-identity
6
163
30,833
203
0.17
0.01
3

Dialect use
1
19
1,947
19
−0.29
0.00
1

Stereotype threat
4
237
21,890
416
−0.19
0.11
3

Total/average
93
5,399
27,708,083
10,942
−0.18
0.05
3

Full-term vs. preterm/low birth weight

Preterm babies weighing less than 1,500g have the highest likelihood of remaining behind their peers in many attributes, such as height, weight, executive processing, and achievement. When combined with one or more neonatal morbidities and a long neonatal intensive care unit, this can stress an already immature system (Sullivan et al., 2008). The 13 meta-analyses based on over 1 million individuals have an overall effect of −0.56 between full-term and preterm babies – with this difference evident throughout their school years. For example, *Maitra et al. (2014, p. 145) concluded as follows:

adolescents who were born preterm, compared with those born full-term, have shown significant deficits in executive functioning; tasks related to verbal fluency, inhibition, cognitive flexibility, planning and organization, working memory, and verbal and visuospatial memory showed the highest levels of impairment.

*Kovachy et al. (2014) showed that children born preterm had significantly lower scores than those born at term for decoding and reading comprehension. *McBryde et al. (2020) found the effects more pronounced in mathematics, particularly higher-order skills of applied problems more than mathematical knowledge and calculation. They suggest this is likely to be related to issues with working memory, as mathematical problem-solving involves processing the problem while simultaneously retrieving contextual information from long-term memory. *Der Spek et al. (2011) provided further support arguing that preterm-born children score significantly lower compared with term-born children on simple, as well as on complex, language function tests throughout childhood. These differences were evident very early, and “for complex language function, group differences between both preterm- and term-born children significantly increased from 3 to 12 years of age.”

Breast feeding

Breast feeding has little or no relation to the intelligence in children (*Der et al., 2006). Many of the oft-claimed benefits are more because mothers who are more likely to breastfeed are more intelligent, come from more highly resourced families, and have higher levels of cognitive stimulation at home. So any effect is likely related more to the socioeconomic resources of the home than to breastfeeding or not.

Illness

Illness, in general, has a negative effect on achievement, and in many cases, this is a function of lesser opportunity to learn (absence) and not necessarily proficiency to learn. These illnesses relate to chronic illness (d = −0.20), diabetes (−0.17), iodine deficiencies (−0.49), asthma (−0.33), sickle cell disease (−0.25), deafness (−0.61), and those with ADHD are markedly different in achievement levels compared to those without ADHD (−0.90).

Physical syndromes

Students with a cleft lip/palate (d = −0.41), deafness/cochlear ear implants (−1.14), poor oral health (−0.23), sickle cell disease (−0.25), and brain injuries (−0.41) all perform, on average, lower than their peers. The variance, however, can be huge as a person is not identified by their illness. As will be shown in Chapter 12, we can learn a lot from the success of many intervention programs for these students that can benefit all students, as there is a focus on acceleration, strategies of learning, appropriate teaching of consolidation, and celebration of the joy of each student as the manifest success in progress to higher achievement.

Maltreated children

Maltreatment of children can result from sexual abuse, physical abuse, or neglect (*McGuire & Jackson, 2018). All have a similar high negative impact on achievement, but there was much variance in the size of effects, indicating a need for more systematic moderators. The effects are similarly highly negative for math (−0.46) and language arts (−0.64), and the various forms of maltreatment (multiple, −0.61; physical, −0.52; sexual, −0.66; neglect, −0.86).

Autism

*Brown et al. (2013) compared students with and without autism in reading comprehension. There was a large overall effect (d = −74), and specifically, the effects were similarly large for semantic knowledge (−0.48), low for decoding (−0.09), and much higher for texts that demand a good understanding of the social world (−1.36) compared to those that did not have autism. *Tiede and Walton (2019) reviewed various interventions, such as community-based and stakeholder-mediated models, which take place in homes and schools. The effects included social engagement (0.69), expressive language (0.32), receptive language (0.28), play (0.23), adaptive behavior (0.09), and attention (0.51). The most important conclusion was the major variability, suggesting other factors than autism impact the success of these programs.

Sleep

Major sleep deprivation has a negative effect on achievement. That the average effect of sleep is not a major factor in the various studies is likely because most students come to school with sufficient sleep for it not to be a problem. *Astill et al. (2012) analyzed 86 studies and reported sleep duration had a small effect on cognitive performance (r = 0.08), and this did not differ when relating to executive functioning (r = 0.07), tasks that address multiple cognitive domains (r = 0.10), with school performance (r = 0.09), or behavioral problems (r = 0.09). The effects are slightly (but trivially) higher for sleep-disordered breathing, such as habitual snoring, upper airways resistance syndrome, or obstructive sleep apnea (r = −0.15) (*Galland et al., 2015). The claims that schools should change their opening hours (e.g., start later for adolescents) make little difference to achievement outcomes, and often, when asked to choose, most adolescents prefer to come to school earlier so that they can enjoy long afternoons out of school (Biller et al., 2021).

Sugar

The common claim is that excess sugar in the diet leads to higher distractibility and, thus, lower achievement effects. *Wolraich et al. (2009) found a zero relation across all their outcomes and concluded that “the strong belief of parents may be due to expectancy and common association” (p. 1617). Enjoy the doughnut.

Exercise

There have been many meta-analyses on the relation of exercise to achievement, most showing very low relations. This should not be surprising, as many obese, physically disabled students can have high attainment, and very healthy individuals can have the opposite. To claim that a healthy body leads to improvements in achievement is as problematic as claiming that we can cure the obesity crisis by asking overweight people to learn more reading and mathematics. The issue should be that knowledge of health issues and enhancement of physical attributes are worthwhile outcomes in their own right, and therefore, we should make health and physical skills a core part of school learning.

Ethnicity

Having positive views of one’s ethnicity is also importantly related to achievement success (d = 0.32), an increase in positive developmental adjustment (d = 0.40), a decline in participation in delinquency (d = −0.23), and an improvement in sociability (d = 0.30). It certainly seems that maintaining a positive image of our cultural background is most worthwhile. Miller-Cotto and Byrnes (2016) cautioned about generalizing these positive but small effects to all ethnic groups, and Webber et al. (2013) have argued that much depends on the nature and extent that each student identifies with and is committed to their ethnic, cultural identities.

Dialect use

Speaking a nonstandard English dialect is one risk factor for reading difficul-ties in English. However, care is needed, as sometimes this use of dialect can be cofounded with students coming from lower-resourced families, immigrants, refugees, and others. *Gatlin and Wanzek (2015) found a −0.29 effect and concluded that the more dialect students use in language production, the lower their literacy scores tend to be. This was the case regardless of student SES background and grade level.

Stereotype threat

Attitudes and beliefs about the impact of race or ethnicity can impact learning. Stereotype threat occurs when there is a negative stereotype about a person’s group, and the student is concerned about being judged or treated negatively based on this stereotype. The claim is that these students face extra pressure than peers – to avoid confirming any stereotype about their group’s inferiority. Meta-analyses in many domains support the negative effects – for women in math (*Flore & Wicherts, 2015, d = −0.22; *Mitchell, 2014, −0.64; *Doyle & Voyer, 2016, −0.03), for African Americans and Latinos on intellectual tests (d = −0.50), and for age-based threats among the elderly (−0.28) (see *Spencer et al., 2016). *Nguyen and Ryan (2008) found that for women and minorities, subtle threat-activating cues produced the largest effect, followed by blatant and moderately explicit cues. The effects were highest on moderately more than higher math-capable females (−0.52 vs. −0.29), and least on low math-identified females (−0.11). Classes need to be inviting to all students, high expectations need to be for all, the message needs to be that performance in all subjects can be improved, and teachers need to evaluate their positive impact on all students – where all means all. There is no place for racism or stereotyping.

Gender differences

Among the most studied physical attributes on achievement is sex. The overall effect is very small (d = 0.07; in all cases, the direction has been changed to boys–girls). *Hyde (1981) has published the most extensive study on the gender effects across many dimensions (more than achievement). She included 124 studies and many millions of students. Across her four major outcomes, the differences slightly favored girls in communication, attention, effortful control, and inhibitory control; and boys in well-being, achievement, helping others, and aggression. Thus, girls display a greater ability to manage and regulate their attention and inhibit their impulses – skills most useful in classrooms. From this work, Hyde proposed the “gender similarity hypothesis,” claiming that males and females are more alike than different. The conclusions are that the variability among males can be large, and the variability among females can be large; the differences between the average male and female are small.

Conclusions

The fundamental claim in this book is that from birth, every child is a learner, is teachable, can grow, and can be taught to love learning.

A lot of learning is trial and error, failure is a constant, and students need to be taught that errors and failures need to be seen as opportunities for further learning, not sources of shame.

All educators and students need to learn coping strategies to deal with situations via emotional coping (crying, venting) and problem-based coping (restructuring what happened, trying different solutions). To do this, they need to develop a sense of persistence and know when to seek help, when to be conscientious, when to stop and learn from others, and when to learn how to seek, hear, interpret, and deal with feedback.

The frequency, breadth, and quality of language interaction between the child and another (adult, peer, sibling) truly matters.

The mantra should not be helping students realize their potential but working with students to exceed what they might think is their potential.

The art of teaching is knowing where each student is, where they need to be, and whether they have the confidence to bridge this gap.

The teacher’s aim is to understand and direct each student’s motivational resources to invest in schoolwork and to teach students to develop adaptive expertise to know when to focus on surface, when on deep, and when on transferring learning.

What matters most is how educators think, diagnose, and evaluate; hence the importance of the mind frames of teachers, leaders, parents, and students.

A child is born into a world of expectations. Such expectations can induce warmth and nurturing but sometimes lead to fear of the unknown, friction between parents, and rejection (see Hattie & Hattie, 2022). The child brings into this world many attributes from genetics and the womb, but the fundamental claim in this book is that from birth (notwithstanding claims about the heritability of intellectual dispositions), every child is a learner, is teachable, can grow, and can be taught to love learning.

So much happens in the four to five years before the child comes to school, and these years set a trajectory that can be very hard to change. During these early years, children hear and then start to learn language, develop a theory of mind about the world around them (which is why 2–3-year-old children are so interested in why questions), learn how to interact and to act in light of others, learn how to develop competence and build an understanding of what competence means, and learn how to make choices. They also learn how to learn, deal with errors and setbacks, work with others or not, and make sense of the world around them.

Like at all phases of learning, especially in these early years, students need people who are more expert than them to guide them to the next levels of challenge in learning. Students need to learn how to cope with situations via emotional coping (crying, venting) and problem-based coping (restructuring what happened, trying different solutions). To do this, they need to develop a sense of persistence and know when to seek help, when to be conscientious, when to stop and learn from others, and when to learn how to seek, hear, interpret, and deal with feedback.

Hence, the importance of language during these early years – talking, listening, reasoning, and explaining. Imagine two children. One, Adam, is born into a family where the child is not exposed to much talking, listening, reasoning, or explaining. Another, Amy, is born into a family where there is a lot of talking, listening, reasoning, and explaining. Now imagine these two children at age 5 when they start school. How many words will Amy have heard more than Adam has heard? That is, how many more words will Amy have been exposed to by the time Amy starts school compared to Adam: 30 million (Hart & Risley, 2003; but see Kuchirko, 2019). No wonder Amy has a head start; no wonder we are saying how critical it is to surround young children with language; talk, talk, listen, listen, show you have listened. Further, the language interaction between the child and another (adult, peer, sibling) truly matters. It is active not passive interactions with languages; not watching television, playing games (without talking), watching screens, but interacting with others – parents, adults, and peers in language.

These years can amplify the inequities of opportunity and access to quality ECE centers, and then the compound interest effect can occur. This compounding or the Matthew effect is based on the notion that the rich get richer and the poor remain poorer. This effect is often very present in learning development. If, for example, students do not attain sufficient levels by age 8 (e.g., level I in PISA), it is highly likely they will never catch up (*Pfost et al., 2014). Worse, teachers can readily identify these students at age 5. So why do we not do something about it? This is why we overemphasize the power of exposure and interaction with language in these early years – they are the building blocks of later learning. It is not going to preschool, playing, staying at home with parents and family – it is the degree of exposure and exploration of language, whatever the setting (see Thunder et al., 2021).

By the time they start school, the child certainly is no blank slate – they bring skills (prior achievement, intelligence, family, and cultural influences), various dispositions to learn, or what will be called will (e.g., confidence to tackle challenging tasks, resilience, resourcefulness, reflection, and relating), a sense of purpose (thrill) as to why they are in this place called school (motivation), and various physical attributes that can impact on learning.

The major skill attributes relate to developing a working memory (0.68), developing the skills of inhibition, shifting and updating (executive functioning, 0.70), being aware of their current achievement and learning skills, and wanting to improve them (1.33), accelerating through the Piagetian phases of development, building various strategies of learning (especially their skills to consolidate and appropriately overlearn), being curious (0.90), and learning how to impose structures and meaning on their world. Students are most aware of their achievement levels, which can be an enabler to higher achievement or a barrier to students being satisfied with their expected levels of achievement. The mantra should not be helping students realize their potential but working with students to exceed what they might think is their potential.

The major will attributes relate to building confidence to take on challenges (0.65) and appraisal skills to use problem-based coping strategies, especially to reduce anger (−0.82), maltreatment (−0.63), frustration (−0.52), procrastination (−0.50), illness (−0.44), boredom (−0.33), anxiety (−0.31) and depression (−0.30). Developing students’ sense of confidence that they can master a task and that it is worth their investment in mastering the task is critical. With a sense of confidence, it is more likely there will be investment, concentration, persistence or engagement, and more positive attitudes toward the content being taught. As will be argued later, this has major implications when developing success criteria for lessons–not only to help students see what success looks like but to help them interpret the teaching and seek feedback to reduce the gap between them and where they need to be next moving toward success. If the gap between their perception of their current status and the success criteria is perceived as too large, it should be little surprise that many students do not bother investing. If the criteria of success are too opaque, it should be little surprise that they resort (at best) to doing the task regardless of quality learning; and if the work is too easy or boring, then what a turn off. Herein lies the art of teaching – knowing where each student is, where they need to be, and whether they have the confidence to bridge this gap.

The thrill attributes relate to building adaptive motivations and strategies (0.57); skills in engagement, concentration, and conscientiousness to attend to tasks (d = 0.53); a desire to wanting to improve and, at all times, to learn to become their own teacher. Students come to class with many motivational resources, and the teacher aims to understand and direct these resources to invest in schoolwork, to teach students to develop adaptive expertise, to know when to focus on surface, when on deep, and when on transferring learning.

The physical dimensions are more variable and relate more to maximizing opportunities to engage in learning and schooling. Given that developing physical skills, health, and knowledge are so lowly related to achievement in other domains, this highlights the critical importance of including teaching these skills in our curriculum.

An aim is to develop positive, proactive, and powerful student ways of seeing their learning world. We have identified ten mind frames we should be aiming to teach students, and while the evidence for each is explored in subsequent chapters also, it is worth concluding this chapter with these ten:

I am confident that I can learn.

I set, implement, and monitor an appropriate mix of achieving and deep learning goals.

I strive to improve and enjoy my learning.

I strive to master and acquire surface and deep learning.

I work to contribute to a positive learning culture.

I have multiple learning strategies and know when best to use them.

I have the confidence and skills to learn from and contribute to group learning.

I can hear, understand, and action feedback.

I can evaluate my learning.

I am my own teacher.

Reflections

The skills

Discuss the following with a group of colleagues and among a group of students.

Are there any barriers to high self-efficacy in your school, such as fear of failure, comparative reward systems, ability labels, ego related praise?

Are students aware of how learning breaks down into aspects, such as resilience, concentration, lack of distractibility, and so on? Do they know that learning involves not only effort but deliberate practice, time, and input? Do they know how to seek help?

Have we exposed students to examples of people like them who have achieved success? If success precedes motivation, how do you ensure that students experience success in their learning to move forward?

Are we using appropriate strategies for normalizing error, such as using positive responses (You are on the brink of new learning!), frequent sharing and analysis of common mistakes with the class – emphasizing how we learn from them, referencing the work of scientists who look for error as a means of further learning and encouraging whole-class celebration of mistakes?

Try providing students with a worked example but with an error clearly identified, and ask them to work out how the student made the error and what feedback they would give to help the student rework the problem.

The will

2. For each student, make a chart of their willingness and confidence to take on challenging tasks. Then intentionally strategize steps to increase their confidence using many influences in this chapter.

3. Use Berry’s six-factor model of engagement on Figure 5.1 to evaluate your students’ engagement. Discuss (a) Kumar and Quinn’s findings that follow the model and (b) ways in which disengaged students can be moved to engagement by experiencing successful learning: “learning begets engagement more than engagement begets learning.”

The thrill

4. Goals that are too easy are poor motivators because the chance of goal attainment is already so high that further efforts do little to move these chances. Conversely, goals that are too difficult fail to inspire because the chances of success are so low that effort expenditure will not make the goal any more achievable. Discuss this statement, referring to reflection 3. As it seems critical that strategies are in place to help students when the goal is too difficult (e.g., peer coaching, knowing how to seek help, modification of the task) or to extend and challenge when the goal is too easy.

5. Which of your students show they have high levels of strategic motivation and know when the task asks for surface, deep, or transfer skills? Which students are more surface-oriented, and consider, how can you structure your teaching and feedback to move them to become more strategic?

6. Discuss the meanings, sharing examples of each mind frames at the end of this chapter that we should aim to teach our students. Discuss possible ways in which these can be shared across the school and owned by each student.

CHAPTER 6 The home and family

DOI: 10.4324/9781003380542-6

One of the fascinations of the COVID pandemic impact on schools is that many parents experienced their children’s school from an entirely novel perspective. Despite most parents experiencing school themselves, many began to appreciate the amazing expertise of teachers to motivate, inspire, and teach 20–30 students at once (and in high school, 200+ a day) for 200 days a year. Parents vividly were reminded that today’s school encompasses much more responsibility for cognitive, social, and emotional health, that it is not merely a matter of getting it right or wrong but appreciating the struggle (as a good word) of learning, and that students work together with many other students throughout the day in their learning (and some have and some do not have the skill to work with others). Further, they appreciated that teachers need to know not only content but also the common misunderstandings to fathom how to then best teach students – and much more. For example, other students had access issues, were not in safe havens, had obligations to siblings, had uninvolved parents, or had reduced or no access to loving parents. As we argue in our book on 10 Mindframes for Parents, parents are not first teachers, as this mixes the role and expertise of parents and teachers, but parents need to be first learners.

A fundamental function of the home is unconditional love. The home can be a nurturing place for the achievement of students, but it can be a toxic mix of harm, neglect, and low expectations. It is the case that schools can be safe havens for many and that parents may all want to help their children, but some do not have the skills. We discovered in the COVID pandemic teaching that we do not, as parents (or teachers), necessarily need to have the students in front of us all the time; that if we cover less, we have the luxury of going deeper, as this can turn students onto learning; and that both teachers and parents need to gradually release responsibility to the student, and this is a necessity, not a luxury.

A major concern is the extent to which parents know how to speak the language of learning and, thus, can benefit their children during school. Unfortunately, some do not know this language, which can be a significant barrier to the home, contributing to the achievement and the realization of parents’ expectations for their children (Clinton et al., 2005). Schools have an important role in helping parents learn the language of schooling so that they can provide every possible assistance to their children in terms of developing their learning and love of learning, and creating the highest possible shared expectations for learning. This occurred during the COVID pandemic, as parents saw teachers’ skills in motivating and providing feedback, and the importance of teaching students to self-regulate.

This chapter investigates the influences of the family resources, the family structure, and the home environment. A theme developed in this chapter is that parents can have a major effect regarding the encouragement and expectations they transmit to their children.

There are 117 meta-analyses relating to the influence of the family, and these are divided into three major topics: family resources, family structures, and the home environment (Table 6.1).

Table 6.1 Summary statistics for the three major home and family variables

Home
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Family resources
24
1,333
5,010,954
4,563
0.13
0.01
3

Family structure
25
1,213
10,770,413
2,954
0.21
0.01
3

Home environment
68
4,130
8,411,275
9,179
0.11
0.06
4

Total/average
117
6,676
24,192,643
16,696
0.15
0.03
3

Here is another example where averages at too high a level can be misleading. The table hints that family structure matters more, but it is more the family environment that has the highest effects (noting that this latter category has more higher effects and some much lower ones, making the overall average somewhat misleading; Table 6.2).

Table 6.2 Summary statistics for each of the home and family influences

Home
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Socioeconomic status
19
1,153
4,774,885
4,377
0.56
0.03
5

Family on welfare/ state aid
1
8
736
8
−0.12
0.03
1

Grant aid
2
68
6,259
53
0.03
0.01
2

Immigrant status
2
104
229,075
125
0.05
0.10
3

Family structure
5
209
59,676
255
0.14
0.08
4

Mother employment
2
88
307,061
1,528
0.05
0.08
4

Engaged vs. disengaged fathers
6
322
9,981,830
459
0.21
0.05
4

Non- vs. divorced families
6
318
29,268
457
0.26
0.08
3

Remarriage
2
122
375,614
139
0.24
0.08
3

Adopted vs. non-adopted children
3
150
13,806
112
0.21
0.04
2

Remove children into care homes
1
4
3,158
4
0.33
0.00
1

Home environment
6
165
57,584
811
0.40
0.08
4

Corporal punishment in the home
1
16
11,428
16
−0.33
0.14
2

Television hours
3
37
1,024,853
540
−0.15
0.08
3

Parental involvement
24
1,947
2,177,233
3,668
0.30
0.13
5

Parental programs
15
781
121,469
1,879
0.39
0.08
5

Parental autonomy support
8
485
388,359
1,217
0.06
0.01
4

Parental expectations
4
429
4,419,155
403
0.50
0.13
4

Home visiting
3
84
7,731
100
0.22
0.28
2

Mobility
4
186
203,463
545
−0.38
0.01
4

Total/average
117
6,676
24,192,643
16,696
0.15
0.06
3

Family resources

Socioeconomic status

Socioeconomic status (SES) relates to an individual’s (or family’s, or household’s) relative position in the social hierarchy and refers directly to the resources in the home. The three leading indicators of SES are parental income, education, and occupation. The overall effect from the 19 meta-analyses based on 1,153 studies (4,377 effects) is d = 0.56, which is a notable influence on the student’s achievement.

*Sirin (2005) reported an effect size between achievement and parental education of d = 0.60, parental occupation was 0.56, and parental income was 0.58: very similar indeed. Further, there was an effect size of d = 0.50 with neighborhood resources and 0.66 with free or reduced-cost lunch (a common measure of SES in the US). There was little variability in the relation between SES and various types of achievement (verbal, d = 0.64; mathematics, d = 0.70; science, d = 0.54). *Sirin made much of a slight increase from preschool through middle school, but the effects are not that different: preschool, d = 0.38; elementary, d = 0.54; middle, d = 0.62; and high school, d = 0.52. The effects were lower for students in rural schools (d = 0.34, where there is likely less variability of SES within a school) than in suburban (d = 0.56) and urban (d = 0.48) schools. Overall, there were not many differences across these effects based on the key components of SES, so the question arises as to how these SES effects influence student achievement.

We need to be careful, however, about the unit of analysis used in these studies: is it the socioeconomic status of the school or the student? *White’s (1982) meta-analysis on the relationship between SES and academic achievement noted the importance of distinguishing between effects based on aggregated units (such as SES of the school, d = 0.73) and effects based on the individual level (such as the SES of the student, d = 0.55). Further, *Sirin (2005) noted that the effect was much lower when the data about SES were provided by the students (d = 0.38) – who probably saw less inequity in the difference due to home resources – than when provided by the parents (d = 0.76). Thus, SES is more important at the school than at the individual level and for the parents more than for the students. The compositional effects of the school may become critical in setting expectations, allowing students to see what is possible, and in how teachers pitch the challenge of learning for most students.

There is still a need for a closer examination of the many roles of SES. Socioeconomic status is also a good indicator of prior achievement; at minimum, the two may be intertwined. So is it really socioeconomic status or prior achievement that is serving here as the predictor? When controlling for prior achievement, the effects fall to low values: for example, *Rumberger and Palardy (2005) estimated 0.05 for math and 0.06 for reading (cf., *Lu & Rickard, 2014).

The view that SES is a delimiter or not or a proxy for prior achievement can markedly affect so many students’ expectations and life chances. Fortunately, as a young child starting school, I did not know the supposed limitations because my dad was a cobbler and my mum a housewife; naivety was my best defense. In a series of articles, Marks and O’Connell (2021; O’Connell & Marks, 2022) notes these problems with the misuse of SES and argue that once a child’s cognitive ability is taken into account, even the modest link between SES and attainment diminishes to slight influence.

Resources, however, buy opportunity, and the effects from socioeconomic resources are likely more influential during the preschool and early years of schooling. The lack of language exposure, the lower levels of parental involvement in teaching and schooling, the fewer facilities to realize higher expectations and encouragement, and the lack of knowledge about the language of learning may mean that students from lower SES groups start the schooling process well behind others.

Well-resourced parents can be more influential in socially stratifying schools, such as choosing what are perceived as higher-achieving schools, choosing schools with tracking, and investing in early childhood education. They can provide “codes of speech” (Bernstein, 1971) that allow their children greater access to school resources, have higher expectations for current and future opportunities to extend their children, engage them in after-school activities that complement their schooling, and provide richer resources in the home to enhance learning opportunities.

One of the ways this influence is manifested is that schooling introduces a language and set of cultural norms with which many parents, particularly those from lower SES families, are not familiar. In a five-year evaluation of five of the lowest SES schools in New Zealand, we found major positive consequences when teaching parents the language of schooling (Clinton et al., 2005). This evaluation (the Flaxmere Project) involved a series of innovations related to improving home-school relations within and among five schools, including giving families computers and employing former teachers as home-school liaison persons. The home-school liaison persons allowed the parents to learn the language of schooling – that is, the parents learned the language about the nature of learning in today’s classrooms, learned how to assist their children to attend and engage in learning, and learned how to speak with teachers and school personnel. Involving parents with the schools via the Flaxmere Project led to enhanced engagement by students in their schooling experiences, improvements in reading achievement, greater skills and jobs for the parents, greater awareness of the language of schooling, and higher expectations, high satisfaction, and high endorsement of the local schools and the Flaxmere community (the effect sizes ranged from d = 0.30–0.60 and occasionally were much higher across many outcomes). The greatest effects were related to the parents’ increased knowledge of the language of schooling and learning.

Either there can be efforts to reduce the barriers between school and home, or the effects of the home on student learning can be compromised as the child is then asked to work in two worlds – the world and language of home, and the world and language of schools. For many children, this is asking too much. It is also difficult for children in these two worlds to build a reputation as a learner, learn how to seek help in learning, and have a high level of openness to learning experiences. It’s a compound issue when the child from lower-resourced families is asked to learn to live and work in classes with peers from middle- and higher-resourced families, and when the race or ethnicity is also different.

One of the important experiences of the COVID pandemic distance teaching was that parents were more appreciative and informed about their children’s world of schooling. How to advance this in the regular in-class teaching, particularly when we note the typical reports sent home to parents are public relations disasters. Hattie and Peddie (2003) analyzed school reports from hundreds of schools and found that more than 95% of students were achieving well, putting in an effort, and being a pleasure to teach. Unfortunately, this is not the experience of students and leads to distrust and distancing between parents and schools.

Welfare policies

*Gennetian et al. (2002) found close to zero effects from students in families who received welfare compared to those not receiving welfare. They make much of an effect size of d = −0.10 by claiming that the effects on adolescents were “significantly worse.” Still, it is difficult to imagine the visible effects of this effect – about 4% fewer mothers in the welfare program group reported that their child performed above average. There are certainly many effects of welfare programs for these families that are beneficial. Still, it seems that there are other more powerful effects on achievement than the welfare status of the family.

Mother employment

Since the 1980s, most United States mothers have been employed, although the claim that this was somehow detrimental to their children is still a widely held belief. *Goldberg et al. (2008) showed that the effects of maternal employment on achievement were indeed trivial (r = 0.03), as did *Lucas-Thompson et al. (2010, r = 0.00). *Goldberg et al. could not find differences with respect to SES (middle/upper, r = −0.04; lower-middle, r = −0.06), ethnicity (white, r = −0.03; majority African American and Hispanic, r = 0.02), child’s age (pre-school, r = 0.02; elementary, r = 0.06; high school, r = 0.02), family structure (one parent, r = 0.15; two-parent, r = −0.01), or whether the work was part-time (r = 0.04) or full time (r = −0.01). It does not matter to a child’s achievement whether a mother works outside the home or not.

Grant aid

*Nguyen et al. (2019) investigated the impact of financial grants in tertiary education. An extra $1,000 USD of grant aid improved year-to-year persistence by 1.2 percentage points and degree completion by 0.01 percentage points. The authors argued that, while small increases, their results point to the value of front-loading aid by providing a greater amount of grant aid in the first year of enrollment than in subsequent years, supplementing grant aid with additional treatments, such as academic or social supports, and moving aid toward students based on financial need instead of subsidizing students who would likely succeed without additional support. Expanding these financial supports would induce more students to attend postsecondary education and increase their educational attainment.

Immigrant status

There are 7.6 million migrants living in Australia, 30% of the population was born overseas, and I am one of them. Care is needed, as immigrants come from similar backgrounds to the new country (as for me), or there can be major cultural, school, and expectation differences. In the US, nearly one in four children have at least one immigrant parent (Fortuny et al., 2011), and by 2050, an estimated one in three children will come from immigrant families (Ho & Kao, 2018). *Dimitrova et al. (2018) explored the achievement effects on immigrant students in European countries. The overall effects were small, d = 0.09, although the effects were negative for adolescents (−0.16) and positive for elementary students (0.14). There were more positive effects in externalizing and internalizing outcomes for studies conducted in Northern (i.e., Denmark, Finland, Norway, Sweden, the UK) and Western (i.e., Austria, Belgium, Germany, the Netherlands, Switzerland) countries compared to those in Southern Europe (i.e., Greece, Italy, Portugal, Slovenia). They claimed this difference was related more to the length of the countries’ active immigration policies and the presence of family reunion programs. They concluded that “immigrants appear to bring important family characteristics to their host countries such as strong family aspirations and expectations, which possibly account for the more favorable school patterns of their children” (p. 157).

There can be a strong stereotype threat for these students (*Appel et al., 2016, d = 0.63), and some have argued for a generational effect. However, *Duong et al. (2016) only found d = 0.01 in the academic achievement of first- and second-generation students, and that second-generation performed slightly better than third-or-later generations (d = 0.12). *Ho and Kao (2016) reviewed US immigrants’ performance on NAEP, high school grades, and school completion. Immigrant students generally do not perform at the same levels, particularly English Language Learners. But then care is needed as to which immigrants – Asian immigrants outperform, and there is a moderator effect of SES and English language skills. Further, there are important roles and responsibilities that teachers and schools can take to ameliorate any lowering of achievement, such as not having ability grouping, having high expectations of the progress of these students, focusing on the impact of the teaching, reducing stereotype threat, attending to developing positive study strategies, being attentive to friendship and peer relations, and ensuring students release and action that enhanced education can lead to upward mobility (Ogbu, 2004).

Family structure

There are many types of families, and the overall effects of these different types could be classified as small compared to many other influences. In most Western countries, about 70–80% of families have two parents, about 10–20% are single-parent, and about 2–10% are other than these structures. The various meta-analyses on different family structures (two vs. single parent, same-sex parents, only vs. non-only children, incarcerated parents) have very small effects. *Pong et al. (2003) found that single parenthood is associated with lower mathematics and science achievement (although the effects are quite small, d = 0.13). They also noted that countries with more generous welfare policies, like Austria, showed the smallest gaps. The greatest gaps were in countries that lag behind other industrialized countries in providing financial assistance, universal child benefits, tax benefits, and maternity leave benefits to single and poorer families.

A review of only-born child literature by *Falbo and Polit (1986) found that only-born children surpassed all except firstborns and children from two-child families concerning achievement and intelligence, but the effects are small. Only-born children exceeded all non-only children, especially those from families with three or more children, in positive character attributes and the positivity of the parent-child relationship – only-borns were more motivated and had better relations with parents (d = 0.13) – otherwise, there were no differences across many affective outcomes.

All meta-analyses on this topic found small effects on achievement relating to whether a father was present or not in the family. *Amato and Gilbreth (1999) found small effects relating to the fathers paying or not paying child support (d = −0.13), contact between fathers and children (d = 0.11), feeling close (d = 0.06), and authoritative parenting (d = 0.17). *Salzman (1987) found d = 0.26 achievement effect of father-presence compared to father-absence. The effects were slightly higher for achievement (d = 0.30) compared to aptitude tests (d = 0.20) for elementary and junior high students than for preschool, but there were no differences between males and females or across socioeconomic groups.

Divorce

Compared to children with continuously married parents, children with divorced parents scored lower (but not by much) on measures of academic achievement, psychological adjustment, self-concept, and social relations. *Amato and Keith (1991) used 92 studies that compared children living in divorced single-parent families with children living in continuously intact families. The overall effect size was d = −0.26 on school achievement for the children in the former group, although this difference was lower for the more recent studies (d = −0.12 for more recent studies, compared to d = −0.23 for studies 30 or more years ago). Other effects were d = −0.23 for conduct, d = −0.08 for psychological adjustment, d = −0.09 for self-concept, d = −0.12 for social adjustment: all small effects. Teachers saw no differences between these two sets of children (d = −0.04), and the effects were greater for girls than boys (d = −0.30). Amato and Keith also found similar achievement effects (d = −0.22) for children who experienced the death of a parent. The other meta-analyses on children from non-compared to divorced families show a similar small effect.

Remarriage

*Jeynes (2006) compared intact vs. parental remarriage and found the effects on the achievement of the former over the latter was d = 0.22, but there were no differences between children from parental remarriage and those children in divorced or widowed families. He argued the increased interactions with two adults are beneficial but also suggested it may be difficult for children to make more than one family transition (to single or divorce to remarriage). *Kunz and Kunz (1995) found d = 0.30 effect, and the effects were slightly lower for the more school achievement outcomes (d = 0.25 for academic achievement, d = 0.16 for verbal achievement, but d = 0.52 for math achievement). The effects decreased with age, and she related many of the decreases more to the economic differences between one- (divorced) and two-parent families. Kunz (2018) was more interested in the effects of divorce on interpersonal influences. From her 53 studies, children who experienced divorce had less positive interpersonal relationships with their mother and father but more positive sibling relationships (although the effects are very small).

Adopted vs. non-adopted children

Non-adopted children had slightly higher school achievement than their adopted siblings; the adopted children outperformed their non-adopted siblings and peers left behind; adopted students did less well in school than non-adopted children – but the effects are small relative to other influences. Most importantly, the age of adoption seems important. Those who were adopted in their first year showed no differences (d = 0.09), and the effects increased if they were adopted in the second year (d = 0.32) and beyond their second year (d = 0.42). Overall, *van Ijzendoorn and Juffer (2005) concluded that it seemed that “adopted children were able to profit from the positive change of environment offered by adoption and subsequent upbringing” (p. 327), but overall, the effects are small. In contrast, *Scott et al. (2011) found larger (d = 0.40) effects on the language skills of internationally adopted children.

Remove children into care homes

*Cassarino-Perez et al. (2018) reviewed the child welfare system’s effects on moving children into care. Having placement stability was related to higher odds of having a high school diploma, but race, gender, mentoring, and maltreatment had no significant association with education pathways for young people who leave care. They note the positive effects of early detection and case evaluation to enable a stability plan for students to stay at the same school with friends and practice the same leisure activities.

Home environment

The home environment includes measures of the sociopsychological environment and intellectual stimulation in the home. *Iverson and Walberg (1982) suggested that achievement is more closely linked to the home’s sociopsychological environment and intellectual stimulation than parental socioeconomic status indicators, such as occupation and education. However, they were not specific about which home indicators were most influential. *Gottfried (1984) completed a meta-analysis on studies using the Home Observation for Measurement of the Environment (HOME) scale, which measures responsivity, restriction, punishment, play materials, involvement, and variety. Maternal involvement, variety, and play materials were the most consistent and highly correlated factors with achievement (Gottfried, 1984). *Strom and Boster (2007) assessed the effect of messages about educational importance in the home (r = 0.21) and school (r = 0.14) on dropping out. Specifically, parental expectations had the greatest impact on a student’s decision to stay in vs. drop out of school (r = 0.32), then parent-child interactions (0.17), and family interaction about the school.

Parental involvement

The effect from 24 meta-analyses on parental involvement is 0.41, but there is much variance in the influence of parental involvement. For example, there are negative effects when the parent’s involvement involves a surveillance approach, low positive effects relating to parental involvement in early intervention in school-related programs, and much higher effects relating to parental aspirations and expectations.

*Casto and Lewis (1984) examined studies relating to parental involvement in early intervention programs. They found little support for the idea that parental involvement leads to more effective intervention programs. They commented that while programs that involved parents could be effective, they were not necessarily more effective than those either not involving parents or involving them in a minor way. Similarly, *White et al. (1992) also argued that parental involvement led to more effective outcomes. However, the effect of parental involvement often, particularly after the variance due to students and teachers being accounted for, is low at best (*Innocenti et al., 1992).

Other metas with low effects included *Erdem and Kaya (2020), with r = 0.09, and similar low correlations across all domains (language, 0.08; math, 0.08; science, 0.12) and levels of schooling (pre, 0.08; elementary, 0.08; high schools, 0.11) but much higher relating to parent expectations (0.29). *Castro et al. (2015) noted r = 0.12, as did *Moerea (2016), r = 0.12, who also reported much higher correlations when families have high academic expectations for children (r = 0.22) and when they develop and maintain communication with their children about school activities (0.20). *Kim (2020) found r = 0.12 for parents living in East Asian countries, but the influences were higher for academic socialization (0.31) and in high school (0.17) compared to elementary schools (0.05). *Kim and Hill (2015) reported r = 0.14 between father involvement in education and child achievement, and 0.15 for mothers. The effects for academic socialization were highest (0.16) and lower for school involvement (0.12), intellectual enrichment (0.11), and home involvement (0.07). *Barger et al. (2019) separated the influences of parents, school, and the home. The correlates were most similar: achievement (parents, 0.13; school, 0.12; home, 0.09), engagement (0.15, 0.13, 0.15), motivation (0.23, 0.12, 0.11), social (0.12, 0.08, 0.11), and emotional (0.17, 0.15, 0.16). These relations were similar across SES.

*Senechal (2006) found that more active involvement by parents was more effective. For example, the effect size from studies where parents taught their children specific literacy skills was twice as effective (d = 1.15) as parents listening to their children read (d = 0.51), which, in turn, was much more effective than reading to the child (d = 0.18). These effects were reasonably consistent from kindergarten to grade 3, for students with (d = 0.38) and without reading difficulties (d = 0.74), and for families from different socioeconomic status groups.

*Crimm (1992) reviewed parental involvement and found greatest effects between kindergarten and grade 3 (d = 0.41) but then decreased with age (grades 3 to 5, d = 0.36; secondary, d = −0.05). The most successful involvement related to tutoring (d = 0.49), home visits, and interactions with teachers (d = 0.48), and the lowest was parent training (d = 0.15). The highest effects were in reading (d = 0.40), while the effects were much lower in mathematics (d = 0.18), which is not surprising given that parents are more likely to read than do mathematics with their children. Jeynes (2005) found that the best predictor was parent expectations (d = 0.58), which was far greater than parental involvement at the school (d = 0.21). In a subsequent study using secondary students, *Jeynes (2007) similarly found greater effects from parental expectations (d = 0.88) than from other parent factors, such as checking homework (d = 0.32), having household rules (d = −0.00), and attending and participating in school functions (d = 0.14). Overall, “the higher the hopes and expectations of parents concerning the educational attainment of their child, the higher the student’s own educational expectations and, ultimately, the greater the student’s academic achievement” (Hong & Ho, 2005, p. 40; see also *Fan & Chen, 2001; *Rosenzweig, 2000). *Ma et al. (2016) found very high correlations (0.53) for the involvement of parents in the early childhood years, particularly for home-school connections, school participation (volunteering work for school), and leadership roles in shaping up social services and educational systems. *Nye et al. (2006) also had high correlates (0.43), and they were similar for reading (0.42) and math (0.34); as did *Ates (2021, r = 0.39) and similar across primary (0.31), middle (0.35), and high (0.33) schooling.

When parents see their role as surveillance, such as commanding that homework be completed, the effect is negative (see also Clinton & Hattie, 2013; *Fernández-Alonso et al., 2022). *Hong and Ho (2005) noted that parental supervision in the forms of monitoring students’ homework, time watching television, and time going out with friends appeared to harm the educational aspirations, particularly of adolescent students. *Rosenzweig (2000) reported low effects with supervising homework (d = 0.19), participating in school activities (d = 0.14), communicating with school and teachers (d = 0.14), monitoring school progress (d = 0.12), providing structure in the home (d = 0.00), and controlling and disciplining parental style (d = −0.09). These effects were the highest in high SES families, in elementary compared to high schools, and in Asian and Latino compared to white and African American families. Negative effects related to the presence of external rewards, homework surveillance, negative control, and restrictions for unsatisfactory grades. *Barger et al. (2019) found a negative effect on parents helping with homework (r = −0.15), *Kim and Hill (2015) found also a negative effect (−0.11), *Barger et al. (2019) found also a negative effect (−0.15), and Kim (2020) found a small positive effect (r = 0.08) of parent involvement in homework in East Asian countries.

Parental autonomy support

Over the past 40 years, the research has made an important distinction among three parenting methods – permissive, authoritarian, and authoritative (Baumrind, 1968). A parent is rarely only one of the three, but many parents have a dominant set of strategies they apply – and more importantly, the child is aware of which set the parent applies. Permissive parenting gives much leeway to the child, allows them free rein, and ignores a lot of misbehavior. Authoritarian or bossy parenting engages in “I am right, you are the child,” sees the child as unable to make good decisions, and engages in verbal hostility and sometimes corporal punishment. Authoritative or reasoning parenting creates warmth and involvement, engages in appropriate reasoning and listening to the child, and creates a climate of trust and fairness. The meta-analysis evidence supports the authoritative (positive), over permissive (zero), and authoritarian (negative).

We (Hattie & Hattie, 2022) aimed to avoid the confusion between the authoritarian and authoritative, and preferred the term “reasoning and listening parenting” to “authoritative.” The desired reasoning and listening strategies relate to developing the child’s autonomy and the skills of relating to others and promoting competence. This is called self-determination theory and has been well and widely researched as a desirable set of attributes in our Western world (Ryan & Deci, 2017). A key to this development of self-determination is for parents to know the right time to deliver the right amount of the right messages to build the child’s confidence and skills to be a learner.

*Pinquart (2015) used 308 studies and reported r = 0.15 for reasoning and listening, 0.01 for permissive, −0.17 for authoritarian, and −0.15 for neglectful parenting. He noted that higher achievement was related to higher levels of parental responsiveness (warmth, r = 0.14), behavioral control (r = 0.11), autonomy granting (r = 0.11), and a reasoning and listening parenting style (r = 0.17). Authoritarian parenting included harsh parental control (r = −0.16) and psychological control (r = −0.11). Similarly, longitudinal studies had similar patterns: warmth (r = 0.17), behavioral control (r = 0.11), harsh control (r = −0.13), psychological control (r = −0.23), autonomy granting (r = 0.15), and reasoning and listening parenting (r = 0.15). Academic achievement showed stronger associations with behavioral control (r = 0.12 vs. r = 0.03) “if child rather than parental reports on parenting, and associations of warmth and autonomy support with achievement were stronger if observer ratings (r=0.24 and r=0.39) rather than child or parent ratings on parenting (r=0.11/0.14 and r=0.10/0.11) were used” (p. 14).

Pinquart and Kauser (2018) used 428 studies to seek any differences between cultures within and then cultures outside the US. Reasoning and listening parenting were related to at least one positive child outcome, and authoritarian parenting to at least one negative child outcome in the US, across white families and those with an African and Asian ethnicity. Regardless of cultural background, young people benefit “from having parents who are authoritative: warm, firm, and accepting of their needs for psychological autonomy” (p. 10). There was no support for the claim that authoritative parenting may be associated with less positive child outcomes in Chinese and other East Asian societies than in Western countries.

Parental expectations

Our first mind frame in our parent book Ten Steps to Develop Great Learners is “I have appropriately high expectations” (Hattie & Hattie, 2022). The overall effect of parental expectations is 0.49, and they have powerful effects on how children will interact, learn, and develop as learners. Supporting children to become great learners means setting high expectations but appropriately so – using the Goldilocks principle of not too hard, not too easy, and not too boring. To achieve these expectations, a child needs to have developed an emotional bond or attachment with the parents, preferably based on trust. This sense of attachment and trust allows the child to make mistakes, learn from them, and develop resilience and other attributes. A child also needs encouragement, a sense of fairness in how you react to them, and your direction to learn right from wrong. The parent’s role is not to make their child depend on them but to help them develop autonomy and a sense of competence.

*Pinquart and Ebeling (2019) reviewed 169 studies with r = 0.30 from cross-sectional and 0.28 from longitudinal studies between parental expectation and achievement, and these correlations were not affected by the SES of the family or prior achievement effects of the child. Similarly, *Castro et al. (2015, 0.22), *Jeynes (2007, 0.40), *Danisman (2017, 0.19), and *Phillipson (2013, 0.37) found similar correlations.

*Phillipson (2013) distinguished between parental expectations of grades (r = 0.39), for language (0.30) and math (0.39), in primary (0.40) and in secondary (0.13), and about future aspirations for their children (primary, 0.19; high, 0.32), and children’s perceptions of parental expectations (which differed in primary, 0.30; and high schools, −0.01). The expectation effects on grades as perceived by students are higher in younger than older students, but parents of high school students are more focused on expectations about future aspirations.

Corporal punishment

What’s wrong with a short, sharp smack to enhance learning? Quite a lot, when you look at the effects. *Paolucci and Violato (2004) located 70 studies and found −0.33 for achievement, −0.35 for affective, and −0.25 for behavior outcomes. These results were not moderated by who did the spanking, whether the child was male or female, SES, age, or discipline technique. So there seems a lot wrong with a spanking.

Engaged and disengaged fathers

It is less the presence of fathers, but how they are involved with their children that matter more. *Amato and Gilbreth (1999) found an effect of 0.07 between non-resident fathers and their children’s academic success (d = 0.13 with child support payment; frequency of contact, 0.02; feeling close, 0.06; and authoritative parenting, 0.15). Similarly, *Adamsons and Johnson (2013) reviewed the effects of nonresident father involvement on academic achievement (0.04), behavior (0.50), psychological (0.03), and social outcomes (0.15). The effects did not differ by father involvement in activities (0.09), contact (0.02), or financial provision (0.06). *Jeynes (2015) also found low (0.16) effects of father involvement both in the US (among children of color and not) and in non-US samples. The effects were similar in younger (1–10, 0.18) and older (11–20, 0.14) and related to the nature of involvement (to foster achievement, 0.14; foster psychological welfare, 0.17; foster positive behavior outcomes, 0.20; and encourage other healthy outcomes, 0.22).

*Salzman (1987) investigated father absence due to employment, military service, death, divorce, separation, or desertion and found a d = 0.26 achievement effect of father-presence compared to father-absence (for absences with military or employment, 0.08; death, −0.21; divorces, separation, or deserting, −0.31). The effects were slightly higher for achievement (d = 0.30) compared to aptitude tests (d = 0.20), higher for elementary (−0.34) than high school students (−0.15), but there were no differences between males and females or across socioeconomic groups.

The involvement of fathers matters more. *Kim and Hill (2015) reported r = 0.14 (d = 0.28) between the father’s involvement in education and child achievement (and the same r for mothers), with the effect higher (r = 0.17) for students in high school than elementary school (0.10). *McWayne et al. (2013) focused on the effect of fathers in the early childhood years and found similar correlations between father involvement with achievement (r = 0.21) and with positive engagement activities (0.08), positive parenting (0.08), and negative parenting (−0.10), but higher effects of positive parenting on prosocial skills (0.19) and self-regulation (0.21). Contrary to their expectations, the correlation with achievement was higher for white (0.27) than non-white students (0.10) and for middle-high SES (0.27) compared to low SES (0.14). They concluded that both the quantity and quality of direct father involvement matter. More specifically, aspects of parenting quality (e.g., warmth, nurturance, and responsiveness reflecting positive parenting, and alternatively, harshness, punitiveness, nonresponsiveness reflecting a negative parenting style) and frequency of positive engagement activities (both general [e.g., playing] and learning specific [e.g., reading to the child]) are important in predicting children’s social and academic success.

Mobility

The effect of student mobility between schools can be quite marked. Transience, or mobility across schools, has become a significant trend in recent decades. My boys changed schools and countries five times in their school days, and no move was easy. In New Zealand, for example, 40% of all students change schools each year (including moving from elementary to middle school, from middle to high school, and from junior to senior high schools), and in the United States, 20% change residence each year. The effects of such mobility on reading (d = −0.27) and mathematics (d = −0.22) are negative (*Mehana, 1997). *Jones (1989) argued that any changing of schools made the negative effect, as mobility was not negatively related to the total number of moves, socioeconomic status, or ethnicity.

*Card et al. (2011) specifically investigated the mobility effects of students with military parents. Using 16 studies, the average correlation between deployment and child problems was r = 0.08; internalizing problems, 0.09 (e.g., depression, anxiety); externalizing problems, 0.04 (e.g., aggression, rule-breaking); and academic problems, 0.08 (grades). The academic problems were greater when mobility happened in the middle years (−0.21) compared to the early years (−0.12) and high school years (0.00).

During the middle school years, the impact of peers increases. One of the important considerations when a child moves to a new class or school is their skills in making friends, especially friends that share an interest in achieving well in school (see Ota, 2014). Galton et al. (1999) estimated that up to two out of every five pupils fail to make expected progress during the year immediately following the change of schools. Too many teachers cling to the principle of the fresh start and, thus, waste time finding out about their students instead of a more seamless transition of teacher knowledge and evaluations over the years or schools, and Hargreaves and Galton (2002) noted that teachers often sought to slow down the quicker pupils and speed up the slower ones up in order to manage the class. Further, after transfer, those underachieving were typically identified as problem students.

Galton and Willcocks (1983) followed students longitudinally, and every school change caused negative effects. They noted that typically, there were adjustment issues, including problems with friendship patterns, particularly friendships to support learning. Whenever there is a major transition in schools, the key success factor is whether a child makes a friend in the first month (cf., Galton, 1995; Pratt & George, 2005). Therefore, it is incumbent for schools to attend to student friendships and ensure the class makes newcomers welcomed. This is likely a similar implication from returning from longer lockdowns of schools during the COVID pandemic to home then back to in-class; peers matter, especially during adolescence (Carroll et al., 2009).

Television hours

The overall effects of television on achievement are small but negative (−0.15); however, given the changes in technologies available to students (video games, computers, and interactive technology), the effects of television on achievement are probably of far less of interest and importance than most other technology influences on achievement (see later on screen time).

There is likely a nonlinear negative effect. *Williams et al. (1982) found slightly positive effects for up to ten hours of viewing a week, while the magnitude was greater when watching over ten hours (cf., Ennemoser & Schneider, 2007). *Razel (2011) reported that the optimal viewing time (that is, when the relationship is at least positively related to achievement) decreases with age – younger children can watch more television with no negative effects, but by age 7, it is down to one hour, and by 17, it is zero hours.

*Neuman (1986) argued that these differences reflected parental characteristics. Parents allow unrestricted and unsupervised viewing tended to have fewer expectations and lower educational aspirations for their children than those who assumed greater control over television viewing. She found no support for the displacement hypothesis, as leisure reading, sports activities, and time with friends all seemed independent of the time spent watching television. Television can also have positive effects on prosocial behaviors (d = 0.63), which outweighs the effects of antisocial behaviors (d = 0.30, *Hearold, 1980).

Parent programs

Parenting programs tend to work with greater impact in the early years, with children struggling with reading and numeracy, when taught by trained teachers or trained parents, and when parents are involved in peer groups for support (*Layzer et al., 2001). There was greater success when the intervention was focused on specific outcomes, and thus, shorter, more directed programs were more effective than longer programs (*van Steensel et al., 2019). When parents engage their children in activities (e.g., reading to them), the effects are higher than when they engage in deliberate teaching methods, showing the importance of the expertise of teachers. *Graue et al. (1983) noted that the more experienced and trained the teachers or parents, the more effective the impact of home programs (*Manz et al., 2010). They also argued that providing home-based math environments, including all math-related activities, attitudes, expectations, resources, and interactions between parents and children in the home, was the more effective way to enhance children’s early math development (*Grindl et al., 2016).

Most home programs were for the early years, and the effects dropped once schooling commenced. *Daucourt (2019) argued this is likely because once formal schooling begins, instruction is more likely to be impactful at school than in any home program. The effects were higher when students started the programs behind their cognitive development for lower than higher SES samples, and there were no differences between boys and girls (*Layzer et al., 2001).

The effects of home programs related to early reading skills were higher than for numeracy programs (see *Daucourt et al., 2021). *Senechal (2006) showed that the effects were higher when parents listened to their child read (0.51) compared to reading to their child (0.18), and *van Steensel et al. (2011) also showed low effects of shared reading 0.05, but the effect increased when they also engaged in other literacy activities (0.21). The largest effects of parents’ reading were increasing students’ knowledge about concepts of print (*Manz et al., 2010, 1.21) and expressive language (0.76).

Home visiting

Home visiting programs have had a long tradition, and there are many reasons for these visits (abuse, prevent injury, cognitive). The effects were higher when professionals than nonprofessionals visited (*Sweet & Appelbaum, 2004), when there were ongoing (1+ a month) rather than single-site ventures (*Grindl et al., 2016), where the visit included active learning for parents and peer support (*Layzer et al., 2001), and when the home visiting was part of a more systematic series of interventions (*Layzer et al., 2001). The effects, however, are typically lower than sending children to ECE programs.

Concluding comments on home effects

Across all home variables, parental aspirations and expectations for children’s educational achievement have the strongest relationship with achievement.

Many parents struggle to comprehend the language of learning and, thus, are disadvantaged in the methods they use to encourage their children to become great learners.

Parents should be educated in the language of schooling so that the home and school can share the expectations, and the child does not have to live in two worlds.

It is not so much the family structure but rather the beliefs and expectations of the adults in the home that contribute most to achievement.

Any classification of a child’s ability or future as a member of a group (e.g., low SES, immigrant, divorce, adoption) should be resisted, as the variance can be large, and love and parental engagement are not reserved for the affluent.

It is more about the skills of parents as first learners: that is, demonstrating and being open to learning, promoting the language of learning, instilling the skills and thrills of learning with their children, developing parents’ ability to listen, setting fair boundaries, and developing particular skills about how children learn.

A child is born into a world of expectations. Across all home variables, parental aspirations and expectations for children’s educational achievement have the strongest relationship with achievement (d = 0.49), while communication (interest in homework and schoolwork, assistance with homework, discussing school progress; d = 0.38) has a moderate size effect and engagement in surveillance of educational activities is negative. Thus, parents need to hold high aspirations and expectations for their children, schools need to work in partnership with parents to make their expectations appropriately high and challenging, and then all work in partnership to have these expectations realized – and even work to raise the child’s expectations.

Many parents, however, struggle to comprehend the language of learning and, thus, are disadvantaged in the methods they use to encourage their children to attain their expectations. Too often, the alienation of the home from school reduces the initial parent expectations. The Flaxmere study, for example, found that when their children started school, 98% of the parents considered that education was very or extremely important to their children’s future. Two-thirds of these parents expected their children to attain diplomas and degrees. By the time they left elementary school, these aspirations had been dowsed, and the parents mainly wanted their children to “get a job” (Clinton et al., 2005).

Parents should be educated in the language of schooling so that the home and school can share in the expectations, and the child does not have to live in two worlds – with little understanding between the home and school. Some parents know how to speak the language of schooling and, thus, provide an advantage for their children during the school years, while others do not know this language, which can be a major barrier to the home, contributing to achievement. Parental expectations are far more powerful than many of the structural factors of the home (e.g., single- or two-parent families, families with resident or nonresident fathers, divorced parents, adopted or non-adopted children, only children, non-only children, birth order). It is not so much the family structure but rather the beliefs and expectations of the adults in the home that contribute most to achievement.

Many have noted that the home postcode should never determine a child’s opportunity or outcomes. Home resources can provide access to greater learning opportunities, but care is needed to interpret the high correlations between socioeconomic status and achievement. SES can be seen as a proxy of prior achievement, any classification of a child’s ability or future as a member of a group (e.g., low SES, immigrant, divorce, adoption) should be resisted as the variance can be large, and love and parental engagement are not reserved to the affluent. Further, we have discussed the Matthew effect, which can increase the impact of early delays. It is more about the skills of parents as first learners, that is, demonstrating and being open to learning, promoting the language of learning (i.e., discussing learning and developing a culture of learning), and instilling the skills and thrills of learning with their children, developing parents’ ability to listen, setting appropriate boundaries, and developing particular skills about how children learn (Hattie & Hattie, 2022). It is working in home-school partnerships, learning the language of learning, and engaging with expertise in teaching (recognizing, accessing, and using expertise in teaching) that matters. Homes are more about loving, learning, listening, and being responsive to all family members, encouraging friends and the skills of friendship, and developing the skills of self-determination.

In our recent book on parental mindsets, we identified ten ways of thinking by parents that can lead to optimally realizing high expectations and impact student learning:

I have appropriately high expectations.

I make reasonable demands and am highly responsive to my child.

I am not alone.

I develop my child’s skill, will, and sense of thrill.

I love learning.

I know the power of feedback, and success thrives on errors.

I am a parent, not a teacher.

I expose my child to language, language, language.

I appreciate that my child is not perfect, nor am I.

I am an evaluator of my impact.

Reflections

Check that your school’s parents, teachers, and students have a shared language of learning, including being resilient, concentrating, not giving up, having a go, having strategies for being stuck, being willing to take on challenges, reviewing and improving, being curious, looking for patterns, and making connections.

Are there ways in which we can introduce parents to the language of schooling: what learning in today’s classrooms consists of and looks like, and how they can best help their children? For example, run open days or online learning using TeachBack ideas – inviting the student to teach their parent what they have been learning over the past week, and then inviting parents to comment on this experience.

To what degree is your class and school inviting to students from various socioeconomic homes, immigrants, different family structures, disengaged parents, or where home is not a safe haven?

After reading the section on “Parental involvement” and “Parental autonomy support,” review the guidance you are currently offering parents about ways they can help their children’s progress.

Turn the ten parental mind frames into a short two-page handout for parents, giving examples of what they might say to their child to exemplify each mind frame.

CHAPTER 7 School and society

DOI: 10.4324/9781003380542-7

Schools do not work in isolation from the society that supports them. Many authors have written on the coupling of society and its education system (Alexander, 2000; Biesta, 2013). Any quest for world-class schools is doomed, as culture beats classrooms and not the other way around. As Clarke et al. (2006) noted, even comparing countries of similar rankings in international tests, there is remarkable variance in what happens across their schools and within their classrooms.

There are core questions in every society about whose knowledge is privileged, there remains major class conflict across neighborhoods and countries, and there are still many vicious inequalities. There is control and power by hierarchies within and outside schools. There are major influences (or sometimes, exclusions based on culture, poverty, gender, and identity issues), and there are continuing amplifications of difference and disabilities. Power is so often validated and supported by systems of knowledge held by experts (e.g., policy makers, teachers, researchers, psychologists), and there are many competing social constructions of childhood, arguments for a child-centered society (or not), and the plea for more recognition of student voice in the purposes and pursuit of schooling. No child leaves poverty, culture, or influences from their home and caregivers at the school gate. For many students, however, school is a safe haven, is a respite from the vicissitudes of home, and certainly, is among the most defining place to develop the mores and values of a society (for liberation or control).

The school’s culture is powerful, and it affects the development of students’ sense of identity and belonging. In some societies, the dominant culture does not always welcome diversity, students from different to majority backgrounds, and those from different racial backgrounds. Make no mistake; racism is a terrible burden. It attacks the spirit. It attacks self-esteem and the soul in ways that those who are not subjected to it would not have an inkling. Racism is a major handicap: Noel Pearson (2021, p. 371) noted that it results in Aboriginal people not having access to opportunities, not recognizing opportunities when they arise, and not being able to seize and hold onto opportunities when they recognize them.

Some have argued that the current corpus of research favors white, Western, and workmanlike students (the gender here is intentional). Indeed, one criticism is that VL is as follows:

closely aligned with agendas of neoliberalism, sexism, and ableism that operate to perform exclusion in schools and de-professionalize teachers in feminized work…. Visible Learning might also be understood as a component of its sexist and masculinist “thrust.” Presided over by a single, god-like male authority, the cult of Visible Learning is about blunt instruments, both at the level of the meta-analysis and what it demands in the classroom. Its fixation on “ejaculatory outcomes,” or rather, “outcums”.

(McKnight & Whitburn, 2020, p. 4)

It is indeed the case that most articles in the original studies are Western-based, from English-speaking countries – if for no other reason than that is where most educational research is undertaken and that the major search engines in the world are in English, and it is not helped by my restricted skills to read other languages (although the translation engines are improving at a fast rate). As shown in Chapter 2, the move of meta-analysis to other countries is increasing. There still needs to be much caution with claims of generalizability to other than Western democratic countries (indeed, to any system where the between-school variance is as large or larger than the within-school variance). Our recent work synthesizing the work in Global South countries has many similar but some critically different messages to this book (Hamilton & Hattie, 2021). We suggest the implications and directions, however, may need to be different, as these countries cannot wait 150 years (as most in the West have) to develop our model of schooling, and maybe they need to bypass our current structures and system to build better (Global South is the more empowering concept compared to terms like the “third world” or “developing countries”; Mahler, 2018).

It is not the case, however, that the studies in the corpus of meta-analyses are mainly white, written by males, nor fail to include diversity among students (poverty rich, white, color, etc.). Because it turns out that many of these moderators rarely change the overall messages of the influences (with important exceptions, as noted in various parts of this book), and this generality to reasonably resourced countries is an empirical finding and not a restriction in the choice of studies. Further, this lack of many meaningful moderators does not mean that teachers should ignore the differences and uniqueness of each student as they implement high-probability interventions (see sections on the centrality of the student and the effects of moderators). Nor does it mean we should not keep searching for moderators.

It does mean that there is a reasonable degree of confidence to interpret the findings from this synthesis in a somewhat generalized manner – what works best tends to work best for all students – provided the interventions are implemented with high levels of fidelity, with attention to the centrality of each student, and with a continual focus on evaluating the impact on each student. I prefer the message “what works best for all students may not work best for minority students, but what works best for minority students is likely to work best for all students.” This adage emphasizes building from the centrality of each child to the whole class, rather than making decisions for the entire class and hoping or adapting the instruction for specific students. The message “know thy impact” is intended to emphasize evaluating the impact of any intervention or choice of intervention on your students. Using evidence from this book is more of a claim about the probability of impact, and maximizing the probability of impact is wise. The core notion is to evaluate the impact than when the intervention is implemented locally.

This sequel, however, is not a book about the sociology of education. But there are remarkable parallels in its essential message to many major social thinkers. Take, for example, the work and writings of Paulo Freire (2018), whose messages are still applicable today in Global South societies and to many mainstream Western classrooms. Freire argued that in typical classrooms of the oppressed, the teacher talks, and the task is to fill the students with the contents of their narration (this surely sounds familiar in many Western classrooms). “The outstanding characteristic of this narrative education, then, is the sonority of words, not their transforming power” (p. 71). The student’s task is to memorize this narrated account. Education is, thus, an act of depositing or banking. The aim is to deposit pearls of wisdom so that the student banks these for later use. Yes, banking implies that knowledge is a gift bestowed by the teacher on the uninitiated, and students should be grateful for these deposits.

Instead, Freire argued that teachers must be partners in learning with the students. The teacher cannot think for the students, they must problem pose and learn with the students, and the student becomes the teacher, and the teacher, the student. This is so close to the major messages in VL about stopping the overemphasis on facts, engaging in dialogue as much as a monologue, being wary about too much tell and practice, reducing the current grammar of privileging facts and content, and engaging in more listening to how students think, process, critique, and know. The themes in Visible Learning relate to acquiring knowledge and then problem posing, understanding the important power of student voice, and seeing learning through the eyes of the students.

A fundamental Freirean notion is that learners must understand how the myths of dominant discourses are, precisely, myths that oppress and marginalize them – but which can be transcended through transformative action (Lankshear et al., 1993). Freire critiqued the then (and still) dominant model of schooling: a “careful analysis of the teacher-student relationship at any level, inside or outside the school, reveals its fundamentally narrative character” (p. 72). That is, the teacher narrates and the student is expected to listen. “The teacher talks about reality as if it were motionless, static, compartmentalized, and predictable…. (The teachers) task is to ‘fill’ the students with the contents of his narration” (p. 71). Education claimed Freire is suffering from narration sickness, and there is a dullness of words and a loss of their transforming power. Not surprisingly, students come to believe that learning is listening to these deposits from the bank of knowledge and ideas of others, which then need to be received, filed, stored – for the day someone checks the status of the bank account in each child’s mind. How accurate that is of too many current Western classrooms.

Some students, particularly those above average, prefer this form of teaching and enjoy building large banks of knowledge to be trotted out when needed. They are good at this game, they are the winners, and they want more of it. Some parents welcome teachers who bestow these gifts of knowledge deposits on their children, and many societies welcome this safe, nonchallenging, and manageable model. Countries with testing cultures love to eat facts for breakfast, dinner, and supper. Freire argued that such a model gives a low priority to developing critical consciousness and is the preferred model of the oppressors to preserve the status quo. (Perhaps it is not surprising that many countries do best on international tests of content that overemphasize content and reduce critiquing and relating ideas; see Zhao, 2014). Most important, this banking model minimizes disequilibrium, rarely engages students in seeing or making relations among ideas, and reduces the challenge of learning to short-term memory, knowing lots, and regurgitating the deposits of teaching. They aim to replicate the society as it was rather than educate students to build their (and our) own futures. These schooling systems are developed more to sort, classify, and select than support learning and critique.

The alternative is to consider teaching as a subversive activity (Postman & Weingartner, 1971), and the aim should be to teach students to challenge knowledge and learning. The aim needs to relate more to entice, educate, and invigorate students and teachers into the thrill of the challenge of building their realities and understandings about themselves and critiques of the worlds around them. That is, to light the spark of learning. We go to school to learn that which we do now know or understand (“powerful knowledge,” Young, 2013), and we are more likely to learn when we are at the edge – the edge between what we know or think we know and what we do not know. This is akin to Vygotsky’s zone of proximal development or Piaget’s disequilibrium. Thus, a core role of schooling is to form new relations between what we know and what we are learning – and then to critique these relationships. Yes, this does require knowing things and building a bank, but it involves then using this bank of knowledge to make relationships among ideas and between what we know and do not know, to discover now what we do not know anew, and to critique what we are taught and know. The use, critique, and interpretation of the bank, not the presence of the bank of knowledge, needs to be the value of schooling. This involves taking risks, being in circumstances where we may not know what to do next, seeing errors and not knowing as opportunities to learn, and having others help us make the optimal connections among ideas.

The claim is not that Visible Learning is an updated Freire, but the claim is there are many parallels in the story of the power of critiquing, knowing thy impact, and developing critical learning skills. Many other sociological theorists have similar messages, often using quite different sources of evidence (e.g., Madeline Hunter, 2004).

Purposes of schooling

Some critics have attacked VL because it is narrowly focused on achievement. Zhao (2018), for example, argued that many have conflated the focus in Visible Learning on achievement as being about the entire education experience. He then proceeds to outline these other experiences, such as critical thinking, problem-solving, social and emotional learning, grit and resilience, and citizenship – and thus, “promoting or discarding educational interventions or treatments based simply on their effects on academic outcomes is not wise” (p. 88). He repeats my own claims – VL is about achievement, there are many other outcomes of educational experiences, we need to consider side effects, we need to consider the interaction of interventions with other outcomes, and indeed, I raised these issues in Visible Learning.

From the outset (chapter 1 in VL1), the following was noted:

It is important from the start to note at least two critical codicils. Of course, there are many outcomes of schooling, such as attitudes, physical outcomes, belongingness, respect, citizenship, and the love of learning. This book focuses on student achievement, and that is a limitation of this review.

(p. 6)

And later, the following was noted:

Levin (1988) has often argued that one of the most powerful predictors of health, wealth, and happiness in adult life relates more to the number of years in schooling than to achievement. Hence, a major goal of schools should be to turn us on to learning (irrespective of where we fall on the achievement ladder) and to assist us to be open to new experiences in learning.

(p. 32, emphasis in original)

But some critics prefer to ignore these clear claims, and there should be no apology for also including achievement as an outcome of schooling. It is hard to imagine schools if there were no focus on achievement (too).

Hattie and Larsen (2020) discuss the various purposes of education. A major claim, as noted in Visible Learning, is to enable students to become their own teachers – and this entails much more than focusing on achievement. This involves students knowing what to do when they do not know what to do, having the skills and dispositions to learn, seeking help, seeing errors and mistakes as opportunities to learn, and knowing how and having the confidence to learn with others. These attributes are often referred to as lifelong learning and self-regulation. The major purpose of education is to make human beings persons or, as Luhmann (2013) elaborated, to minimize the improbability of social communication. We develop through socialization and upbringing. One aim is to develop the ability to learn, learn with and from others, reshape one’s knowledge system, and be creative and competent to improve oneself (Qvortrup, 2006).

There have been long debates about the purpose of schooling, such as passing on core notions of humanity and civilization (or at least one’s own view of these matters), preparing students to live pragmatically and immediately in their current environment (as children), equipping students to live independently, participating in the life of their community, learning to give back, and developing personal growth and occupational preparation. In the past 30 years, however, many Western education systems have prioritized the purpose of schooling as aiming to enhance achievement in domains such as reading, mathematics, and science. Many interpretations of the international testing movement (PIRLS, TiMMS, PISA) have reinforced this narrowness. Such narrow excellence leads to curricula being based on achievement in a few privileged domains, and great students are considered those who attain high levels of proficiency (banking) in these domains. We need a much richer, wider, and worthwhile basket of goods that define the value of schooling (Hattie, 2018; Hattie & Larsen, 2020).

The day-to-day focus of schools also needs to be about learning – how to know, how to know more efficiently, and how to know more effectively. Of course, learning and achievement are not dichotomies; they are related, and learning is always about something (Biesta, 2012). Through growth in learning in specific domains comes achievement. The question is how enhancing a student’s learning skills translates into higher achievement and many other purposes of schooling. As Perkins (2014) noted, the current employment world is less enamored with students knowing heaps of information “because the world we are educating learners for is something of a moving target, itself is much unknown as known” (p. 21). The requirement is more on knowing how to learn and even more on knowing how to learn with others, how to teach others, and how to work in teams sharing your knowledge. Recall that learning is always about something (hence, debates about curricula).

This distinction between learning and achievement has been presented in many different ways over recorded history. Aristotle (350BC/1908), for example, claimed there were at least four ways of knowing: by doing, by demonstration, by intuition, and by practical reasoning. More recently, Ryle (1949) argued that there were at least two types of knowledge: knowing-how and knowing-that. Knowing-how is when you claim you know how to do something (e.g., riding a bike), whereas knowing-that is when you claim you know that something is correct (e.g., explaining the physics of how to ride a bike). You can know how without knowing that, and vice versa. In recent debates, there has been a revisiting of these notions under the banner of twenty-first-century skills. There are many permutations of twenty-first-century skills, but they typically involve creativity, critical thinking, problem-solving, communication, and collaboration. The claim here is that these thinking skills need to be developed within the various curricula content, as (a) they can differ in meaning depending on the content (critical thinking in math can be different from critical thinking in arts); (b) students can be good at critical thinking in one domain but perhaps not in another, as critical thinking is most dependent on the knowledge to then critically think about; and (c) there is no need to develop separate curricula domains of these thinking skills (which some countries are now doing, possibly to perform well on international tests of these skills), but these skills need to be an integral part of each domain.

School influences

There are five major headings under influence of the school (Table 7.1). The current penchant for creating new models or, often, new names for schools (e.g., charter, trusts, academies) has had a minor impact on student learning but absorbs so much debate and financial issues that this is a true example of the politics of distraction (Hattie, 2015). What matters most is a great start to schooling and leadership quality.

Table 7.1 Summary statistics of the five major school influences

School
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

School resourcing
10
925
2,353,777
1,882
0.24
0.20
3

Types of schools
19
941
1,759,886
1,953
0.08
0.03
3

Preschool interventions
50
3,033
448,158
15,875
0.37
0.08
5

School compositional effects
20
537
167,039
1,427
0.13
0.05
2

Leadership
42
1,893
5,731,055
4,299
0.46
0.05
4

Total/average
141
7,329
10,459,915
25,436
0.26
0.08
3

Across the 22 influences related to the school, the average effect of 0.26 is small relative to the averages of the other major influences. These school influences range from policies supporting school choice, creation of different types of schools to more positive effects of school leadership, school climate, and attendance in (and the quality of) preschools. The school influences that matter most relate to what happens within a school than structures, timetables, or types of schools. What matters is located less between- and more within-schools (Table 7.2).

Table 7.2 Summary statistics for the various school and society influences

School
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Finances
6
228
2,289,626
1,048
0.19
0.08
4

Building quality
1
594
54,671
594
0.24
0.08
3

Systems/accountability systems
3
103
9,480
240
0.27
0.20
2

Teacher performance pay
1
40
3,682
40
0.05
0.00
1

Charter schools
6
347
31,937
780
0.04
0.03
3

Religious schools
3
127
64,368
127
0.23
0.00
3

Single-sex schools
2
205
1,601,933
135
0.07
0.03
3

Summer school
6
183
54,377
809
0.17
0.02
4

Summer vacation length
1
39
3,589
62
Ȓ0.09
0.00
1

Modifying school calendars/ timetables
2
77
46,761
674
0.10
0.04
3

Preschool with at-risk students
12
1,330
183,135
9,193
0.51
0.04
5

Preschool programs
22
880
156,966
4,056
0.33
0.16
5

Head Start programs
6
412
46,307
1,288
0.37
0.02
4

Early intervention types
10
411
61,750
1,338
0.26
0.09
5

Desegregation
10
335
37,196
723
0.23
0.06
4

College halls of residence
1
10
11,581
23
0.05
0.00
2

Diversity of student body
3
55
80,526
333
0.09
0.00
3

School size
2
30
2,761
129
0.33
0.00
2

School choice
1
22
2,025
28
0.02
0.00
1

Consolidating/ merging schools
2
47
4,326
169
0.05
0.03
2

Middle school interventions
1
38
28,624
22
0.18
0.00
2

Principals/school leaders
28
1,198
1,416,750
2,993
0.37
0.06
5

School climate effects
13
527
4,298,843
1,138
0.53
0.06
5

School effects
1
168
15,462
168
0.48
0.02
2

Total/average
143
7,406
10,506,676
26,110
0.21
0.04
3

Finances and accountability

Finances

Although the meta-analyses seem to indicate that money does not seem to matter much, this would be a misleading conclusion. It is not the amount of money spent that is important but how it is spent. When more is spent on instruction, the effects are higher – this means investing in professional development (of the right sort, see later), instructional supplies, and teacher time to collaborate on enhancing their collective impact. In contrast, when more is spent on structural solutions (reducing class size, extending the school year, creating new schooling structures), the effects are much lower.

The seemingly limited effect on finances can be related to the fact that (a) most studies have been conducted in well-resourced countries (e.g., the United States, the United Kingdom), where increased funding is on top of a solid base; (b) the variance in resources to schools within a country is not as substantial as it is in developing countries; (c) most finances in schools are tied up not in discretionary but in fixed costs (such as teacher’s salaries, busing, and buildings), and these do not vary in proportion of costs across schools within any country; and (d) if the school composition effects are much more significant at the within-than between-school level, then costs could make a difference within schools and less of a difference between schools (and too much additional spending is currently focused more on between and not within schools).

In a rebuttal to the claims about the limited effect of increased finances, *Green-wald et al. (1996) analyzed the effects of differential school inputs on student outcomes. Their analysis showed systematic, positive patterns in the relations between educational resource inputs and student outcomes. An increase in per-pupil expenditure of $500 increased the effect on achievement by d = 0.15 for per-pupil expenditure, d = 0.22 for teacher education, d = 0.18 for teacher experience, d = 0.16 for teacher salary, and d = 0.04 for teacher/student ratio. Thus, we can expect “comparable and substantial increases in achievement if resources were targeted to selecting (or retaining) more educated or more experienced teachers” (Greenwald et al., 1996, p. 380). It seems a major disjunction that teachers’ salary (for those who stay in the classroom) is more a function of age and experience than expertise. As Hanushek (2013) has long argued, a teacher has few incentives to maximize achievement. Most financial incentives are related more directly to the school than teacher expertise.

The emphasis may need to be not on the notion of “Does money make a difference?” but on “How does money make a difference, particularly above and beyond the fixed costs of running a school (capital, lighting, salaries)?” It is difficult to imagine money does not make a difference at these critical margins. Jonathon Kozol (2005), in his scathing analysis of the existence of apartheid schooling in America, The Shame of the Nation, claims that he will believe money is not important when wealthy parents stop spending so much money on their children’s educations.

*Davis-Beggs (2013) analyzed 20 studies of investment from the US 2009 Race to the Top and American Recovery and Reinvestment Act as to whether or not a positive or negative effect relationship existed between school funding expenditures and achievement. While an overall effect (d = 0.16) is small, there were clear outliers, but the number of studies in this meta-analysis is too small to make definite conclusions.

Some have recommended vouchers for parents to spend on their children’s schooling. *Shakeel et al. (2016) reviewed the effects of voucher programs (on reading and math) in schools that allowed students to attend private schools. They used RCT studies from around the world and found, overall, low effects (d = 0.17) with math trivially higher (d = 0.15) compared to reading (d = 0.08) within the US (d = 0.00) and higher in non-US countries (Columbia and India, d = 0.24). When one program was excluded (from Columbia), the overall effect reduced to d = 0.04. Certainly, it is hard to see support for vouchers in the more well-resourced countries, but maybe some programs might have a small impact particularly where there is a larger gap between private and public schools (as in some developing countries).

The greatest recent investment in some countries is in paraprofessional or teaching aides. In the UK, for example, close to 30% of the salary budget is on aides, who have a zero to a negative impact on students (Blatchford et al., 2012). There has been no meta-analysis of the effects of teaching assistants, and given their prevalence and the overall negative impact, this is well worth doing. Webster et al. (2015) conducted the most extensive study, which found that teaching assistants have serious unintended negative effects on students’ academic progress. The more TAs support a student receives, the greater the negative impact. Compared with teachers, TAs were more likely to prompt pupils and supply answers, their explanations were sometimes inaccurate or confusing, their focus was more on task completion than on learning or understanding, and they often did the work for the students. The assigned students most often needed the teacher expertise but were more likely to be outsourced to the TA, which led to increased levels of learned helplessness. Blatchford et al. (2012; Webster et al., 2015) have written extensively on how better to use TAs, advocating that they should be less used with students with learning difficulties and should be deployed in ways that add value to what teachers do and not to replace them. Maybe we need to think of other roles for TAs, such as working with the more self-regulated students so teachers can attend more to those with special needs, grading (and teachers set up the rubrics and engage in moderation), and lesson preparation.

Does money matter? Yes, if reducing inequities in school outcomes matters; yes, if investment in teacher expertise matters; yes, if we focus on investing in time for teacher collective efficacy and PL sessions; and yes, if we investigate how it is spent wisely to improve instruction. The overall message of VL is about investing in expertise, and this is where the debate about resources needs to focus. There seems to be value in investing in advancing teachers’ expertise from graduate to proficient to highly accomplished to lead (AITSL, https://www.aitsl.edu.au/standards) and allowing for collective efficacy about the impact of teaching on students to be maximized. It costs to have teachers work together outside the classroom and within the school day, and it costs to have principals know how to build collective efficacy in their school and with other principals across schools.

Systems/accountability systems

“Accountability” seems the buzzword for many; based on the Ronald Reagan principle, trust but verify (although some systems have little trust in educators’ expertise). Many states in the United States have high-stakes testing, and there is also much testing embedded in the No Child Left Behind and more recent imperatives. There have been arguments that such frequent testing is akin to a test-coaching effect, whereas others consider that any gains are due to narrowing the curriculum and teaching to the test, and because too many students are excluded who may not perform so well.

Amrein and Berliner (2002) raised much debate with their analysis of the performance of 18 states with high-stakes testing systems and found little effect on student achievement. This conclusion, not surprisingly, was contested (e.g., Braun, 2004; Raymond & Hanushek, 2003; Rosenshine, 2003). *Lee (2006) used meta-analysis to compare different state policies on the National Assessment of Educational Progress examination. He found six studies favored high-stakes testing states, five were mixed, and one favored low-stakes testing states. The effects were extremely varied (d = −0.67 to d = 1.24), and the overall effect was d = 0.36. It made no difference, however, as to the focus of the accountability – that is, whether the focus was a combination of schools and students, d = 0.38; schools alone, d = 0.39; or students alone, d = 0.31. The effects on mathematics (d = 0.38) were slightly higher than on reading (d = 0.29), and higher for elementary (d = 0.44) and middle schools (d = 0.35) than for high schools (d = 0.03). States with the strongest accountability programs made greater gains over the years than those with weaker accountability measures. Lee, however, noted that these gains mapped to similar trajectories from the years before these accountability policies were brought into law. He concluded that “to argue that states adopting strong accountability policies significantly improved student achievement is not convincing until substantial improvements in schooling conditions and practices occur” (p. 26).

*Hendriks et al. (2018) reviewed the impact of evaluation at the class and school levels. Their overall correlation from only seven studies was r = 0.07. The studies often related to implementing an evaluative cycle of inquiry, and the results depended on whether the data collected are accurate, that “teachers and school leaders have the necessary skills to analyze and interpret the data effectively, that educators can provide relevant feedback, … that teachers and school leaders can make appropriate adaptations to teaching and learning” (p. 156), and the attitudes, knowledge, and skills that teachers and school leaders have in evaluation and assessment as well as the strategies they use and the quality of the capacity building.

Higher-stakes accountability may likely have positive effects on those schools in the bottom half of the distribution but little and, possibly, negative effects on those in the top half. If there is to be high-stakes accountability, maybe the reward for being in the top half (on whatever basket of goods) is to be left alone for three to five years. This would be a badge of honor showing the quality of leadership and teaching, creating opportunities to explore and improve without continual fear of outside accountability models and major promotion of the school’s success to parents (and voters and politicians).

Teacher performance pay

A meta-analysis on teacher merit pay concluded the following:

merit pay has the potential to improve student test scores by either motivating teachers to improve their performance or by attracting and retaining more effective teachers. We find that aggregating results from multiple studies across different cultural, economic, and political contexts suggest that incentive pay, as argued by merit pay advocates, offers a promising strategy to improve student test scores. (*Pham et al., 2021, p. 28)

But the effect size was a meager 0.05. Yes, statistically significantly different from zero but so small to be worth worrying about. It may also have equity effects, as Hill and Jones (2021) showed white students performed (slightly) better than Black students under performance pay conditions.

Building quality

Schools need to be healthy, safe, clean, and well-maintained. *Gunter and Shao (2016) found r = 0.12 between the quality of school building conditions and achievement. The effects were similar across curricula, but the effects were higher for elementary (r = 0.20) and middle school levels (r = 0.26) than for high (r = 0.10) and K–12 (−0.21) schools. While these effects are low, there is no reason why school buildings should not be aesthetic, be fit for purpose, meet high standards for quality, and be well maintained.

Types of school

Never underestimate the desire to invent a new schooling model. There are many proposals for schools of the future, charters, trusts, and academics. All you need is a philanthropist, a famous person, an educator with an ego to proclaim, “this is an XXX school” – and you will find advocates. The assumption is that the current model is not working; therefore, we need a new one – despite the evidence that many excellent examples of high-impact schools are under the current model. The aim should be to reliably identify these great schools, understand them, and upscale them. Instead, there is a penchant for inventing a new model and believing it will be better. The reality is that you are running a school within six months of opening a new form of schooling.

Charter and religious schools

The overall effect of different forms of schooling is small indeed (0.04 for charter and 0.23 for religious schools), although the variation is large. *Jeynes (2012), for example, found the overall effect for religious (0.25) larger than for charters compared to public schools (0.01), but the effects for religious schools ranged from 0.68 to −0.07, and for charters, from 0.75 to −0.87. The effects tend to be larger in math than in reading, although *Betts and Tang (2008) found no differences for reading but slightly higher effects for math. More importantly, they concluded that the “estimated charter school effects are highly variable, which likely reflects variations in the quality of education provided at charter schools and comparison schools, namely, local traditional public schools.”

Even within one form of charter school, there is remarkable variation. For example, *Krowka et al. (2017) investigated No Excuses charter schools that emphasize high academic expectations, rigid and consistent discipline, extended instructional time, intensive teacher training, and increased parental involvement. The schools mainly work with students from low-income and minority backgrounds. There were lower effects for reading (d = 0.06) than for math (d = 0.20) in No Excuses compared to public school peers (see also *Cheng et al., 2017). However, when two schools were removed from the sample (one in Idaho and one in San Diego), these math average effects reduced markedly toward zero differences compared with public schools.

Others have found the opposite effects favoring public schools. *Erickson (2013), from 11 studies and over 1 million students, found a negative effect of charters (d = −0.27), with lower achievement scores in reading (−0.25) and mathematics (d = −0.29) when compared to traditional public schools. The advantage of public schools increased in middle and high schools.

The bottom line is that the nature or name of the school seems minor; it is the local implementation that may or not matter; this is the case for public, religious, charter, trusts, private, single-sex schools – or whatever name you want to call them. The variance within schools is markedly higher and more important than the variance between schools. We are unlikely to improve education by inventing new labels for schools.

Single-sex schools

From time to time, there is a resurgence of advocacy for single-sex schools. Much of the interest comes from writers exhorting the advantages for girls citing the differential nature of teacher interactions, intimidation of girls by boys, marking and assessment bias, and the content and presentation of subjects.

When comparing types of schools, it is critical to acknowledge that there may be prior selection effects. For example, it is not uncommon to see studies and newspaper headlines proclaiming that students from single-sex schools are more likely to outperform on national assessments, take academically selective subjects, and are more likely to gain university entry than their counterparts at government co-educational schools. But in so many cases, the selection effects of single-sex schools admitting students who already were high achievers need to be considered. For example, *Pahlke et al. (2014), using only controlled studies (random assignment or controls for selection effects), showed only trivial differences between students in single-sex and co-ed schools: for mathematics performance (g = 0.10 for girls-only, 0.06 for boys-only) and science performance (g = 0.06 for girls, 0.04 for boys), and in some cases, showed small differences favoring co-educational schooling (e.g., for girls’ educational aspirations, g = 0.26), and both boys and girls had slightly higher self-concepts in co-ed schools (0.08 for girls, and 0.11 for boys). Thus, it matters little whether the schools are co-ed or single-sex.

Summer vacation effects

In the early years of formal schooling in America, the school calendar (including the long summer break) was designed to meet the needs of agricultural communities (*Cooper et al., 1996). Today, with only about 3% of Americans’ livelihood being linked to agricultural cycles, there have been calls to reduce the effect of the long summer break on learning. Supporters of change consider a three-month break too long, as children learn best, they claim, when learning is continuous, and the long break means significant time needs to be spent reviewing previous material in order for learning to commence again (*Cooper et al., 2003). *Fitzpatrick and Burns (2019) found single-track year-round calendars with shortened summer breaks (< 20 weekdays) but with similar numbers of total days (about 180) per year were slightly more effective (d = 0.13) than longer summer breaks (40–50 weekdays).

On average, students do lose some achievement gains over the summer (d = −0.09), and the effects were slightly larger in mathematics (d = −0.14) than in reading and language (d = −0.05) (*Cooper et al., 1996). Compared to all other effects, however, these are minor indeed. Middle-class students appeared to gain on grade-level equivalent reading recognition tests over summer (d = 0.13) compared to lower-class students (d = −0.14). There were no moderating effects for gender or race, but summer’s negative effect increased with grade level. It may be that if teachers were more attuned to the proficiencies that students came into their classrooms (e.g., reviewing the reports from the previous teacher, starting a new year in the last month of the previous year so teachers and students could form relations and understand what they bring to the new year) before they started the school year, then the first month of the school year could be used to recapture the losses from the summer break reasonably quickly (and not used to reassess what the previous years’ data could have shown them).

In a large-scale study, but not a meta-analysis, of 18 million students, Atteberry and McEachin (2021) showed the remarkable variation in summer effects. “A little more than half the students exhibit summer learning loss, while the other half exhibit summer learning gains” (p. 269). The average learning loss that a student loses is 39% of their total school year gains during the summer, and these negative effects tend to accumulate over the years. Like Attenberry and McEachin, the implication is not more summer school or more year-round schools but possibly more effective diagnosis to identify these students before or immediately after summertime, ensuring there is more evidence about all students passed from one year’s teacher to the next and moving from a year-based curriculum to a levels-based curriculum.

Summer school

Does going to summer school makes a difference? In general, not much, but it is difficult to ignore even these small gains if they are critical to students who may be already marginal (as that is often the criteria for selection). *Cooper et al. (2000) analyzed 93 summer programs, and their students scored about d = 0.23 greater than those not in summer schools, although the effects were more favorable for middle-than lower-class students. Higher effects were found for programs more specifically tailored to the student needs, when parents were involved, and for mathematics more than reading, and the effects were the same across all grade levels. Both *Cooper et al. (2000) and *Kim (2002) found small effects from both remedial and acceleration summer programs.

Given the length of time students already spend in the regular school year, maybe more attention to the efficiency of teaching during this school year might mitigate these students’ lack of effective teaching, thus giving them their summer back.

Changing school calendars

Among the more common emails I receive from school leaders and teachers (especially math teachers) is about changing the structure of the school timetables, vacation length, and the effects of adding more days to the school year. My typical answer is “make a decision, any decision,” as it does not greatly matter – it is what you do with the time you have that matters. More important are strategies such as deliberate practice time, interleaving, spaced vs. massed practice, and purposeful use of time for learning (see Chapter 11). *Cooper et al. (2003) found low to no effects of modifying the school calendar – lengthening school days, changing vacation length (d = 0.06), and modified and traditional calendar schools (0.01). *Fitzpatrick and Burns (2019) found small effects of all-year classes (0.13).

Note, students in Finland spend 900 hours in school per year; US students, 1,224 hours; South Korean students, 1,600 hours; Poland students, 6,327 hours; and Australian students, almost double, at 11,000 hours. There is little relation between hours in class and (say) PISA rankings (OECD, 2016). If anything, there is a curvilinear relation: the PISA report noted that performance in reading improved with each additional hour of language-of-instruction lessons per week up to three hours, but the relation weakened among students who spent more than three hours per week in these lessons. More important, students who had access to a room and help for homework at school and were offered more creative extracurricular activities (e.g., music, art) performed higher in reading – even after accounting for per capita GDP.

The notion of efficiency was added to the usual effectiveness measures of the impact of schooling. We may pay more attention to doing better with the time we already have rather than believing that more of the same is necessarily going to make a difference. The COVID pandemic demonstrated to many students that they could achieve the same amount at home in far less time than they spend in school.

School compositional effects

With one exception (school size), the compositional effects of schools are small, although all positive.

Desegregation

Desegregation is a process aimed at reducing racial segregation, typically used in the United States, and started in earnest following the Civil Rights legislation in the 1960s. Since Brown vs. Board of Education of Topeka (1954), American schools have been desegregated (which also occurs in other countries), but the resistance has never abated. There is a rise of districts being granted unitary status, which ends their legal obligation to desegregate schools and limits their potential to desegregate voluntarily. Fiel and Zhang (2019) document these impacts, and they hypothesized the following:

processes of race-based status competition related to white resistance and subsequent black dissatisfaction make court-ordered desegregation less effective, less popular, and thus less stable. We expect these dynamics to be most pronounced beyond tipping points where large black population shares make predominantly black schools likely and give blacks greater political influence.

(p. 1695)

During their elementary years in North Carolina, my own children attended a magnet school (these were established to attract white students into predominantly Black neighborhoods), and we could not comprehend why our neighbors cherished the all-white local school where they claimed their kids could be with like-thinking friends. Our local school principal claimed he would welcome our white Australian boys, as this would add “much-needed diversity” into the school. No surprise, we choose the magnet and not the local school.

All the meta-analyses were from the 1980s when desegregation was a hot topic, so the studies are quite dated. The effects are generally positive but small. The effects were larger when students moved into desegregated schools for longer times, more so in reading than math achievement, and for better designed studies. The success of desegregation may be better assessed by the degree it provided opportunities and diversity than achievement effects.

Diversity of student body

The question is whether a high proportion of ethnic minority students in a school leads to more positive or negative educational and social consequences. All three metas on this topic show close to zero effects. There may be negative effects if teachers have lower expectations because these kids are unlikely to learn; they need easy, not challenging material; or lower-quality teachers are assigned to schools with greater diversity. However, there may be more critical reasons to favor diverse student bodies than the effects on achievement, and any low effect indicates that diverse cultural schools can be similarly effective in achievement and likely to excel in other dimensions of communication, socialization, self-respect, and respect for others. Our students will live in a diverse society, so building the skills, attitudes, and disposition to do so effectively can start in our schools.

College halls of residence

This is a university-related topic, and it does not matter whether a student lives in a college, at home, in a fraternity or sorority house, or in off-campus housing or flats.

School size

The optimal high school size is about 600 to 900 students, which typically leads to a school offering a strong core curriculum to all students. There is less likelihood of using electives to stream/track and, thus, diluting the curriculum. Lee and Smith (1997) found that achievement gains in mathematics and reading over the course of high school were largest in middle-sized high schools (600–800 students). Consolidating very small (high schools) may be more important than consolidating into large ones (*Stekelenburg, 1991), particularly in lower SES areas.

The major arguments for addressing school size relate to economies of scale, curricular diversity, student achievement, and sense of belonging effects. Friedkin and Necochea (1988) studied California students in the 3rd, 6th, 8th, and 12th grades. They found that large schools were associated with greater achievement for 12th-grade students, but small schools were associated with greater achievement for students in the 3rd, 6th, and 8th grades. Meier (1996) found that in large schools of over 400 students, about 30% feel a sense of belonging, whereas in smaller schools, about 70% feel a sense of belonging. This increased sense of belonging occurred in small schools because (a) people in small schools are more likely to know and respect each other, (b) the anonymity of large schools increases anger and physical violence, and (c) small schools were less intimidating for parents. But many larger schools set up schools-within-schools or other methods to engender this sense of belonging. There can be immediate savings in moving to larger schools, but once n > 1,000, these gains disappear quickly. It seems the optimal school size is not related so much to a number but how schools, regardless of size, attend to the student’s sense of belonging, the availability of optimal curricula, the non-use of electives to amplify the low to negative effects of tracking, and the use of resources to maximize the collective efficacy and intentional alignment skills of the teachers.

Luyten et al. (2014) found mixed effects for elementary schools and concluded that “one would be tempted to conclude that school size does not matter for cognitive outcomes.” However, the optimal size for secondary schools is about 500–1,000 (cf., Leithwood & Jantzi, 2009), and school size seems to matter more for noncognitive outcomes, such as social cohesions, safety, well-being, and involvement; smaller schools are better for students with disadvantages, during the early years; and there can be diseconomies of scale when > 1,000. PISA and TIMSS showed very small positive effects for smaller schools (but four countries showed high positive correlations – Austria, Belgium, Germany, Italy), and these effects usually disappear when other resource-related variables are added to the analyses.

Consolidating/merging schools

School decision-makers often view school districts’ consolidation or merging as an effective way to save money, increase offerings (both academic and extracurricular), deal with poor-performing schools, and increase student achievement. *Hall (2019) found d = 0.00 for elementary and −0.06 for secondary schools. There can be an immediate negative effect on the first year following the merger (d = −0.27) but no differences thereafter. There are no differential effects on African Americans (0.05), Hispanic students (0.06), or by grade level, sex of student, or socioeconomic status.

School choice

It is hard to avoid the parental debates about the choice of schools and the associated claims that the power of competition leads to improved schools. In my home town of Melbourne, over 60% of students pass their local school to attend to a school of choice. Go to any dinner with parents of school-age children, and the debate turns to the school their children go to, and comparisons are made about the schools. The Australian Government has provided a MySchool web page with much information on schools so that parents are better informed about the school – although the critical information about progress is rarely used, and the all-important information about the nature of the friends, socialization, safety, and fairness of the school is not provided. Further, the results in this book show that it is more the choice of teachers, not schools, that matter, and parents usually have no choice of teacher. *Jabbar et al. (2022) claim to have found “small positive effects” of competition between schools. When the outcome was at the district level, the effect was 0.01; at the school level, 0.01; and at the student level, 0.002. These effects are so close to zero that it is safe to say competition does not enhance students’ achievement.

There are other more indirect but still powerful effects on student achievement from school choice policies (Kornhall & Bender, 2019), particularly residualization in many schools, as the more resourced parents move their children up the ladder of privilege and access. Increased opportunity for school choice leads to increased school segregation (Brandén & Bygren, 2018) and more white-flight (Trumberg & Urban, 2021), and a major reason for the choice is families wanting to avoid their children having to share schools with “low-status children” or children “not like ours” (Smith & Meier, 2016). Such choice can lead to negative spirals for lower socioeconomic schools, and countries (like Sweden) that have moved extensively to school choice models have had major adverse impacts on their overall country achievement results (Kornhall & Bender, 2019).

Middle school interventions

*Grant (2011) reviewed interventions at the middle school level and found very low effects. In the ORACLE study, the largest and longest study of transfer into and out of middle schools, Galton reported that when we compare what teachers did 20 years before and now, they still talk at rather than with their students; in science, there is now even greater emphasis in the middle schools on transmitting information rather than on problem-solving or hypothesis generation; 19% of instruction in classes was whole-class instruction in 1980, and this almost doubled to 35% in 2011 – and this increase came at the expense of teachers listening to students. Teachers still make four times more statements to questions, and for the most part, classrooms are “dominated by teacher talk and most of this talk was of a cognitively low level” (Hargraves & Galton, 2002, p. 104). In many senses, middle schools are, thus, preparing students for high school – as they are dominated by telling what to do and imparting facts, and there are high levels of repetition of work across middle and high school years, which leads to the phenomenon known as the hiatus in progress (or cruising, given the difficulty of work stays the same over the years) – which has been noted throughout the world as a consequence of school transfer: leading to a drop in attainment, less positive attitudes toward subjects, increased dislike to teachers and school, and decline in self-esteem. While many students recover, about 10% do not, and those in academic need are mostly represented in this 10% (Eccles & Midgley, 1997). Once again, debating forms of schooling are of low consequence if the teaching and impact on students do not improve.

Preschool intervention effects

There have been 50 meta-analyses on preschool programs based on 3,033 studies, about 721,719 students, and 15,875 effects, with an overall d = 0.38. The effects are somewhat larger for preschool with at-risk students. The benefits of early intervention are evident over various outcome variables (including IQ, motor skills, language, and academic achievement) and across a wide variety of children, conditions, and types of programs. The best early predictors of achievement in these meta-analyses are teaching students not to be so distracted, addressing behavior problems, developing language, and enhancing cognitive functioning (including executive functions).

Early intervention programs are more effective if they are structured, they include about 15 or more children, and the children are in a program for up to 13 hours a week. This effect accrues similarly for regular and at-risk, disabled, and special education students. The effects, however, reduce over time (by age 8, it can be difficult to determine who did or did not attend preschool programs). Thus, there is a need for systematic, sustained, and constant attention to enhancing learning if these early gains are to be maximized (*Casto & Mastropieri, 1986; *Cooper et al., 2010). For example, for disadvantaged populations, the immediate benefits decline rapidly and largely disappear after 60 months once they start school (*Casto & Mastropieri, 1986; *Kim et al., 1996; *White & Casto, 1985). *Gilliam and Zigler (2000) synthesized the effects of preschool attendance across 13 American states and claimed there were sizable effects (d = 0.20–0.30) on achievement by the end of preschool, although these effects were not evident by the end of first grade. This lack of longer-term effect is greater for preschool programs that overemphasize play and social and emotional development but far less for those programs that also focus on developing language (e.g., concepts about print), numeracy (ordering, patterning), and the skills of learning (attending, reducing distractions, and working with other peers) – all can be developed using play.

Care is needed not to imply that a low correlation between preschool and later learning necessarily means that preschool is not predictive or powerful. Instead, the sustaining hypothesis claim is that the long-term effects can be a function of the lack of success of teachers and environments subsequent to the child leaving preschool to capitalize on head starts of ECE.

One exemplar study illustrates the typical findings. The Effective Early Educational Experiences (E4Kids) was a five-year longitudinal study designed to assess the impact of everyday, approved early childhood education and care programs on about 2,500 children’s (age 3–8) learning and developmental outcomes (Tayler et al., 2013). The study did not find a clear result for the relative, independent contribution of individual structural factors to children’s outcomes (such as adult-child ratios, physical space, personal care/hygiene routines, and materials) but did confirm the importance of intentional teaching that focused on the development of children’s language and thinking: building vocabulary, having language-rich back-and-forth exchanges between adult and child, and promoting the expressive aspects of children’s language usage.

Overall, the study found that too many early childhood programs delivered small effects on children once they started school, and there remained large gaps in quality and outcomes related to SES. These gaps were not narrowed from attendance at early childhood settings – unless there were high levels of intentional teaching with high-quality adult-child engagement in years 1 and 2. Only 13% of settings met the high-quality bar in 2015, and there were few to no quality services in the low SES areas (Cloney et al., 2016). This study led to nationwide quality standards for settings, services, and staff (see www.acecqa.gov.au/nqf/national-quality-standard), and this has massively improved the quality and impact of the centers. By 2020, over 75% of all services met the national quality standards based on educational program and practice, children’s health and safety, physical environment, staffing arrangements, relationships with children, collaborative partnerships with families and communities, and governance and leadership.

Type of preschool program

The type of preschool seems to be a significant moderator. *Chambers et al. (2016) reviewed 28 early childhood programs and found six with the highest impact (d > 0.20): Curiosity Corner, Direct Instruction, ELLM, Interactive Book Reading, Let’s Begin with the Letters, and Set Leap!) and argued that the common denominator was that they all focused on language, literacy, and phonological awareness. Head Start programs were reasonably systematic in finding higher effects (d = 0.33). For example, *Mares and Pan (2013) analyzed 24 reports on the impact of Sesame Street, with an overall d = 0.29: cognitive, 0.28; learning about the world, 0.24; social reasoning and attitudes, 0.19; and much higher for those watching from low-SES populations (0.41).

*Nelson et al. (2003) reported that the effects of these preschool programs were greater if students were in them for at least a year and were particularly higher for minority students. Performance on standardized mathematics assessments (d = 0.25) and reading (d = 0.20) were higher for participating than for nonparticipating children. *Fusaro (1997) found that children attending full-day kindergarten showed greater achievement (d = 0.18) than those attending half day (*Cooper et al., 2010), and this was the case for both boys and girls (*McCoy et al., 2017). *Cooper et al. (2010) found an effect from full-day kindergarten compared to half day on achievement of 24, but this effect fades by the end of third grade. By the upper grades (grades 7–11), a slightly higher percentage of underachieving students who had participated in preschool intervention programs did not need special education and were not held back a grade (*Goldring & Presbrey, 1986). *Brunsek et al. (2017) were more interested in how a well-established scale of classroom quality (ECERS) could help understand the more salient influences in early childhood settings. The ECERS rates structural (e.g., staff/child ratios, physical environment) and process (e.g., interactions that occur within the child’s environment) quality, and neither was related to child outcomes. The only one they highlighted was the subscale that assessed the quality of language used in the classroom rather than materials available to children. Thus, “encouraging children to communicate and use language to develop their reasoning skills” is a core predictor of success in ECE settings.

There is little support for the widely held belief that the involvement of parents leads to more effective early intervention (*Casto & Mastropieri, 1986; *Casto & White, 1984), except that there is support for the claim that those most in need (disadvantaged students, for example, students from lower socioeconomic areas, or minority students) had higher impacts when parents were more involved (*Collins, 1984; *Harrell 1983).

Focusing on developing language is the core skill to provide children with the best start to learning. This means developing their facility to hear, decode, understand, and use language, language, and language. Play does not appear in these studies as critical unless it is play that involves language (see Chapter 13). Lillard et al. (2013) conducted a major systematic review of pretend play on young children’s development and concluded that “existing evidence does not support strong causal claims about the unique importance of pretend play for development” (p. 1). There is little convincing evidence for showing that play affects creativity (there is some support for play on developing solving problems that involve construction), conservation, theory of mind, or social skills. Their conclusion: “Taken together and examined closely, these studies present a dim view for the oft-made claim that pretend play importantly enhances cognitive development,” and their “truth” is “that we do not have a good basis of evidence from which to claim that pretend play is crucial to development.”

While these effects show the advantages of preschool learning experiences, maybe they are not as large as they should be – especially for students who most need the language, theory of mind, concepts about numeracy, and so on. There is also the effect of the early years in school that can dissipate any early gains from early childhood. For example, Chetty et al. (2011) estimated that there could be decays of over 75% in these first few years of schooling.

Leadership influences

In his biography outlining his experiences of starting and leading schools, John Marsden (2021) noted that “I’ve seen a bad principal destroy a good school in less than a year. I’ve seen a great principal turn a bad school around in a couple of weeks” (p. 99). There are major contrasts between those leaders who can make things happen in their schools and leaders who cannot (Macklin & Zbar, 2020). While the direct impact of school leaders on students is not as high as that of teachers, leaders can still have an enormous positive difference in the learning lives of students and teachers. The average effect from 17 meta-analyses on leadership (principals, superintendents) on student achievement, based on over 600 studies, 1,350+ effects, is 0.40, which is appreciable for an indirect effect on student achievement.

The argument from a review of these 17 meta-analyses is that those can-do leaders share five major attributes:

Developing a shared narrative about all in the school, maximizing impact on students.

Having high expectations and promoting challenging goals for all.

Providing supportive processes and structures to do this work.

Being proficient in leading the collective to maximize this impact.

Knowing how to effectively implement, evaluate, and improve the impact of interventions across the school and within classes.

Thus, there are reduced impacts for charisma and charm, for the hero or great person model, or for the cajoler and transformation leader preaching the vision and raising the flag that all should march behind. It is less about providing individualized support to staff members, modeling behavior, providing contingent rewards, being aware of the personal aspects of teachers, adapting their leadership behaviors to the needs of the current situation, and managing by exception. It is also less about the transformational leader who is more concerned with the mission and ensures all are behind this mission. As *Robinson et al. (2008) demonstrated, it is more about instructional leadership and the enhanced impact of such leaders on student outcomes (d = 0.42; which were much greater than the effects of transformational leadership, d = 0.11). A high-impact leader may need transformational attributes, but they are preconditions for developing a shared mission about impact. It is leaders who place more attention on the collective power of all, focusing on the impact of teaching, leading, and learning on the students, and creating safe, inviting, and fair environments and structures for all (teachers, resource staff, and students) to succeed that has the greater impact.

In Visible Learning, I distinguished between “instructional” (powerful) and “transformation” (not so powerful) notions of leadership, but these terms are becoming so muddled that it may be time to move past them. High-impact school leaders have their major focus on creating a school learning climate free of disruption, that is fair and safe for all, developing a powerful sense of collective efficacy, collaborative expertise, and high teacher expectations for all teachers and students. High-impact leaders engage with their teaching staff in ways that inspire them to new levels of energy, commitment, and moral purpose; find ways to preserve the time for teachers to be adaptive experts in their focus on student impact; and spend much attention on developing and enacting a vision for the school about impact – and enable the collection of evidence to realize this vision. Robinson (2011) argued that these attributes are more present and effective in elementary schools (and among department leaders in high schools) and, particularly, in low-performing and turnaround schools. In a similar vein, Bendikson et al. (2011) demonstrated in schools needing major improvement that high-impact leadership is critical, whereas in already well-performing schools, it may be less an imperative (but still worthwhile).

To become this high-impact leader, there are prior conditions to attaining high impact, such as supportive, fair, and safe management in the same ways teachers need to have reasonable levels of supportive, fair, and safe classroom climates to then make the difference. In addition, high-impact leaders also need to be good problem-solvers, have good management, can build the skills and confidence among their teachers to collectively have and realize high expectations about progress to achievement, and be able to influence their teachers to deliver on the school’s mission.

The prime attribute of highly successful school leaders (including principals, superintendents, department heads, deputies) is that they can and have developed a shared narrative about the mission of all in this school toward maximizing impact on all students (where all means all). In their overview of school leadership research, Hallinger and Kovačević (2019) noted the move from a focus on management to leadership, and more recently, they argued that the cognitive anchor of the research is “leadership for student learning and development.” These leadership qualities are highly attuned to the notion of equity as enabling all students to gain at least a year’s growth for a year’s input (and have shared, validated, and agreed notions on what this year’s growth means and what evidence of this growth looks like) and removing the predictability of success or failure that currently correlates with any social or cultural factor (National Equity, 2018). “The more leaders focus their influence, their learning, and their relationships with teachers on the core business of teaching and learning, the greater their likely influence on student outcomes” (*Robinson et al., 2018, p. 23).

Such high-impact leadership relates to making the school and classes inviting places for students and teachers to come and invest in learning, growth, well-being, and learning about respect for self and respect for others. It is focusing all on the core business of teaching leading to learning, having clear visions about what desirable learning is about, promoting appropriately challenging goals for the teachers and their students. This often requires leaders who are willing to challenge the status quo.

This high sense of moral purpose is lived, is demonstrated to all, and is the font of building trust and relationships, as is the leader’s perceived and real knowledge about current curriculum, instruction, and assessment practices. These leadership qualities translate into establishing clear success criteria, especially for teachers; knowing how to hone in on the most critical and challenging goals; supporting teachers with high probability strategies that are well implemented; and setting and delivering on high expectations that all students can improve and gain at least a year’s growth in learning.

High-impact school leaders know, can create, and support the conditions of work for these high expectations to be realized. They ensure an orderly and supportive environment, such as protecting time for teaching and learning by reducing external pressures and interruptions, and establishing an orderly and supportive environment both inside and outside classrooms (d = 0.49), which can excite and intellectually stimulate teachers about maximizing impact, learning the high probability impact innovations (d = 0.64), and establishing safe environments for teachers to critique, question, and support other teachers to reach these impact goals together. This focuses on learning where teachers meaningfully collaborate, which leads to collective efficacy (see Chapter 12).

One of the critical implementation factors is building the collective (and thus, moving away from a hero or great person leadership). For example, building collaborative structures (d = 0.35); promoting and participating in teacher learning and development (d = 0.91); planning, coordinating, and evaluating teaching and the curriculum (e.g., direct involvement in the support and evaluation of teaching through regular classroom visits and provision of formative and summative feedback to teachers, d = 0.74); and strategic resourcing (aligning resource selection and allocation to priority teaching goals, d = 0.60 – all effects from Robinson et al. (2008). *Sun and Leithwood (2012) also noted an effect of 0.35 from building collaborative structures.

Perhaps one less researched topic about effective leadership is implementation and evaluation skills. That is, whether school leaders monitored the effectiveness of school practices and their impact on student learning (r = 0.56) – implementation is key. A bad implementation can ruin a school climate because it may create moments of distrust and confusion. Conversely, a positive implementation can bring staff together and build collective efficacy. We are struck by our work with leaders, how few have implementation models, and our education literature also has few models (unlike in business, engineering, and medicine; see Hamilton et al., 2022).

But do teachers welcome this notion of leadership? Moir et al. (2014) interviewed high school teachers about what they perceived as effective leadership. The teachers identified their department heads more than the principal as more influential leaders. They described effective leaders as those who created positive relationships, communicated effectively, led by example, and supported the teachers. The major theme was that they preferred transformational leaders: leaders who understood and respected them, and preserved their autonomy about how they taught. For these high school teachers, effective leaders were less involved in collective efficacy, interpretation of evidence, or promotion high expectations. This was a small study but hints at why it can be so hard to implement high-impact leadership: it requires expertise, it requires building trust, it requires appreciating and knowing about the content and curricula to ground the narrative then around impact, and it requires a commitment to longer-term focus on moving the narrative away from debates about teaching, resources, or conditions to also focusing on the impact of all in the school on the student’s learning lives.

School climate

Schools have climates – some inviting, some less so. *Bulris (2009) reported an effect of d = 0.74; and *Bektas et al. (2015) reported d = 0.47 between school culture and achievement. *Scheerens et al. (2013) investigated many school climate factors, and the highest effects were an orderly climate, opportunity to learn, effective learning time, and orientation to achievement across the school. Very low effects were found for consensus and cohesion among staff, the presence or not of homework, parental involvement, and differentiation.

From Turkish studies, *Kocyigt found d = 1.12 of climate on achievement, with the greatest impact from the perception of culture, collaborative leadership, program development, collegial support, and unity of purpose. Also from Turkey, *Karadağ et al. (2016) reported an effect (d = 0.36) on the climate developed by school leadership on achievement. They argued that the higher impacts include support, communication, trust, and respect developed by focusing on continuous learning and teaching, and establishing intentional, positive, and confidential relationships with their school managers, colleagues, and stakeholders.

At the college level, *Credé and Niehorster (2012) showed that the greatest impact on students’ GPA (i.e., the degree to which students have adapted to the academic demands reflected in their attitudes toward their course), their engagement with material, and the adequacy of their study efforts (r = 0.39), then institutional attachment – the degree to which students identify with and have become emotionally attached to the education community (r = 0.19). These all are aspects of a sense of belonging (see Chapter 8). There were similar effects for personal-emotional adjustment (r = 0.17) and social adjustment (r = 0.11). Thus, they claimed this shows the importance of coping strategies and social support for students’ ability to adjust to college.

Coping styles that reflected engagement with the problem (especially a proactive coping style) were moderately positively related to adjustment whereas negative relationships of similar strength were found for avoidant coping approaches…. Social support from peers was most strongly positively related to social adjustment whereas support from faculty and the institution was most strongly positively related to academic adjustment and institutional attachment.

(p. 157)

School effects

There has been a long debate about how much of the variance in achievement can be attributed to the differences between schools. Rutter (1979) noted that successful schools tend to differ in the attitudes of their staff and pupils and, hence, in their overall climate or atmosphere compared to unsuccessful schools. Schools did not vary as a function of the overall expenditure per pupil, age of school buildings, amount of space available, number of books in the school library, teacher-pupil ratio, class sizes, and teacher qualifications. What matters, claimed Rutter, was academic emphasis (high expectations, clear academic goals, an appropriate degree of structure, active instruction, and a task-focused approach).

The variance between schools is often far smaller than the variance within schools – even adjusting for students’ socioeconomic status – but this can vary by country (Konstantopoulos, 2006). For example, Lietz (2009) used the Australian national assessment data to show that about 20% of the differences in performance at the 15-year-old level are found between schools, while around 80% are among students within schools – similar to Canada, the United Kingdom, and New Zealand, but much greater than Finland (about 5% between-school variance) and smaller than Germany (52% between-school variance).

There are many models to address the question of how much variance can be attributed to between-school differences, and these models can vary depending on whether they account for differences in the intake characteristics of students at each school, whether achievement or learning gains are the focus (again with or without contextual controls). The effects hover around 18–25% overall and half this when adjusting for intake differences (*Sheerens & Bosker, 1997). The effects are larger for mathematics and composite academic performance measures than language and larger for secondary than primary schools. There can be considerable variations in these estimates. For example, in science, across 69 countries, the average is about 30% and within, about 70%, but note the between-schools estimates for different countries: Iceland (4%), Finland and Norway (8%), compared to Australia (25%), Hong Kong (22%), New Zealand (21%), the United States (21%), and the larger effects for the Netherlands (65%), Bulgaria (59%), and Hungary (57%).

Marks (2006, 2016) found the between-schools estimate for Australian schools was 18%, declining to 11% when considering students’ socioeconomic backgrounds. This means, for example, using the national assessment scales (which have scores ranging from 200 to 700), a difference of only about 25 score points separates schools close to the top and the bottom of the continuum of school effects and the effects (not surprising given how small they are) bouncing about over years (c.f., Timmermans & van der Werf, 2017). Lloyd and Schachner (2021) estimated 0.17, Jennings et al. (2015) estimated an effect of 0.14 between-school variance, and Deming (2014), about 0.10.

Among the more systematic studies of the size of this school effect is *Scheerens and Bosker’s (1997) meta-analysis. They used hierarchical linear modeling based on 168 studies and found that, on average, schools account for 19% of the achievement differences among students and 8% when adjusting for initial differences among students. The school effects were higher for mathematics than language, and less developed countries had higher between-school variance than Western countries. They also concluded that “schools matter most for underprivileged and/or initially low-achieving students” (p. 96). I conclude that we spend too much time debating differences between schools, whereas our focus should be on the differences within a school.

Concluding comments

We do not create our students’ future; they do and will.

A major purpose of schooling is to educate students to become their own teachers, as this maximizes the probability; they will be more informed to choose the content they wish to learn, know how to learn, be curious about the content, and critique this content, and learn how to engage others in this learning.

It is how the resources are used within the school, what the ethos are within the school led by the school leaders, and what the expertise of the school leaders and teachers are that matter.

The climate of the school is critical to ensure it is an inviting place to come and be and learn, and they need to ooze fairness, predictability, opportunity to learn, and excellence.

What matters are leaders who can develop a shared narrative about all in the school, maximizing impact on students; who have a moral purpose and knowledge about the desired impact across the school; who have high expectations and promote challenging goals for all; who provide supportive processes and structures to do this work; who are proficient in leading the collective to maximize this impact; and who knows how to effectively implement, evaluate, and improve interventions across the school and within classes.

It is not charisma and charm but competence to reliably identify concerns, deal with difficult problems, engender trust, and develop a collective for all in the schools to ensure every child gains more than a year’s growth for a year’s input (and making this goal concrete).

A major role of leaders is to develop, esteem, and highlight the culture, climate, and collective impact for all educators and students to feel invited to thrive and learn.

The greatest source of variances is within, not between, schools in most Western countries.

We do not create our students’ future; they do and will. I am sure my parents said that the world would be different when I grew up, there will be jobs not thought of today, and what I needed was a slight updating from what they were taught in schools (and they were told the same). Schools, then and now, do not work in isolation from the society that supports them, and they can instill values that we want our children to emulate, but there are no guarantees. The claim is that a major purpose of schooling is to educate students to become their own teachers, as this maximizes the probability that they will be more informed to choose the content they wish to learn, know how to learn, be curious about the content, and critique this content, and learn how to engage others in this learning. But becoming a teacher is still about something: the moral purpose questions of the purposes of the school need continual debate.

Throughout this chapter, there have been claims about the sources of between-school variance. Go to a school in a poor area of the city and ask if it looks the same as one in a leafy suburb, ask if poverty is left at the school gate, see the resource differential among some schools, and ask parents about their views on differences between schools, and so often, the answer relates to the shape, structure, size, and similarity to their desired ethnic/SES dreams. Note how policy makers continue to invent new forms of schools and soon discover they are schools. A major finding throughout this chapter is that what happens within a school is so much more powerful than the differences between schools – but while the policy mavens and parental clamor and fuss about between-school differences, they create politics of distraction away from where we should be putting our time, energies, and expertise.

But then the common plea is to throw more money at everybody. Such lolly scramble solutions (https://en.wikipedia.org/wiki/Lolly_scramble) are aplenty, but more often, it is more about how the money is spent than how much is spent. It is how the resources are used within the school; what the ethos are within the school led by the school leaders; and what the expertise of the school leaders and teachers are that matter; and what matters more is the choice of the teacher, not the school. This does not mean that schools are sorely needing a major upgrade (and this costs) either in the quality of the buildings, ability to attract high expertise, and excellent leaders. South Australia, for example, assigns extra financial resources to schools in need for them to advertise specifically for a highly accomplished or lead teacher who retains this salary increase while they stay at that school, and this is redistributing expertise across the system most effectively. Does money matter? Yes, if reducing inequities in school outcomes matters; yes, if investment in teacher expertise matters; yes, if we focus on investing in time for teacher collective efficacy and PL sessions; and yes, if we investigate how it is spent wisely to improve instruction.

To ensure schools are high quality, there are good claims to trust but verify, and reward these high-quality instructional schools with earned autonomy to continue high-quality impact. Instead, we love to invent new names and types of schools; classify students based on their ability, sex, length of the school year, religion; tinker with school calendars; and consolidate and merge schools. We then permit parents to choose schools and create contingencies to create competition to be the best, whereas the neighborhood school should be the best in the neighborhood, and the next best is the school just down the road. Many parents choose schools based on “kids like mine” and who they want their child to be friends.

Apartheid in many schools is alive and well – in many schools, walk the corridors and count the color in the top and bottom tracks, walk the playground and watch the friendship groups, and discover whether the school welcomes diversity in race, gender, culture, and ability by asking the students. The school climate is critical to ensure it is an inviting place to come, be, and learn (Purkey et al., 2020), and school climates need to ooze fairness, predictability, opportunity to learn, and excellence for all.

Where there are structural effects, it is often a proxy for a deeper concern. For example, the size of high school seems to matter, which is fascinating, as it is not the size but the opportunities and restrictions of size. It is the opportunity to offer a core of subjects and not use electives as yet another form of streaming or tracking. Going to preschool matters when this learning focuses on developing language and a student’s curiosity to understand their world, explore patterns and numbers, and learn to reduce distractions. But the greatest start needs to be accelerated and built upon. This requires intent to teach, development of skills in working with others, and expertise in teaching. Developing an invitational climate is the prime responsibility of leaders across the school.

Leaders matter. Leaders who can develop a shared narrative about all in the school, maximizing their impact on students; who have a moral purpose and knowledge about the desired impact across the school; who have high expectations and promote challenging goals for all; who provide supportive processes and structures to do this work; who are proficient in leading the collective to maximize this impact; and who knows how to effectively implement, evaluate, and improve interventions across the school and within classes. It is not charisma and charm but competence to reliably identify concerns, deal with difficult problems, engender trust, and develop a collective for all in the schools to ensure every child gains more than a year’s growth for a year’s input.

At the annual Visible Learning conference in Denver, 2022, we elaborated on the ten mind frames for culture and climate.

Equity

1. We discover, correct, and disrupt inequities.

2. We embrace diverse cultures and identities.

3. We recognize and disrupt biases.

4. We create equitable opportunities and eliminate barriers to opportunities.

Identities

5. We cultivate fortifying and sustaining environments for all identities.

6. We acknowledge, affirm, and embrace the identities of all our students.

7. We remove barriers to students learning, including barriers related to identities.

Belonging

8. We strive to invite all to learn.

9. We value engagement in learning.

10. We collaborate to learn and thrive.

Reflections

After reading the first section and “Purposes of schooling,” do we need to review the balance in our teaching between getting students to acquire knowledge and getting students to use knowledge, by making relationships among ideas and between what we know and don’t know? It is the use, critique, and interpretation of the bank of knowledge, not its presence, that needs to be the value of schooling.

Is there an agreed statement of basket of goods you want to develop in your school, and do parents and student share them?

Can we use the five major attributes of effective leaders on p. 172 as a vehicle for ensuring the implications are included in school policies, both in the rhetoric and in reality? Read the whole section on “Leadership influences” to unpack what we need to emphasize, what we need to underplay, and what we need to review in our leadership.

Debate: Principals need to know what it means to be a great teacher, preferably have been one themselves, feel it, esteem it, and knowing this, grow and esteem and promote great teachers.

Rate your school using the ten culture and climate mind frames. Then devise strategies to enhance the culture and climate of your school. Be sure to ask students their views on these ten mind frames as part of your evaluation.

CHAPTER 8 Classroom effects

DOI: 10.4324/9781003380542-8

The structure and climate of classes set the scene for what happens in them. The various influences are hotly debated, and teachers and students (and parents) wait each year for assignments to larger or smaller classes, high or lower track classes, mainstreamed or special classes, hoping for highly engaged and well-behaved students and friends. The desire is for the class to have social cohesion, great classroom management, minimal disruptive behavior, high levels of friendship and no bullying, and appropriate interventions and counseling for those in need or most likely to affect other students.

There are three major categories of within-class effects: the composition of the class, management of the class, and classroom climate (Table 8.1).

Table 8.1 Summary statistics for the three major classroom influences

Classroom
No. metas
No. i studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Class composition effects
57
1,970
5,593,250
4,870
0.19
0.06
3

Classroom management
33
1,588
411,706
5,929
0.24
0.07
3

Classroom climate
30
1,194
4,681,462
7,890
0.24
0.03
3

Total/average
120
4,752
10,686,418
18,689
0.23
0.06
3

The highest classroom effects relate to classroom cohesion, management, teacher-student relations, friendship, and a sense of belonging (Table 8.2). The lowest effects relate to the more structural aspects of classes: open vs. traditional, tracking or streaming, reduction of class size, and classroom grouping. The more negative effects are bullying, retention, feeling of being disliked, overdependence on the teacher, and suspension and expulsion.

Table 8.2 Summary statistics for the various school and society influences

Class composition effects
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Reducing class size
9
181
570,652
1,242
0.17
0.06
4

Open vs. traditional classrooms
4
315
28,992
333
0.02
0.03
3

Tracking/streaming
14
469
43,166
1,136
0.09
0.05
4

Detracking
1
15
15,577
22
0.09
0.08
2

Within-class grouping
3
144
28,662
209
0.16
0.08
3

Small-group learning
7
197
18,755
278
0.46
0.07
3

Multigrade/age classes
2
67
6,167
45
−0.01
0.08
2

Mainstreaming/inclusion
11
445
4,868,670
1,189
0.52
0.07
5

Ability grouping for gifted students
6
137
12,609
416
0.21
0.06
3

Classroom management
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Classroom management
4
179
16,247
84
0.43
0.09
3

Classroom breaks
1
18
1,657
18
0.16
0.02
1

Bullying
5
81
290,204
84
−0.28
0.07
3

Decreasing disruptive behavior
2
140
5,940
315
0.82
0.03
3

Suspension/expelling students
1
24
7,579
42
−0.20
0.08
2

ADHD treatment with drugs
10
476
28,609
1,875
0.30
0.06
4

Intervention programs for ADHD
5
350
32,213
3,224
0.44
0.24
4

Counseling effects
5
320
29,256
287
0.29
0.08
3

Class climate effects
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Class climate effects
3
80
582,941
761
0.29
0.02
4

Strong classroom cohesion
2
76
11,187
438
0.66
0.08
3

Teacher-student relationships
5
428
590,784
1,718
0.62
0.04
5

Teacher-student support
1
93
8,560
93
0.32
0.05
2

Teacher-student dependency
1
8
3,808
8
−0.24
0.04
1

Friendship
3
60
5,522
229
0.35
0.05
2

Belonging
3
97
78,931
174
0.46
0.08
3

Students feeling disliked
1
5
1,776
5
−0.26
0.08
1

Relative age within a class
1
8
3,347,259
8
0.45
0.00
2

Retention
10
339
50,694
4,456
−0.24
0.04
5

Total/average
119
4,659
10,677,858
18,596
0.22
0.04
3

Class composition effects

Reducing class size

There is more heat generated by the class size question than all others. I have received hate mail, disbelieving emails, and harassment and boos. The common refrains are “of course it matters” and “it does affect my workload and stress levels,” and I have often been invited to take a class of 30 year 9s on a Friday afternoon (and yes, I have done this many times). So let me be clear, an overall effect from eight meta-analyses of 0.18 simply means reducing class size enhances achievement. Simple. I have never said, “class size does not matter” but have expressed surprise that such an obvious contender for influence of the epoch is so low. Surely, if reducing class size meets all the claims attributed to it, the effect size should be higher.

Blatchford and Russell (2020) have the best synopsis of this topic. Their book takes a deep dive to understand the nuances, the dilemmas, the arguments, and the moderators, and their plea, like mine, is to understand more the dynamics within smaller and larger classes to explain the rather low influence of such a costly intervention. They examine the effects of class size on many outcomes, explore how reductions in class size influence classroom processes, and then focus on how to make the most of any reduction in, or changes to, class size to enhance student achievement.

They could not find nonlinear effects, such as greater effects when class sizes reduce below 20 (as *Glass & Smith, 1979, reported), although *Bowne et al. (2017) investigated the effects of class size in early childhood settings and found a nonlinear effect. For child-teacher ratios 7.5:1 and lower, the reduction of this ratio by one child per teacher predicted an effect size of d = 0.22. For class sizes 15 and smaller, one child fewer predicted an effect size of 0.10. But no discernible relationship was found for larger class sizes and child-teacher ratios. They concluded that “class size and child-teacher ratio may both need to be very small to facilitate higher quality interactions at the level necessary to yield modest increases in child cognitive and achievement outcomes” (p. 422; see also *Perlman et al., 2017). There may be positive effects on reducing chronic absences, and students are less likely to withdraw in smaller classes (Tran & Gershenson, 2021).

When we investigate the effects on what happens in classes, then we see any important effects. In smaller classes, there is more (but not a lot more) individual attention and one-to-one interactions between the teacher and the student, and students become less passive (Finn & Cox, 1992), but even in smaller classes, whole-class teaching and individual work still dominate. Reducing class size on its own is not enough to guarantee effective teaching, and learning will follow.

Teachers do not necessarily change the way they teach when faced with smaller classes, even though they may feel they change…. changing the number of pupils in a class may not be enough for teachers to change the curriculum and style of teaching very much.

(Blatchford & Russell, 2020, p. 298; cf., Hattie, 2005)

A typical response to this lower-than-expected effect of reducing class size is to note that many of the more powerful influences identified in this book could be more effective if the class sizes were lower. With smaller classes, goes the plea, there could be more feedback, more interaction with students and among peers, more diagnosis, and so on. But the evidence indicates that when class sizes are smaller, if these influences are indeed implemented, there is still no great difference in student outcomes. Maybe professional learning to adjust to teaching smaller class sizes might be needed to optimize the avowed benefits more fully. It appears that the effects of reducing class size may be higher on teacher conditions (Smith & Glass, 1980; d = 0.49 on workload, morale, attitudes toward students), which then may or may not translate into effects on student learning.

The effects of class size reduction on achievement are relatively small compared to all effects and about average for classroom effects. As outlined in Blatchford and Russell (2020), we need more sophisticated investigations of the influences of the size of classes on classroom practices, teaching methods, and on more than academic outcomes. I also note the increasing number of adults in classes, such as teaching assistants, and the exciting possibilities of classes of 80–90 students in a class with 3 teachers working together. There are many comparisons to make about the ratio of adults to students, and we need more understanding of these effects on students.

Open vs. traditional classrooms

The four meta-analyses are quite dated and come from an era where open vs. closed classrooms was a popular debate. My own children went to an open magnet school in North Carolina, but there was little openness left, as teachers used book cases, filing cabinets, and pot plants to create their own spaces. The conclusion from these older meta-analyses is that yes, open spaces are evident, but few principles of open teaching exist. No surprise that the overall effects are zero.

More recently, the language has changed to innovative learning environments (ILEs), which include much more than the shape or size of the classroom, and it is core to the notion that the nature of the teaching and learning needs to align with the opportunities provided by the space (Byers et al., 2018; see www.iletc.com.au/). ILEs aim is to move beyond the transmission of knowledge to include collaboration, communication, critical thinking, creativity, citizenship, and character (Fullan & Langworthy, 2014).

Murphy (2020) demonstrated that the ways teachers think about their work differ in that those in ILEs are more likely to have mindsets as follows: “I engage as much in dialogue as monologue,” “I focus on learning and the language of learning,” “I give and help students understand feedback,” and “I collaborate with my peers.” His studies showed that these mind frames follow from their being enabled by the physical environment to have more regular interaction, critique, and evaluation discussions with colleagues who share the teaching of the students. Byers et al. (2018) conducted a systematic review of more recent ILE studies, and the average effect was d = 0.31 but from only four studies. They noted that contemporary research seems to repeat the errors from the 1970s open-plan research, where the quantity and quality of studies were poor, the wishes and claims outshouted the evidence, there was too much focus on the environment and not the interaction of the teaching methods and environment, and it is, thus, not surprising that few have been convinced to make the move to the opportunities provided by ILE.

Tracking/streaming/between-class ability grouping

Differentiating classes by ability grouping remains popular worldwide, especially in math classes. For example, the OECD (2010) reports that 46% of students across OECD countries group by ability into different classes – and 75% receive instruction in at least one subject in an ability-grouped class in Australia, Canada, Hong Kong, Ireland, Israel, Malta, New Zealand, Singapore, Thailand, the United Kingdom, the United States, and Vietnam.

The outcomes can be broadly grouped into achievement effects and equity effects. *Steenbergen-Hu et al. (2016) reviewed 13 meta-analyses that showed students did not benefit from between-class grouping (−0.03); the effects did not vary for high- (0.06), medium- (−0.04), or low-ability students (0.03). Tracking has minimal effects on learning outcomes; no one profits. The overall effects on mathematics and reading were similarly low (reading, d = 0.00; mathematics, d = 0.02), and the effects on self-concept were close to zero. Castejón and Zancajo (2015) noted a negative relationship between student motivation levels and the degree to which systems sort and group students into ability groups. Why do we persist with a failed intervention? Who benefits? Not the students.

The effects on equity outcomes are more profound and negative. The most influential in-depth study of teaching and learning in tracked classes is Oakes’s (2005) Keeping Track: How Schools Structure Inequality. Her study was based on an intensive qualitative analysis of 25 junior and senior high schools. The major finding was that many low-track classes are deadening and non-educational environments. Oakes (1992) concluded that “the best evidence suggests that, in most cases, tracking fails to foster the outcomes schools value” (p. 13). Ability grouping fosters friendship networks linked to students’ group membership, and these peer groups may contribute to polarized track-related attitudes among high school students, with high-track students becoming more enthusiastic and low-track students more alienated (Oakes et al., 1992). Oakes (1992) commented that tracking limits “students’ schooling opportunities, achievements, and life chances. Students not in the highest tracks have fewer intellectual challenges, less engaging and supportive classrooms, and fewer well-trained teachers” (p. 20). Shanker (1993), then president of the American Federation of Teachers, in a commentary on Oakes’s research, was more earthy: “Kids in these [lower] tracks often get little worthwhile work to do; they spend a lot of time filling in the blanks in workbooks or ditto sheets. And because we expect almost nothing of them, they learn very little” (p. 24). In a similar qualitative design, Page (1991) provided a detailed account of the daily activities of eight low-track classes and found that teachers and students came to understand how not to push each other too hard so that they could cope, that low tracks were used as “holding tanks” for students with the most severe behavior problems, and that teachers focused on remediation through dull and repetitious seatwork (see also Datnow & Park, 2018). When tracked, the composition effects of peers working together can reinforce the lower performance levels (Thrupp et al., 2002).

Oakes and Wells (1996) claimed that tracking guarantees the unfair distribution of privilege in that white and wealthy students benefit from access to high-status knowledge that low-income students and students of color are denied. Oakes et al. (1990) analyzed 1,200 public and private elementary and high schools in the United States and found that minority students were seven times more likely to be identified as low-ability than high-ability students. Those schools that track often explain this ethnic subdivision by referencing past achievement and, thereby, argue that tracking can maximize opportunities to alter this. On the contrary, if tracking leads to proportionally more students from lower socioeconomic backgrounds or from particular ethnic groups being placed in lower tracks, then tracking may increase divisions along class, race, and ethnic lines.

Datnow and Park (2018) show the questionable use of data to make tracking decisions and noted that if this practice continues, then there are more regular assessments (e.g., at least three times a year), ensuring that all students have similar opportunities to learn challenging curricula to enable them to profit from whatever track they are assigned and when they move tracks. They concluded the following:

problematic practices of tracking and ability grouping with long-term consequences continue to abound in schools and are legitimated with data. In fact, tracking remains one of the most enduring practices in American high schools, in spite of a robust research base denouncing it.

(p. 148)

In his survey of tracking policy in California and Massachusetts, Loveless (1999) concluded that there are massive contradictions in that detracking is taking place in low-achievement schools, poor schools, and urban areas; whereas suburban schools, schools in wealthy communities, and high-achieving schools are staying with tracking – indeed, they are embracing it. “This runs counter to the notion of elites imposing a counterproductive policy on society’s downtrodden. If tracking is bad policy, society’s elites are irrationally reserving it for their own children” (Loveless, 1999, p. 154). In addition, Braddock (1990) found that schools with more than 20% of their rolls from minority groups were more likely to track than those with fewer minority students.

Oakes et al. (1992) found that Asian students were more likely to be assigned to advanced courses than Hispanic students with equivalent test scores. A disproportionate number of low socioeconomic status and disadvantaged minority students occupy the lower and noncollege tracks (Hanushek & Woessmann, 2006; Oakes et al., 1992). Students of average ability from advantaged families are more likely to be assigned to higher tracks because of actions by their parents, who are often effective managers of their children’s schooling (Gamoran, 2009). Further, schools with a larger proportion of minority and lower socioeconomic students are less likely to have sufficient higher-level courses, affecting the probability of students entering higher classes (Caro, 2009; Ferrer-Esteban, 2016). Moreover, the higher-track programs in these schools are often less rigorous than higher-track classes in schools with fewer minorities and higher socioeconomic students (Oakes et al., 1992; Kalogrides & Loeb, 2013). Modica (2015) noted that academic tracking strengthened racial boundary keeping and reinforced the idea among students that whiteness and academic success are correlated in a fixed and natural way.

Given that many teachers prefer less variance in the abilities of their students within a class, this can reduce students’ life chances by lowering expectations (of teachers and students). At a minimum, there needs to be reviews by school leaders and independent evaluators (or inspection systems) to ensure that the assignment to middle- to lower-track classes are defensible, that all students are exposed to challenging curricula, and that tracking does not lead to apartheid in schools. In Italy, for example, the regulations stipulate that all students must be exposed to the same curriculum and the same quality and quantity of instruction (Dupriez et al., 2008; Duru-Bellat & Mingat, 1997). Further, Parker et al. (2021, p. 343) demonstrated that “ability stratification biases academic self-concept for children at the top and those at the bottom – albeit in different directions. This is to the detriment of every student.”

There is a further conundrum in this research. The empirical evidence leads to a conclusion that there is a close to zero effect from tracking, but the qualitative literature indicates that there may be quite different teaching and interactions in the low- vs. high-tracked classes. The qualitative evidence suggests that low-track classes are more fragmented, are less engaging, and taught by fewer well-trained teachers. If these lower tracked classrooms were more stimulating, were more challenging, and were taught by well-trained teachers, there may be gains from tracking for these students. It seems that the quality of teaching, the transmitted expectations of teachers and students, the differential curriculum of easier and harder work, and the nature of the student interactions are the key issues, rather than the compositional structure of the classes.

Let us seriously stop tracking: no student is the winner.

Detracking

The detracking movement grew from the concern about the inequitable distribution of educational resources, and the premise is that all students should have access to a rigorous curriculum. Given the zero effect of tracking, then it is worth noting that detracking (removing the tracking and moving to more heterogeneous classes) has a much higher effect compared to students in tracked classes (*Rui, 2012). Critically, nearly all the positive effects relate to students in the lowest tracks (d = 0.28), and these students are the greatest beneficiaries of removing tracking. The middle and higher students are neither advantaged nor disadvantaged.

Multigrade/age classes

Multiage classes include students from more than one year level who are taught in the same classroom by the same teacher (also called multigrade, combination, split-grade, vertically grouped, mixed-aged, family group, and nongraded). These are common in very small schools, in many developing nations, and where there are uneven numbers of students at different year levels. The average effect across three meta-analyses is −0.02: it does not matter.

*Veenman (1995) conducted a meta-analysis of the cognitive and affective outcomes of multigrade and multiage classes in primary schools across a variety of English-speaking and non-English-speaking countries (see also Mason & Burns, 1996; *Veenman, 1996). In reviewing 34 studies comparing multigrade and single-grade classes, and 8 studies comparing multiage and single-age classes, Veenman found no differences in achievement (d = 0.00 and d = −0.03 for multigrade and multiage classes, respectively) and small effects on student’s attitudes toward school, self-concept, and personal adjustment favoring these classes (d = 0.10 and d = 0.15, respectively). There were small positive effects of multigrade classes for students in grades 1 to 3 (d = 0.06), a near-zero effect for grades 4 to 5 (d = 0.01), and a small negative effect for grades 6 to 7 (d = −0.08). There was some support for the possibility that student achievement may suffer in subjects such as science (mean effects size, d = −0.19) and mathematics (d = −0.25), but there was no support for the notion that there might be small negative effects in schools where selection factors would not be operative (e.g., in rural schools the mean effect size was d = 0.10). As a consequence, *Veenman (1995, p. 367) concluded, “parents, teachers, and administrators need not worry about the academic progress or social-emotional adjustment of students in multigrade or multiage classes. These classes are simply no worse, and simply no better, than single-grade or single-age classes.”

Part of the reason for this small effect is that the opportunities offered by teaching across grades/ages are not realized. *Veenman (1995) and Mason and Burns (1996) both noted that teachers rarely capitalized on multigrade or multiage arrangements to promote learning from peers. Instead, teachers tend to teach distinctly different curricula, maintain grade levels, and deliver separate lessons to each grade-level group.

Within-class and small-group learning

*Steenberger-Hu et al. (2016) completed a meta-synthesis of five meta-analyses of within-class grouping, with an average of d = 0.25. The effects were greatest for high (0.29) and low average (0.30), and lower for medium-ability students (0.19). Within-class grouping adds value, and part of their appeal is that they are rarely permanent and often heterogeneous.

Small-group learning typically involves assigning a task to a small cohort of students and then expecting them to complete this task as a group. The effects from the seven meta-analyses on small-group learning are systematically between 0.4 and 0.5, showing that using small groups within a class session can be most effective. *Tomcho and Foels (2012) showed that using small groups over a brief period is more effective than over a longer term, when there is no group accountability, and when participant interdependence was high than when participant interdependence was low. *Lou et al. (2001) noted that the effects of small-group learning were significantly enhanced when students had group work experience or instruction, where specific cooperative learning strategies were employed, and when group size was small.

A consistent message from studies of the effectiveness of within-class and small-group sessions is that students may need to be taught skills of working in groups. We termed these “student collective efficacy skills” (Hattie et al., 2021). The major skills related to the I and we skills. The I skills include seeing oneself as a learner, not being constrained by beliefs about whether one is a high or low achiever, and being able to stand in the shoes of other students and appreciate their processes of thinking and ways of understanding. This involves accepting others as they are, decoding and understanding what others are thinking and feeling, becoming social problem-solvers, empathizing with others and the group’s moods and feelings, and listening and demonstrating that they have listened to others in the group. Group work is more successful when students have been taught listening and turn-taking skills (e.g., not being dominating or socialized to silence or loafing); identifying, confronting, and welcoming challenges to success; and being skilled in conflict resolution, negotiation, and communication skills. The we skills include having confidence that the group can come up with a better solution than any one person (including yourself), a sense of trust so that alternative perspectives and errors are seen as opportunities, and skills in group problem-solving.

Group work also requires ensuring the task is more open than closed, allowing for multiple interpretations, having success criteria for individuals and the group, and having an assessment criteria that should welcome individual and group contributions. It is important to ensure that all students have sufficient knowledge, confidence, and motivation to tackle the deeper learning that can occur in groups – so preteaching the surface or content knowledge before grouping – otherwise, many can be left behind, not valued, and less able to contribute to the group.

Mainstreaming – detracking special education students

The notion of the least restrictive environment for students with special needs has often led to these students being mainstreamed – that is, placed in regular school classes. The least restrictive environment, however, does not mean mainstreaming but refers to modifying content, materials, classroom management, instructional techniques, and strategies for these students, even within the regular class. Full inclusion, claim the advocates, leads to increased expectations by teachers, more peer interaction, more learning, and greater self-esteem.

There was a flurry of movement from special education classes to mainstreaming in the 1970–1990, and thus, most meta-analyses were conducted around this time, showing that, in general, it was a better solution. The average effect for the five meta published up to 2000 was 0.28. *Kavale and Carlberg (1980) found small but positive advantages for mainstream over special classes (d = 0.12) and a similar effect for achievement (0.15) and social/personality outcomes (0.11). *Baker (1994) reported similar effects (d = 0.08) in favor of mainstreamed students, with more positive outcomes for mathematics (d = 0.22) than for reading (d = 0.12). He also found similar effects for those classified as mentally retarded (d = 0.47) than learning disabled (d = 0.13), and *Wang and Baker (1985) found similar effects across various grades. The more recent studies reinforce the advantages of regular classes. *Oh-Young and Filler (2015) reported 0.31 for academic and 0.26 for social outcomes, indicating an advantage for more than less-integrated settings. *Zumeta (2009) found small differences for achievement for students with disabilities educated in inclusive settings (0.09).

But how are students without special needs affected by the presence of special-needs students? *Szumski et al. (2017) found a d = 0.12 across nonspecial-needs students; thus, the presence of students with special educational needs positively impacted all students. This advantage was similar across all levels of schooling, regardless of whether the special needs were mild or severe. Similarly, *Ahmad (2016) specifically investigated the effect on mathematics and reported a d = 0.38 increase for students with special needs and d = 0.81 for those without special needs. So all benefit.

In our teacher education program, we abolished the focus on special needs and introduced a learning interventions program. All students need learning interventions, and this focus privileges the centrality of all students – building their learning skills, building their I and we skills, working with others, and reinforcing the edict that what works best for all students may not work for students with special needs, but what works for students with special needs work best for all students.

As Mitchell and Sutherland (2020) have demonstrated, there is “little evidence to support the notion of disability-specific teaching strategies” (p. 7). Instead, what is required is the systematic, explicit, and intensive application of a wide range of effective teaching strategies – day-by-day, minute-by-minute – in classrooms. In their major synthesis of studies for students with special needs, they identified the same effective strategies for all students, as outlined in this book: behavioral strategies (DI, review and practice, formative assessment, feedback), social strategies (cooperative group, peer tutoring, social skills training, classroom climate), cognitive strategies (self-regulated learning, memory strategies comprehension strategies), and mixed strategies (assistive technology, early intervention, RTL, universal design).

Ability grouping for gifted students

If any group of students is a winner from grouping, it is gifted students. They receive programs that typically involve a faster pace of instruction and more challenging tasks within the same curriculum frameworks as medium- and low-ability students. Herein lies a key distinction. Where there are specific curricula aimed at challenging students at the appropriate level, then there seems to be a greater likelihood of success in engagement and learning (*Steenenberg-Hu et al., 2016). Goldring (1990), for example, found that gifted students, when placed in homogeneous classes with challenging curricula, achieved more than gifted counterparts in regular classes, although Redding and Grissom (2021) found close to zero effects, and even among those underrepresented groups in gifted classes (African American, low SES), the effects question how effective these programs are serving these students. For students in special classes, the greatest advantages were in science and social science, and the smallest was in reading and writing. There was no evidence of negative or differential social effects: there were no differences in general self-concept or creativity for students in special classes and those in regular classes (although see Craven et al., 2000). *Vaughn et al. (1991) found positive effects from various creativity programs on self-concept (d = 0.11), achievement (reading and vocabulary, d = 0.65), and creative thinking (d = 0.44).

Classroom influences

Classroom management

Classroom management is a necessary but not sufficient condition for raising achievement. It typically involves establishing a positive climate, good relations among students and with the teacher, a sense of fairness and an invitation to learn, and appropriate interventions for reducing disruptions to learning. However, it does not necessarily mean quiet, ordered, and compliant focused. Care is also needed about the levels of issues that lead to management issues: between low-level behavioral issues, such as talking out of turn or task refusal, and high-level behaviors, such as violence (verbal, via social media, physical).

*Korpershoek et al. (2016) noted that 85% of classroom management interventions focused on changing students’ behavior, 74% partly focused on improving students’ social-emotional development, 54% focused on changing the teachers’ behavior (54%), and only 4% on improving the teacher-student relationships. They reported an overall d = 0.22 among positive classroom management, 0.17 on academic achievement, 0.24 on behavior, 0.21 on social-emotional, 0.08 on motivation outcomes, and 0.24 on the quality of teacher-student relationships on achievement. The effects of classroom management programs were greater on boys (0.23) than girls (0.10), similar across grades and socioeconomic background, and for students with (0.27) or without (0.20) behavior problems.

*Marzano et al. (2000) investigated and found the effects of various classroom management processes on achievement in well-managed classrooms was d = 0.52 and on heightened engagement was d = 0.62. The attribute of teachers that had the greatest influence was the teacher’s with-it-ness (see “Mindfulness,” d = 1.42); that is, the teacher could identify and quickly act on potential behavioral problems and retained emotional objectivity (d = 0.71). The next most effective methods were disciplinary interventions (d = 0.91), which included verbal and physical behaviors of teachers that indicated to students that their behavior was appropriate or inappropriate (d = 1.00); group contingency strategies, which required a specific set of students to reach a certain criterion level of appropriate behavior (d = 0.98); tangible recognition, which included those strategies in which students were provided with some symbol or token for appropriate behavior (d = 0.82); and interventions that involved a direct and concrete consequence for misbehavior (d = 0.57). The major factors included what *Marzano et al. (2000) termed “high dominance” (clarity of purpose and strong guidance) and “high cooperation” (concern for the needs and opinions of others and a desire to function as a member of a team). Rules and procedures (d = 0.76) involved stated expectations regarding behavior and well-articulated rules and procedures that were negotiated with students.

Thus, setting clear expectations and boundaries, creating positive relations with and among students, seeing the problem as difficult behaviors more than difficult students (emotional detachment), ensuring there is appropriate challenge in the tasks to engage students in learning rather than resisting learning, and having the ability to identify and quickly acting on potential behavioral problems.

Classroom breaks

*Huimin et al. (2021) investigated the effects of active breaks across the school day. Across 18 studies, breaks improved on-task behavior (0.69) but had far lower effects on cognitive functioning (0.31). As will be shown later, some form of exercise throughout the day is beneficial to cognitive learning, although the effects are small. What may be more critical is to understand the levels of distractibility and attention of all students, and to construct lessons to motivate students to expend their working memory on the relevant tasks, processes, and learning.

Bullying

If classes are to be inviting places that are fair and safe, and places where students are respected as learners, then there is no place for bullying. No place. But therein lies the dilemma when discussing bullying, as the place is not just the physical space called classroom and school (and after-school and home) but also cyberspace. Bullying also has short- and long-term (sometimes very long-term) negative impacts, and can adversely affect the behaviors (and learning) of both the bullied and bully, and bullying is all about control – as the bully aims to confront via deliberate attacking and intentionally depriving the other of self-regulation, self-control, and self-respect.

There is a negative correlation between achieving and being bullied, but this may mean that those with good academic achievement were less likely to have been exposed to bullying victimization (*Moore et al., 2017). *Gardella et al. (2017) found being bullied was negatively related to achievement and school attendance.

There are many meta-analyses of programs to reduce bullying. Most find these programs to have, on average, important positive effects (Ferguson et al., 2007, d = 0.24; Fraguas et al., 2021, d = 0.15 in reducing bullying and 0.21 on improving mental health problems), and these programs reduced reports of being bullied (in person and via cyberspace) by 15–20% (Gaffney et al., 2021; White, 2019). In addition, while prevalence rates for cyberbullying were lower than for traditional bullying, there were high correlations in the incidence between cyber and traditional bullying (Modecki et al., 2014; Walters, 2021).

More intensive programs were more effective, as were programs including parent meetings, firm disciplinary methods, and improved playground supervision (Gaffney et al., 2021). A whole-school policy (including positive school and class climate, supervision in playgrounds, and anti-bullying policy) does work but needs to be augmented with other interventions and programs (White, 2019). These augmented programs need to include training in emotional control, peer counseling, or social skills training, and schools need to regularly collect and interpret data on student behavior and promote a school culture that does not tolerate such behavior (Lee et al., 2015). Other aspects of effective programs include involving nonbullied peers, as this increases the likelihood they would intervene to stop bullying or provide support for the bullied (Lee et al., 2015; Gaffney et al., 2021; White, 2019); having parenting strength-based programs and inviting parents to join in anti-bullying programs can reduce both bully perpetration and victimization (Chen et al., 2021; Huang et al., 2019; Gaffney et al., 2021). Programs are more effective in high than in elementary schools, probably because the elementary teacher has more direct effects, and students are more compliant with teachers (Huang et al., 2019; Lee et al., 2015; Gaffney et al., 2021; Walters, 2021; Yeager et al., 2015).

Decreasing disruptive behavior

The presence of disruptive students can have negative effects on their own and all other students’ achievement outcomes. Thus, reducing disruptive behaviors must be a core competency of any successful teacher. There have been many meta-analyses of the effects of various programs on decreasing disruptive behaviors (although these are not included in Metax™, as they do not report on achievement effects). For example, Weisz et al. (1987) found an average effect of d = 0.79 from psychotherapy studies conducted with school-age children. Kazdin et al. (1990), in a larger set of studies, found d = 0.77 and, in particular, that behavioral interventions (d = 0.76) were much more effective than non-behavioral interventions (d = 0.35). The effects were highest for self-control (d = 0.87), then treatment of delinquent behaviors (d = 0.42), noncompliant behaviors (d = 0.42), and aggressive behaviors (d = 0.34) (see also *Prout & DeMartino, 1986).

*Skiba and Casey (1985) found higher effects for programs based on aversive reinforcement procedures (i.e., time-out, response cost), cooperation, cognitive behavior modification, and behavioral consultation, and much lower effects for programs based on biofeedback training or psychodynamic counseling (making students more self-aware of the impact of their behavior and their concerns about what led to the disruptive behaviors). The most successful programs included social or token reinforcement (d = 1.38), cooperation (d = 1.05), behavioral consultation (d = 1.09), cognitive behavior modification (d = 1.0), and the least social skills training (d = 0.44). These results indicate that targeting classroom disruptions via a behavioral approach is the most efficacious. The effects were not related to the curricula domain, age, or level of schooling.

A major message is that poor behavior needs fair, clear, and understood consequences. The effects of the various types of consequences are quite similar: use of positive consequences (0.86), negative consequences (0.78), combined positive and negative consequences (0.97), and interventions that had no immediate consequence (0.64) (*Stage & Quiroz, 1997). Consequences matter. The effects of understood consequences are high (0.65) for reductions in the regular classroom and higher in self-contained special classes (self-contained or resource rooms, 0.97). The different interventions did not differ in their overall success in reducing disruptive problems: programs using group contingencies (−1.02), self-management (−1.00), differential reinforcement (−0.95), and cognitive-behavioral interventions (−0.50). Teachers, however, are less likely to see evidence of reductions in disruptive behaviors (0.37) compared to behavioral observation methods (0.83) (*Stage & Quiroz, 1997; see also *Reid et al., 2004). Perhaps this suggests that teachers can build expectations of behavior from specific students, and when their misbehavior actually reduces, they need to be made more aware of this and revise their expectations.

Suspension/expelling students

Suspension is usually a limited time-out of the regular class (in-school suspension) or out of the school, and expulsion is removal from the school roll completely. *Nolte-meyer et al. (2015) noted the effect of in-school suspension on subsequent school achievement was d = −0.10, and the effect of out-of-school suspension was −0.24. Students suspended were more likely to come from low-income and urban schools, among African Americans, lower achievement, and lower SES homes.

suggests that students who may experience heightened risk from the outset may be doubly disadvantaged by their schools’ use of disciplinary practices that may further exclude them from instruction that they need to progress educationally and alienate them from the school setting.

(*Noltemeyer et al., 2015, pp. 234–235)

Alternatives that can help reduce the behaviors that led up to suspension can include school-wide positive behavior interventions, conflict resolution, and social skills programs (Lester et al., 2017; Valdebenito et al., 2018). The overall effects of these programs are positive but small, and more success might come from enhancing these student’s achievement outcomes and making classes inviting for all students to thence reduce their feelings of exclusion and unfairness in class.

ADHD treatment with drugs

So often, drugs are seen as the answer to reducing behavior problems and for enhancing attention, and there are claims they can increase achievement. The evidence is more equivocal. The effects of medication are quite low. *Kortekaas-Rijlaarsdam et al. (2019) reported low effects of Ritalin on math accuracy (0.03), math productivity (0.08), reading accuracy (0.06) (cf., *van der Oord et al., 2008). *Purdie et al. (2002) investigated the effects of various drugs on students with attention deficit hyperactivity disorders, and while there was evidence that the various drugs (both stimulants and depressives) seemed to reduce behavior problems when rated by teachers and parents, although not by the students or independent observers (see also *Crenshaw, 1997), there were limited effects on achievement. *Ottenbacher and Cooper (1983) also found much greater effects on behavior (d = 0.90) and increased attention span (d = 0.84) compared to academic performance (d = 0.47) (see also *Thurber & Walker, 1983).

*Kavale (1982) found positive effects for stimulants, but these effects were higher for lower-level tasks (memory and copying, d = 0.41) than for higher-order tasks (reading d = 0.32 and mathematics 0.09). *Purdie et al. (2002) noted that the effects of stimulants (d = 0.35) were not that different from school-based psychological and educational interventions (d = 0.39), social skills training (d = 0.31), cognitive and self-regulation programs (d = 0.58), and parent training (d = 0.31). Similarly, *Snead (2005) compared the effects of behavior interventions (0.23), medication (0.17), and both (0.19). Compared to behavior interventions, medication had greater effects on reducing aggression (0.78 vs. 0.42) and internalizing behaviors (0.51 vs. 0.34), but little difference on social problems (0.33 vs. 0.29). *Van der Oord et al. (2008) also found similar effects of behavioral or drug treatments (0.19).

Students diagnosed with ADHD are more likely to be lower-achieving students (d = −0.99, difference between those diagnosed with ADHD and their peers; *Fowler, 2015), and this lower achievement and distractibility can be a combined powerful source of angst and distraction for teachers. Assigning teacher assistants can help reduce the pressures and attention of the teacher on these students but does little to enhance the learning of ADHD students (Blatchford et al., 2012). Medication does seem to reduce behavioral problems and distraction, but this does not necessarily translate into academic gains.

ADHD treatment not using drugs

The meta-analyses that have reviewed the effects of nondrug interventions with ADHD students led to consistently high effects. *DuPaul and Eckert (1997) found that school-based programs had a greater effect on ADHD students’ achievement than drug treatments. They investigated in-school treatment programs for ADHD students, and their effect on behavior was d = 0.78 and achievement, d = 0.58. *Daley et al. (2014) found an overall treatment of behavioral interventions of 0.28 on achievement. The most effective programs for enhancing positive behaviors in class relating to social skills programs (0.47) then conduct problems (0.26). However, programs that focused more on positive parenting seem not to be sufficient to produce positive changes in children’s behaviors. *Du Paul and Eckert (1997) found high effects for contingency management interventions (d = 0.94) and academic interventions (d = 0.69), and they were more effective than cognitive-behavioral procedures (d = 0.19) in improving classroom behavior. The latter, however, was most effective in enhancing achievement effects. *Du Paul et al. (2012) reported that academic intervention had greater effects (0.56) than contingency management (0.24) programs.

Counseling effects

The effects of school counseling are much higher on behavior than achievement. *Prout and Prout (1998), for example, found an overall effect of 0.97. The effects were highest for cognitive-behavioral interventions (1.45), relaxation (0.84), and skills training (0.50) programs. These programs led to effects on depression (1.96), attitude (0.85), anxiety (0.74), self-esteem (0.74), and social skills (0.38), but zero on improving academic performance. Similarly, *Grant (2011) reported an overall effect of 0.18 on counseling programs in middle schools on achievement. The effects were higher on reducing aggression (0.23) than on achievement (0.12), negative on reducing conduct problems (−0.10), and negative when teachers rather than counselors delivered the program (−0.27). *Whiston et al. (2009) found lower effects on GPA (0.15) and achievement (0.16), compared to improving discipline (0.83), anxiety (0.40), depression (0.37), social skills (0.33), attendance (0.30), self-esteem (0.29), reducing of altercations and fights (0.27), and social development (0.24). There are meta-analyses targeting social and emotional learning and the development of social skills, such as Murano et al. (2020), who found an effect of 0.34 for enhancing social skills and 0.32 for reducing problem behaviors.

Another school-wide program, Student Success Skills, is a focus of the meta-analysis by *Villares et al. (2012). This program involves teaching all students cognitive and metacognitive skills, such as goal setting, progress monitoring, and memory skills; social skills, such as interpersonal skills, social problem-solving, listening, and teamwork skills; and self-management skills, such as managing attention, motivation, and anger. The overall effects on raising achievement were 0.29, but no moderators or more details were provided.

*Reese et al. (2010) updated the *Prout and deMartino (1986) study of the effects of school-based psychotherapy and counseling research, with an average effect of 0.23. Larger effects related to skills training (0.55), then cognitive-behavioral interventions (0.24), and the lowest, with play therapy (0.03). The effects were larger for elementary students (0.65), compared to high (0.30) and middle (0.07) students; and when focused on externalizing behaviors (0.44) relative to internalizing behaviors (0.23). *Baskin et al. (2010) found larger effects on non-academic (0.50) compared to academic outcomes (0.38). The effects were higher from student reports on gains (0.59) compared to teacher reports (0.26) and achievement measures (0.36), and similar across elementary (0.41) and high schools (0.45).

Overall, counselors seem a most worthwhile addition to the school team, and as will be seen later when discussing response to intervention, and critical in these teams. As expected, counselors have higher effects on non-achievement-related variables, which are often the correlates of involvement in class.

Classroom climate

Classroom climate

*Wang et al. (2020) reported that classroom climate had small to medium positive associations with social competence (r = 0.18), motivation and engagement (0.25), and academic achievement (0.12), and negative with externalizing behavior (−0.18) and socioemotional distress (−0.14). *Demirtas-Zorbaz et al. (2021) also found similar correlations with achievement (r = 0.18). As we shall see, there are particular aspects of classroom climate that make the class safe, fair, and inviting for students to learn, not know, and work with peers. Similarly, such positive climates are more likely to develop the students’ needs for competence, autonomy, and relatedness.

Classroom cohesion

A key factor in developing a positive classroom climate is classroom cohesion – the sense that all (teachers and students) are working toward positive learning gains. Classroom cohesion includes goal-directedness, positive interpersonal relations, and social support. For example, *Haertel and Walberg (1980) found that learning outcomes were positively associated with cohesiveness, satisfaction, task difficulty, formality, and goal direction; and negatively associated with friction, cliques, apathy, and disorganization. In classrooms with greater cohesiveness, it is more likely that there is co-peer learning, tolerance, and welcoming of error and, thus, increased feedback, and more discussion of goals, success criteria, and positive teacher-student and student-student relationships (*Evans & Dion, 1991). *Mullen and Copper (1994) argued that group cohesion was more related to commitment to task rather than interpersonal attraction or group pride. It is co-learning that makes the difference.

Teacher-student relationships

I do get frustrated when I hear that the major finding from VL1 is that the most important aspect of a classroom is teacher-student relations. Not because they are anything but important but because they are a means to an end. Further, the expertise of the teacher to engender positive student-to-student relationships is as critical – peers can be harsh critics, sometimes intolerant of others, and the cause of exclusion (e.g., bullying, not liking). The reason for developing positive relations is so that the class is an inviting place to come and learn, errors and not knowing are welcomed as opportunities to learn, and students can feel safe that there are fairly treated by all.

Building relations with students implies having agency, efficacy, and respect by the teacher for what the student brings to the class (from home, culture, peers), and allowing the experiences of the child to be recognized in the classroom. Further, developing relationships requires skills by the teacher – such as the skills of listening, empathy, caring, and positive regard for others. Students are great detectives of messages that indicate they are not welcomed, they are not going to be treated fairly, and the probability that they will advance in their learning with this teacher (see teacher credibility).

Thus, a key message is to understand relations through the eyes of the students. Bishop (2019) noted that minority students particularly noted whether their teachers liked them. For example, did the teachers create caring and learning environments, have high expectations of them as learners, invite them to engage in cognitively challenge or easy tasks, and pathologize the problem in the class as the students, the race, the resources, and the home, or engage in pathologizing practices (remedial, limited curricula, simplified language, ability grouping, transmission teaching methods).

*Cornelius-White (2007) completed one of my favorite meta-analyses: on teacher-student relations. He embedded his review in person-centered education based on Carl Rogers’s (cf., Rogers et al., 2013) claims about the qualities needed to build personal relationships between the teacher and the learner. These qualities include genuine trust in learners and an accepting and empathic climate. Can the teacher see the world through the student’s eyes, show flexibility in their teaching methods, demonstrate a willingness to change to meet the students’ learning needs, and learn along with the student in a safe and trusting learning environment? He found a correlation of 0.34 (d = 0.72) across all person-centered teacher variables and all student outcomes (achievement and attitudes).

In classes with person-centered teachers, there is more engagement, more respect for self and others, fewer resistant behaviors, more teacher release of responsibility, and higher achievement outcomes. *Cornelius-White noted that most students who do not wish to come to school or dislike school do so primarily because they dislike their teacher. His claim is that to “improve teacher-student relationships and reap their benefits, teachers should learn to facilitate students’ development” by demonstrating that they care for the learning of each student as a person (which sends a powerful message about purpose and priority) and empathizing with students – “see their perspective, communicate it back to them so that they have valuable feedback to self-assess, feel safe, and learn to understand others and the content with the same interest and concern” (p. 23).

Teacher-student dependency

One extreme of teacher-student relations is an overdependence on the teacher. Such clingy, overly compliant attention to teachers may be more evident in the early years (when many are already highly rule-governed) but can appear throughout the school years. Unfortunately, it reduces the student learning of what to do when they do not know what to do, reduces their autonomy when learning, reduces their skills and confidence to engage with challenging tasks, and can lead to poorer peer relations.

*Roorda et al. (2011) reviewed 28 studies and found teacher overdependence correlated to r = −0.12 with achievement, −0.13 with engagement, and −0.17 with prosocial behavior. These effects were maintained over time; it does not matter whether the informant of dependence was the teacher, student, or independent observer, and the effects were the same regardless of age level. They concluded the following:

teachers and other school practitioners need to be made aware that high levels of student-teacher dependency might be just as harmful to students’ school adjustment as low levels of closeness and high levels of conflict. This seems to be especially true for students at risk for maladjustment due to their ethnic minority status or for students in higher grades.

(p. 12)

Student friendship

Friends can contribute to social and emotional well-being; build trust to work together (particularly on challenging tasks and problem-solving); mutually aid, unite, and encourage; provide social status in a classroom; and create community. Students in friendship groups can have more mutual discussion, higher levels of social sensitivity, and greater conflict resolution. However, the reverse can also occur, as some friendship groups can be more distracted, engage in off-task interactions, converse about non-achievement-related topics, and be formed based on conflict and betrayal.

*Wentzel et al. (2018) defined friendship as working with mutual friends on academic tasks and, more simply, as having friends in the class. They reported d = 0.29 between friendship and achievement. The effects were similar in elementary and secondary levels, twice as high among boys (0.24) than girls (0.13), higher when friendships were positive and reciprocal (0.38), and lower as the number of friends increased (0.25). Similarly, *Newcomb and Bagwell (1995) reported d = 0.37 for friends vs. nonfriends on task performance, and *Chung et al. (2018) found d = 0.31 for friendship on group performance. There have been many references through this sequel to the importance of friendship as a predictor of academic success, especially when there is a change in class (a new year, moving schools).

Belonging

A sense of belonging in the class is a powerful precursor to learning. Belonging refers to the extent that students feel personally accepted, respected, included, and supported by others in the school social environment (*Allen et al., 2016). Teachers who develop their students’ beliefs and feelings about being personally accepted, respected, included, and encouraged by others are likelier to have students who feel they belong, indeed invited into learning. *Moallem (2013) found r = 0.22 (d = 0.53) between a sense of belonging and achievement, and this did not differ for males and females or race/ethnicity background. The higher effects were related to conceptualizing school belonging more in terms of peer group acceptance than emotional engagement (e.g., quality of teacher-student relationships, school safety, relationship with peers, and harmony among the different racial groups). *Allen et al. (2016) found teacher support (autonomy, support, and involvement, r = 0.46) had the greatest effect, then peer support (r = 0.32) and parent support (r = 0.33).

*Nurmi (2012) asked about teacher-student relations when there were children with high levels of problem behaviors. They found negative associations with conflict (r = 0.20) and dependency in the teacher-student relations (r = −0.19), and positive effects with closeness (r = 0.21). While not a meta-analysis relating to achievement, *Gerber and Wheeler (2009) found large negative effects of rejection on mood (−0.50), self-esteem (−0.70), and belonging and control or frustrated by rejection. Allen et al. (2018) noted the major benefits of a positive sense of belonging, especially for high school students (*Korpershoek et al., 2020), and that about one-third do not feel a sense of belonging at school. From a systematic review, the optimal interventions were reducing bullying, having social skills training, planning goals, having mental health support, and fostering adaptive learning strategies to enhance a sense of achievement.

Students feeling disliked

Most of us encountered peers we did not like or who did not like us. *Card (2010) found that about one-third of students claimed these antipathetic relationships, and they lead to externalizing problems (e.g., aggression), internalizing problems (anxiety, loneliness, withdrawal), peer victimization or rejection, and negative effects on academic achievement (d = −0.26). There were hardly any differences in the prevalence for males and females, and across elementary and high schools.

Relative age in the class

There is a reasonable high effect for relative age within the class, although much higher in elementary than high schools (Mavilidi et al., 2021). There can be significant developmental differences in physical, emotional, and cognitive skills within any class of students. For example, in systems where students start school in the year they turn 5, there can be 11 months’ difference (some at 5 years, 0 months; some 5 years, 11 months, which is 20% of a lifetime difference). At this age, older children are generally bigger, faster, stronger, and more coordinated than their younger peers. This can lead to some teachers seeing older children as being more intelligent, being more emotionally mature, having a greater vocabulary, and generally being more ready for school; and indeed, they can be. *Maynard (2013) synthesized 12 relative age studies based on over 3 million students and found a correlation between relative age and achievement of r = 0.22 (d = 0.45) – with the effects larger in math than reading, and the effects increased from grades 1 to 3, then began to decline (close to zero by grade 9). An alternative policy is to start school closer to the age you turn 5 (e.g., the day they turn 5, in the term they turn 5, as in many jurisdictions), and then the relative age effects become much lower.

Mavilidi et al. (2021) used a longitudinal study showing the relative age effects were larger in elementary than high schools. Taking into account various covariates (gender, SES, type of school, percent of girls in the school, preschool cognitive achievement), the effects were 0.29 for year 3, 0.22 for year 5, 0.10 for year 7, and 0.07 for year 9. In the early years, relative cognitive and social immaturity play a major part, and there is a social comparison process, as students compare their performance with their peers. This big-fish-little-pond effect occurs when the relatively younger students compare their accomplishments with relatively older classmates who are physically, cognitively, socially, and emotionally more mature.

Retention

Retention is the practice of not promoting students up a grade level in school (that is, the student repeats the level), and it is based on the belief that children learn more academically by repeating a grade. This is one of the few areas in education where it is difficult to find any studies with other than a zero or negative effect. Overall, there are negative effects for retained students, and there are more positive effects in the long term for promoted students than for retained students – even when matched for achievement at the time of the decision to retain or promote.

There are two comparisons – grade comparisons compare those retained with their grade peers, and age comparisons compare repeaters with their age-mates who were promoted (most studies). The effects from grade comparisons are d = 0.05 and for age comparisons, −0.11. When students are retained, they do as well as peers but still fall behind their peers who were promoted (Goos et al., 2021).

Retention has been found to have a negative effect across all subjects on social and emotional adjustment, and behavior, self-concept, and attitude toward school. For example, *Jimerson (2001), based on 169 achievement effects, found a mean effect of d = −0.39, and this negative effect was mirrored across many subjects: language arts (d = −0.36), reading (d = −0.54), and mathematics (d = −0.49). An additional 246 effect sizes related to socioemotional and behavioral outcomes were also systematically negative (d = −0.22), and attendance was lower for the retained students (d = −0.65). The negative effect is across all ages, and while it is often claimed that retention may be beneficial in the early years, the evidence is to the contrary. *Bright (2011) specifically investigated retention effects for K–6 students and from 26 studies, and found that students scored −0.50 standard deviations lower than promoted students on academic outcome measures. *Gottfried (2013) also showed the negative spillover effects on grade-retained peers.

*Holmes (1989) synthesized the results from 63 studies on retention effects and reported an overall effect of d = −0.15. This negative effect increases over time such that after one year, the retained groups were scoring lower than the comparison groups who had gone on to the next grade (d = 0.45); and this difference became larger each subsequent year, with the difference reaching d = −0.83 for measures taken four or more years after the time of retention. Clearly, the threat of nonpromotion is not a motivating force for students. Moreover, being retained one year almost doubled a student’s likelihood of dropping out, while failing twice almost guaranteed it.

These negative effects are partly caused by schools and teachers not providing optimal interventions for the retained students the first time they were in the grade. Therefore, the students are retained in programs that were not beneficial to them in the previous year. Marsh et al. (2017) did show that the students, when they repeated a grade, did perform better after the second year of the same grade level compared to their new peers in their retained grade, but they still fall well below their (equivalent) peers that progressed to the next level.

*Holmes (1989) concluded that finding another educational practice on which the evidence is so unequivocally negative would be difficult. The effects are bad enough for achievement, but when the negative equity and social and emotional effects are added, the situation is dire for retention (retention is highest among boys, racial/ethnic minorities, or less advantaged children) (Manacorda, 2012). Consider two students of the same achievement, and it is four times more likely that the student of color (African American, Hispanic) will be retained and the other (white) student promoted (Cosden et al., 1993; Meisels & Liaw, 1993). The only question of interest relating to retention is why it persists in the face of this damning evidence. The extra (repeated) year produces no benefit for retained children over controls on achievement tests or teacher ratings of social maturity, learner self-concept, or attention at the end of first grade (Shepard & Smith, 1990).

Conclusions

It is less about the structure and the composition of the students within the class and more about the climate, the culture, the sense of belonging, and the class being an inviting place for students to want to come (back) to.

Students learn much from each other, and we need to be more deliberate in teaching the skills of students working in teams with peers with diverse backgrounds and skills – as this is the world they live in outside school and will do so increasingly when becoming adults. Given that students, not us, will create their futures, modeling how to work with diversity is one of the more powerful messages and heritages we can leave with our students.

Maybe we need to return to the notions of the least restrictive environment, the maximal exposure to appropriate challenging work, the centrality of the student, and the focus on accelerated progress to achievement for all.

The climate needs to be seen by students as fair, led by a teacher who they believe can enhance their learning, has high expectations of each student’s success, shows their with-it-ness, works quickly to suppress disruptive issues, has clarity of purpose, and shows excellent listening skills.

Creating a sense of cohesion and positive relations between teacher and student and among students leads to high trust such that students can fail quickly, learn from failure, and feel comfortable not knowing and, thus, receptive to the impact of the teacher.

If students want anything from a teacher, they want to know us, they want to be safe to learn, they want to see the teacher as competent, and they want fairness in reducing misbehavior and promoting learning.

It is less about the structure and the composition of the students within the class and more about the climate, the culture, the sense of belonging, and the class being an inviting place for students to want to come to.

Many have argued that the structure of classes sets the scene for what happens in them, but the evidence is thin. The various structural influences are hotly debated, and teachers and students (and parents) wait each year for assignments to larger or smaller classes, higher or lower track classes, mainstreamed or special classes, multiage or single-age classes, hoping for the highly engaged and well-behaved students. Given students, not us, will create their futures, modeling how to work with diversity is one of the more powerful messages and heritages we can leave with our students. The desire is for the class to have social cohesion, great classroom management, minimal disruptive behavior, high levels of friendship and no bullying, and appropriate interventions and counseling for those most likely to affect other students. But these assets are not reserved for smaller classes, higher track, single-age, or students with special needs. Variance is a positive, is the norm, is an asset, and can be an opportunity to create an open-to-learn class climate, where there is a focus on every student, making at least a year’s growth for a year’s input – no matter where they start in their learning journey. Students learn much from each other, and we need to be more deliberate in teaching the skills of students working in teams with peers with diverse backgrounds and skills – as this is the world they live outside school and will do so increasingly when becoming adults.

The overemphasis on class structural considerations begs why these effects are so low. Only by asking this question can we understand how to improve their influence. For example, class size has a small effect despite the intuitive and seductive claims because teachers too rarely change their teaching methods but tend to do more of the same (ditto open vs. traditional classrooms and multigrade classes). We love to group, marginalize, separate, and reduce variance among students to make teachers’ lives easier. The effects of ability grouping are small on student’s achievement, but teachers measure success by their ease and satisfaction of teaching in more homogeneous classes; if they focused on the achievement and equity effects, this practice would or should soon disappear from our schools. It is fascinating that school size, while a high impact, is not the issue but the underlying reasons – enough students to offer a rich core to all but not too many to use electives as quasi-streaming.

When the opportunities from structural influences are realized, then the effects can increase. Within-class and small-group learning are not fixed, but they change frequently; the curriculum and teaching are honed to the current grouping; and all students have similar success criteria, but there may be different ways and times to attain them. A major reason why ability grouping for gifted students has higher effects is that the curriculum and teaching are modified and not more of the same. But this maxim should apply to all, as noted in the success of acceleration for all students. How do we cover less content to go deeper and allow enrichment for all? But too many prefer a mile-wide and an inch-deep curricula exposure (especially those anchored to national tests, which typically are of this mile-wide inch-deep form).

Too often, classroom composition also is more impacted by equity or other than achievement effects. The effects of mainstreaming are small, but the equity effects can be highly negative. This begs a return to the notion of the least restrictive environment, the maximal exposure to appropriate challenging work, the centrality of the student, and the focus on progress to achievement for all. Holding students back a year should be done with guilt at best, and where there is evidence of improvement (relative to similar peers allowed to advance), then these cases need to be published, as we need to better understand such rare occurrences of success. The achievement effects of retention (and suspension and expulsion) are bad enough, but the equity effects are damning.

The message in this chapter is that developing the class climate is far more critical regardless of how many and which students are present. Classrooms need to be inviting to students to come and want to learn. Classroom management is a necessary but not sufficient condition for raising achievement. The climate needs to be seen by students as fair, led by a teacher who they believe can enhance their learning, has high expectations of each student’s success, shows their with-it-ness, works quickly to suppress disruptive issues, has clarity of purpose, and shows excellent listening skills. Creating a sense of cohesion and positive relations between teacher and student and among students leads to high trust such that students can fail quickly, learn from failure, and feel comfortable not knowing and, thus, receptive to the impact of the teacher.

Disruptive students can be the bane of a teacher’s (and peers’) enjoyment of learning. When I ask teachers to choose either me randomly choosing 10–15 students to remove to make the class size smaller or they choose any 5 students they want to remove, the latter is always the winner. These disruptive students take a disproportionate amount of time, and often, focusing on them increases their disruptive behaviors. A major message is that there needs to be fair, clear, and understood consequences to poor behavior, or better fair, clear, and understood consequences for focusing on learning. McDonald (2021) outlined interventions and strategies to create high trust, appropriate regulatory disciplines, and high levels of engagement in learning. Teachers need to communicate their beliefs about behavior in learning in a way that is understood, focus on prevention and instruction to develop quality learning environments, develop strategies for dealing with challenging behaviors and vulnerable students, and be consistent and fair. If students want anything from a teacher, they want to know us, they want to be safe to learn, they want to see competence by the teacher to reduce misbehavior and promote learning, and they want fairness.

It is not reasonable, however, to assume that compliant, nondisruptive students are learning. Compliance may be a condition for teaching, but withdrawn, passive, avoiding, and non-engaged students have similar negative profiles to disruptive and naughty students. Drugging students into compliance may lead to fewer behavioral issues, but do not presume that learning is happening. School counselors can have important influences on these and many other students, and have become important resources in schools – especially when they work with teachers to then work with students (see RTI later, Chapter 13).

Schools can be lonely places, particularly during high school years, as students are developing their reputations as learners (or not) and friends feed our beliefs, expectations, engagement, and as Aristotle (350 BC / 1995) remarked, “community depends on friendship and when there is enmity instead of friendship, people will not even share the same path” (1295b23–25). Having friends, or at least a friendly class, leads to a sense of belonging, an important precursor to engagement in the hard work of learning. Being not liked or bullying or being bullied, not surprisingly, is negatively related to achievement, and tough as it is to reduce bullying (especially cyberbullying, which crosses the school gate), it must remain a high priority.

Reflections

Read the section on ability grouping and discuss these important findings in relation to your current practice as a school. It concludes as follows:

The qualitative evidence indicates that low-track classes are more fragmented, less engaging, and taught by fewer well-trained teachers. Clearly if these lower-tracked classrooms were more stimulating, challenging, and taught by well-trained teachers, there might be gains for tracking for these students: but there are not. It seems that the quality of teaching, the transmitted expectations of teachers and students, the differential curriculum of easier and harder work, and the nature of the student interactions are the key issues, rather than the compositional structure of the classes.

Are there ways in which changes could be made in your school (e.g., the most qualified teachers teach the lowest-tracked classes)? Discuss what would need to be implemented to support teachers in making these changes.

After reading the section on small-group learning, organize a trial period during which teachers use the conditions for effective collective efficacy; then report back the findings and impact at a staff meeting.

In the “Classroom management” section, the teacher attribute of with-itness is described as having a high positive influence (0.71): (a) decide together how to interpret potential behavior problems in the classroom, giving real examples, to ensure consistent expectations; then (b) agree on specific ways in which we should quickly act once these problems have been identified. Share words and phrases used. Where does with-it-ness fit in these discussions?

Read “Teacher-student relationships.” Do we communicate by words, tone of voice, and body language that we truly care for the learning of each student as a person? Do we empathize with our students: “see their perspective, communicate it back to them so that they have valuable feedback to self-assess, feel safe and learn to understand others and the content with the same interest and concern”? Try observing each other teach, just focusing on those three elements, and looking for how it appears to affect the self-efficacy of each student. Then ask the students whether their teacher listens, likes them, feels safe to acknowledge not knowing, and cares.

After reading the section “Decreasing disruptive behavior,” discuss whether your school has fair, clear, and understood consequences to poor behavior, bearing in mind the sentiments of the last reflection, as this is the stand-out message.

The key message of this whole chapter is that developing the climate of the class rather than its organization is far more critical. Read the conclusion, then discuss current strengths and areas to work on in your school in this regard. Are you consistently celebrating error as being on the brink of new learning, giving students strategies for what to do when they don’t understand, being clear about learning intentions, having high expectations for all students and modelling empathy, kindness, and fairness?

CHAPTER 9 The teacher

DOI: 10.4324/9781003380542-9

There are approximately 80 million teachers in our world’s schools, and UNESCO predicts another 69 million are needed in the next ten years. Over the past 50 years, there have been major transformations in who becomes a teacher. Ingersoll et al. (2014) used the US Schools and Staffing Survey from the 1980s onwards and noted seven trends in the US (and added three more), which are likely emulated in many of the OECD Western countries (see also https://ourworldindata.org/teachers-and-professors):

Larger: The teaching force has ballooned in size, and when you add teaching assistants and leadership positions, the ratio of adults to students has sharply increased. Since 1987, student enrollment has increased by 19% and the teaching force by 46%.

Grayer: The percentage of teachers over 50 has increased (33% > 50 in OECD countries), but this trend will likely reverse as we seek replacements for the growing number of students. Younger teachers are familiar with the gig economy, seeing teaching as a shorter and not lifelong vocation. As budgets become squeezed, the cheaper, less experienced teachers will be favored over the more expensive and experienced.

Greener: While the post-50s have increased, the other trend is the more signifi-cant proportion of beginning teachers. But noting that the average age of starting teacher education is more likely to be 25–30 and the average age (across Australia) of first registering as a teacher is now 30+, indicating larger numbers of mid-career switching into (and out of) teaching, the number of teachers in their 30s is almost as high as the post-50-year-old numbers. Indeed, in the 1980s the modal teacher had 15 years’ experience, but the current modal teacher is now in the 5th year of teaching.

Female: The teacher workforce has long been dominated by females (in the US, 67% in 1980 and 76% in 2012; across the OECD world, 70% in 2017; in Australia, 80%). This is not due to a decline in males entering; females now are increasing at double the rate of males. A more recent trend is the much faster rate of females becoming school leaders.

Diverse: While the major ethnicity within a country still dominates the profession, this is fast changing. The proportion of white students is decreasing in our schools, and the proportion of minority teachers is also increasing.

Academically okay: In the US, the best and brightest are less likely to become teachers, and those majoring in becoming a teacher have lower SAT scores than those not going into teaching – this has a continuing downward trend. In Australia, the average achievement estimates to enter teaching have been on a 12-year downward trajectory, only starting to stabilize and maybe reverse in the past two years as governments impose minimum achievement profiles for entering teaching.

Less likely to stay a teacher: Over the past 40 years, the annual attrition rose by 41%, from 6.4% to 9%, and higher in high-poverty, high-minority, urban, and rural public schools. Gundlach (2022) meta-analyzed 186 studies and found teachers leave due to conflict with leaders and stay because of students. School leadership and collegiality have a large effect on intentions to stay or not in schools (r = 0.39) and stay in the profession (0.28). No surprise, teachers who are satisfied with their jobs are much less likely to intend to leave or actually leave (0.20); and satisfaction is a function of autonomy (0.30), presence of school behavior systems (0.37), and quality and quantity of professional development (0.22); and teachers stay for the salary reasons (0.47; the alternatives jobs are not necessarily as secure or as highly paid). In addition, there were lower relations for mentoring (0.06), staff-student ratios (0.18), and parent relations (0.00). They are less likely to leave because of the students: academic standards (−0.13), special education students (0.00), limited English proficiency students (0.06), lower socioeconomic status (−0.10), or student misbehavior (0.05).

Prepared online: More teachers are coming into our schools, having been trained in online and not face-to-face programs. In Australia, for example, 25% of students in training are completing their course primarily by online courses (yes, they still have practice in real classrooms but often with no one from the program involved in supervision other than the class teacher) – and there is no reason to believe they are better or worse than face-to-face programs. These teachers are likely to have quite a different perspective of how teaching can occur in their future classrooms.

Start keen and hungry: Teachers in their first ten years are more willing to embrace professional standards and engage with other teachers in and outside their school. However, these same teachers are often the least served by professional learning and less likely to experience great induction programs (over 90% of Australian principals claim to have excellent induction programs, but less than 40% of their teachers know about it). They are also striving for opportunities for early leadership.

Paydeprived: Given their expertise, the salaries for teachers may start okay (top ten highest graduate salaries in Australia; many on short-term contracts), but after ten years, the increases dwindle, and thus, relative to occupations requiring similar levels of expertise, they start to slip well behind.

To be a teacher is to engage in a practice of human improvement, and teachers succeed when their students succeed. Teachers work with students who, in the main, are made by law to attend (and many would not go to or stay in school if it was not compulsory); they need to develop high levels of positive teacher-student relationships while also being in a position of authority; they need to establish and actively manage an emotional connection with students but have clear boundaries; they need to be respected, be clear, be trusted, and be seen by students as able to have positive impacts on them; they often work alone where they are the only professional in the room; they are allowed remarkable flexibility to choose how they teach, and many believe that their evidence of experience is more powerful than evidence from other sources; they deal with balancing the simple with the complex, aiming to make sense out of the myriad of instances and experiences that confront them each day in the class; and they work in a profession where experience more than expertise is a significant driver of salary, esteem, and opportunity. No wonder the differences among teachers are a major source of variance in teachers’ impact on their students.

The majority of those who choose to be teachers do so to have an impact on their students. Richardson and Watt (2016) reported that the highest-rated motivations for choosing a teaching career were intrinsic values, perceived teaching ability, the desire to make a social contribution, the work with children and adolescents, and positive prior teaching and learning experiences. Lower-rated motivations included job security, the desire to enhance social equity, and increased provision for family time. It is, thus, not surprising that the theme of Visible Learning, know thy impact, merely reminds educators to return to one of the primary reasons they entered the profession.

Given the ten points earlier, it might be time to ask if teaching will remain a career. The gig economy is a free-market system where workers work in short-term arrangements. In the US, this accounts for about one-third of all workers and includes freelancers, consultants, Uber drivers, Airbnb landlords, on-call workers, and relief teachers – and the majority who choose this type of work are happy with it primarily because they are self-employed; are their own bosses; have better balance in their home, private, and work lives; and can seek the highest bidder for their expertise. As the demand for teachers increases (relative to supply), those with hard-to-find expertise will likely become gig workers. It will, however, make it harder to maintain a teacher workforce. We have written about the importance of the profession and the system supporting the profession, seeing that expertise is the essence of teaching; the alternative is not pleasant (Rickard et al., 2021).

To have the desired impact on student achievement, there are many attributes and roles for teachers, such as building safe and respectful environments where it is okay to fail, learn, and work together; demonstrating that they listen to how students think and engage; showing their passion for learning and developing curiosity; and providing multiple opportunities to learn. In addition, we recall those teachers who saw something in us that allowed us to exceed our expectations, teachers who helped students to have different and better strategies or processes to learn and engage with the subject, and teachers who demonstrated patience and willingness to explain material and help students with their work (Clinton et al., 2018).

Teacher effects

The teacher accounts for the greatest source of within-school variance. There is, however, a wide estimate of the variance of the impact of different teachers. Hanushek and Rivkin (2006) reported that the average variance for reading teachers is 13% and for mathematics is 17% (c.f., Ladd, 2008). Rowan et al. (2002) estimated teachers account for 4–16%, and Nye et al. (2004), a range of 3–16% of the variance. This implies that a teacher at the 25th percentile of the quality distribution vs. one at the 75th percentile would mean a difference in learning gains of about 0.2 standard deviations in a year. Experiencing the higher impact teacher would move a student to the middle of the achievement distribution from the 50th to the 59th percentile. A major study evaluating the impact of teachers of varying quality showed that a 1 SD increase in a series of high-quality teachers was positively related to an increase in students’ 12th-grade mathematics achievement of d = 0.26; −0.29 above those students in non-high-quality classes (Lee, 2018).

But Grasby et al. (2020) estimated the classroom and teacher effects on literacy and numeracy by comparing the similarity of 3,402 twins who shared or did not share classrooms. There was a sobering low percentage of variance (2–3%) explained by teachers/classrooms. They commented that this says a lot about genetic factors, family environments, and expectations, or the commonality of what is taught in classes given a national curriculum focused on year expectations.

This chapter has two sections: teacher attributes and teacher professional learning (Table 9.1). Teacher attributes (Table 9.2) include personality, verbal ability, expectations, not labeling, estimates of achievement, teacher credibility, clarity, collective efficacy, and student rating of the quality of teaching. Teachers’ professional learning includes initial teacher education programs, teaching qualifications, professional development, coaching, microteaching/video review of lessons, and subject matter knowledge.

Table 9.1 Summary statistics for teacher attributes and teacher education

Teacher
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Teacher attributes
47
2,238
3,688,095
4,838
0.59
0.08
3

Teacher education
48
2,272
3,693,349
5,259
0.36
0.07
3

Total
95
4,510
7,381,443
10,097
0.47
0.08
3

The key notions relate to teachers working together critiquing their impact, their credibility in the students’ eyes to improve learning, the clarity of what is expected of students in the lesson, what they believe success looks like, teachers not labeling students, and each others’ expectations. There were lower effects for many methods to educate preservice teachers and professional learning.

Part A: Teacher attributes

Table 9.2 Summary statistics for teacher attributes

Teacher
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Teacher self-efficacy
3
44
4,050
122
0.27
0.04
2

Teacher verbal ability
1
21
1,933
58
0.22
0.03
1

Teacher effects
4
299
3,233,771
1,207
0.38
0.01
4

Teacher expectations
9
648
59,641
778
0.58
0.09
4

Teacher expectations -physical attractiveness
1
12
1,104
12
0.36
0.00
1

Teachers not labeling students
1
79
7,271
79
0.61
0.08
2

Teacher estimates of achievement
4
151
45,873
224
1.30
0.08
3

Teacher credibility
1
51
14,378
51
1.09
0.05
2

Teacher collaboration
1
46
3,975
67
0.38
0.04
1

Teacher self-efficacy
2
28
15,319
129
0.22
0.03
2

Teacher clarity
3
101
18,442
239
0.85
0.04
3

Collective teacher efficacy
3
85
5,699
85
1.34
0.05
2

Personality

There are few dimensions of teacher personality systematically related to achievement effects. From the breakdown in Table 9.3, the highest personality correlates with maximizing student achievement related to having personal confidence in oneself as a teacher, enthusiasm for the job, and judgment skills. The other personality factors have low to zero correlates.

Table 9.3 Correlations between teacher personality attributes and student achievement gains

No. rs
Aver
SE
Source

Personal efficacy
21
0.21
0.22
Klassen et al.

Teacher enthusiasm
98
0.16

Maddocks et al.

Judgment-perception
7
0.16
0.13
Klassen et al.

Expressiveness of teacher
12
0.14

Abrami et al.

Teaching efficacy
21
0.09
0.23
Klassen et al.

Teacher efficacy
83
0.08
0.18
Klassen et al.

Sensing-neuroticism
8
0.06
0.12
Klassen et al.

Conscientiousness
7
0.06
0.05
Klassen et al.

Extraversion
19
0.03
0.13
Klassen et al.

Ego defense
8
0.03
0.20
Klassen et al.

Neuroticism
8
0.01
0.07
Klassen et al.

Openness to experience
5
0.00
0.04
Klassen et al.

Self-efficacy
2
−0.03
0.06
Klassen et al.

Agreeableness
6
−0.06
0.06
Klassen et al.

Feeling-thinking
8
−0.13
0.18
Klassen et al.

Total
313
0.05
0.13

Teacher verbal ability

Teachers talk a lot, and verbal ability seems to be a somewhat important attribute. Verbal ability and its close correlate, intelligence of teachers, relate to student achievement (Kaplan & Owings, 2002; Darling-Hammond, 2000) but only for teachers during their first few years in the classroom. Hanushek (1989) wrote, “Perhaps the closest thing to a consistent conclusion across studies is the finding that teachers who perform well on verbal ability tests do better in the classroom” (p. 48). *Aloe and Becker (2009), in their meta-analysis, noted that the relationship is not as strong as many have claimed, and indeed, the relationship declines with teaching experience. They concluded that “having teachers with high verbal ability alone will not likely make a meaningful difference in our classrooms” (p. 620; note the word “alone”). It may be that more generalized academic or intellectual proficiency is a critical determinant in the early years as teachers grapple with the buzzing classrooms with massive expectations and load on them. As they become more experienced (say > 5 years in the classroom), the effect of intelligence and verbal ability decreases as other factors assume more importance (cf., *Greenwald et al., 1996).

While *Aloe and Becker (2009) reported a decline in teacher impact with experience, this is still contested. Indeed, teachers can improve their effectiveness by gaining more experience but maybe not at the same rate as during the first few years (Jackson & Bruegmann, 2009; Podolsky et al., 2019). Moreover, the effect sizes of this enhanced effectiveness are positive but small: Ladd and Sorensen (2017) reported that a teacher with 12 years of experience raised test scores from 0.08 standard deviations in ELA to 0.18 standard deviations in math as compared to teachers with no prior experience. Papay and Kraft (2015) reported gains about 0.05 but noted the nonlinear curve in improved effectiveness (Figure 9.1). The claim here is that we do not necessarily accept these small gains by the more experienced (many who may have reached a high level of effectiveness) but invite school leaders to structure schools so that all collectively and collegially make improvements in their effectiveness – independent of experience.

Figure 9.1 Relation between years of experience and student mathematics achievement.

Expectations

Teachers form expectations about student abilities and skills, affecting student achievement (*Dusek & Joseph, 1985). The question is not “Do teachers have expectations?” but “Do they have false and misleading expectations that lead to decrements in learning or learning gains – and for which students?” and “Do they have high expectations for all their students?” Across the eight meta-analyses, there is a systematic high effect, indicating that teachers’ expectations can make a difference in the student’s performance.

Care is needed to interpret the implications of these effects on teacher expectations. Weinstein (2009) has shown that students know they are treated differentially in the classroom due to expectations held by teachers and are quite accurate in informing on how teachers differ in the degree to which they have higher expectations of some children over others. There are differences in classrooms where teachers aim to select talent for different educational pathways (such as schools with tracking) compared with those where achievement cultures seek to develop talent in each child. There are differences in classes where teachers believe achievement is difficult to change because it is fixed and innate compared to teachers who believe achievement is changeable (Dweck, 2008). Weinstein also demonstrated many institutional practices (such as tracking) could lead to beliefs that preclude many opportunities to learn: “Expectancy processes do not reside solely ‘in the minds of teachers’ but instead are built into the very fabric of our institutions and our society” (Weinstein, 2009, p. 290). Fox (2016), for example, found that same-race or same-sex teachers had little effect on expectations to complete more schooling or be recommended for advanced courses – with one noted exception. The same-race effect for teacher expectations to complete more than high school ranges from 11 to 17 percentage points.

What is more important is whether the teacher has high or low expectations. Rubie-Davies (2014, 2017; Rubie-Davies et al., 2006) found that teachers with high expectations tend to hold them for all the students in the class and successfully raise achievement, whereas those with low expectations are, sadly, also successful in the opposite direction. Further, these expectations can dramatically affect what happens in their classrooms. Table 9.4 shows the contrast from observations of classrooms of low- and high-expectation teachers. High-expectation teachers think of their role as change agents and not facilitators, have growth not fixed notions of ability, are more mastery than performance goal-oriented, have a better balance of open and closed questions and challenges, are focused on learning more than doing activities, were more explicit about the nature of success more than a procedural focus on completing tasks, gave more effective feedback to students, and were more open to receiving feedback from students about their interests and prior knowledge. A major difference was in their conception of differentiation, with highs rarely mentioning the term but, when pressed, argued that all students can attain the success criteria of the lesson and that differentiation means different times and ways of attaining success. The lows saw differentiation in terms of grouping, different activities, and levels of challenge; different opportunities, therefore, are exposed to the curriculum and different expectations of the low- and high-ability student’s chances of success. No wonder the outcome effects are so different.

Table 9.4 Differences in classrooms of low- and high-expectation teachers

Low expectation (d= −0.03 to 0.20)
High expectation (d= 0.50 to 1.44)

See role as facilitators, constructivists, socializers
See role as director, active change agent, academic instructor

Expect low performance and see low performance, and this reinforces their views about low performance
Expect improvement, see the errors, and seek negative evidence to enact improvement

Hold fixed view of ability
Hold incremental notions of intelligence

Argue that some are expected to improve
Argue that all are expected to improve

See great differences among students in class
See lower differences among students in class

Place students in fairly inflexible ability groups
Mainly use interest-based grouping

Clearly differentiate the instructional activities between high- and low-ability students
Have students participate in similar activities

Have high- and low-ability students participate in quite discrete learning activities
Have all students participate in similar learning experiences

Stress performance goals
Emphasize task-mastery goals

Provide much repetition and lower-level activities for lower-ability students
Engage all in advanced activities

Often speak about differentiated activities in class
Hardly speak about differentiated activities in class

Spend more time with low and ask high to work independently
Work with all equally

Comment on low effort, class behavior, in-class relationships
Comment on developing confidence, motivation, persistence, and attitude to work

Constantly remind students of procedures and routines
Have procedures in place that students manage themselves

Focus on activities and behaviors
Focus on learning

Communicate details of the activities to be completed
Speak more frequently about goal setting and success criteria

Ask mostly closed questions
Ask more open-impact and closed questions

Make more negative statements about learning and behavior, and has low trust
Make more positive statements about learning and creates high trust

Give dollops of praise
Give more feedback, critique, and a focus on specific where-to-next feedback

Make less observations and interviews assessing and monitoring student learning
Make more observations and interviews assessing and monitoring student learning

Expose students more to lower-level, repetitive activities
Make all students feel challenged in completing activities

Give less attention to student interests and minimal choice
Have learning opportunities centered around student interests

Make little use of student prior knowledge
More often use student prior knowledge to make connections

Keep to their script for the lesson
More likely modify lessons according to student needs

Based on this evidence, teachers must stop overemphasizing ability and start emphasizing progress (steep learning curves or acceleration are the right of all students regardless of where they start) to emphasize individual progress made against previous attainment, rather than comparing students to one another (you could be last in a race but be pleased that you’ve beaten your personal best); to stop seeking evidence to confirm any low prior expectations but to seek evidence to surprise them about student talents, skills, and progress; to find ways to raise the achievement of all; to stop creating schools that attempt to lock in prior achievement and experiences (e.g., via ability grouping between or within classes); and to be evidence-informed about the talents and growth of all students by welcoming diversity and being accountable for all. “Be prepared to be surprised” and “Do not presume” seem appropriate mantras to avoid adverse expectation effects. If teachers and schools have expectations, make them challenging, appropriate, and checkable so that all students are achieving what is deemed valuable.

As seen in Chapter 5, student expectations are possibly more influential than teachers’ expectations. They know about the teacher’s expectations, feel it in the grouping within the class, and soon, learn their place in the school’s learning environment. The teachers’ role is to help each student exceed what they think are their expectations, not reify them, douse opportunities to flourish, and provide mundane, repetitive activities that maintain instead of raise their level of achievement.

Teacher expectations – physical attractiveness

When we meet people, we quickly form expectations about them; physical appearance is in your face. We are more likely to consider physically attractive people as higher in social skills, intelligence, and health, and have warmer and friendlier dispositions (Patzer, 2012). We are more likely to employ and vote for the more physically attractive candidate, and students claim they learn more from more attractive teachers (Hosoda et al., 2003; Olivola & Todorov, 2010; Westfall et al., 2016). *Ritts et al. (1992) reported d = 0.36 between student attractiveness and teacher judgments for academic measures (e.g., intelligence, future academic potential, grades). They argued this came via their biases that physical attractiveness is related to positive social skills and less behavior misconduct. Teachers are more willing to recommend special-needs programs for unattractive than attractive children (d = 0.48). Regardless of the student’s gender, race, or past performance, this was the case.

Not labeling students

We are very good at inventing labels and assigning these students to groups – often, funding follows these groups. The problem is that the label of the group can lead to low expectations for students in the group and that every child is seen as equal within these groups, and we then personify the problem as the students (or home or resources). The key is using evidence-based interventions (e.g., intentional alignment, RTI, MTSS) and designing optimal learning interventions for all students. Slavin (2019) notes that schools are being held to lower accountabilities for these students and that special education may need to be reinvented for these students, and many of the other diagnosed students are seen as victims of poor regular class instruction, and this is what needs fixing. Hettleman (2019) has written a damning indictment of the problems of mislabeling the disabled. He is passionate about services for students with special needs. There is no simple answer, but the love of labels needs to be checked, as lower expectations often have devasting consequences.

Grouping, thus, can lead to a particularly invidious form of low expectations, leading to nasty negative effects from labeling or grouping students. These low expectations lead to questions about why these students cannot learn, engage, and be successful. For example, the label “learning difficulties” can lead to low performance expectations. The label “ADHD” can assume every student diagnosed with ADHD is similarly a low performer, and those with special needs must, therefore, be treated as if they have some handicap. *Fuchs et al. (2002) found 79 studies that compared students with lower reading achievement but not labeled in any way with those labeled as having learning difficulties. The effect of d = −0.61 indicated that the reading scores of 73% of low achievers without the label were above the average reading score of low achievers with the label – clearly, labeling leads to lower performance.

The conundrum is that successful diagnoses can be the first step to appropriate intervention. Imagine feeling ill and your doctor asks you to complete a series of tests and then proclaims you have illness x; that is the end of the discussion; you will now be seen as an illness x person, and you will be invited to modify your behavior and interactions, and others will be told to respect your illness and accommodate it. No, diagnosing illness x is the first critical phase to deciding on the appropriate intervention for wellness, flourishing, and health. In education, too often, diagnoses provide an explanation for why a student cannot, and accommodations are made to the diagnoses rather than vigorously intervening based on this diagnosis to lead to flourishing. Diagnosis is but the first and should never be the last step.

The Singapore school system refuses these labels, preferring “low progress students,” and this is a step in the right direction, but even so, this term also can be a proxy for blaming the child or a code for special needs, low SES, or migrant backgrounds. Earlier, it was noted that Italy has regulated that all students must be exposed to the same curriculum and the same quality and quantity of instruction. Perhaps we need a robust debate on either ensuring labels are used only for diagnosis of specific learning issues and never attached to the child or mandating that every child needs accelerating and has a defensible intervention program premised on acceleration.

Teacher estimates of achievement

Teachers continually make judgments about student abilities to then decide on the most appropriate learning and task requirements (Artelt & Gräsel, 2009) – and these judgments can be relative to other students, relative to each student’s actual achievement, or relative to some absolute judgment, such as standardized test scores or IQ measures. Teachers with more accurate judgments are more likely to calibrate the optimal difficulty of tasks and be more able to provide stretch feedback (Urhahne & Wijnia, 2021). The meta-analyses on teacher judgment indicate very high estimations of judgment accuracy of teachers’ estimates with actual performance (*Südkamp et al., 2012, r = 0.63; *Hoge & Coladarci, 1989, r = 0. 66) but less so about creativity (*Gralewski & Karwowski, r = 0.23).

*Machts et al. (2014) related teacher judgments to intelligence, r = 0.54; giftedness, 0.37; and creativity, 0.36 (the latter included non-achievement factors, such as motivation and dispositions). The accuracy did not differ by school level. Where there is a discrepancy between student’s current achievement and their intelligence, teachers tend to default to achievement, which could mean many students who are underachievers are less likely to be identified. The conclusion of *Machts et al. was that “Teachers appear to be good enough judges of school performance and a general ability level; however, expecting every teacher to have the competency to judge underlying intellectual potential may be unrealistic.” These results certainly give credence to trusting and including teacher estimates of achievement, provided they are triangulated alongside the more standard methods (test scores, etc.).

These judgments also convert to more formal aspects of education, such as students’ report cards, diplomas, course placement decisions, and school tracking decisions. *Machts et al. (2020) showed that when teachers write report cards, they tend to have an overall judgment about the students’ achievement that influences all subject estimates of achievement, as well as a highly correlated dimension of learning orientation (e.g., use of methods, autonomy, engagement), but are more able to distinguish among various social dimension (teamwork, ability to handle conflict).

Even though the overall correlations are high, about two-thirds of the variance in teachers’ judgments cannot be explained by student performance. There can also be a halo effect where overall beliefs about a student are generalized to all achievement domains. Carvalho and Abreu (2018) showed how teachers’ estimates of achievement could be wrongly influenced by family support, social skills, disciplined behavior, and gender. As Sanrey et al. (2021) noted, it is, thus, important for teachers to maintain a degree of uncertainty and extensively seek and process information about students before forming judgments, as well as to ask about the achievements of a student in a particular area and not ask the more generalized question, “How good is this student?” Do not presume. Therefore, the judgments must be continually triangulated, tested, and reconsidered. It also should not mean that students with higher achievement are, thus, likely to grow and learn, and those with lower achievement are doomed to stay at the bottom. The job of teachers relates more to ensuring every student, no matter where they start, makes high levels of growth during the lessons.

This discussion also speaks to a major lack in many of today’s classrooms. The best information about student achievement is their achievement, and if this is underestimated or students are not working to their optimal, this can create a self-fulfilling prophecy. If other independent information were available, it might raise the expectations of teachers (and students) to then realize higher achievement via providing more challenging tasks and fostering higher expectations of success. Such measures could include intelligence measures, achievement batteries, standardized tests, or ability. But of course, some of these measures have a dirty history, and there is no desire to replicate some undesirable uses (see Fletcher & Hattie, 2011).

Teacher credibility

Teacher credibility is the students’ belief that they can learn from a particular teacher because the teacher is believable, convincing, and capable of persuading students that they can be successful. If a teacher is not perceived as credible, the students just turn off. This credibility is a function of beliefs about teacher’s trust, competence, passion for their subject, and desire for their students to learn in their class.

*Finn et al. (2009) investigated the effects of students’ beliefs about teacher credibility. Students who perceived their teachers as caring, understanding, and responsive to students were more likely to sign up for another course with the same instructor and were more likely to recommend the teacher to other students. In addition, they reported greater motivation to learn and increased cognitive learning in classes where they considered the teacher as credible. Thus, if you ask a student before they enter a class, “Is this teacher credible in their desire, ability, and caring about your learning?” and they say no, then the impact of that teacher is likely to be seriously compromised.

Teacher clarity

Bolkan (2017) outlined a five-factor model of instructor clarity: disfluency, overload, interaction, coherence, and structure. Disfluency refers to instructors who have difficulty explaining class concepts in a simple manner, who cannot create examples to explain course concepts, and who deliver course lessons in a convoluted fashion. Overload refers to learning situations where the pace of instruction outstretches students’ ability to absorb course material. Interaction indicates that explicit instruction must include working with students to determine their levels of comprehension and adjusting class lectures to adapt to student understanding. Coherence is a function of not providing redundant information, which can confuse students and direct their focus to unimportant aspects of their course lessons. Finally, structure refers to the organized presentation of information.

*Fendick (1990) explicitly explored the effects of the teacher’s clarity of speech (speaks loud enough so all can hear, does not use few vague terms), organization (starts with success criteria, covers all topics on post-test, reviews student work), explanation (explains simply and interestingly, explains at the right pace, gives examples of how to do the work, answers student questions, gives enough time), and assessment (asks questions, encourages discussions, provides feedback). The overall correlation of these dimensions of clarity and achievement is high (r = 0.35; d = 0.67) and consistent across the various dimensions: organization, 0.32; explanation, 0.35; examples and guided practice, 0.23; and assessment of student learning, 0.32. The effects were similar across verbal and numerical subjects, larger and smaller class sizes, and for less and more experienced teachers. They increased across grade levels (elementary, 0.26; high school, 0.30; college, 0.41) and were higher when rated by the students (0.40) than by independent observers (0.28, as surely, they do not experience the class as the students do). *Titsworth et al. (2015) more recently reported r = 0.52 on affective learning and r = 0.34 on cognitive learning, and this did differ when measured by self-reports or via achievement tests as the outcome. They wisely warned that this should not mean over-orchestrating learning and that ambiguity may be warranted occasionally.

It seems a hands-down misère that if students do not understand the teacher’s instructions or lessons, there is unlikely to be much comprehension and engagement. But clarity is more than comprehension. It is about “being explicit about precision-in-practice” (Sharratt, 2018). It is about teachers having high expectations; sharing their notions of success criteria with their students; ensuring there is intentional alignment among the lesson, the tasks, and assignments; providing that the delivery of the lesson is relevant, accurate, and comprehensible to students; giving worked examples to illustrate the degree of cognitive complexity desired from the students; providing welcome feedback about where to move to next (Fisher et al., 2018).

Teachers’ collective efficacy about their impact

Since Bandura’s pioneering work (1993), the notion of collective efficacy has been influential. Bandura observed that a group’s confidence in its abilities seemed to be associated with greater success, and when a team of individuals shares the belief that through their unified efforts, they can overcome challenges and produce intended results, groups are more effective. This phenomenon has been observed often in schools with large impacts on student achievement.

There is much more to collective efficacy than just feeling influential or having high expectations. These efficacy beliefs need to focus on what impact means and how to diagnose and evaluate the levels of impact, and the beliefs need to be fed with the evidence of impact. A shared language about learning and impact is more critical than a shared way of teaching, and the perceptions that influence the actions of educators in cultures where efficacy is present include “We are evaluators,” “We are change agents,” and “We collaborate.”

It can be difficult to engender such efficacy, as too often, staffrooms and professional learning experiences rarely engage in collaborative inquiry about teacher’s theories of teaching and learning, and rarely aim to develop some consensus about what high impact means and how it can be illustrated; and Little (1990) noted that collaborations in schools often “appear contrived, inauthentic, grafted on, perched precariously (and often temporarily) on the margins of real work” (p. 510). Teachers can prefer to solve problems that inquire into the problem; engage in storytelling; and deflect discussion to the curriculum, attributes of students, resources, time, and workload rather than engage in pedagogically productive inquiry. As Lefstein et al. (2020) noted, such talk requires expertise of sensitivity, requires noticing that a critical event requires attention, requires interpretation to make sense of the event, requires reasoning using evidence to move beyond opinions, requires a repertoire of strategies, requires allowing for multiple voices to make interpretations, requires skills to navigate tensions, requires challenging and critiquing of one’s own and others’ viewpoints, requires confidence that the collective can lead to a better interpretation and direction, and requires judgment to choose the best course of action.

Donohoo et al. (2018) noted that school leaders play a key role in creating nonthreatening evidence-based instructional environments. By promoting a culture of collaboration focused on knowing thy collective impact, leaders have the potential to support school improvement in ways that positively influence teachers’ collective efficacy beliefs and, thus, promote student achievement. Leaders do this by creating conversations about what impact and effort mean, the difference between progress and achievement, and the use of reliable evidence. They can also influence collective efficacy by setting expectations for formal, frequent, and productive teacher collaboration, and creating high levels of trust for this collaboration to occur.

There are skills in developing collective efficacy, such as high levels of social sensitivity, high levels of turn-taking, high levels of social awareness, confidence in each other’s abilities, and the belief in the impact of the team’s work. We have explored in more depth these notions in our translation of this high-level influence in our book Student Collective Self-Effi cacy (Hattie et al., 2021c; cf., Datnow & Park, 2018; DeWitt, 2017; Hargreaves & O’Connor, 2018). Much also depends on the nature of the task for the collective, with a preference for more open-ended and information-rich problems, particularly if the complexity permits complementary divisions of labor (Stasser & Abele, 2020). There needs to be attention to ensure collective efficacy is not seen as mere teachers working together and meeting, and the focus of the efficacy needs to be clearly on maximizing the impact on students.

Teachers’ professional learning

Initial teacher education programs

Teachers learn 50% of what they come to know about teaching in their first year in the classroom and half as much again in the second, and it becomes relatively flat after this – with little variance in knowledge attributed to teacher education programs. The majority of those teachers (62%) in their first year who had the lowest effects on students are the same teachers with the lowest effects five years later. Conversely, the majority of those (73%) with the greatest effects in their first year are the same teachers with the highest effects five years later (Atteberry et al., 2015; Henry et al., 2011).

Further, more than 90% of principals claim they have an excellent mentoring program for new teachers, but less than 40% know about these mentoring programs (AITSL, 2017). My experience is that these new teachers are among the hungriest for learning and most exciting to work with, but too many quickly believe that a great teacher is a busy one – and they spend an excessive time seeking and creating resources, marking, and commenting. For example, one teacher I shadowed was told, “your class is unique, and if you have a problem, come and see me,” which was code for the requirement to tailor and create all resources anew, and any seeking assistance was a symptom of a problem/failure. Another teacher I shadowed moved from a regular to an open classroom (94 years 1–3 students with three teachers), and each teacher was assigned a term planning for the team (and thus, two terms with no planning). He was delighted to do a third less planning, had more time to engage in the acts of teaching, but was warned, “Don’t tell anyone you do not plan for two-thirds of the year” – as if being efficient is a secret and an admission of not being a good teacher.

Levine (2006, p. 109) described teacher education as the “the Dodge City of the education world. Like the fabled Wild West town, it is unruly and disordered.” There is no standard approach to where and how teachers should be prepared. Walsh (2006, p. 1) also claimed that “the nation’s leading teacher educators … concede that there is presently very little empirical evidence to support the methods used to prepare the nation’s teachers”. Yet for those working in many teacher education institutions, there is a strong claim that there is a standard approach, order, and core knowledge and understandings that all future teachers should have. I have sat through many meetings where colleagues have decided on the essential core knowledge and experiences that should be taught to teacher education students. This has been a long and often vexed discussion in every place and every time the group has decided on this core – but in every place, this core has been different. There is no set of essential experiences that must be taught, let alone a correct order for teaching students to become teachers.

Moreover, it seems surprising that the education of new teachers seems so data-free; maybe this is where future teachers learn how to ignore evidence, emphasize craft, and look for positive evidence that they are making a difference (somewhere, somehow, with someone). Spending three to five years in training seems to lead to teachers who are reproducers, teachers who mimic the teacher they liked most when they were at school, and teachers who too often see little value in other than practice-based learning on the job. The common refrain that “the best part of college was practice teaching” or that the real learning occurs in situ points to the lack of effect of the college experience (a refrain often advocated by teacher educators who do not seem to realize how ineffectual it makes them sound). This is not claiming that there should not be teacher preparation programs or that there are no exemplary teacher education programs. They are exemplars (Darling-Hammon & Lieberman, 2013), and for a review of ITE evidence, see Hattie et al. (2015).

We have outlined the clinical model of teacher education successfully implemented at the University of Melbourne (modeled on Darling-Hammond at Stanford; Rickards et al., 2021). While there is some debate over the term “clinical” (some seeing as cold, focused on a deficit), its meaning is more a balancing of evaluative thinking, reasoning, and critical thinking focused on students’ impact. As Sackett et al. (1996) claimed in medicine, it is the “conscientious, explicit, and judicious use of current best evidence in making decisions about the care of individual patients” (p. 71); it is neither the tyranny of evidence nor the opinions or reflections of the teacher. Neither judgment nor evidence is enough, but it is the judicious balancing and integration of clinical expertise (proficiency and judgments) with the best available external evidence from systematic research. We now prefer the notion of developing evaluative thinking and the core attributes are the centrality of clients/students, specialized knowledge and skills, use of evidence and judgments in practice, accountability to professional standards, and rigorous academic and practical training before registration as a teacher (Alter & Coggshall, 2009).

A signature aspect of the UoM model is the assessment: the Clinical Praxis Exam asks teacher candidates to present the diagnosis, intervention, and evaluation of their practice concerning specific student learning needs and development – mapped to the following:

What is the student ready to learn, and what evidence supports this?

What are the possible and preferred evidence-based interventions?

What is the expected impact on learning, and how will this be evaluated?

How will the preferred intervention be resourced and implemented?

What happened, and how can this be interpreted?

How do I collaborate with others (colleagues, students, research) to gain insight into my impact?

Mini versions are interspersed throughout the program (often presented orally in a 5–10-minute presentation), culminating in a significant capstone presentation. The task draws explicitly on and requires the synthesis of theory and content knowledge from core subjects and learning areas or disciplines with evidence from teaching practice, thus enabling meaningful engagement between the coursework site (the university) and the sites of practice (the schools) (see Rickards et al., 2021, for details).

The meta-analyses relating to teacher education show that the effect size of teacher education on subsequent student outcomes is negligible (about 0.10). *Qu and Becker (2003) reported a very small effect from only 24 studies – not many studies considering how important this topic should be (and they acknowledged the difficulties in finding even these studies). The effects of four-year college training compared with alternative certification are d = −0.01 and, compared with emergency licenses, d = 0.14. The effect for those training in one subject but teaching out-of-field is d = 0.09, but when compared to teachers with full certification who have several years of teaching experience as opposed to emergency teachers, the effect rises to d = 0.39 – probably reflecting the influences of teaching experience (pedagogical subject matter) and not subject matter knowledge per se. Similarly, *Sparks (2005) found few (N = 5) studies with 18 effects and commented on how little is known about such an important, well-discussed, and advocated topic. She reported that fully certified teachers had slightly more effect on student achievement than those with probationary or emergency licenses (across mathematics, science, and reading, d = 0.12); and that teachers trained in the field they are teaching in were more effective than those not so trained (d = 0.38). While not a meta-analysis, one of the rare random controlled studies involved assigning students to 44 teachers with emergency licenses and 56 trained teachers (Glazerman et al., 2006). They found no differences in reading and d = 0.15 in mathematics. They concluded that Teach for America teachers were “an appealing pool of candidates” (p. 95), mainly as they serve low-income and often difficult-to-staff schools. At best, it was concluded that teacher education programs make some difference compared to emergency licenses. However, so much more is needed on this topic.

Over the past generation, there have been over 100 reform committees and documents here in Australia, most recommending higher entry standards, different curricula, more time in schools, and specific requirements to include x or y – and thus, it is no surprise that most have had limited to no effect. The 101st review (TEMAG, 2014) did the opposite. Following the NCATE/CAEP model in the US, it started by asking the 400+ programs to put the evidence on the table that the graduates could change the learning lives of students (i.e., be classroom-ready). Since then, the regulations have been changed, the debate about being classroom-ready has been in full flight, and the first programs have been accredited based on their evidence of impact on their graduates. Institutions are now cooperating to provide external moderation of their standards of classroom-readiness. There are Teacher Performance Assessments that cross program and university boundaries, and there is accumulating evidence of the success of many of these programs. The message is not because the overall effect size is low, therefore, we should abandon teacher education, but we should celebrate the successful programs based on the evidence of their impact and learn and scale these programs. We can raise teacher education standards only when we know what these standards are and what they relate to, when we build an evidence base of successful programs, and when the profession is involved in setting the standards.

What may be needed is more study of the best programs, so there is a focus on scaling up success. Darling-Hammond (2000), for example, studied exemplary teacher education programs and identified seven features of these programs: coherence based on a common, clear vision of good teaching that permeates all coursework and clinical experiences; well-defined standards of practice and performance that guide and evaluate coursework and clinical work; curriculum grounded in knowledge of child and adolescent development, learning, social contexts, and subject matter pedagogy; extended clinical experiences carefully developed to support the ideas and practices presented in simultaneous closely interwoven coursework; explicit strategies to help student teachers to confront their own deep-seated beliefs and assumptions about learning; strong relationships, shared knowledge, and shared beliefs that link all who are teaching these prospective teachers; and assessment based on professional standards that evaluates teaching through demonstrations of critical skills and abilities using performance assessments and portfolios.

Teacher qualifications

*Manning et al. (2019) reviewed the effects of qualifications within early childhood education and care. These qualifications of teachers/caregivers can vary in terms of the level of education (e.g., high school, post-high school, university degree), degree of specialization (i.e., general, special, or early childhood education), and type of training (e.g., preservice, on-the-job, or continuing). The only qualification across their 49 studies was the caregivers’ education level, which had low but positive relations for the quality of childcare learning (r = 0.19).

*Klassen and Kim (2018) reviewed 32 studies concerning selecting teachers for teacher education preparation courses. The correlation of both academic (r = 0.13) and non-academic (r = 0.10) predictors of later teacher effectiveness (external measures, observation ratings, achievement gains) was small. These averages did not vary across the level of schooling, selection type, or selection for ITE or employment. They noted that these correlations are much lower than in many other vocations.

Professional development

Too often, professional learning is evaluated in terms of the effects on the teacher (their satisfaction, liking, and perceived learning from the course) and much less regarding the effects on actual improvements to classroom practice and impact on students. *Wade (1985) divided the outcomes into four groups: reaction, how the teachers felt about the professional development; learning, the amount of learning the teachers accrued; behavior, whether teachers changed their behavior because of the professional development; and student outcomes, impact on students. Professional development is more likely to shift teacher learning (d = 0.90), but these learnings have lower effects on teachers’ actual behavior (d = 0.60) and teachers’ reactions to the professional development (d = 0.42), and even lower influence on student learning (d = 0.37) (see also *Batts, 1988; *Egert, 2015; *Harrison, 1980; *Kraft et al., 2017). The effects are larger when the outcomes are more directly tied to specific success criteria from professional learning (*Blank & Alas, 2010) and when the program has already been field-tested or used extensively (*Harrison, 1980).

Across the 23 meta-analyses, there is much variance in the overall effects (from 0.10 to 0.66), reflecting the variability in the quality of impact of PL. Metas with more studies had higher effects, and more recent studies had lower effects – possibly because more recent metas are more rigorous and focus much more on the effects of PL on students than on teachers. Gore et al. (2022) argued that the current consensus on effective PD needs to be questioned. Thus, they questioned if the identification of the current consensus includes a focus on curriculum content, including discipline-specific content knowledge and pedagogy; active learning; collaboration among teachers across grade levels, schools, or districts; a sustained duration over an extended time; modeling of best practice; the use of coaches or expert support; opportunities for feedback and reflection; and coherence with school and teacher priorities so that PL is aligned with broader policies and/or reform efforts. They offered no evidence of the enhanced effects of an alternative. There have, however, been many studies and meta-analyses showing the key components, whereas maybe more research could be spent on implementation models (e.g., fidelity, dosage, adaptation, quality) (see Hamilton et al., 2022). I agree with Gore et al. that identifying the key components is the first step, but quality implementation is less understood.

Still, the most comprehensive review is by *Timperley et al. (2007), who reported an overall effect of PL of 0.66, with high effects in science (1.10), writing (0.81), language/literacy (0.95), and math (0.54); and higher in high school (0.66) and elementary (0.58) than in middle school years (0.32). They provided many findings of the key components, including the following:

Extended time for opportunities to learn was necessary but not sufficient.

External expertise was typically necessary but not sufficient (as they became the honest brokers).

The prevailing discourses challenged (related to enhancing the mind frames more than tips and tricks).

Opportunities to participate in a professional community of practice were more important than place.

Consistency with wider trends in policy and research (the need for alignment).

Active involvement of school leadership.

A clear alignment between the intended learning goals and the activities was evident.

Teachers were provided with a variety of ways with multiple opportunities to understand and learn the content.

The student perspective was maintained.

This notion of challenging prevailing discourses is critical not only for the student but also for teacher learning. This disequilibrium upsets our cognitive biases of continually seeking evidence that we are having an impact and asks instead for higher than lower expectations for all students, and seeking evidence where we may not be having an impact, thence leading to deeper considerations of the effects of our teaching. This disequilibrium is a powerful precursor to effective professional learning. This can take time (*Tinoca, 2004), the involvement of school leaders in this process, and the involvement of external experts. These experts are most impactful when they are attentive to the fidelity of implementation, ensure the impact of PL on students, and call low performance and engagement.

Teachers talking to teachers about teaching (involvement in a professional learning community; PLC) was necessary but not sufficient. The focus of PLCs needs to relate to the evidence of impact on the students from professional learning. PLCs are more effective when problematic beliefs are challenged when there is testing of the efficacy of competing ideas, when high expectations for all are developed, and when discussions were grounded in artifacts representing growth and challenges in student learning. Professional development was more effective when the school leadership supported opportunities to learn (often indicated by the leader’s presence in the professional learning), where access to relevant expertise was available, and when opportunities were provided to meet to process new information (see also *Patrick, 2014). Factors unrelated to the impact of professional learning include funding, release time, and whether the involvement was voluntary or compulsory.

Coaching

Coaching teachers is quite popular, but there are so many varieties (e.g., peer, cognitive, instructional coaching) that it is not surprising that there is much variability in the effects (Atteberry & Bryk, 2011; Deussen et al., 2007). Many coaching studies aim to improve classroom practice or implement some teaching processes with fidelity, and fewer evaluate the impact on student achievement outcomes. The common denominator of many coaching models includes analyzing the baseline current situation, setting goals, identifying the intervention, and providing support until the goals are met (often with classroom observation and feedback). Coaching tends to be “individualized, time-intensive, sustained over a semester or year, context-specific and focused on discrete skills” (*Kraft et al., 2018, p. 2). Coaching can be expensive to implement and scale up, and so often, it is the particular skills of the coach more than the method or focus that makes the difference.

Kraft et al. (2018) reviewed 60 studies of coaching on instructional practice (d = 0.49) and student achievement (0.18). The achievement effects were higher in science (0.35) than reading (0.16) and math (0.04); there are smaller effects for general coaching programs (0.07) than content-specific programs (0.20); the effects were larger in K–12 (0.26) than ECE programs (0.12), were larger for smaller programs (< 100 teachers, 0.28) than larger programs (0.10), and had no differences when coaching was virtual or in-person (see also Powell et al., 2010).

Unlike most meta-analyses of teaching interventions, there are studies comparing coaching with other methods. For example, *Kraft et al. (2018) found that pairing coaching with group trainers (0.12) was more effective than providing teachers with a video library (−0.13). *Dunst et al. (2019) compared four enhancing adult professional learning methods. The effect size for coaching on teaching practice (0.91) was highest, then just-in-time training or individualized tailored training to specific immediate concerns (0.52), guided design or problem-solving sessions (0.49), and relaxation and role-playing learning (0.05). They also found the effects of coaching were highest for teacher skill acquisition (0.69), self-efficacy beliefs (0.49), learning attitudes (0.41), and knowledge acquisition (0.33). It seems these influences of coaching do not translate well to students. *Dietrichson et al. (2017) reported coaching (0.04) was hardly different from after-school programs (0.02), providing incentives (0.010), summer programs (0.03), personnel development (0.07), technology (0.11), and increased resources (0.08), and much lower than cooperative learning (0.22), feedback and progress monitoring (0.32), and tutoring (0.36).

Knight (2019) has used these findings to refine coaching into three phases: identify, learn, and improve. During the identify stage, coaches partner with teachers to identify a clear picture of reality; agree on a powerful, compelling, reachable, and student-focused goal; and agree on a strategy the teacher will implement to hit the goal. During the learn stage, the coach helps prepare the teacher to hit the goal by clearly describing the strategy to be implemented, often with the help of a checklist, and then provides a model of the strategy in one or more ways. Finally, during the improve stage, the coach supports the teacher as they make adaptations until the goal is met.

Thus, coaches need to deeply diagnose the current baseline reality (context, beliefs, and skills of the teacher), work in partnership with the teacher, listen and engage in dialogue not monologue, be precise not provisional (which requires deep skills in addressing the fidelity, dosage, adaptations, and quality of implementation), anchor the coaching in the class (e.g., use videos, share their interviews with students, use student artifacts and scores), offer powerful tools for analyzing student work to show the impact of the intervention and teacher skills, keep to the goals (otherwise, “it is just a nice conversation”; Campbell & Van Nieuwerburgh, 2018), develop checklists to aid implementation and explanation, model or co-teach strategies (but do not run whole lessons as the strategy can be lost), focus on beliefs and how the teacher is thinking, and jointly plan next steps. It is also important for coaches to have role clarity, work alongside and support teachers, and become evaluators. When school leaders add coaching to their role, this diminishes the trusting relationship that is so important for successful coaching.

Microteaching/video review of lessons

Microteaching typically involves videoing teachers conducting (mini-) lessons and then engaging in post-discussions about the lessons. There is also a rich history of analyzing videos of lessons (Hiebert et al., 2003). For example, from a detailed analysis of videos, *Seidel and Shavelson (2007) found that outcomes were related to teacher goal clarity and coherence, students working together was related to motivational development, and students perceived themselves as being more self-determined and motivated in classrooms with high-quality classroom discourse.

*Morin et al. (2019) concluded that students benefit when their teachers engage in video analysis activities to improve professional practices. Regardless of the length of videos, number of intervention videos, primary evaluator, or dependent variable, the effects were higher with immediate rather than delayed feedback. Their model involved recording, reviewing, and revising, but they noted that much more than self-analysis is required. This is not self-reflecting on your own videos, as we see what we have seen, but working with knowledgeable other critics to provide alternative interpretations, hearing the teacher thinking aloud and considering alternatives, watching for unintended consequences, and planning where-to-next improvements in their impact on student learning.

Video analysis has larger effects for more specific aspects of teacher education preparation. *Metcalf (1995), for example, carried out a meta-analytic review of studies of on-campus clinical experience for teacher education and found that laboratory experiences produced moderate to strongly positive results for teacher affect, knowledge, and instructional behavior (d = 0.70). Such experiences included microteaching with analysis, reflective teaching, and videotaped role-play with debriefing. *Bennett (1987) reviewed the effects of various teaching methods within teacher education programs and reported higher effects for demonstration (1.65) and information (0.63) than for theory (0.15) on trainee teachers’ knowledge. He found a similar pattern for attitude outcomes (0.48, 0.15, −0.08, respectively) but the reverse for effects on skills: theory (d = 0.97), information (d = 0.35), and then demonstration (d = 0.26). The conclusion was that all components should be included – theory, demonstration, and practice – as well as feedback and coaching, preferably in a distributed rather than condensed manner across many sessions.

Teacher subject matter knowledge

There has been a long debate about the importance of teacher subject matter knowledge, with the seemingly obvious claim that teachers need to know their subjects to teach them (van Driel, 2021). Teaching, according to Shulman (1987), “begins with a teacher’s understanding of what is to be learned and how it is to be taught” (p. 7), and Ball and Forzani (2010, p. 41) added that content knowledge needs to be thought of as “explicit knowledge and skills beyond simple expertise.” A more specific depiction of teacher subject matter knowledge was developed by Ball et al. (2008) based on two main categories: subject matter knowledge and pedagogical content knowledge (Figure 9.2). In mathematics, for example, they argued that teacher knowledge related to mathematical reasoning, knowledge of students, and “perhaps even on a special kind of reasoning about students’ mathematical thinking.” Others have included knowledge and beliefs related to the topic, specific knowledge of subject assessment, instructional strategies, and knowledge and beliefs about the purpose and goals for teaching the content at particular grade levels (Magnusson et al., 1999).

Figure 9.2 Ball et al. model of subject matter knowledge and pedagogical content knowledge.

Despite the plausibility of this claim, there is not a large corpus of evidence to defend it. *Ahn and Choi (2004) found a very low correlation (r = 0.06) between teachers’ knowledge of mathematics and student outcomes. *Dupuis and Karl (2017) similarly found low correlations (r = 0.11) between teacher knowledge in math and student achievement; *Druva and Anderson (1983), r = 0.03 between teacher background in science and achievement; *Kelley and Camilli (2007), between teacher qualification in early childhood; and *Manning et al. (2017) reported r = 0.16 between teacher educational attainment and language reason, and r = 0.11 for language interactions. It is not that convincing that subject matter knowledge matters.

A related notion is out-of-field teaching. Out-of-field teaching is not a unidimensional construct. An English graduate teaching math is not the same as a physics graduate teaching math, a junior high English teacher assigned to year 1 reading, a biology major teaching chemistry, a physical education teacher teaching music – all assume there is no necessary commonality in quality of teaching or that the graduate is only versed in their major. (I learned the clarinet from my geography teacher.) Shah et al. (2020) estimated that 19% of math, 5% of science, and 17% of technology classes (in year 10 or above) in Australia are taught by those who did not have this subject as part of the teacher education or other professional qualification. These teachers are more likely in schools that report staff shortages, so the comparison of effects could be with not teaching the subject. Hoxby (2004) argued that there is a lack of evidence from carefully evaluated policy experiments showing that out-offield teaching has, in fact, an adverse effect on student outcomes. *Dunst et al. (2019), in their meta-analysis on teacher background influences, reported r = 0.02 between in-field certification and teaching practices, and −0.01 with classroom quality. Inand out-of-field tend not to differ on these dimensions.

So how do we reconcile the fundamental belief of many that knowing your subject matters when teaching and exciting students? One argument we have been exploring is to relate the presence of surface and deep knowing in classes and subject matter knowledge. Shulman (1987) claimed that content knowledge must go beyond the superficial to a deeper, analytical understanding. If the classroom is dominated by surface content knowledge, there seems little value if the teacher has or has no deeper conceptual understanding. Only if the classes have sufficient demand for deeper conceptual knowledge may teachers’ deeper subject knowledge matter. If, as identified in Chapter 3, classes, assignments, assessments, and feedback are dominated by surface content knowledge, it is less surprising that teacher subject matter knowledge does not matter. One example could be the millions of dollars spent on upskilling science teachers to find that little changes when they return to classrooms. It is highly unlikely that their classes would change if this upskilled knowledge is not accompanied by changing classroom practice to have a different proportion of surface to deep knowing. This argument is in line with the claim by Magnusson et al. (1999, p. 96), who noted that “teachers with differentiated and integrated knowledge will have a greater ability than those whose knowledge is limited and fragmented, to plan and enact lessons that help students develop deep and integrated understandings.”

Picker (2022) focused on the teaching of reading to grade 1 students. She developed a Reading Content Knowledge survey with five factors (phoneme blending, phonics rules, knowledge, morpheme counting, and phoneme manipulation). She then invited 17 teachers who were either −1 SD (the lows) or +1 SD (the highs) from the overall mean to provide example lessons that were transcribed and analyzed. The high content knowledge teachers more frequently taught all content-focused teaching practices (especially explanation and meta-language components). The lows were far less responsive to opportunities that could take the lesson and learning deeper (the teacher continued with the planned lesson despite the students’ understanding or progress). Highs were more involved, engaged in more scaffolding, and invited students to read and seek help from peers. In contrast, the lows were more imparting, had higher levels of telling students what to do and what they needed to know, and were more focused on completing the reading activities and asking closed questions. Picker concluded that the highs moved beyond learning the facts to a deeper level where thinking deeply, brainstorming, making connections, and justifying reasoning seemed to be a regular part of classroom dialogue. At times, the teacher scaffolded class discussions by repeating, extending, or expanding comments that affirmed and reinforced the point of the learning, and questioning and feedback were used by both the teacher and children for clarification and extension of understanding. The lows were too focused on completing the lesson as designed, missed opportunities to make children’s thinking visible and scaffold it to a deeper level, had shorter discussions, and used questions more to stimulate correct responses based on factual information.

We need to understand why the overall effect of content knowledge is low, not simply dismiss the importance of content knowledge. Picker’s study supports the claim that when there is a balance of surface (knowing-that) and deep experiences (knowing-how) in the classes, the impact on student learning is maximized. But when there is a high proportion of surface learning in the classroom, then it probably does not matter the degree of teacher content knowledge.

Conclusions

The most critical teacher influences relate to their having high expectations for all students, not labeling students, seeing differentiation as allowing for different paths and different times for all to attain the success criteria, being seen by students as highly credible influences on their learning and achievement, having high levels of clarity such that student all know what they are learning and why they are learning, and knowing their progress in their learning.

The greatest influences are when teachers work together to evaluate their beliefs and evidence of impact, seek critique and alternative explanations of their impact, and use microteaching to collectively review this impact on students.

A great teacher is defined by their impact on all students, engaging in both knowing-that and knowing-how/with notions in lessons, and ensuring and knowing that all are gaining more than a year’s growth for a year’s input. We know quality teachers more by how they think and seek critique about their impact on students than by what they do and know.

The low quantity and quality of evidence of teacher training should be a major embarrassment for these institutions who constantly ask for more – more years, more resources, more influence. Instead, they need to provide evidence that their graduates are classroom-ready.

It is difficult to find evidence that subject matter knowledge is important. This is a conundrum.

There needs to be alignment between the higher levels of content knowledge and the proportion of surface and deep learning in the class lessons and assignments. It is likely that where surface dominates the teaching and assignments, then subject matter knowledge matters less.

How to invoke high expectations seems critical, and this may require more in-school discussion of appropriate benchmarks across grades and agreement on evidence of performance before starting the class or year.

The greatest single issue facing the further enhancement of students is the need for teachers to have a common conception of progress and standards. When a student moves from one teacher to another, there is no guarantee that they will experience increasingly challenging tasks, have a teacher with high expectations of how to progress up the curricula, work with a teacher who will grow the student from where they are as opposed to where the teacher believes they should be at the start of the year, and experience a teacher with high levels of evaluative thinking.

The most critical teacher influences relate to their having high expectations for all students, not labeling students, seeing differentiation as allowing for different paths and different times for all to attain the success criteria, being high in verbal ability (especially in their early years of teaching), being seen by students as highly credible influences on their learning and achievement, having high levels of clarity such that student all know what they are learning and why they are learning, and knowing their progress in their learning. Finally, the greatest influences are when teachers work together to evaluate their beliefs and evidence of impact, seek critique and alternative explanations of their impact, and use microteaching to collectively review this impact on students.

While there are no metas on the nature of thinking used by teachers, the claim is that there are particular ways of thinking that demarcate the more from lesser effective teachers. We call this evaluative thinking (Rickards et al., 2021). As Scriven (1991) noted, the core evaluative thinking skills are critical thinking, reasoning, and understanding others (how they think, reason, judge, interact). In education, this evaluative thinking is focused on maximizing our impact on the learning lives of the students, which thence includes making classrooms inviting places to learn, developing a climate of fairness, developing respect for self and others, and ensuring the focus is on improvement in learning and student character (see Hattie & Larsen, 2020). We identified five core evaluative thinking skills:

Evaluative thinking involves invoking reasoning and critical thinking in valuing evidence, leading to where-to-next recommendations.

Evaluative thinking involves addressing the fidelity of implementation, continually checking for unintended consequences, and allowing for adaptations to maximize the value of the outcomes.

The focus of evaluative thinking is specific to the major decisions each profession aims to make: for teachers, it is maximizing the impact on the learning lives of students.

Evaluative thinking involves investigating potential biases and confounding factors that may lead to false conclusions.

Evaluative thinking relates to understanding others’ points of view, leading to judgments of value or worth.

Thus, educators with high levels of evaluative thinking have greater pattern recognition, are more adept at checking biases and constraints, are more able to monitor their and others’ (e.g., their peers and students) judgments, and are more likely to seek alternative actions. As a result, they can be slower to come to problem representations and conclusions, try to see the world through other eyes, check back with the problem statement more regularly, and are focused on becoming a learning expert.

Every year, a student engages in a huge gamble as to who is at the front of their class – will it be a teacher who has a major positive influence or a teacher who has a less-than-average although positive influence? Will it be a teacher who engages in evaluative thinking or one who wants to cover the curriculum? The mantra is that every student should have a great teacher by design, not by chance. A great teacher is not defined by the teaching methods they use, their personality, their initial teacher education program, or necessarily their subject matter knowledge. Instead, a great teacher is determined by their impact on all students, engaging in both knowing-that and knowing-how notions in lessons, and ensuring and knowing that all are gaining more than a year’s growth for a year’s input. We know quality teachers more by how they think and seek critique about their impact on students than by what they do and know.

Teachers’ initial teacher training programs have little impact on how well teachers influence their students’ achievement. Maybe subsequent effects (e.g., the experience of the first classroom years) wash out this earlier training. The low quantity and quality of evidence of teacher training should be a major embarrassment for these institutions, who constantly ask for more years, more resources, and more influence. Teacher education may be more successful if it places more emphasis on learning and teaching strategies; developing teachers’ conceptions of teaching as an evidence-based profession (learning from errors as much as from successes); creating an appraisal system that involves a high level of trust and dependence on observed or videotaped reflection/evaluation of practice; and providing beginning teachers with a range of different teaching methods to use when current ones do not work. In addition, it may be more successful if it reintroduced microskills teaching methods, which have demonstrably positive effects on new teachers; developed teachers’ understanding of different ways to teach surface, deep, and conceptual knowledge; demonstrated how teachers could build positive relationships with all students; and showed how evaluation and assessment of students provide effective feedback to teachers about how well they are teaching, who they have not taught so well, and where they need to reteach. One way forward is to ask teacher education programs to articulate a set of graduating standards and then evaluate the nature and quality of evidence that all students meet these standards. Suppose employers and independent educationalists sit with the education program academics in making these decisions. In that case, there is a higher likelihood that these programs will improve to concentrate on training new teachers to affect students’ learning (see www.aitsl.edu.au/deliver-ite-programs/teaching-performance-assessment).

It is difficult to find evidence that subject matter knowledge is important. This is a conundrum. The claim is that those with high levels of subject matter knowledge are more optimally placed to understand the content, the likely errors and misconceptions, and the optimal progressions of surface and deep learning in that content. Such teachers are better placed to provide feedback as students struggle and help move them from their current understanding to deeper and more richly constructed views of the content. It would be expected that students are more likely to become passionate about and enjoy the subject as they master the content, as opposed to those students who learn in a minimax fashion to gain knowledge and facts to pass the test, complete the assignment, and move on to whatever is the next content prescribed for them. But the evidence is lacking for these claims. We may need to ask what minimum subject matter knowledge is needed to be an accomplished teacher and how we can optimize the teaching strategies of those teachers with greater subject matter knowledge. The hunch is that there needs to be alignment between the higher levels of content knowledge and the proportion of surface and deep learning. It is likely that where surface dominates the teaching methods, then subject matter knowledge matters little.

So it is teacher beliefs more than the methods or content that matter. It is less about what teachers do and more about how teachers think about what they do. Teachers walk into a classroom with conceptions of teaching, learning, curricula, assessment, and their students (Brown, 2011; Witter, 2021). We need to understand these conceptions better, as it seems they are powerful moderators on the success of these teachers. Low expectations of the students’ success are a self-fulfilling prophecy. It appears that expectations are less mediated by between-student attributes (gender, race, physical attractiveness) than by whether they are held (high or low) for all students. How to invoke high expectations seems critical, and this may require having more in-school discussion of appropriate benchmarks across grades and seeing evidence of performance before starting the class or year (Nuthall, 2007, shows half of all material taught in any class is already known by the students); so much of the early months of the year involves trial and error, as teachers find out proficiencies of students – information which could have more readily been garnered by reviewing student records and having a discussion with previous teachers.

The greatest single issue facing the further enhancement of students is the need for teachers to have common conceptions of progress and standards. Moderating standards and expectations across teachers (and schools) is essential to student growth. When a student moves from one teacher to another, there is no guarantee that they will experience increasingly challenging tasks, have a teacher with high expectations of how to progress up the curricula, work with a teacher who will grow the student from where they are as opposed to where the teacher believes they should be at the start of the year, and experience a teacher with high levels of evaluative thinking.

My colleague Russell Bishop (2019) moves around classes asking students, “Does your teacher like you?” and so many ethnic minority students (in New Zealand) say no, but the white students say yes. When teachers are shown the results of surveys, they are often astonished – primarily because they assumed that the relations were positive, looked for cues that all was well, and rarely saw the classroom through the eyes of the students. The powerful effect of Bishop’s work is that, after seeing these results, the teachers were quick to change their practices. Professional learning that questions assumptions and the teacher’s theories of practice can be very effective. The power of positive teacher-student relationships is critical for learning to occur. This relationship involves showing students that the teacher cares for their learning as a student and can “see their perspective, communicate it back to them so that they have valuable feedback to self-assess, feel safe, and learn to understand others and the content with the same interest and concern” (*Cornelius-White, 2007, p. 123). Then the powers of developing a warmer socioemotional climate in the classroom and fostering effort and, thus, engagement for all students are invoked. This requires teachers to enter the classroom with certain conceptions about progress, relationships, and students. It requires them to believe that their role is that of a change agent – that all students can learn and progress, that achievement for all is changeable and not fixed, and that demonstrating to all students that they care about their learning is both powerful and effective.

Reflections

Use the low-expectation/high-expectation table (Table 9.4) to review your current beliefs and practice and to discuss and decide which aspects need change. What is needed to enable teachers to make those changes?

Identify all the labels currently used in your school? Labels should be used only for diagnosis about specific learning issues as the first step for appropriate intervention and never attached to the child. Are your labels proxy for blaming the child and code for special needs, low SES, migrant backgrounds?

Do our students perceive us as caring, understanding, and responsive? What are the ways in which we demonstrate these characteristics? Are there teacher behaviors that show the opposite? After reading the final paragraph of this chapter, try getting a neutral outsider to ask Bishop’s question of your students: Does your teacher like you? And after each teacher has reviewed the findings, share and discuss the general implications.

After reading the section “Microteaching/video review of lessons,” ask volunteer teachers to set up a video capture to view either the teaching and/or the learning of a small group of students. Next, decide on focus questions (e.g., How many student questions? What is the percent of teacher talk? How much focus on knowing-that, knowing-with, and knowing-how?). Subsequent analysis with a trusted colleague is invaluable for teacher development.

Are your professional learning communities exemplars of collaborative culture or contrived collegiality? In these sessions, is there evidence of turn-taking, does the task invite multiple evaluative interpretations, and does the group come up with a better solution than if you had resolved the matter yourself?

Share experiences and strategies for students who aim to be silent, not engage with the teacher, and maybe are confused about the lesson.

Why do you think the effects of subject matter knowledge are so low? Work with colleagues to check these reasons in your class, ensuring the lesson deliberately covers both knowing-that and knowing-how/with. Analyze the tasks and assignments to check the cognitive complexity of the tasks asked of students and also the students’ completed work.

CHAPTER 10 The curriculum

DOI: 10.4324/9781003380542-10

In December 2019, I was in Alice Springs when the Ministers of Education from around Australia launched the Mparntwe Education Declaration, updating the goals of schooling. This was the first time a national government report was titled with an Aboriginal language name (the local Alice Springs newspaper duly noted this milestone but failed to include the word “Mparntwe” – clearly, more work needs to be done). The Mparntwe Declaration starts by asking educators to help students “realize their potential by providing skills they need to participate in the economy and society, and contributing to every aspect of their wellbeing.” The goal should be to work with students so they can exceed their beliefs about their potential; be decent human beings; learn, understand, critique, and appreciate precious knowing and understanding; and learn to create their and our future. It is about their lives now and less about preparing for work or the economy. Biesta (2020) has eloquently noted that Adolf Eichmann realized his potential: the schools in Germany at the time provided him the skills to successfully participate in the economy and society, and he looked after himself in every aspect of his well-being. The same cannot be said for Rosa Parks, who dared to say no to develop a moral purpose contrary to the norms of her society, to appreciate the we more than the I, and to use her power of no to refuse to participate in supporting aspects of governance and society of the time. We know which one contributed to society, the economy, and the well-being of the nation and world.

Like most curricula manifestos, Mparntwe argues for strong literacy and numeracy skills; the development of broad and deep knowledge across a range of curriculum areas, and the enhancement of the intellectual, physical, social, emotional, moral, spiritual, and aesthetic development and well-being of young people. Students are asked to deal with information abundance (like learning all in the typical overpacked curriculum); navigate questions of trust and authenticity; and have flexibility, resilience, creativity, and the ability and drive to keep learning throughout their lives. There is no comment about what parents and whānau (New Zealand: extended family or community of related families) are supposed to contribute, other than being partners in learning; and rarely do these lofty goals answer, “Why teach this rather than that?” with any specificity or clarity.

The curriculum is among the hottest contested areas in schools – as it should be. The contest “always entails values and commitments about what students should learn, reflecting choices and purposes about the kinds of outcomes we want to produce for young people” (Yates, 2017, p. 87). A fundamental premise in a democracy is that the nature of the curriculum is where schools meet the voters – and the ensuing debates and often wars are easily ignited. There are many major curriculum tensions, such as the sometimes competing commitment for both equity and excellence; the false dichotomy between precious knowledge and twenty-first-century skills; the fight between back to the basics and a deep and meaningful curricula; the notion that all curricula should be sprinkled over all students as if all of them need the same exposure and content; the sorting by age as if all same-age students are ready, want, and need to learn at that time; the lack of alignment between what the nation tests and what aspirations are claimed in the curricula; the never-ending adding to the curricula every time there is a social crisis and rarely deleting anything; the notions of college readiness to vocational or university destinations, given so many more students are staying till the end of high school; and the emphasis on building people and not just abstracted minds. As knowledge has expanded, the challenges about what to include become more contested. When will the curricula developers ever learn to say no?

The curriculum has to sit within a highly diverse cohort of students and deal with increasing demands on schools by parents, employers, politicians, and students. The new normal is the massive number of students’ diverse backgrounds in schools and the rapid increase of new knowledge, new domains, and new technologies. These have all moved the curriculum from what young people should know or learn to who they are intended to become and what they need to be able to do at the end of school. This is why “‘the ability to communicate,’ ‘to be flexible,’ be a ‘problem-solver,’ and be ‘a lifelong learner’ is seen in so many documents” (Yates et al., 2011, p. 430).

It is impossible to teach everything in the curricula – those who say no to teaching everything are the teachers, as they prioritize, ignore parts, and make judgments about which topics and skills relate to high accountability tests. It is hard to find evidence that changing curricula changes much (but see the section “Science curriculum” later), especially when the debate is phrased in terms of breadth vs. narrowness/focus, knowledge vs. dispositions. There is a need to reduce the breadth to allow for more depth.

The majority of the meta-analyses relate to reading, writing, mathematics, and science. Other curricula domains include spelling, drama, arts, music, coding, integrated curriculum, financial programs, motivation programs, moral judgment programs, outcome-based education, social skills, diversity training, creativity, chess, outdoor education, juvenile delinquency programs, play programs, bilingual programs, special college programs, and career education programs (Table 10.1).

Table 10.1 Summary statistics for teacher attributes and teacher education

Reading, writing, and the arts
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Comprehensive reading programs
22
821
495,289
2,970
0.57
0.09
5

Reading programs for specific groups
10
367
30,127
1,016
0.82
0.16
4

Conjoint reading programs
7
204
20,222
273
0.42
0.07
3

Music-based reading programs
6
149
5,747,987
313
0.38
0.07
4

Reading strategies
13
444
55,002
960
0.47
0.09
4

Phonics instruction
12
426
43,577
1,980
0.48
0.07
4

Phonological awareness
11
563
80,114
6,275
0.75
0.16
5

Sound or inference
5
81
7,634
436
0.43
0.08
3

Vocabulary programs
12
419
35,918
824
0.62
0.09
4

Visual perception programs
9
830
418,124
5,564
0.67
0.03
5

Repeated reading programs
3
72
6,627
164
0.80
0.08
2

Sentence-combining programs
2
35
3,221
40
0.13
0.09
1

Reading comprehension programs
11
630
94,405
2,160
0.50
0.08
5

Exposure to reading
16
606
424,119
1,785
0.50
0.15
5

Whole language approach
5
81
12,620
288
0.14
0.06
3

Reading Recovery
3
68
8,630
1,496
0.47
0.09
3

Writing programs
17
801
94,874
1,375
0.53
0.08
5

Spelling programs
3
163
15,826
230
0.53
0.05
3

Drama/arts programs
12
845
3,503,113
1,109
0.55
0.11
5

Music programs
4
84
5,793,132
84
0.33
0.09
3

Total/average
183
7,689
16,890,561
29,342
0.50
0.09
4

Mathematics and science
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Mathematics curricula and programs
15
832
353,322
2,513
0.38
0.12
5

Mathematics direct or guided
4
196
20,351
228
0.59
0.05
3

Mathematics problem-solving
5
155
33,938
433
0.98
0.09
4

Mathematics teaching specific skills
14
843
421,882
2,348
0.73
0.08
5

Mathematics programs for LD
8
231
21,023
355
0.60
0.27
3

Mathematics programs - other
2
49
8,960
64
0.42
0.03
2

Manipulative materials on math
9
368
40,286
683
0.38
0.09
4

Use of calculators
6
238
21,905
1,100
0.25
0.05
3

Coding
2
170
15,121
971
0.46
0.04
3

Science programs
21
1,234
472,945
2,892
0.50
0.08
5

STEM programs
10
422
186,265
452
0.90
0.08
4

Science lab programs
4
126
11,597
474
0.43
0.08
3

Science conceptual change programs
2
112
10,308
168
0.94
0.08
2

Total/average
102
4,976
1,617,903
12,681
0.58
0.08
3

Curricula for gifted students
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Enrichment programs
5
240
51,462
569
0.49
0.08
4

Acceleration programs
4
117
14,096
462
0.53
0.18
3

Total/average
9
357
65,558
1,031
0.51
0.13
3

Other curricula programs
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Foreign languages
1
99
14,128
17
0.85
0.00
2

Integrated curriculum programs
2
61
29,551
80
0.39
0.05
3

Financial programs
1
37
3,405
177
0.16
0.02
2

Motivation programs
2
91
42,097
109
0.45
0.02
3

Moral judgment programs
2
49
12,019
49
0.40
0.05
2

Character/positive programs
6
395
439,543
1,211
0.23
0.03
4

Outcomes based education
1
20
16,160
20
0.97
0.22
2

Tactile stimulation
1
19
505
14
0.36
0.14
1

Spatial skills/training
2
49
5,606
109
0.56
0.09
2

Social skills programs
16
799
1,064,848
2,539
0.40
0.05
5

Diversity courses
1
307
28,256
307
0.09
0.01
2

Creativity programs
19
1,235
147,683
1,952
0.51
0.11
5

Chess instruction
1
24
5,221
40
0.34
0.00
1

Outdoor/adventure programs
5
211
28,369
531
0.49
0.04
4

Juvenile delinquent programs
1
15
1,381
134
0.12
0.00
1

Play programs
6
137
12,000
151
0.52
0.00
3

Bilingual programs
13
515
48,730
1,552
0.40
0.07
5

Career interventions
3
143
166,238
243
0.34
0.05
3

Total/average
83
4,206
2,065,740
9,235
0.42
0.05
3

Curricula programs: reading

Reading is one of the most contested curricula areas, as many have made strong claims about the best way to teach reading (Castles et al., 2018). There is the perennial phonics vs. whole language/balanced literacy debate, although the evidence leads to the conclusion that when you want to teach a new set of skills to students, you should deliberately teach those skills. Those skills relate to the Big Five – phonics, phonemic awareness, fluency, vocabulary, and comprehension. The purpose of developing these is to turn students onto the joys of reading and enable them to read to learn. Among the highest predictor of later success in schooling is enjoyment of reading; reading opens the door to learning (Gutiérrez-de-Rozas et al., 2022).

Whatever the best methods, if students do not develop sufficient reading acumen by middle elementary school, they are handicapped from learning in other curricula. They need reading skills to take advantage of the positive Matthew effects from successfully learning to read. This phenomenon of Matthew effects comes from the parable of talents (Matthew 25:29) – such that those who score higher on some attribute gain relatively more than those who score lower – the rich get richer, and the poor stay poorer (Stanovich, 2000). Perhaps it is not surprising that better readers are more motivated to read more, as those with skills are more likely to enhance and use them. The key problem is that after teaching certain curricula skills and content in the first two years of school, many teachers move on to new skills and content (requiring the skills to read), leaving behind those who did not master the first set of skills – a virtuous cycle for the rich, a vicious cycle for the poor. In a meta-synthesis of the impact of Matthew effects, *Pfost et al. (2014) found much support for the fan-spread effect, which supports positive and negative Matthew effects in reading. These effects were most evident for the highly constrained skills (e.g., letter knowledge, phonics, concepts of print), and they occurred regardless of orthographic regularity across various languages. There should be no excuse for not successfully teaching the fundamentals of reading by age 8, even though it may entail Tier 2 and 3 programs (e.g., response to intervention, specialist small-group coaching, and one-to-one tutoring).

Shanahan (2017) appropriately cautioned reading teachers and researchers about merely looking up the effect size and their rankings in meta-analyses and not critically looking deeper. He recommended, and I recommend as well, that readers use Visible Learning as a resource, not as a definitive decider: that is, go further and read the original meta-analyses (and sometimes it will be worth even going back to some of the original studies to find out what was really happening in the studies). There are few simple answers, and as is argued in the “Teaching Strategies” (Chapter 13) chapter, excellent diagnosis is the beginning before choosing any intervention; then select the best intervention relative to the diagnosis, engage in quality implementation (attending to fidelity dosage and adaptations), and then evaluate your teaching relative to the diagnosis.

Since Visible Learning was published in 2009, the number of meta-analyses has markedly increased relative to reading (52 to 124), as has the number of studies (2,143 to 4,914), but the average effects for all and each subcategory have barely moved (from 0.46 to 0.49 overall). No surprise that the messages have also not changed, emphasizing the importance of teaching a set of learning strategies to construct meaning from text. Such active teaching of reading skills needs to be planned, deliberate, and explicit. Successful reading requires the development of decoding skills, the development of vocabulary and comprehension, and the learning of specific strategies and processes. Programs based on skills and strategies are successful, whereas others without such emphases have minimal effects on learning to read. Continuing to develop one’s proficiency in reading depends on acquiring these skills as well as learning to derive meaning and enjoyment from the skills of reading.

Comprehensive reading programs

Comprehensive programs typically include many aspects of reading instructional methods. Some investigate whole class or Tier 1 (nonremedial) reading instruction (much of Slavin’s reviews), and others cover the Big Five in a comprehensive way – phonemic awareness, phonics, fluency, vocabulary development, and reading comprehension. If teachers employ excellent diagnostic methods, one or a few of these Big Five are likely needed to then move to the other deeper comprehension-based methods, but for students who have fluency and listening skills (phonemic awareness and phonics), they probably need more vocabulary development and comprehension. The public spat between two camps (e.g., phonics vs. whole language) is of little use to students who have low levels of listening to sound or higher levels of fluency and comprehension.

*Slavin et al. (2009) reviewed many Tier 1 comprehensive approaches to teaching beginning reading (lasting at least 12 weeks, with an average of just over a year in implementation). Across the 36 programs they identified, involving 32,075 students, the average effect was 0.32 for decoding, 0.23 for comprehension, and 0.24 overall. They concluded that instructional process programs designed to change daily teaching practices have substantially greater research support than programs that focus on curriculum or technology alone. They found similar findings in parallel syntheses for upper-elementary reading programs and middle and high school reading programs (*Slavin et al., 2007).

Overall, the more successful programs included cooperative learning, daily instruction, and extensive teacher professional development; these were much more successful than focusing on innovative textbooks or instructional technology alone. “Programs focusing on professional development in structured activities that engage children in discussions about reading, giving them opportunities to help each other learn and use metacognitive skills, may promise to enhance reading achievement from the second grade onward” (*Slavin et al., 2009, p. 70). Cross-age (0.26) and same-age tutoring (0.26) also showed that structuring interaction among students on reading strategies was worthwhile, as did programs emphasizing strategy instruction (0.32), such as teaching skills, such as prediction, summarization, and self-evaluation (the essence of reciprocal teaching).

In particular, *Slavin et al. (2009) reported numerous studies showing positive achievement effects for their own program, Success for All. This comprehensive school reform program ensures reading success for children in high-poverty schools (*Slavin & Madden, 2011). It provides schools with a K–5 reading curriculum focusing on phonemic awareness, phonics, comprehension, and vocabulary development, beginning with phonics-controlled mini-books (e.g., decodable books) in K–1. It uses cooperative learning, one-on-one tutoring, frequent progress assessment, regrouping students according to current reading level and across grade lines, extensive professional development, and a full-time facilitator to help teachers effectively apply all program elements. In addition, a Solutions Team works with parents to help them support their children’s achievement and deal with issues, such as attendance and behavior problems.

Reading programs for struggling readers

Initial reading instruction of specific skills with struggling readers, students with learning difficulties, or ELL students seems to succeed. For example, *Scammacca et al. (2015) focused on struggling adolescent readers. They reinforced the overall message about teaching specific skills and recommended interventions at word and text levels, and knowledge of word meaning and concepts. The message from these studies is that the teaching of core knowing-that skills are essential before moving to the knowing-how skills (comprehension), but making both sets of teaching interesting, engaging in the richness of the consequences of reading (e.g., the storytelling), and iterating between immersing in reading and updating the attention to teaching the skills are the key.

Conjoint reading programs

These programs are less about teaching reading but influencing the focus on reading on other curricula content. The conjoint teaching of reading with other subjects typically produces much greater effect. For example, *Swanson et al. (2014) reported an effect of 1.02 from reading interventions delivered using social studies, with the largest effect on adolescent students. These interventions included graphic organizers, mnemonics, reading and answering questions, and using guided notes. *Graham et al. (2018) explored conjoint teaching of reading and writing with an effect size of 0.33 from balanced programs on reading and 0.37 on writing. In their earlier work, *Graham and Hebert (2011) found that teaching writing increased reading with a smaller but positive effect of 0.13, and *Graham and Santangelo (2014) reported an effect of 0.54 by using spelling instruction to raise word reading and comprehension.

More specifically, the effects on reading from summary writing were higher than answering questions (0.24) and note-taking (0.05); note-taking was higher when compared to answering questions (0.20); answering questions was little different from extended writing activities (0.01) (*Gillespie & Graham, 2014). These claims support shared knowledge theory, which says that reading and writing are not identical skills but draw on the same knowledge and cognitive systems (Shanahan, 2016). Both draw on general and meta-knowledge about written language; entail procedural knowledge about how to access information purposefully; involve setting goals, questioning, predicting, summarizing, visualizing, and analyzing; ask for knowledge of text attributes (text, words, syntax, and usage); and are based on communicative processes.

Reading: music-based reading programs

There has been much hype about the Mozart effect, whereby playing a Mozart sonata for ten minutes increased spatial reasoning, lowered blood pressure, and much more. This led to the endless playing of classical music to babies, and in one US state, the governor proposed to give every newborn a CD of classical music. The argument is that learning music impacts sound perception, increases working memory, and develops language and fluid intelligence (Moreno, 2009). However, there is little evidence from five meta-analyses to defend this notion. Indeed, the higher the quality of the research designs, the more the effect drops to zero. *Sala and Gobet (2018) concluded, “the findings refute all the theories claiming that music training causes improvements in any domain-general cognitive skill or academic achievement” (p. 15).

Reading strategies

This is more the other category, as these meta-analyses include grouping, databased decision-making, and methods that could invoke variable reading instruction methods. As noted earlier, the instructional strategies chosen by teachers make the difference, such as cooperative learning to help each other learn and use meta-cognitive skills, peer tutoring, daily instruction, structured activities that engage children in discussions about reading, and reciprocal teaching. Success seems augmented with excellent diagnosis, progress monitoring, and adjustments to teaching in light of collecting evidence about student progress in reading. *Filderman et al. (2019) investigated data monitoring systems involving frequent progress monitoring, administering diagnostic assessments with students who are not making adequate progress, and adapting instruction based on progress monitoring and diagnostic evaluation data. Most of the studies collected data once a week or more, but there were disappointing adjustments made to teaching in light of the evidence, and the overall effect was important but small.

Expository text helps build background knowledge necessary to understand content information, and often, this information is the basis for reading to learn various curricula domains. Meyer et al. (2014) identified five text structures, including description, compare/contrast, sequence, cause/effect, and problem/solution. *Hebert et al. (2016) found that instructing students about text structure improves expository reading comprehension, especially when multiple text structures were taught. It also involved writing activities, such as note-taking; sentence writing, such as answering short-answer questions; and writing paragraph-length responses to a text.

Phonics

If there is ever a war in reading, just mention the F-words. The meta-analysis published by the *National Reading Panel (2000) showed the power of phonics and phonemic awareness in learning to read. This has not changed across the 13 meta-analyses published since Visible Learning in 2009 for phonics across 500+ studies with d = 0.57, 8 meta-analyses for phonological awareness across 250+ studies with d = 0.75, and 6 meta-analyses for related sound or inference teaching across 200+ studies with d = 0.53.

As with most surface, content, and vocabulary teaching, intentional instruction methods have been most powerful in teaching phonics skills. For example, *Swanson (1999b) synthesized empirical evidence from research on reading intervention for students with learning disabilities and found that models that used Direct Instruction and taught students strategies for recognizing words improved their reading comprehension performance.

Phonological awareness

The National Reading Panel concluded that there are many tasks commonly used to assess and improve phonemic awareness, such as phoneme isolation (What is the first sound in paste?), identification (Which sound is the same in bike, boy, bell?), categorization (Which sound does not belong in bus, bun, rug?), blending (Which word is s/k/u/l?), segmentation (How many phonemes in ship?), and deletion (What word remains when s is removed from smile?). The panel argued that an essential part of learning to read involves the following:

learning the alphabetic systems, that is, letter-sound correspondences and spelling patterns, and learning how to apply this knowledge in their reading. Systematic phonics instruction is a way of teaching reading that stresses the acquisition of letter-sound correspondence and their use to read and spell words (Harris & Hodges, 1995). Phonics instruction is designed for beginners in the elementary grades and for children having difficulty learning to read.

(*National Reading Panel, 2000, pp. 2–89)

The National Reading Panel found an overall effect size of phonological awareness on phonological outcomes of d = 0.86, reading outcomes of d = 0.53, and spelling of d = 0.59. Teaching focused on one or two types of phonemic awareness led to larger effects than teaching many more. Teaching to manipulate phonemes using letters led to greater effects than teaching without letters; as did synthetic phonics programs that emphasized teaching students to convert letters into sounds, blend the sounds to form recognizable words, and analyze and blend larger subparts of words and phonemes. These phonics effects were present whether classroom teachers or computer formats were used, were higher for preschool than in higher grade levels (that is, more powerful in learning to read), and were more effective when delivered through tutoring, in small groups, or through teaching classes of students. These National Reading Panel findings were hotly contested because so few studies were used from the myriad of reading studies that were available. However, the findings from other meta-analyses on the teaching of reading are not dissimilar in conclusions from those of the *National Reading Panel.

For example, *Pan et al. (2019, d = 0.77), *Ruan et al. (2018, d = 0.87), *Fischer and Pfost (2015, d = 0.36) have all provided strong support for the importance of teaching phonological awareness (PA) as a major skill in learning to read. They reinforce the earlier claims by *Ehri et al. (2001), who only considered controlled experiments that included phonemic awareness instruction on learning to read. They concluded that the benefits of phonemic awareness instruction were replicated multiple times across experiments and, thus, provided solid support for the claim that PA instruction is more effective than alternative forms of instruction or no instruction in teaching PA and helping children acquire reading and spelling skills. *Swanson et al. (2003) also found that predictors, such as rapid naming, word attack skills, and letter identification, were highly related to reading – especially reading comprehension (d = 0.93) – and this finding was replicated more recently by *Araujo et al. (2015, d = 0.95). It should be no surprise that students who cannot read at age 8 lack these skills.

*García and Cain (2014) asked about the relative importance of decoding skills to reading comprehension during reading development. Word reading accuracy (r = 0.86) is a better predictor of reading comprehension than the speed of word reading (r = 0.65), and the next two important correlates are accuracy of pseudoword reading (0.56) and accuracy of word reading in context (r = 0.48). The correlations were higher for narrative than expository texts when silent reading was more than reading aloud. They noted that better decoders devote greater cognitive resources to the processes involved in constructing meaning from text because they have fast and efficient word recognition skills – and this was the case across all school ages. There is a virtuous circle in that those with better comprehension read more and, through this exposure to print, increase their decoding skills. These results confirm the relevance of instruction in both decoding and language comprehension in the classroom, even with adolescent readers. They concluded that “it is important to combine different measures of decoding and different materials and procedures to assess reading comprehension” (p. 100).

Overall, phonics and phonological awareness instruction are powerful in learning to read – both for reading skills and comprehension.

Sound or inference programs

Sound or inference programs are highly related notions to phonics programs, such as spelling to sound regularities, morphological teaching, rapid automatized naming, and inference and vocabulary instruction. Rapid naming refers to the time required for a child to quickly and accurately name an array of well-known visual stimuli (usually letters, digits, objects, or colors). *Araújo et al. (2015) argued that this process is one of the best and perhaps one of the most universal, longitudinal, and concurrent predictors of reading ability and a core deficit in reading disability. Using 126 studies, they reported high effects on word reading (r = 0.45), text reading (r = 0.45), non-word reading (r = 0.40), and reading comprehension (r = 0.39). They concluded that rapid naming performance “reflects, from very early on, underlying cognitive processes that are relevant for learning to read and, consequently, reflects the usefulness of these measures in predicting reading competence and its failure” (p. 881).

Vocabulary programs

Training in vocabulary acquisition seems to be a valuable part of a more comprehensive program to develop reading, comprehension, and especially oral language development. *Marulis and Neuman (2010) located 67 studies with an average effect of 0.88, leading them to argue that vocabulary training is an essential component of early reading development. This effect was high regardless of whether it was the experimenter (d = 0.96), teacher (d = 0.92), or parent (d = 0.76) who was undertaking the teaching, but was much lower for child care providers (d = 0.13).

The effect did not vary by group size (individual, 2–5, or 6+) and was more effective in short interventions (< 1 week), and even small high-intensity interventions showed vocabulary gains. This was the case particularly if there were explicit (detailed definitions and examples given before, during, or after a storybook reading) more than implicit (word embedded in an activity without intentionally stopping or deliberately teaching word meaning) methods. The teacher also needs attention to ensure transfer from vocabulary instruction to reading within texts. *Stahl and Fairbanks (1986) showed that vocabulary instruction and knowledge of word meanings help growth in reading comprehension of texts with the taught words (d = 0.97), and the impact then on transfer to other texts was lower (d = 0.30).

There can be large differences in outcome effects depending on whether the vocabulary taught in the program was used in the reading comprehension measure to evaluate the impact of the vocabulary teaching or if the outcome measure was more general (not intentionally aligned to the instruction). For example, *Stahl and Fairbanks (1986) found a 0.60 difference favoring experimenter-made and standardized achievement measures, and the differences tend to be more pronounced with vocabulary. They suggested that “this may indicate that vocabulary instruction generally facilitates growth in reading comprehension, both on measures containing and not containing taught words, possibly by increasing the student’s interest in learning new words” (p. 100).

The most effective vocabulary teaching methods explicitly provided defini tional and contextual information and provided students with more than one or two exposures to the words they were to learn. The mnemonic keyword method also positively affected the recall of definitions and sentence comprehension (see *Mastropieri & Scruggs, 1989). This method involves students first learning a concrete word that sounds like the target word and then creating an image linking the target word to its definition. *Klesius and Searls (1990) found similar high effect sizes for this keyword method and reported rapid fading effects, with the highest delayed post-test effect of only d = 0.19. *Fukkink and de Glopper (1998) also examined the effects of instruction in deriving word meaning from context and claimed that the derivation of word meaning was indeed amenable to instruction, with even relatively short instruction having positive effects.

Visual perception programs

Visual perception is the process of differentiating written letters and words, and is an important aspect of early reading. *Peng et al. (2016) used meta-analysis to distinguish between readers and those with reading difficulties, and visual perception was one of the major discriminators (d = −0.86), along with phonological awareness (−1.06), rapid naming (−1.17), orthographic knowledge (−1.03), and morphological awareness (−0.20). *Kavale and Forness (2000) found that both auditory and visual perception were important predictors of reading for both average students (d = 0.36) and students with learning or reading disabilities (d = 0.38). There was little difference in the accuracy in predicting reading proficiency for many of the auditory perceptual skills (auditory comprehension, d = 0.40; memory, d = 0.38; blending, d = 0.38; discrimination, d = 0.37) or visual perceptual skills (memory, d = 0.47; closure, d = 0.43; discrimination, d = 0.39; association, d = 0.38; motor integration, d = 0.36; spatial relation, d = 0.33), but it was lower for figure-ground discrimination (d = 0.25). It seems visual perception is one of a group of major deficits that, if remedied, could improve reading skills.

Repeated reading programs

Tasks that require high levels of automaticity are important for all high-performance skilled processes, such as music and sports – and reading. Repeated reading is one method of helping to improve the automaticity of reading skills. It involves rereading a short and meaningful passage until a satisfactory level of fluency is reached. These skills of automaticity in word recognition and decoding (the move from accurate to automatic word reading) need to be specifically assessed and taught, especially to learning disabled students. It is perhaps not surprising that such automaticity, or overlearning, is a major feature in many second- and third-chance reading programs (see later).

*Lee and Yoon (2017) found that reading a passage at least four times increased reading fluency more than two to three times, previewing by listening to a passage before repeated reading. Both strategies help model and enhance understanding of the text, thus reducing any negative emotion. They, as did *Therrien (2004), found further that the effects of repeated reading had marked positive effects on reading comprehension as well as reading fluency – although the effects on near transfer (immediate influence and comprehension) were greater (d = 0.76) than on far transfer (ability to fluently read or comprehend new passages), although the latter effect (d = 0.50) is still substantial.

Sentence-combining programs

Sentence combining is an instructional strategy to improve reading that requires students to combine one or more sentences into one compound, complex, or compound-complex sentence (see *Fusaro, 1993, p. 228). The effects are small. *Neville and Searls (1991) found that sentence-combining as an instructional technique was much more effective at elementary than at high school levels, whereas *Fusaro (1993) found that the effects of sentence-combining on reading comprehension were ambiguous across all levels. Overall, the various reading meta-analyses, sentence-combining methods do not seem to have high value in the tool kit of reading instruction (0.15).

Reading comprehension programs

Again, the intentional teaching of strategies led to the greatest impact on reading comprehension. Many of the needed skills may need to be explicitly taught: *Suggate (2014) argued that the evidence from his synthesis indicated that there might be an ordering of interventions such that the earlier programs should target phonemic awareness alone, leaving decoding skills to grades 1 and 2, then to reading comprehension interventions.

*Rowe (1985) found programs with a dominant focus on processing strategies (e.g., inferential reasoning, rules for summarizing, and chunking texts) produced a higher effect (d = 1.04) than did text programs (e.g., repetition of concepts, explicitness, d = 0.77) and task programs (d = 0.69). Programs that empathized with teaching visually dependent strategies (0.94) and those that depended on auditory or language strategies (1.18) were successful (*Sencibaugh, 2005). *Pyle et al. (2017) reviewed the evidence for teaching students to use the structure of the text to help organize their memory for text-based content. These structural components include the organization of ideas, the relationship among the ideas, and the vocabulary used to convey meaning to the reader. The effects were highest for graphic organizers (1.71), then comprehension questions (0.93), and teaching recall and summarization (0.86). As important, the effects on student comprehension were higher when students were taught text structures using a gradual release approach when there were increasingly complex expository texts. Students improved their use of the text structure strategy when students were explicitly taught clue words associated with text structures and drew attention to the organization of the test to identify key ideas. When students worked with the same text structures embedded in a more complex text, and when current learning was related to prior learning (such as using texts that contain familiar topics so students have sufficient background knowledge and can focus on the text structure strategy).

*Guthrie et al. (2007) reviewed a specific program to enhance reading comprehension – the concept-oriented reading program. This program works because the engaged reader is internally motivated to read. Hence, the aim is to engage students and attend to motivational and reading instruction strategies. The engaged reader is cognitively active in using strategies and seeking to link old to new information; is behaviorally active in task participation, effort, and persistence in the face of difficulty; and is frequently reading for pleasure and learning. The concept-oriented reading program is a 12-week intervention, and each lesson has various segments: oral reading, a mini-lesson on comprehension strategies (inference, asking questions during reading, summarizing, and comprehension monitoring), independent writing and reading, and teacher-guided reading in small groups (for modeling, scaffolding, and guided practice). The effects are positive on multiple text comprehension (d = 0.93), fluency (d = 0.73), and story comprehension (d = 0.65), as well as on motivation outcomes (curiosity, d = 0.47; willingness to engage in challenge, d = 0.31; task orientation, d = 0.28; self-efficacy, d = 0.49).

Exposure to reading

More exposure or time reading does not necessarily lead to greater reading skills. *Lewis and Samuels (2003) found an effect of only 0.10, although the effect was somewhat higher for students with low reading achievement. They recommended that up to 30 minutes a day; and it is regular reading for shorter periods that is more effective than longer times on a less frequent basis.

There are now multiple meta-analyses on shared reading, mainly by teachers sharing interactions with the students about the text. *Blok (1999) found an overall effect of d = 0.63 for effects on oral language and d = 0.41 for effects on reading. The effects were higher with younger students and when the groups were small, as this fostered more interactions. *Stephenson (2011) found that shared book reading is more related to receptive language than expressive language, letter knowledge and print concepts than listening comprehension and phonological awareness, and word identification than decoding and reading comprehension. It seems learning reading skills are not so advanced, but listening, expressive language, and word identification improve with shared reading. The effects are higher for younger children, and the effects start to diminish as children become conventional readers and can read on their own (*Bus et al., 1995). The effects are similar for mono- and bilingual students (*Fitten et al., 2018) and students from all socioeconomic backgrounds (*Bus et al., 1995).

The effects depend on who is doing the reading – the parent to engage in the excitement of reading, the teacher who uses reading aloud as a teaching tool, or a volunteer who seems to have little or any effect. A meta-analysis on inter-generational transmission of literacy affirmed the positive effect of parent-child interactions in supporting children’s literacy orientation (*Bus et al., 1995). Parent-preschooler joint book reading experiences were shown to be related to positive outcomes in language growth (d = 0.67), emergent literacy (d = 0.58), and reading achievement (d = 0.55).

*Kim and Quinn (2013) reviewed summer vacation reading programs, showing little impact (0.10), although the effect was double for students from lower-income families. The effects were higher on reading comprehension than reading vocabulary. This suggests a low probability that a reader will learn a new word during normal reading (*Swanborn & de Glopper, 1999). This increased effect from low-income families reflects the low number of books in the home and parents less time reading and discussing books with their children. The increased presence of book programs has a major gap to fill – and has an effect.

Blatchford et al. (2012) have shown teaching assistants’ low to negative impact on many learning domains, including reading. *Torgerson et al. (2002) investigated the effects of unpaid classroom volunteers to provide extra support to children learning to read. Almost half the studies showed negative effects, and the overall relationship was small. They concluded that “there is little good evidence that the policy of encouraging volunteers to help teach children to read is effective” (p. 443). Blatchford et al. go further and say the effect of teaching assistants can be “toxic.”

Shared reading by teachers and parents can have positive effects, primarily on oral and receptive language, and reading skills, such as decoding, comprehension, and expressive language, may be best taught via other methods. Shared reading is more effective when spaced on shorter bursts over time than massed in longer sessions. More is not necessarily better unless there is specific instruction in teaching reading. Sustained silent reading, time on task alone, and nonprofessionals have much lower effects.

Whole language/balanced literacy approaches

The whole language approach to reading instruction is based on the following idea:

acquisition of reading skills depends on the context in which these skills are presented. Individual words are learned more easily and fluently when presented within a particular context. The words gather meaning from other words around them and the structure of the story.

(Gee, 1995, p. 5)

Gee found that such whole language approaches in reading instruction positively influenced reading achievement. He did qualify this finding by noting that studies with larger sample sizes produced smaller effects, and interventions shorter than one year did not provide the depth of instruction to produce measurable outcomes.

At first look, there is remarkable divergence in the overall effects from the four meta-analyses on whole language approaches. The average effect from the Gee study has an overall positive effect compared to *Jeynes and Littell (2000), which has an overall negative effect. There was much overlap in the articles used in these two meta-analyses. The difference is how the authors classified some key studies and the coding of what constituted the whole language. On the latter, Gee included programs with systematic phonics and word study as whole language programs and, thus, had what he considered a more “balanced” program of both sets of skills. For example, the study by Tunmer and Nesdale (1985) is classified as whole language, which is opposite from what Tunmer and Nesdale claimed for their program. Tunmer and Nesdale’s three whole language classes, according to Gee, included a “heavy emphasis on the teaching of phonological recoding skills” (p. 421). Trachtenburg and Ferruggia (1989), also classified by Gee as whole language, used various strategies, including word lists, letter-sound naming, and decoding techniques. Similarly, Gee classified Uhry and Shepherd’s (1993) study as whole language, and they used segmentation and spelling instruction, a 10–20 minute phonics lesson, and word copying. If these three studies were reclassified so that they were not included as whole language studies, then the Gee average would shift from d = 0.65 to zero – and the average across all whole language meta-analyses to d = 0.06.

*Stahl and Miller (1989) also had zero effects (d = 0.17 for word recognition and d = 0.09 for reading comprehension) from whole language programs. Whole language approaches may be more effective in kindergarten than in first grade, produce higher (but still close to zero) effects on measures of word recognition than reading comprehension measures, and are more effective when used instead of a reading readiness program. They compared the effects to basal readers, and there was a trend toward higher effects for basal reading programs. *Stahl et al. (1994) updated *Stahl and Miller’s (1989) meta-analysis on whole language instruction and again reported small effect sizes. They did add that whole language approaches were effective in improving children’s attitudes toward reading, and they concluded that there was a slight advantage for traditional approaches on measures of decoding.

The most recent meta-analysis found a substantial negative relationship between whole language interventions and basal readers and learning-to-read programs. *Jeynes and Littell (2000) investigated the effect of whole language instruction on the literacy of low socioeconomic status students from kindergarten through to grade level 3. They found that low socioeconomic status children receiving basal readers did consistently better than their counterparts receiving whole language instruction.

In summary, whole language programs have negligible effects on learning to read – be it on word recognition or comprehension. Such methods may be of value to later reading but certainly not for the processes of learning to read – it appears that strategies of reading need to be deliberately taught, especially to students struggling to read.

The earlier conclusion is a repetition of the 2009 text, as there has been no new meta-analysis on whole language. O’Connor (2020) critiqued my claims about whole language, arguing that I was explicitly hostile and biased against constructivist and progressive pedagogies and in favor of Direct Instruction and related conservative pedagogies. He argues that *Stahl and Miller (1989) conflated language-experience literacy approaches with whole language teaching and learning, and noted *Stahl et al. (1994) later published a meta-analysis excluding these debatable studies. The average effect size crept up from 0.09 to 0.15. He dismissed *Jeynes and Littell (2000) with an overall negative effect (−0.65) because Jeynes is a right-wing evangelical Christian who is biased toward phonics, and I should have noted this bias. O’Connor noted that I omitted three studies from *Gee’s (1995) meta-analysis because they were clearly not whole language interventions, and this reduced the overall effect from 0.65 to 0. I stand by the exclusions.

There have been advances in that whole language has morphed into a new name, balanced literacy (and there is one meta-analysis on one program; *Abrami et al., 2020; d = 0.20). Balanced literacy claims to incorporate elements of phonemic awareness and phonics, teaching vocabulary and fluency. It involves reading aloud, class shared readings, independent reading, shared writing, and leveled books, and all teaching (e.g., of phonics) is within the context of stories and not separately. The work of Pressley (1998) is often used to defend balanced literacy, yet he specifically included classroom instruction in “phonemic awareness, teaching of word recognition, vocabulary development, inculcation of comprehension strategies, prior knowledge development and instruction about how to use prior knowledge, and teaching of self monitoring” (Pressley et al., 2003). This explicit teaching sits alongside more holistic experiences “while flooding their classrooms with motivation.” It seems that whole language may not be as effective as phonics because it is very eclectic. As Gee’s (1995) interpretation of whole language indicates, it means many things to many people. This may be why its effects are weak: it tries to do too much. It tries to cover everything and may not cover anything very well in the process. In contrast, phonics approaches are much more specific and intensive, focusing mainly on word decoding skills. These have more impact in the crucial stages of learning to read, especially those first few years of school.

In a recent meta-analysis, *Abrami et al. (2020) reviewed the effects of a computer program ABRACADABRA (A Balanced Reading Approach for Children Always Designed to Achieve Best Results for All) that aims to balance code emphasis and a literature-rich context. The digital tool includes 33 alphabetic, fluency, comprehension, and writing activities linked to 20 interactive stories of various genres and 15 stories written by students. It uses a gaming method to motivate, provides (game) rewards when students meet milestones, and is a teacher and parent support process. Yet across 17 studies, the average effect was 0.20; including phonics (0.19), phonemic awareness (0.09), reading fluency (0.18) comprehension (0.18).

Reading Recovery

Reading Recovery (RR) is a second- or third-chance program implemented by specially trained teachers working individually with students in daily 30-minute lessons lasting 12–20 weeks (Clay, 2000). The meta-analysis evidence seems consistent – it makes a difference to these students. *D’Agostino and Murphy (2004), for example, found that Reading Recovery students outperformed control students, especially on scales in the observation schedule (a key part of the program). They concluded that Reading Recovery “was reaching its fundamental goal of increasing the lowest-performing first graders’ reading and writing skills to levels comparable with their classroom peers” (p. 35), and there was a “lasting effect, at least by the end of second grade, on broad reading skills.” *Elbaum et al. (2000) concluded that “well-designed, reliably implemented, one-to-one interventions can make a significant contribution to improved reading outcomes for many students whose poor reading skills place them at risk of academic failure” (p. 617). They also critically noted that the outcomes from Reading Recovery were similar for those studies that used small-group interventions, “while serving 3–4 times the number of student per instructor” (p. 615). *Slavin et al. (2011) also found similar effects from Reading Recovery studies and those using paraprofessional or volunteer tutors using structured and intensive programs (+0.24 vs. +0.23), despite the much more intensive and expensive training that RR teachers receive; and they concluded that emphasizing phonics greatly improves student literacy learning outcomes.

After a major review, Chapman et al. (2018) concluded that students who have less well-developed foundation skills required for successful literacy acquisition derive limited or no benefit from placement in the RR program. When designing the Reading Recovery program, Clay (2000) stated that it does not meet the needs of all struggling readers. She developed early diagnostic tools and quicker exit for those who did not show improvement during the program. These students need systematic and explicit instruction in phonological awareness and letter-sound relations, along with strategy training in how and when to use this knowledge while reading text and spelling words while writing messages.

Further, many students who enter these programs come from classes with little or no reading instruction to help them catch up – it is third chance compared to nothing. This is a reason for these programs to exist, as students are so often not explicitly taught the basic skills of reading in regular classrooms once they pass about grade 2, with teachers moving from learning to read to reading to learn (and finding alternative ways for nonreaders who lack basic skills to cope). By the end of elementary school, these accumulated adverse Matthew effects and lack of reading skills are not as surmountable by substitute activities and strategies. It would be wonderful to have a synthesis of alternative third-chance reading programs that might help these students.

The claim is that third-chance programs work, particularly programs with excellent diagnoses, and attend to the appropriate Big Five skills needing remediation. This is the key, not that Reading Recovery is the only such program (as other one-on-one or small-group instructional methods also seem to have positive effects). However, it is also clear that third-chance programs do not necessarily work long term for all students and that there may need to be a fourth chance.

Overall comments on reading meta-analyses

There is much meta-analysis support for the five pillars of good reading instruction: phonemic awareness, phonics, fluency, vocabulary, and comprehension – and attending to all is critical. These need to be taught explicitly as part of a daily routine using methods such as cooperative learning, peer tutoring, reciprocal teaching, and teaching text structures (including note-taking, and sentence and paragraph writing), differentiating written letters and words, as well as acquiring a more extensive vocabulary as a focus on teaching reading. Teaching reading conjointly with other subjects, such as science (*Kaldenberg et al., 2015) and social studies (*Swanson et al., 2014), and especially writing, helps build structured activities that engage children in discussions about reading.

When the National Reading Panel identified the Big Five instructional components, they did not elaborate on their inter-relations, nor how or when they impact the cognitive capacities underlying learning to read. Tunmer and Hoover (2019) have built a model based on the relations among the Big Five. They claim that effective beginning reading teachers and intervention specialists can answer two key questions about their practice: What am I doing, and why am I doing it? This requires three competencies: first, having a broad understanding of the cognitive capacities involved in learning to read; second, determining what beginning or struggling readers know and what they still need to know to become skilled readers; and third, providing their students with targeted, evidence-based instruction that directly addresses their individual literacy learning needs.

Hoover and Tunmer (2020) have developed the Cognitive Foundations Framework to help reading teachers better understand what their students face as they learn to read in alphabetic writing systems. Here’s an example:

if beginning readers are not progressing satisfactorily (i.e., at rates similar to their peers) in reading comprehension, is it because they are having problems recognizing printed words, problems understanding the language being read, or both? Weakness in word recognition skills may stem from a lack of automaticity or weak alphabetic coding skills, or the difficulty might lie with inadequate knowledge of the alphabetic principle, weak letter knowledge, weak phonemic awareness skills, or a failure to appreciate how print operates.

(p. 90)

Similarly, weaknesses in aspects of language comprehension can be detected. Such students may be as follows:

may have weaknesses in vocabulary or syntactic knowledge, or perhaps weaknesses in phonological knowledge that prevent them from hearing the differences between words with different meanings (e.g., thin and fin). Likewise, children having difficulty understanding stories and passages or making inferences may have weak background knowledge, difficulties in activating relevant knowledge, or weak understanding of the structures used in integrating meaning across sentence boundaries. Any of these circumstances could lead to specialized instructional activities that target the underlying cognitive skills.

(pp. 90–91)

The most effective programs for teaching reading are to attend first to the pre-reading skills of alphabet knowledge, phonemic awareness, and oral language. Then a combination of vocabulary, comprehension, and phonics instruction with repeated reading opportunities is the most powerful set of instructional methods. The least effective methods are whole language, sentence combining, and the assumption that students will learn vocabulary incidentally when reading. Perhaps the strongest claim is the need to explicitly teach skills relating to phonological recoding, especially phonemic and phonological awareness, as this finding has been among the most replicated findings in the teaching of reading. Students who struggle with reading typically do so because of lower skills in these areas, and learning how to decode is critical to comprehending text. A teacher who combines the power of phonics to develop a skill base of decoding with a focus on reading comprehension that focuses on comprehension strategies, vocabulary, and fluency development through reading practice will be much more effective than the teacher who concentrates only on phonics or whole language. Once students have these automated skills, they can devote greater cognitive resources to the processes involved in constructing meaning from text because they have fast and efficient word recognition skills – and this is the case across all school ages. Programs that assume children will naturally learn to read and that assume teachers will only interfere with this natural process if they try to teach skills, such as in an extreme whole language approach, which merely exposes students to books and more reading, are hardly likely to make an impact on learning to read.

Given the phenomenon of the Matthew effect on learning to read, with its positive and negative snowball effects on children’s reading futures, it seems critical for students to have sufficient acumen and fluency in reading by about age 8, then to move to reading to learn, enjoying reading, and maximizing comprehension from reading. Effective first-chance teaching will be important to ensure that all children succeed and none fall by the wayside. In the unlikely event that this happens, second- and third-chance programs will be critical for those left behind. Developing comprehension programs that focus on processing strategies (e.g., inferential reasoning, graphic organizers, shared reading, comprehension questions, recall and summarization, and chunking texts) will ensure that students move beyond decoding to the higher levels of reading sophistication that is characteristic of the true reader.

Writing programs

The number of meta-analyses on writing have doubled since 2009 to 16 (overall 0.48), and a lot of this renewed interest in writing is due to Graham and colleagues. In their first and largest study, *Graham and Perin (2007) recommended that it is most powerful to teach planning, revising, and editing strategies (0.82), particularly if the students are struggling writers. Effect sizes for various strategies were as follows: strategies for summarizing reading material (0.82); working together to plan, draft, revise, and edit (0.75); setting clear and specific goals for what students are to accomplish with their writing product (0.70); using word processing (0.55); and teaching students strategies to write increasingly complex sentences (0.50). The results show the power of teaching students the processes and strategies involved in writing, structuring the teaching of writing by having students work together in an organized fashion and setting clear and specific goals, especially the purpose of each piece of writing.

*Graham and Perin (2007) identified 12 elements of current writing instruction found to be effective for helping adolescent students learn to write well and to use writing as a tool for learning. *Koster et al. (2015) found four interventions had the greatest effects on the learning to write. Goal setting, strategy instruction (especially from the SRSD approach, see *Graham et al., 2013), text structure instruction (e.g., the explicit teaching of elements of text structure, in different types of texts: narrative, persuasive, and compare-contrast texts), and peer assistance (and grammar instruction yield negative average effect sizes). I have added the effect sizes for elementary and secondary students for each element. It is likely these same 12 also are most powerful for students with learning disabilities (cf., *Graham et al., 2013). *Gillespie and Graham (2014) found an overall effect of 0.74, and four processes were statistically different from zero (i.e., strategy instruction, ES = 1.09; dictation, ES = 0.55; goal setting, ES = 0.57; and process writing, ES = 0.43). Again, time and explicit instruction were needed (systematic instruction, teacher support, and scaffolding). *Graham and Perin (2007) identified 11 strategies for teaching writing:

Writing strategies (1.20), such as explicitly and systematically teaching steps necessary for planning, revising, and editing text, such as brainstorming, collaboration for peer revising, self-assessment, goal setting, self-regulated strategies, such as general and task-specific writing strategies, the background knowledge needed to use the strategies, and procedures (goal-setting, self-monitoring, self-instructions, and self-reinforcement) for regulating the strategies, the writing process, and writing behaviors.

Summarization (0.82), how to summarize texts, providing fading models of good summaries.

Collaborative writing (0.75), whereby learners work together to plan, draft, revise, and edit their compositions.

Specific product goals (0.70) include identifying the purpose of the writing (persuade, instruct, narrate, describe, explain, recount, creative), as well as being explicitly about the desirable characteristics of the final product (i.e., provide rubrics of success criteria).

Word processing (0.55), compared to paper and pen, particularly for low-achieving writers, as it allows the writer to add, delete, be legible, check spelling, and move the text easily, increases how much students write. *Morphy and Graham (2011) specifically investigated the use of word processing programs compared to pencil and paper methods, especially for weaker writers and readers. Their overall effect (d = 0.52) related to writing quality (0.52), length (0.48), development and organization of text (0.66), mechanical correctness (0.61), and particularly, motivation to write (1.42) and preference for word processing over writing by hand (0.64). Especially powerful writing quality effects were associated with word processing programs that provided text quality feedback or prompted planning, drafting, or revising (d = 1.46).

Sentence combining (0.50) involves teaching students to construct more complex and sophisticated sentences through exercises combining two or more basic sentences into a single sentence.

Prewriting (0.32) engages students in activities designed to help them generate or organize ideas for their composition – such as group and individual planning before writing, organizing prewriting ideas, prompting students to plan after providing a brief demonstration of how to do so, or assigning reading material pertinent to a topic and then encouraging students to plan their work.

Inquiry activities (0.32) to help develop ideas and content for a particular writing task by analyzing immediate, concrete data (comparing and contrasting cases or collecting and evaluating evidence) and explicitly teaching students how to be more creative or how to produce visual images (cf., *Hillocks, 1984, d = 0.56). *Atkinson (1993) also argued that these inquiry activities might be the result of the relationship with an “audience,” “either because of the presence of a real and immediate audience as in writing workshops or because of a need to collaborate with others to complete a task” (p. 105). The importance of including the purpose of writing in the lesson is thus emphasized.

Process writing approach (0.32), such as emphasizing writing for real audiences; encouraging cycles of planning, translating, and reviewing; stressing personal responsibility and ownership of writing projects; facilitating high levels of student interactions; developing supportive writing environments; encouraging self-reflection and evaluation; and offering personalized individual assistance, brief instructional lessons to meet student’s individual needs, and in some instances, more extended and systematic instruction.

Study of models (0.25) provides students with good models for each type of writing that is the focus of instruction to analyze these examples and emulate the critical elements, patterns, and forms embodied in the models in their own writing.

Feedback (0.80 for teacher and 0.37 for peer): teacher feedback on their progress learning a specific writing skill, peer feedback, and student self-assessment (teaching students to use rubrics). Students do not always welcome peer feedback (DeLuca et al., 2018). *Gersten and Baker (2001) noted that when feedback was combined with instruction on the writing process or text structure, a common vocabulary was created that gave teachers and students a meaningful way to engage in dialogue, which resulted in improved written products. The prompts helped give teachers or peers concrete suggestions for providing appropriate feedback.

The weakest relation was traditional grammar instruction (−0.41), and it is unlikely to help improve the quality of students’ writing, and indeed, too many effects were negative (*Graham et al., 2012).

Specific writing programs, such as the self-regulated strategy development model (1.17) and process approach to writing instruction (0.40), improved how well students wrote (*Graham et al., 2012). These programs involve explicitly teaching strategies for planning, drafting, or revising different types of text and teaching how to apply self-regulation procedures, such as goal setting and self-assessment, to help them manage the writing strategies they were taught, such as forming images and being more creative (0.70); structuring and forming different types of text (0.59); using the skills of spelling, handwriting, and keyboarding (0.55); planning, drafting, revising, and editing their papers (0.89); setting clear and specific goals (e.g., add three new ideas when revising) for what students are to accomplish when writing (0.76); previewing and organizing ideas for their papers before they write a first draft (0.54); and assessing students’ writing and progress in learning to write (0.42).

*Santangelo et al. (2016) specifically investigated self-regulation processes that, when taught, most improved writing. The three most powerful were cognitive strategies instruction (1.06), mental imagery (0.76), goal setting (0.51), prewriting (0.55), and self-evaluative standards (0.51). *Graham and Sandmell (2011) specifically investigated the process of writing or writing workshops approach. These programs typically involve students engaging in cycles of planning (setting goals, generating ideas, organizing ideas), translating (putting a writing plan into action), and reviewing (evaluating, editing, revising). They write for authentic purposes and audiences, taught with some of their writing projects occurring over an extended time. Students’ ownership of their writing is stressed, as is self-reflection and evaluation. Students work together collaboratively, and teachers create a supportive and nonthreatening writing environment; although the effects on motivation were low (0.19).

There is also much support for the reciprocal effects of teaching reading through writing, teaching writing through reading, and teaching writing about content areas (e.g., science, social studies, and mathematics) – and in all cases, the mutual benefits are substantial (Graham et al., 2020). This begs for more combined reading and writing interventions to benefit both, as the respective skills tend to complement; although the strong message is that each set of skills (reading and writing) needs to be explicitly taught (*Graham et al., 2018). Balanced reading and writing programs, for example, that focused on common sources of knowledge and cognitive processes, such as meaning-making, had an effect of 0.39 on reading (reading comprehension, 0.39; decoding, 0.53; reading vocabulary, 0.35) and 0.37 on writing (writing quality, 0.47; writing mechanics, 0.18; writing output, 0.69) (*Graham et al., 2018).

While computers seem to help motivate and improve writing, there are still many occasions for pen-and-paper writing. *Santangelo and Graham (2016) reported that teaching handwriting led to greater legibility (0.59) and fluency (0.63), quality (0.84), length (1.33), and fluency of students’ writing (0.48). Motor instruction did not produce better handwriting skills (0.10 for legibility and −0.07 for fluency). Ironically, teaching handwriting via technology (0.85) resulted in statistically significant improvements in legibility. Students who received 10 hours or more of handwriting instruction made greater legibility gains than those provided 8 hours or less. They noted that older students (grades 5 and up) made greater gains in fluency than younger students (grades 4 and below), suggesting that a certain level of competence in handwriting may be needed before fluency gains can be maximized.

Spelling programs

Misspelled words can make a text more difficult to read and devalue the quality of the writer’s message, and teachers give much lower scores (d = −0.38) for writing with misspellings even when the task is about the quality of the ideas (*Graham et al., 2011). In addition, lack of spelling skills can detract from motivation, willingness to express ideas, choice of words for ideas, and impact on working memory; and *Graham and Hebert (2010) showed that spelling instruction enhanced children’s word reading skills (0.62).

Spelling remains a mystery to many students, and there can be high linguistic demands not only to spell but to detect spelling errors. Devising a learning progression for learning to spell would include the prephonic phase of linking the sounds to symbols, the emergent phase where spelling is related to acoustic and articulations, the semiphonic or early letter name spelling, the pattern layering when learners come to understand that spelling does not always work in a strictly left-to-right fashion, the within-word pattern moving away from one letter = one sound, the syllable patterning, the morphemic phase recognition and use of phonemic spelling conventions, to the deviational constancy spelling when students are spelling almost all the words correctly in their spontaneous writing (Deacon & Sparks, 2015).

Some have argued that spelling can be “caught” (Westwood, 2018), and proficiency is best acquired incidentally through reading and writing by having teachers model and correct when they see misspelled words. Others have argued spelling can and should be deliberately taught using methods such as spelling specific words, teaching skills rules and strategies, and being exposed to high-frequency misspelled words. There is evidence for both approaches in that spelling skill does seem to relate to increased exposure to reading and writing (*Graham et al., 2018).

Graham et al. (2008) surveyed USA teachers about their instructional practices in spelling. Nearly all claimed to teach spelling, with just over half using a commercial program and over 90% requiring learning a weekly spelling list of words. The major spelling activities were praise for correct spelling (94%), teaching phonics skills for spelling (92%), instruction in phonological awareness (88%), miniles-sons to teach spelling skills and strategies (86%), teacher feedback on misspellings (84%), use of spelling games to teach skills and strategies (83%), spelling rule instruction (83%), encouragement of invented spellings (80%), teaching strategies for spelling unknown words (77%), conference with students about their spelling (77%), student use of spell checkers (76%), students helping each other with spelling (76%), student proofreading their and others’ compositions (71%), reteaching spelling skills and strategies (70%), reinforcement and motivational strategies to teach spelling (63%), and use of word sorting to teach spelling (55%).

All three spelling meta-analyses supported the claim that the highest impact derived from explicitly teaching spelling strategies or systematic study and word practice methods that include multiple practice opportunities and immediate corrective feedback after a word is misspelled (*Wanzek et al., 2006). *Graham and Santangelo (2014) showed that increasing the amount of spelling instruction received had a positive impact on spelling performance (0.70), formal spelling instruction was superior to spelling-is-caught approaches for learning to spell (0.43), and these formal programs improved spelling when writing (0.94), reading skills (0.44), and these effects were maintained over time (0.53). *Lin (2013) showed that systematic code-based literacy instruction is more effective at improving spelling outcomes than non-code-based or less systematic code-based instruction (0.58). The highest effects came from programs focused on phonemic segmentation, and writing out spellings for words had the highest effects – so it is the act of generating spelling and attending to the segmentation of phonemes in words that seem to play important roles in the effectiveness of interventions concerning spelling outcomes – and this was the case across all ages from young to high school.

Drama/arts/music programs

As an ex-music teacher, I have never believed that the teaching of music (or drama) needs to be justified because it improves learning in other areas – it is a worthwhile curricula topic in itself. However, this has not stopped the search for the impact of using music or drama in other domains. The claim is that drama- or music-based teaching can facilitate improvements in both nondrama content (history, math) and drama-specific content.

*Kardash and Wright (1987) found that creative drama activities positively affect children’s achievement at elementary grade levels in oral language skills, self-esteem, moral reasoning, role-taking abilities, and drama skills. *Conard (1992) found an effect size of 0.48 for studies in which creative drama was used as an instructional tool. Creative drama tended to be more effective at the preschool and elementary school levels than for older children. Both regular and remedial students appeared to benefit from and enjoy participating in creative drama. *Butzlaff (2000) argued that practice in reading music notation makes the reading of linguistic notation an easier task; this is, he claimed, because the skill of listening to music requires sensitivity to tonal distinctions, which can assist in acquiring sensitivity to phonological distinctions; reading the repetitive and, hence, predictability of lyrics of songs helps train reading skills, and working together in music groups instills a sense of personal responsibility, which in turns leads to heightened academic responsibility and performance. He was more cautious about the effect size between participation in music programs and achievement in reading, claiming that causality could not be determined from his meta-analysis.

Numerous theses and doctoral dissertations conducted in Turkey show the relationship between creative drama and students’ achievement. The effects of these studies are surprisingly much higher than in other countries. For example, *Günhan (2016) found a 0.99 effect of creative drama-based learning on mathematics success of elementary students, and *Erden et al. (2016) found an effect of 0.87 across many subjects.

*Hetland (2000) related the listening to music by college students and improvement in spatial-temporal reasoning (d = 0.49). Still, she was hard-pressed to find any importance of this finding to education and noted it was remarkably variable and, at best, temporary. *Vaughn (2000) found a medium relation between the voluntary study of music and mathematic achievement, but the effects of training in instrumental or vocal music performance were much lower. Playing music in the background while students are taking mathematic tests has only a small positive effect, perhaps an arousal effect, at best. *Winner and Cooper (2000) found a very small relation between studying arts and achieving (d = 0.10); the relation to verbal was higher (d = 0.39) than mathematics (d = 0.20). They were careful not to assume causality and suggested that studying the arts may lead to greater engagement in schooling, leading to greater academic achievement.

This begs for more synthesis on understanding common underlying skills or reasons for these links, more control over the flow from one subject to the other, and understanding of the dynamics of the teaching that leads to these effects. *Moga et al. (2000, p. 102) suggested the following:

we find some transfer when the bridge is narrow: from experience in the arts, which includes the visual arts, to performance on tests requiring drawing. However, we find no transfer when the bridge is wide: from experience in the arts to performance on tests requiring one to generate ideas, concepts, or words.

Mathematics curricula and programs

A significant theme from the 70 meta-analyses on mathematics is that the explicit and intentional high-impact teaching methods are once again dominant in terms of the most effective interventions. There is little support for readjusting the curriculum in terms of an impact, although it can make a difference to the scope and sequence of topics, and the major foci of intentional teaching. It seems surprising that there is not more on the focus of math interventions – whether math should begin with basic facts and move to problem-solving or the reverse. There are strong hints that more attention to the development phase of cognitive development may make big differences to the success of various interventions.

There are common themes when I review the last ten reviews of mathematics curricula across Australia. There are the usual pleas for more evidence-based lesson planning and interventions, more professional learning for teachers (Jensen et al., 2016), and sharing of best practices (Watt et al., 2017), and university entrance into many degrees should demand math courses (Wienk, 2017). Most importantly, the Stanley report (2008) argued for teaching maths “in ways that help them interpret information or solve practical problems and apply their knowledge.” The typical claim of learning the facts (they called it a rush to apparent proficiency) too often leads to poor problem-solving skills, and when students begin to encounter problems (nearer the end of primary school), there is a drop in interest, motivation, and the start of a withdraw from the study of mathematics. The UK Williams (2008) report recommended that upper elementary school and lower levels of high school should have a mathematics specialist with deep subject and pedagogical content knowledge to oversee the math program across the school, emphasizing math problems. These recommendations were implemented in the UK, and I note the turnaround in their PISA numeracy means, but this initiative was shelved in Australia.

Curricula reform in mathematics relates to changing content, textbooks, and professional development, and overall, the effect was 0.10. So often, these low effects were followed by defenses that implementation of curricula reforms may take some time to realize more positive effects, but it also may be that we need to question the fidelity of implementation. A more sober conclusion is that “despite all the heated debates about the content of mathematics, there is limited high-quality evidence supporting differential effects of different math curricula” (*Slavin & Lake, 2015, p. 445).

Many programs teach mathematics, with publishers and authors promoting book series, packages, and online programs. Slavin and colleagues published reports on programs in elementary (*Slavin & Lake, 2007) and high schools (*Slavin et al., 2008). They noted the marginal methodological quality, small sample sizes and number of classes, and overall low (0.10) effects. They concluded that their review does “not suggest differentially strong impacts on outcomes such as problem-solving or concepts and applications that one might expect” (p. 17). Examples of effects from the programs in their review included three National Science programs; Everyday Mathematics; Math Trailblazers; and Investigations in Number, Data, and Space (d = 0.10); Everyday Mathematics (d = 0.12, 0.35, 0.26); Math Trailblazers (0.06); Saxon math (0.02); Scott Foresman-Addison Wesley (0.07, −0.06); Houghton Mifflin Mathematics (0.14); Growing with Mathematics (0.20); Excel Math (0.13); MathSteps (0.03); and Knowing Mathematics (0.10). Similarly, in high schools, the overall effect was 0.03 (Slavin et al., 2008). It also seems that the choice of textbook the schools choose hardly matters (0.10). It is less the programs or textbooks, they claimed, and more the daily teaching practices and student interactions that have larger impacts on achievement.

*Lloyd (2013) claimed that four math processes are crucial in developing math proficiency: acquisition (0.48), fluency (0.71), generalization (0.34), and application (not reported). He argued that math fluency interventions were more effective than math acquisition interventions in improving basic math skills, but more focus on problem-solving generalization interventions was found to have a greater effect on word problem-solving skills than math fluency and math acquisition interventions.

Mathematics instructional processes

Programs designed to change daily teaching practices appear to have more promise than those dealing primarily with curriculum or technology alone (*Slavin et al., 2009). Across many of these reviews, shorter interventions were more effective than longer interventions (*Jacobse & Harskamp, 2011; 0.62 vs. 46). For example, *Sowell (1989) found that interventions of fewer than 14 days were more effective than longer programs, and *Codding et al. (2011) found that shorter bursts were more effective than longer intervention programs in learning basic math ideas.

The debate between minimally guided and direct approaches is alive in the teaching of math – with mixed evidence probably reflecting the variability in what is meant (particularly by Direct Instruction), sometimes small sample size, and the fidelity of implementation. *Malofeeva (2005) found that a combination of direct and guided instruction (0.48) was the most beneficial instructional approach, followed by guided instruction (0.43; and direct, 0.15). But two meta-analyses showed no differences. *Jacobse and Harskamp (2011) reported no difference between Direct Instruction methods (0.58) and indirect (guided) instruction methods (0.61), nor among whole class (0.51), small groups (0.63), and individual interventions (0.60). *Xie et al. (2018) investigated the effects of constructivist (0.55) and transmission methods (teacher-centered teaching and learning model in which the teacher’s role is to design lessons aimed at predetermined goals and to present knowledge and skills in a predetermined order, and students’ tasks are to acquire teacher-specified knowledge and skills, 0.63, passively) – hardly a difference.

The effect of these methods may likely differ depending on the cognitive complexity of what is being taught (see Chapter 11), but this is rarely investigated in the meta-analyses. *Kaplan et al. (2015), for example, reviewed realistic mathematics programs that start with real-life problems, teach students to use models and discovery, and then work backward to teach skills (they claim using didactic methods). They contrasted this with teaching abstract principles and rules first, then moving the problems. The overall effect is 0.61, but there is too little detail about the methods, few moderators, and a small sample of ten Turkish theses. It does hint, however, that more investigation of this ordering of teaching may be worthwhile.

When math is taught as algorithms and facts, students struggle when math ideas are couched into words. Both meta-analyses on this topic are with LD students, and both show the importance of teaching the skills of decoding these words and numbers to then deal with the math ideas. *Lein (2016) investigated word problem-solving instruction, and the effect was 1.03 (but there were two outliers, and when removed, the effect was 0.77). She found the effects were largest for the whole class (1.60) then small group (0.70), and least for individual instruction (0.34). The key components of effective word problem instruction for students were extensive modeling, think-aloud procedures, scaffolded instruction, review with corrective feedback, use of visual and verbal representations, and identification of specific problem types that share underlying common problem features. *Zhang and Xin (2012) had a similar high overall effect of 0.89 from teaching skills in word problems and reported interventions implemented in special education settings displayed a smaller effect size than those implemented in inclusive classrooms. Integrating writing into mathematics classrooms may be a useful instructional strategy to increase students’ mathematics problem-solving success (0.42) (*Bicer et al., 2017).

There is support for the claim that numeracy and number sense are more effective than math fact fluency in developing numerical competence. *Cason et al. (2019) found an effect of 0.88 for the effects of numerical competency development on mathematics achievement. They concluded that “numeracy and number sense are more effective than math fact fluency concerning developing numerical competence” in the earlier years, but “the effect sizes are reduced drastically after second grade.” The effect sizes for numerical competence development are approximately twice as large in grades EC-2 compared to grades 3–5 and middle grades. Again, more attention to the cognitive development phases of students seems critical before generalizing from these math influences. There is likely a Matthew effect at work in math, although there is far less focus on this phenomenon in mathematics than reading.

In so many reviews, the use of intentional or deliberate teaching wins out repeatedly, and there are many forms of these methods. The effects are higher when students are provided with clear success criteria (the modeling) and feedback relative to these notions of success. For example, *Gertson et al. (2009) were more specific about the features of explicit instruction. Such instruction involved the teacher demonstrating a step-by-step plan (strategy) for solving the problem; the lesson plan was problem-specific and not a generic, heuristic guide for solving problems; and students were actively encouraged to use the same procedures demonstrated by the teacher. Such teaching led to an effect of 1.22, and their multiple regression analysis showed that explicit instruction consistently contributed to the magnitude of effects regardless of whether it was paired with other instructional components. *Codding et al. (2011) focused on basic math fluency, and there were higher effects for drilling than for practice with modeling, and lowest for practice without modeling. Similarly, there are high effects for cooperative learning (0.67), problem-based learning (0.58), inquiry-based learning (0.52), variation teaching (0. 49; i.e., the teacher shows many examples of the concept, changes problem situations from simple to complicated), script-based learning (0.47), and autonomous learning (0.30). Similarly, *Chodura et al. (2015) found support for the more intentional strategies, as did *Lee (2000) – strategy-based methods (0.85), guided practice (0.86), peer tutoring (0.76), teacher modeling (0.73), use of specific forms of feedback (0.62), use of mastery criteria (0.63), sequencing examples (0.58), and change in instruction based on feedback (0.42). The least effective were working within peer groups (0.15), using technology for independent practice (0.16), using alternative representation modes (0.26), and identifying and teaching relevant preskills (0.28). Again, it may be more valuable to see the effects of these different methods depending on where the student is working in the instructional cycle – from acquiring knowledge to problem-solving with this knowledge.

Cognitive model-based interventions, such as explicit instruction on problem-solving strategies (0.68), concrete-representational-abstract sequencing using hands-on manipulatives, and solving (0.52) at both elementary (0.62) and high school (0.64) (*Hughes et al., 2014). Similarly, *Ennis and Losinski (2019) reviewed the teaching fractions, and the effects were highest for strategy instruction (1.48), graduated instruction (1.60), explicit instruction (1.25), and anchored instruction (i.e., framing of mathematical problems within relevant and practical contexts to help facilitate problem-solving and computation skills, 0.35). *Moin (1986) compared four techniques for teaching calculus: self-paced (0.54), formative evaluation (0.29), and computer-assisted instruction (0.23). *Haas (2005) investigated teaching high school algebra, with the greatest effects were from Direct Instruction (d = 0.55) and problem-solving (d = 0.52), and the lowest effects were from technology-aided (d = 0.07) and communication and study skills methods (d = 0.07). There were medium effects from cooperative learning (d = 0.34) and manipulative, models, and multiple representations (d = 0.38). Haas concluded that the higher effects of Direct Instruction were because of its focus on desired learning outcomes, decisions about pacing and curriculum emphasis, and the emphasis on seeking enhanced learning for all students.

*Baker et al. (2002) highlighted the power of feedback to students learning mathematics. They found that the highest effects accrued when teachers provided feedback data or recommendations to students (d = 0.71), then for peer-assisted learning (d = 0.62), explicit teacher-led instruction (d = 0.65), and Direct Instruction (d = 0.65). The lowest effects occurred when teachers emphasized real-world applications of mathematics (d = −0.04). One consistent finding was that providing teachers and students with specific information on how each student performed seemed to enhance mathematics achievement consistently. Similarly, the power of peers seems worth considering. *Dennis et al. (2016) found a higher effect for peer-assisted learning (0.82) and teacher-led explicit instruction (0.76), lower when using computer programs or tablets (0.39), and lowest when using paraprofessionals (0.11).

Teaching aids: manipulative and calculators

The major interests in the mathematics meta-analyses have related to aids, such as calculators, manipulative materials, and graphing aids. There are three major themes in these studies. First, the effects of many of the interventions are greatest with the lower-compared to higher-ability students; second, when aids are provided to reduce cognitive load (such as using calculators to lessen the load of calculation in problem-solving); and third, where there are high levels of quality feedback from teachers to students and students to teachers.

*Carbonneau et al. (2013) favored manipulatives when compared with instruction that only used abstract math symbols, but both were more effective when there were high, not minimal, levels of instructional guidance (0.46 vs. 0.29). The effects were higher for fractions (d = 0.69) than for arithmetic (d = 0.27) and algebra (d = 0.21). More critically, the effects were greatest for students in the concrete operational (0.45) compared to those in the formal operations phase of thinking (0.16). Similarly, *Sowell (1989), *Holmes (2013), and *Parham (1983) found that the use of manipulative materials in mathematics had the highest effects on the concrete rather than the more abstract instructional components (−0.09). Even with children in the concrete phase, many seem to struggle with the concept that an object can stand for the item while simultaneously representing a larger mathematical concept and can be deviated by unnecessary detail.

Calculators constitute a significant form of manipulable aids, and they have been subject to much debate, although with the improvement and ubiquitousness of technology, this question seems to be fading. With one exception, the meta-analyses show a low but positive (about d = 0.20) effect from the presence of calculators in mathematics. The key findings supporting the use of calculators seem to be (a) when they are used for computation, drill, and practice work, and for checking work; (b) when they reduce the cognitive load on students so they can attend to other, more mathematical, concepts; and (c) when used for a pedagogical purpose in which they are to be an important element in the teaching and learning process.

*Ellington (2000) found that the effects of calculators were greatest for lower-ability students (d = 0.30), most variable for average ability students (d = 0.20), and negative for high-ability students (d = −0.23). The argument was that calculators could assist in reducing the cognitive load for the lower-ability students, whereas the higher-ability students are less constrained by the additional requirements of knowing the computational aspects when learning mathematics. Further, Ellington found that the effects were much higher when calculators were involved in the teaching process; for example, when used for composition problem-solving, the effects were d = 0.72: “When compared with students who did not use calculators, students in treatment groups were able to solve more problems and make better decisions concerning selecting methods for generating solutions” (*Ellington, 2000, p. 169). *Ellington (2006) specifically investigated graphing calculators, and the overall effects were quite low, although the effects were larger for conceptual (d = 0.72) compared with procedural (d = 0.52) skills. When students were allowed to use these calculators as part of instruction but tested without them, the effects were negative (d = −0.21). The conclusion was that the calculator is neither a help nor a hindrance to students’ overall mathematics achievement; at minimum, their use helps reduce cognitive load and enhance students’ attitudes toward the study of mathematics. *Smith (1996) also reported a positive effect of calculators on attitude (d = 0.37) but lower effects on computational skills (d = 0.21), concept development (d = 0.19), problem-solving (d = 0.15) and graphing skills (d = −0.05). He found greater effects for calculators in high school compared to elementary school students – except for the use of graphing calculators in high schools. *Nikolaou (2001) found a higher effect on problem-solving skills (d = 0.49, particularly in pre-algebra classes). He found no differences in the effects relating to socioeconomic status, gender, grade level, student ability level, student ethnicity, or student calculator expertise.

*Hembree and Dessart (1986) found that the pedagogical use of calculators improved students’ basic skills in completing exercises and problem-solving. Across all grades (and particularly above grade 5, when calculators become more prevalent) and across all ability levels, students using calculators led to greater effects on students’ basic skills in operations, particularly in problem-solving. The effects on problem-solving seem to relate to improved computation and lower cognitive workload demands. They also found a better attitude toward mathematics and an especially higher self-concept for those using calculators than those not using calculators. They suggested that this enhancement in attitude was probably because the use of calculators helped relieve students’ traditional dislike of problems expressed in words (by reducing the cognitive load of having to compute and problem-solve).

Coding

In 1969, I needed to learn Fortran to undertake some multivariate analyses for my master’s thesis. I went to the computer center to enroll, but they claimed, “We do not teach that language. We teach the language of the future.” So my first computer language was APL, which has now completely gone, although remnants of Fortran remain. There is a discipline in coding, similar to algebra, and a beauty and joy in the efficiency of coding (that I still enjoy, having recently become an R convert). But some claim that teaching coding is not worthwhile as coding will soon be redundant, and at best, students can code second best to many current computer-generated coding algorithms. Nevertheless, the recent research on coding is fascinating, as it is a case study of the power (or not) or transfer – does the problem-solving and skills of programming transfer to other tasks?

*Liao and Bright (1991) found an effect of 0.41 for transfer from coding; and *Liao (1999), in an updated synthesis, reported 0.76 with the greatest effects for near transfer (2.48) and lowest for far transfer (e.g., creative thinking situations, 0.13). *Scherer et al. (2019) also found a similar overall effect of 0.49, with higher effects in elementary (0.56) than high school (0.36) or college (0.35). The effects were high for near transfer (0.75) compared to far transfer (0.47); and the cognitive skills most impacted were creative thinking (0.73), math skills (0.57), meta-cognition (0.44), and reasoning (0.37); and lowest for school achievement (0.28) and literacy (0.02). They argued that the cognitive skills in coding are also relevant to other skills, but in areas requiring specific knowledge and understanding (e.g., literacy), there is unlikely to be much transfer from learning to code.

Concluding comments on math curricula and teaching

Overall, the presence of feedback, Direct Instruction, strategy-based methods, peer cooperative learning, high levels of challenge, and mastery have much effect on mathematics learning. That is, intentional teaching makes the difference when teaching mathematics, provided teachers consider the cognitive complexity of the material and how it is aligned to challenge students given their phase of cognitive development. There are many hints that teaching math as problem-solving from the earliest age and then ensuring appropriate skills to complete these problems is more powerful than teaching basic facts and then later introducing problem-solving. In dealing with problems, it is worthwhile to use manipulative materials, calculators, and other resources to help reduce students’ cognitive load and allow them to devote their energies and attention to problem-solving. It is the deliberate and intentional, compared to minimal guidance, that matters, and this is so both for students with LD and non-LD. Much more is needed to see if these are differential teaching methods for specific math skills compared to math problem-solving or for different levels of cognitive complexity.

Science curriculum

No new science curricula meta-analyses have been comparing various science curricula since 2009. The earlier metas compared the traditional curricula (which emphasized knowledge of scientific facts, laws, theories, and applications) with more skills-based, analytic, and process curricula. The latter had higher average effects (*Kyle, 1982; *Shymansky, 1984; *Shymansky et al., 1983). The more recent studies typically confirm this move from textbook approaches but couch the work more in terms of debates about STEM. Most writers, however, actually mean math and science, as engineering is rarely taught in high schools, and technology often relates to computing technology (and there is a convincing argument that technology crosses many domains, including the social sciences and humanities).

*Becker and Park (2011) argued that students exposed to integrative approaches demonstrated greater achievement in STEM subjects, more so for younger students. When integrating into engineering notions, then science and mathematics is the STEM subject that benefits least from integrative approaches. Similarly, *Sire-gar et al. (2020) found a much lower effect of integrating STEM into mathematics (0.23), with the lowest effects with elementary students (0.16) than secondary (0.34) and university (0.26) students; and shorter (0.37) interventions were more effective than longer interventions (0.19).

Teaching methods in science

Many studies have reviewed discovery learning, science inquiry, and design projects in science. *Pellegrini and Heroux (2011) reported high effects of student design projects, particularly when working in teams:

they must communicate and consolidate their thinking about the design, redesign and the execution of the hands-on experience within their group. Team members have to listen, relate and appreciate other points of view, in an active rather than passive way.

(p. 33)

*Donnelly and Linn (2014) recommended more evidence about the role of teachers in these projects, particularly exploring the trade-off between too much guidance (thus making science appear formulaic) and too little guidance (thus encouraging superficial or mindless explorations).

It seems hardly surprising that these methods can outperform textbooks. However, most studies of more innovative teaching methods certainly do not always exclude the textbook as part of the teaching – but focus more on the nature of teaching the ideas in these books. For example, *Therrien et al. (2011) focused on LD students and compared, using a science textbook, focus with three forms of interventions: structured inquiry (0.73); supplemental mnemonic instruction, whereby the keywords are paired with an illustration demonstrating how the keyword is related to the vocabulary word (2.00); and supplemental instruction, such as peer tutoring, explicit instruction, and teacher-provided or student-generated explanation (0.42). They noted all three methods provided significantly more structure than is seen in a pure discovery learning approach, and shorter bursts (1.13) were more effective than sessions lasting longer times (> 12 sessions, 0.59). *Schmidt (2016) reported a focus on similar teaching methods (0.81) was more effective than instructional technology (0.45). *Kaldenberg et al. (2015) reported that students with LD benefit from explicitly teaching vocabulary instruction to increase science reading comprehension (1.25) and the use of more broad interventions, such as how to use a graphic organizer and how to better understand paragraph structure (0.64). They argued that effective vocabulary instruction should focus on semantic mapping, use Direct Instruction or precision teaching, supply the students with mnemonics, and explicitly teach definitions of vocabulary words. Once again, explicit, international teaching works, particularly ensuring the students have the science vocabulary before moving to relating and investigating relations among science ideas.

Students can come to science with misconceptions and misunderstandings, and there are many programs that deal with conceptually changing these beliefs. Texts are the most effective way to eliminate misconceptions, either when text is refutational or when text is used in combination with other strategies that cause cognitive conflict. These refutational texts created a form of cognitive dissonance in students’ thinking. Thus, students could be taught to explain why the misconception was incorrect: “Augmented activation activities facilitated cognitive conflict by directing the reader’s attention to contradictory information in the text or by providing illustrative demonstrations that caused incongruity with extant beliefs.” (Guzzetti et al., 1993, p. 134). *Guzzetti et al. (1995) found that learning charts (d = 0.43), discussion webs (d = 0.51), and augmented activation (d = 0.43) were more effective than activation of prior knowledge (d = 0.10) and question-answer-explanation (d = 0.02) in reducing misconceptions from reading science texts. *Horak (1985) found positive effects (d = 0.57) from students reading science texts that helped students select important aspects of the written material and those that aided in building internal connections within the textual materials. Armagan et al. (2010) investigated conceptual change science texts and found high (< 1.00) for physics, chemistry, and biology, across all year groups.

There is probably a right time for hands-on learning, hinted at the large variability in the findings on this topic. In a synthesis of mainly Turkish studies, *Caglak (2017) found an overall effect of 1.55 when comparing teaching with hands-on coursework to teaching science in a teacher-centered or solely lecture-based atmosphere. Others have reported lower but still worthwhile effects (*Weinstein et al., 1982; *Wise & Okey, 1983). *Bredderman (1983) cautioned that the advantages of these activity-based curricula for elementary students might be lost when they later enroll in classrooms where more traditional methods prevail. *Kyle (1982) reported that students in science classes with a low rather than high emphasis on laboratory activities had higher outcomes. *Rubin (1996) partly explained these lower effects by introducing an important moderator – laboratory experiences increased outcomes relating to manipulation skills (d = 1.26) rather than reasoning skills (d = 0.06) or concepts and subject learning (d = 0.33). *Rubin also showed laboratory experiences that aim to question, explain, and encourage thinking at higher levels. Using a variety of sources to discover answers to questions has higher effects than experiences aiming to verify what has been previously presented.

There are specific teaching strategies with a higher impact, which mirror their effectiveness across many subjects. *Schroeder et al. (2007) also investigated the effects of various science teaching strategies on achievement. The highest effects came from enhanced content strategies (e.g., relating topics to previous experience or learning and engaging students’ interest, d = 1.48), collaborative learning strategies (d = 0.67), inquiry strategies (d = 0.65), manipulation strategies (d = 0.57), assessment strategies (d = 0.51), and instructional technology strategies (d = 0.48). They concluded the following:

if students are placed in an environment in which they can actively connect the instruction to their interests and present understandings and have an opportunity to experience collaborative scientific inquiry under the guidance of an effective teacher, the achievement will be accelerated.

(p. 1452)

*Wise (1996) examined a number of teaching strategies and found the following effects: teacher questioning (d = 0.58), focusing strategies (d = 0.57), manipulation strategies (work or practice with physical objects, d = 0.58), enhanced materials (teacher modification of instructional materials, d = 0.52), use of immediate or explanatory feedback (d = 0.32), inquiry strategies (d = 0.28), enhanced context strategies (e.g., field trips, games, self-paced learning, d = 0.26), and instructional media (d = 0.18). He concluded that active construction of meaning is most likely to occur “when science teachers use strategies that require students to be both physically and mentally engaged” (*Wise, 1996, p. 338).

Two meta-analyses explore a specific science concept: teaching control-of-variable strategies (*Ross et al., 1988; *Schwichow et al., 2015). This aims to help students identify the specific causes of effects and interactions in science work (that which may need to be controlled) and evaluate evidence generated from controlled experiments to then relate this evidence to a hypothesis or theory. Both found similar high effects from specifically teaching this notion (0.73, 0.61). They identified five key moderators – the need to teach this concept specifically (and not add in additional skills), the provision of feedback during the instruction, the presence of cognitive conflict (the teacher directing students’ attrition to their thinking and experimental strategies to think about the validity of their interpretations rather than on the task content of measurement problems), the use of demonstrations, and the exclusion of hands-on activities (as they detract from the concept, add cognitive overload).

Other curricula domains and topics

Enrichment programs

Enrichment programs aim to broaden the educational lives of students, promote higher levels of thinking and creativity in a subject area, and allow students to explore subjects in depth. The overall effects support these programs, but the results show major variance. *Kim (2016), for example, noted that the enrichment effects were higher in high school (but there is only one study) than middle and elementary, and some care is also needed as 3 of her 12 studies had outlier effect sizes (> 2.0).

*Henfield et al. (2017) specifically investigated gifted programs on Hispanic and African American students. The nonschool enrichment programs (0.55) had higher effects than school-based programs (0.19). *Wallace (1989) reported that enrichment was stronger in mathematics (d = 1.10) and science (d = 1.23) than in reading (d = 0.59) or social studies (d = 0.23). Programs, where the aim was more depth, had higher effects than when the aim was broadening the range of ideas. Teachers with more years of teaching gifted students had greater (d = 0.88) effects than those with no or limited experience (d = −0.06). There is much need to be more specific about the nature of enrichment programs, and also, it is noted later that acceleration programs are systematically far more effective.

Feuerstein’s Instrumental Enrichment program is a specific enrichment program (Feuerstein, 1980). This program aims to teach critical thinking skills via 13 to 15 sessions to be completed in one-hour lessons three to five times a week for two to three years. Each session concerns a specific cognitive deficiency, such as blurred and sweeping perceptions; unplanned impulsive exploratory behavior; lack of receptive verbal tools; lack or impaired conservation of constancy, such as size, shape, or quantity; deficient need for precision and accuracy; impaired capacity for considering two or more sources of information at once; inadequacy in experiencing the existence of an actual problem and then defining it; inability to select relevant as opposed to irrelevant cues; lack of or impaired need for pursuing logical evidence; and so on. *Shiell (2002) reviewed the effects of Feuerstein’s programs, and the overall effects on achievement was d = 0.26, and *Romney and Samuels’s (2001) meta-analysis found a d = 0.35 effect on achievement.

Acceleration programs

An alternative to special classes and enrichment for gifted children is to accelerate students through the curricula: it typically involves progress through an educational program at rates faster or ages younger than is conventional, although there are many options, such as curriculum compacting or telescoping, concurrent or dual enrollment classes, single-subject acceleration, and advanced placement (*Rogers, 2019). From their synthesis of six metas, *Steenbergen-Hu et al. (2016) reported an overall effect of 0.81, and *Kulik and Kulik (1984, 2004) claimed this was equivalent to advancing one grade level (and also used a comparison of accelerated students with an older control group, d = −0.04) to provide support for this claim.

*Rogers (1991, 2019) concluded that “in general there is a powerful academic effect to be gained from engaging in a variety of forms of acceleration” (2008, p. 2), and *Steenbergen-Hu and Moon (2011, p. 39) concluded that “acceleration influences high-ability learners in positive ways.” In addition, the evidence for enhancing social effects is also compelling (Kent, 1992; Bernstein et al., 2020), although negative social effects are often used to attack acceleration programs (especially skipping a grade), but it is more likely there are negative social impacts on gifted students if they are not accelerated (Wardman & Hattie, 2018).

Levin (1988, 2017) asked that if acceleration is so beneficial for gifted students, why could it not also be used with nongifted students? Hence, his Accelerated Learning program aims to accelerate the learning of at-risk students so they can perform at grade level by the end of elementary school. These programs involve having high expectations, specifying deadlines for meeting educational requirements, stimulating instructional programs, planning by all staff, and using all available community resources. The evidence, however, is limited from a meta-analysis standpoint: Borman and D’Agostino (1995) claimed Accelerated Learning had “highly promising evidence of effectiveness,” although the overall effect size was only d = 0.09.

Wardman and Hattie (2018) identified many myths relating to acceleration. First is the claim that teachers are against acceleration. Wardman (2009) collected quantitative and qualitative data from 455 New Zealand secondary school teachers and found a high level of support for full-year acceleration instead. Second, acceleration leads to negative emotional reactions. It is hard to find a single study showing significant social or emotional harm caused by acceleration. Wardman (2015) conducted a longitudinal study of 12 high school students who had been full-year accelerated at a secondary school in a low socioeconomic area compared to 12 peers of similar very high ability that were not accelerated. She found no evidence of emotional harm. On the contrary, the report from accelerated students was of “an extra year of life,” which gave them more options; for example, completing double or second degrees. Some of the parents commented that if their children had not been accelerated, they would have become so bored with the pace of the curriculum that they would have exited school early, and the students agreed. Third, there are negative social outcomes for the non-accelerated students.

Similarly, Gross (2006) reported that profoundly gifted students in her study who were not provided the opportunity to accelerate suffered from social isolation, as they were locked in age-grade classes with no intellectual peers. Gross observed that in those permitted to accelerate, “In every case, the young people who have been radically accelerated have found both outstanding academic success and the ‘sure shelter’ of a warm and supportive friendship group” (p. 281). The parents in Wardman’s study claimed that their accelerated adolescents were more mature and responsible than their age peers. In a longitudinal study starting with accelerated 13-year-olds in the top 1% of cognitive ability through to when they were in their 50s, Bernstein et al. (2021) showed that these adults were above the average in psychological well-being, and there were no reports of regrets from acceleration; indeed, they often wished they had been accelerated more.

Principals often argued that they did not want to lose a year of these students, which led to a loss of their year’s income. Further, accelerating these students would mean their success was not included in the school’s performance measures relative to their year, and they lamented the possible negative social effects. The questions surely are, what damage are we doing to students by not accelerating, how many lost gifted do we cause, and why should all students have to lock step in our current age-bound system? It is also worth noting that Wardman’s study was conducted in New Zealand, where they long ago abolished age- or year-based curriculum and assessment, preferring a levels-based curriculum instead. This acknowledges the range of proficiencies in a class, forces teachers not to teach at grade level, and monitors progression for each student up the levels of challenge in the curriculum.

Integrated curriculum programs

*Hartzler (2000) investigated 30 studies using integrated curricula and found different effects by subject: science (d = 0.61), language arts (d = 0.42), social studies (d = 0.38), and mathematics (d = 0.42). The most important elements in integrated programs were thematic instruction (d = 0.46) and an emphasis on process skills (d = 0.36). Integrated programs were more successful in elementary (d = 0.56) and middle school (d = 0.57) compared to high school (d = 0.27); for lower compared to middle- and higher-achieving students; for ethnically diverse students; and when more experienced teachers implemented the programs.

*Hurley (2001) investigated integrated mathematics and science programs. The effects were higher on mathematics (d = 0.37) than science (d = 0.27). The conclusion was that integrating mathematics into science might be good for science, but the effects for mathematics was greater when taught in sequence with science, particularly when science was taught before mathematics – as the effects were the greatest on the subject taught last (probably because of the greater level of integration from the first to the last taught).

Financial programs

*Kaiser and Menkhoff (2018) used 27 random control studies investigating the effects of financial education programs in schools (d = 0.25). Similar effects held for financial knowledge and behavior; more intensive interventions lead to higher effect sizes on financial knowledge, low retention effects over time, and greater effects in elementary compared to middle or high schools.

Motivation programs

We reviewed the relation between various motivation attributes and achievement in Chapter 5. Here, the question is whether introducing programs to enhance motivation makes a difference in achievement. The three meta-analyses asked about the nature of the optimal intervention, but the sample sizes were too small to be clear in each case. A Romanian synthesis by *Wagner and Szamoskozi (2012) claimed that motivation training (changing the motive, e.g., from fear of failure to hope for success, 0.53) was more effective than attributional retraining (reframing how students think about success and failure and encouraging responsibility for academic outcomes, 0.32). *Lazowski and Hulleman (2016) found the effects of motivation training were similar across age, ethnicity, culture, socioeconomic status, and ability levels, but had greater differences depending on the focus of the motivation training (e.g., self-determination, 0.70; attribution, 0.54; implicit theories, 0.56; expectancy-value, 0.39; self-affirmation, 0.38). *Berg (2003) found larger effects for knowledge (0.20) than for behaviors (0.14) and attitudes (0.07).

Moral judgment programs

The major outcome of moral education programs is the facilitation of moral judgment, that is, how people define decisions or actions as morally right or wrong (Schlaefli et al., 1985), and as this is not strictly achievement as typically defined, these are not included in the tables. The overall effect of 0.28 from the moral education programs that emphasized moral dilemma discussions had slightly higher effects (d = 0.41) than those based on personality development (d = 0.36). The effects were greater for adults (d = 0.61) than for college students (d = 0.28), senior high school students (d = 0.23), and junior high school students (d = 0.22). *Zierer and Sanders (2005) reviewed the use of moral dilemmas to improve, and the effects with much higher (0.76) when the teacher participated in the discussions by providing a challenging and stimulating discussion by pointing out a weakness in reasoning than when they did not (0.45).

Outcomes-based education

Outcomes-based education (OBE) is popular in many systems as it is based on accountability, often inspection, and is premised on the notion that school outcomes are measurable and improvable. There are many forms of OBE, most involving clear outcomes from curriculum statements, structuring, and operating schools so that these outcome scans be achieved (with autonomy to deliver on these outcomes within a class), but there is a renewed focus on what students can do after they are taught (Spady, 1998). *Yusoff et al. (2014) found 20 studies, mainly from non-Western countries, with a large 0.97 effect size. Perhaps not surprising when there is clarity on what is to be learned, attention to high expectations to realize these outcomes, and then measurement of what is stipulated. The criticisms typically relate to narrowing outcomes to the measurable, encouraging gaming to reduce the influence of students who are less likely to succeed, and assuming that the actions between stipulating the outcomes and then measuring them are all in the hands of schools.

Social skills programs

The relations between social skills and achievement were reviewed in Chapter x, and this section concerns social skills programs that aim to impact student (social and academic) learning. Social skills or social competence programs are usually provided for learners whose behavior is either highly internalized or highly externalized (e.g., socially isolated, withdrawn, or disruptive). The aim is to develop higher levels of social appropriateness, social problem-solving skills, self-control, or social perspective training. Overall, these meta-analyses involving social skills programs show the effects are stronger on enhancing peer relations (0.80 to 0.90) and social outcomes (about 0.5 to 0.6), lower when the students are initially identified as social problems (0.20), and lowest when academic achievement is the outcome of the social skill programs (0.10 to 0.20). In all these programs, there were primarily short-term gains, indicating a need to provide social skills training regularly and sustainably.

Three meta-analyses are more concerned with the effects of social skills training on social outcomes. *Beelmann et al. (1994) found that social competence training was an effective intervention in children’s social outcomes in the short term (d = 0.61), whereas long-term effects were weak. The effects were greatest on social-cognitive skills (d = 0.77) compared to social interaction skills (d = 0.34), social adjustment (d = 0.18), and self-related cognitions (e.g., self-concept, control beliefs). The greatest effects were for at-risk children and younger children, and larger effect sizes were only found when direct goal criteria, such as social-cognitive skills, were evaluated. There were low effects on broader constructs, such as social adjustment. *Denham and Almeida (1987) were particularly interested in the effects on interpersonal social-cognitive problem-solving (ICPS) and found similar high effects (d = 0.62) (see also *Beelmann et al., 1994). An increase in ICPS skills is related to improvement in behavioral adjustment, particularly for elementary-aged children, particularly when they focus on developing dialogue between teacher and student on social problem-solving. *Hanson’s (1988) review of social skills training literature found that the average social skills training program participant was more socially skillful than 74% of those who had not been given training (d = 0.65). There were greater effects from measures based on behavioral observation, followed by self-report and role-play, and then teacher ratings.

The domain with the highest effects from social skills training related to peer relations among all students (*Schneider, 1992, d = 0.98). He found that the more effective programs included coaching and modeling, mainly when focused on individual peer relation issues. They suggested that students receiving social skills training benefit from reduced social anxiety, increased comfort in social situations, or enhanced motivation, as they are made to feel that meaningful improvements in social behaviors are within reach. However, the effects of these programs directly on academic achievement were very low (d = 0.19).

Two meta-analyses were more concerned with the effects of social training on students with emotional or behavioral disorders. *Quinn et al. (1999) were particularly concerned with students with emotional or behavioral disorders and reported low overall effects (0.20). This effect was similar when rated by teachers, self, peers, experimenter, or parent; and for prosocial behavior (social relations, behavior, problem-solving, competence), problem behaviors (family relations, school social behavior, social communication, and disruptive behavior) and specific social outcomes (anxiety, adjustment, cooperation, self-concept, and aggression). There were no differences in the duration of the intervention or whether the program was established, published, or created for the particular group. *Forness and Kavale (1996) found similar low effects when social skills programs were implemented with children with learning disabilities and social skill deficits (d = 0.21). Again, there were no differences in length of treatment; whether peer-, teacher-, or self-assessed; and across social outcomes – with one exception: students in social skills programs believed their social status was enhanced. Forness and Kavale found that the most effective social skills training included a combination of modeling, coaching, and reinforcement programs, particularly when the social skills training related directly to the student’s social skills deficit – although the persistence of the effects over time was problematic.

Students with learning disabilities have lower social skills than their nondisa-bled peers (d = 0.60 to d = 0.70), and thus, perhaps not surprising, the effects of social skills programs had the greatest impact when these students were the focus. *Kavale and Forness (1996) found high degrees of differences in social skills between learning disabled and nondisabled comparison groups. The differences in social skills were quite marked (d = 0.65). *Kavale and Forness claimed that students with learning disabilities were generally assessed as having social skills deficits irrespective of who assessed them – teachers, peers, or self (although self-assessments were somewhat harsher). Teachers indicated that they considered students as having learning difficulties if they had lower academic competence, particularly if they also had social skill deficits. The dimensions of social skills rated highest by teachers were interaction, adjustment, hyperactivity, and distractibility; and the lowest were aggression, conduct disorder, dependency, and personality problems. Peers were more likely to reject and have limited acceptance of disabled students, particularly those with lower social competencies. Learning disabled students were less popular, were less often selected as friends, and were viewed as less cooperative. *Swanson and Malone (1992) found that in social situations, disabled children were more likely than nondisabled students to be rejected by peers, were less well-liked, were more likely to be rated as aggressive and immature, were perceived as suffering from personality problems, and were perceived as having difficulty staying on task. Children with learning disabilities accurately perceive their status within the classroom.

Diversity training

A particular focus of social skills programs is to decrease racial and ethnicity-related differences in the school. *Densen (2009) examined the impact of curricular and cocurricular diversity activities on various outcomes and racial biases within university students. She reviewed programs aimed at enlightenment (e.g., learning about other groups), contact approaches (bringing groups together), diversity awareness (e.g., reducing deficit thinking), multicultural courses, peer-facilitated interventions (peers instructing peers on racial bias issues), living-learning communities, and intergroup dialogues (students from diverse groups conversing together) – all aiming to influence attitudes (prejudice), cognition (stereotypes), emotions (negative affect), and behavior (discrimination).

Overall, these activities positively reduce racial bias among college students (0.48). The percentage of students of color in the classes did not seem to matter, but the pedagogical method did matter. The most successful programs aimed to reduce racial bias by expanding the content-based knowledge that people have of other groups, altering people’s perspective of their relations with others, and using intergroup contact approaches that diminish racial bias by capitalizing on the benefits of cross-racial interaction. The greatest impact was on white more than on students of color and when the interventions included a cross-racial interaction component. But again, teaching to enhance the achievement of racial and ethnic students in the class can occur by design and should not be left to change.

Creativity programs

Creativity programs are grounded in a common idea that training, practice, and encouragement in using creative thinking skills can improve an individual’s evaluative thinking skills of interpretation, analysis, evaluation, and inference, and development of explanations of the basis of these critical judgments (*Cohn, 1986; *Rose & Lin, 1984).

Like most other programs, an emphasis on instructional strategies and Direct Instruction makes a major difference in the effectiveness of creativity programs. *Scope (1998), for example, reported that creativity programs had a high level of structuring (d = 0.80), questioning (d = 0.73), and responding to student questioning (d = 0.70). These effects were constant across all subject areas. *Higgins et al. (2005) undertook one of the more complete reviews of programs to enhance thinking and creative processing. Across all outcomes, the effect size was d = 0.74; for cognitive outcomes, d = 0.62; for curricula achievement, d = 0.62; and for affective outcomes, d = 1.44. The greatest effects came from meta-cognitive strategies (d = 0.96), cognitive acceleration (d = 0.61), and instrumental enrichment (d = 0.58). This team completed an extensive review of various strategies and developed a four-part thinking model. Thinking consists of information gathering, building of understanding, productive thinking, and strategic and reflective thinking. They argued that developing strategic and reflective thinking is a major goal of schooling. The other three cognitive skills can happen in unplanned and unreflective ways (*Higgins et al., 2005). Another part of the model could be developing motivational and high expectations about developing creative thinking (*Bangert-Drowns & Bankert, 1990; *Cohn, 1986; *Kardash & Wright, 1987).

The CPS (creative problem-solving, Osborn, 1963) is consistently listed among the more successful programs and often delivers higher effects in less time (Cohn, 1986; *Huang, 2005). This model has three phases: understanding the problem, generating ideas, and planning for action. Similarly, programs that focus on activation of prior experience (0.61), promotion of new knowledge that is demonstrated to the student (0.67), demonstration (0.67) and application of skills (0.60), and integration into real-world activities (0.62, what Merrill, 2002, termed the “first principles of instruction”) maximized creative thinking (*Bertrand, 2005). The more of these principles, the greater the effect. *Huang (2005) also reported that school-developed programs had the lowest effects, possibly because many of these are generic and not embedded in the curricula domains. *Scott et al. (2004) found effects for similar processes and phases: focus on problem identification (d = 0.78), idea generation (0.45), implementation planning (0.39), solution monitoring (0.35), and conceptual combination (0.33). He also found that the effect of creativity programs was much higher for nongifted (0.72) and low-achieving (0.68) students than for gifted (0.38) students.

*Berkowitz (2006) found that various communication strategies enhanced critical thinking outcomes; for example, participation in public speaking courses (d = 0.29), argumentation-type classes (d = 0.26), and various types of competitive forensic methods of creatively working through problems (d = 0.41). *Abrami et al. (2015) updated their earlier meta on programs to enhance critical thinking but still found overall effects about 0.30, and these effects were consistent over K–12, across subject disciplines. Shorter programs (one hour to two days) have double the effect of longer programs. The effects of teachers engaging in dialogue (0.32; posing questions, 0.38; whole-class discussions, 0.42; small groups led by the teacher, 0.41) and anchored instruction (0.34; role-playing, 0.61; applied problem solving, 0.35) were more effective than mentoring. Content-specific outcomes produced a higher average effect than generic CT outcomes (0.57 vs. 0.30), and the overall effects on achievement were 0.33.

Developing these creativity principles within a curricula domain rather than generically seems optimal. For example, Cohn (1984) noted that creativity training is not as effective in enhancing creative performance on tasks dissimilar to those employed during the training. A shorter, more intensive burst was more effective than longer programs (*Liu & Chang, 2017; 0.94, < 1 month were more effective than longer, 0.71). The grade effect is far from clear. *Yu (2012), using Chinese studies, found higher effects in elementary (0.88), high (0.66), and middle schools (0.66); *Nui et al. (2013) found similar effects for under- and postgraduates, but *Huber and Kuncel (2016) suggested nonlinear effects among college students but were cautious about indicating a year level when the increase occurs probably because they also noted that there was short-term rather than long-term gains.

Chess instruction

*Sala and Gobet (2016) reviewed the transfer effects of chess instruction, with an effect of 0.34. They conclude that “chess instruction is no more effective in enhancing children’s cognitive and academic skills than many (at least more than 50%) other possible educational interventions” (p. 53), and instead, “learning activities should be as close as possible to the skills to train; for example, mathematics instruction should be used to teach mathematical skill” (p. 54). Further, the transfer may start after a minimum of 25 hours of instruction, and maybe this is better spent directly on the academic subject.

Outdoor education

It seems a stretch to consider why canoeing a rapid, climbing a rock face, and completing a solo three-night camp could affect reading or math scores. *Hattie et al. (1997) reviewed 96 studies and found an average of 0.34 increase across outcomes on academic outcomes. Perhaps most exciting is that this is one of the few areas in education where the follow-up effects (d = 0.17) were positive and were in addition to the effects at the end of the program (so 0.34 + 0.17 = 0.51). It is rare to find such increasing after-effects from an education intervention, as too many have a diminishing return. Specifically, the effects of adventure programs on academic outcomes were d = 0.46; leadership, d = 0.38; self-concept, d = 0.28; personality, d = 0.37; and interpersonal outcomes, d = 0.32. There was, however, much variance between programs, with the Australian far exceeding the American programs – which we suggested was because the former was more oriented to teaching (e.g., only those with social science degrees could be employed as instructors), whereas the latter was more oriented to the outdoor experience. The link is that students develop skills of problem-solving skills and peer and cooperative learning, and there is an enhanced level of seeking and interpreting feedback via these programs. Major reasons for the success are the way activities are structured to emphasize very challenging learning intentions, the success criteria are clear, the peer support optimized, and the feedback not only is given throughout the program but actively sought by the participants. Many of the coping strategies students had when they entered the program were deficient and needed to be replaced with other more cognitive and peer supportive strategies to ensure that the team overcame the many challenges.

*Cason and Gillis (1994) found that longer programs were more effective, and younger participants gained more from outdoor programs than older participants. The effects on grades were d = 0.61, and on school, attendance was d = 0.47, which is above the overall average effect of d = 0.31. *Laidlaw (2000) found an effect of d = 0.49 for wilderness and d = 0.39 for school camping programs, but longer programs were more effective. We also found that programs longer than 20 days were much more successful than shorter programs (*Hattie et al., 1997). Learning about confronting challenges, seeking feedback, adapting to peer cooperative learning, and enhancing self-regulation about one’s skills and strengths seems to last beyond outdoor experience.

Juvenile delinquent programs

*Sander et al. (2012) investigated the transfer effect of attending juvenile delinquent interventions on school achievement. The effect is small (0.12), on attendance (0.11), and attitudes to school (0.19), and these effects were similar for the length of intervention, whether the program was in or out of school, and severity of reasons for being in the program. They conclude that “the effectiveness of those programs that have been tested for improving academic outcomes seems to be lackluster” (p. 1707).

Play programs

Lillard et al. (2013) completed a traditional review of play research, and they conclude that the “existing evidence does not support strong causal claims about the unique importance of pretend play for development and that much more and better research is essential for clarifying its possible role” (p. 1). They could not find convincing evidence that pretend-play enhances creativity, problem-solving, conservation, or most cognitive development aspects. Nor does pretend play help develop the child’s theory of mind, executive functioning, emotional regulation, or development of skill to build more coherent and interpreted concepts about their world and explain and predict behavior. Providing toys does not enhance new stories but does help with story retelling. Children who pretend more also appear to tell more elaborate stories, but when older, they are not better at story comprehension. This does not mean that play is not fun nor worthwhile as an activity, but it does ask for caution about causal claims and invites other ways to enhance positive developmental outcomes.

*Spies (1987) examined play, problem-solving, and creativity in young children and found a small relationship between play and originality for familiar objects but not for unfamiliar objects, and no effect of play on problem-solving. Fisher (1992), in an investigation on the effects of play on development, found stronger evidence to suggest play promotes improved performance outcomes both in cognitive-linguistic and affective-social domains. He found somewhat larger effect sizes for ideation fluency (originality or flexibility of association and the kind of divergent thinking characteristic of creative imagination) and perspective-taking (empathetic role assumption related to greater co-operative behavior, sociability, and heightened peer-group popularity). Fisher found some differences in effects among different types of play, with sociodramatic play having the most striking effect and the smallest effect in imaginative play. The adult-directed play showed no more gains than other play conditions. *Ray et al. (2015) were more interested in the effect of child-centered play therapy and found d = 0.36 on academic outcomes for those students receiving and those not receiving play therapy. They argued that this supported “that CCPT releases a child from emotional limitations hindering the child from performing at full potential in learning endeavors” (p. 119).

The message is to be cautious about the effects of play, have more causal explanations about which forms of play, and not see play as a generic notion relating to improved learning. The explanations are likely related to learning about peer relations and learning how to learn from peers, facing and meeting challenges, knowing the consequence of deliberative practice in play, and knowing the satisfaction from deciding or becoming aware of both the learning intentions and success criteria from being involved in play. Play may be a worthwhile activity, regardless of any association with achievement.

Bilingual programs

Bilingual education involves two languages as the medium of instruction in contrast to structured immersion programs where students are instructed solely in one language. There can be wide variation in the allocation and organization of language use across the timetable and curriculum in bilingual education. This is reflected in high levels of variability in bilingual programs (*Willig, 1985) – especially as some are remediation programs for immigrants (*Oh, 1987) and some are programs to teach a second language. Some are related to preserving cultural principles (e.g., Māori immersion in Kura Kaupapa schools in New Zealand). The variation in the effect sizes for these programs seems to relate to the quality of teaching competence and the explicit attention to teaching strategies of learning. *Slavin and Cheung (2005) end their review by reinforcing this claim that the language of instruction is but one aspect that may be as important as the quality of instruction. They also highlighted the need to separate second-language immigrants’ findings from bilingual programs within a country (like French immersion in Canadian schools). They found French immersion had no adverse effect on English reading achievement and that two-way bilingual programs for English-proficient children are not likely to harm English reading development. For immigrant students, the strongest effects were for paired bilingual approach, in which children were taught reading in both English and their native language at different times each day from the beginning of their schooling, compared to the transitional bilingual education approach in which children are first taught to read primarily in their native language and only then transitioned gradually to English only instruction. They found that “nothing suggests that it is harmful to children’s reading performance to introduce both native language and English reading instruction at different times each day” (p. 274).

*Willig (1985) found that participation in bilingual education programs compared to submersion in English showed small to moderate differences favoring bilingual education for tests administered in both English and Spanish. Outcomes were positive for bilingual education for tests of reading, language skills, mathematics, and total achievement administered in English. The results also favored bilingual education when tests were in other languages for reading, language, mathematics, writing, social studies, listening comprehension, and attitudes toward self and school. *Adesope et al. (2010) found d = 0.41 from cognitive outcomes associated with bilingualism compared to monolingualism, including metalinguistic awareness (0.33), metacognitive awareness 0.32), and problem-solving (0.26). The effects were higher in preschool and high school (c.f., *Powers & Rossman, 1984). These results show little support for language learning inhibiting one or the second language. Indeed, they appear to give bilingual speakers an enhanced capacity to appropriately control and distribute their attentional resources, develop abstract and symbolic representations, and solve problems. *Prevoo et al. (2006) examined relations of typically developing bilingual immigrant-background children’s oral language proficiency in their first and second language with the school outcomes. They found that within-language relations between oral language proficiency and literacy, reading, and math, and the school outcomes were higher (r = 0.43) than cross-language relations (r = 0.27). This evidence shows that “stimulating oral language proficiency in both languages can be a key factor in improving the school outcomes of bilingual children with an immigrant background” (p. 266). *Reljic et al. (2014) asked similar questions using European studies and found support for including the home language of language minority children in school instruction (d = 0.23).

Special college programs

There have been many remediation programs for college students. *Kulik et al. (1983) claimed that special college programs for high-risk students led to them staying in college longer (62% vs. 52% for control students, although the authors note this as a very small effect). The more successful programs related to academic skills (d = 0.28) and guidance sessions (d = 0.41), but the effects of remedial programs have been limited to zero effects (d = 0.05). The effects were stronger in new programs and weaker in institutionalized programs. Thus, colleges seem more proficient at setting up programs for high-risk students than at keeping these programs going. *Valentine et al. (2011) reviewed the effects of college retention interventions, ranging from comprehensive interventions, tutoring, and journaling. The effects were low on achievement (d = 0.08), although students were more likely to stay in college.

Career education programs

Career education involves activities and experiences designed to increase knowledge of occupations, training paths, job-search skills, and decision-making strategies that include integrating work, family, leisure, and community roles. Career education programs seem to positively affect student outcomes (*Baker & Popowicz, 1983). *Oliver and Spokane (1988) found that career counseling generally has positive effects, with class interventions being the most effective but requiring the greatest number of hours. Individual counseling produced more client gain per hour than other intervention models. The intensity of treatment was the only significant contributor to more positive outcomes. *Evans and Burck (1992) found that career education interventions did contribute to academic achievement, but the interventions only improved student academic standards by an average of d = 0.16 over alternative or control conditions. Elementary-age students of average ability benefited most academically, particularly if they were randomly assigned to groups; the intervention was coupled with mathematics and language arts subject matter. The program averaged 151 to 200 hours per school year.

Conclusions

It is the alignment of appropriate teaching methods with the nature of the curriculum objectives that matters most. That is, using teaching methods that focus on the content, the knowing-that, and the skills, then moving to adopt teaching methods that focus on the deeper knowing-how, conceptual, and relational understandings. The knowing-that is firmly embedded within the content of the curriculum, and this makes the transfer of knowing-how or deeper skills all the more dependent on knowing these content and skills.

It is the intentional teaching of the knowing-that, knowing-how, and knowing-with skills of the subject that needs to be planned, deliberate, and explicit.

There can be much success teaching curricula domains from a problem-solving perspective from the earliest age (and ensuring they have sufficient content to problem-solve) and, thus, ensuring students learn appropriate skills to complete these problems.

In all cases, it is more powerful to be concerned about connecting the lesson with prior learning and understanding than attend to the oft-made claims of relating the lessons to the real or some future world.

It is a major mistake, and indeed can be the killer of a child’s later schooling, not to achieve mastery before introducing more challenging tasks. A common theme is the Matthew effect – if by age 8, students do not acquire sufficient content, skills, and relational ideas; and do not acquire the coat hanger or bigger ideas to interpret incoming new information; then the gaps between the haves and have-nots will increase.

If you put all the various systems curricula in any one subject on the table, the most amazing finding is that they are so different. There is no such thing as essential knowledge, there is no such thing as perfect scope and sequence, there is no such thing as one world agreed curricula, and there is no one optimal progression to success. As a result, curricula are, and should be, a most contested topic, the essence of democratic debate, and the core debate about what is taught and valued in schools.

When developing the New Zealand assessment system, we identified 147 specific objectives underpinning the reading curriculum. These were initially grouped into five higher notions that formed the basis of the curriculum (processes and strategies, purposes and audiences, ideas, language features, and surface [grammar, punctuation, spelling]). A curriculum review switched these to six other higher notions (finding information, knowledge, understanding, connections, inference, and surface [grammar, punctuation, spelling]) – but the switch merely constituted a reordering of the 147 objectives. We then implemented the assessment model in New York City. Again, their higher-order notions were seen as perfect by the curriculum developers – but again, it was a mere rearrangement of the same 147 objectives. Oh, the fights and preciousness about the labeling of these higher notions. Reading is reading.

Similarly, the New Zealand teachers were shocked that statistics was not a major part of the high school curriculum in the United States, and the United States teachers were shocked that finite math was not in the New Zealand curriculum. There is probably more variance in the other curriculum domains across systems (i.e., health and physical education, humanities and social sciences, the arts, technologies, and languages). This is because so much energy and emotion are spent on curriculum restructures, but the essence of the skills and understandings and the optimal teaching across these curricula are similar.

It is the alignment of appropriate teaching methods with the nature of the curriculum objectives that matters more. That is, using teaching methods that focus on the content, the knowing-that, and the skills; then moving to adopt teaching methods that focus on the deeper knowing-how, conceptual, and relational understandings. The knowing-that is firmly embedded within the curriculum’s content, which makes the transfer of knowing-how or deeper skills all the more dependent on knowing these content and skills – which is why each curriculum area is unique.

There are many more studies on reading, writing, math, and sciences, and too few on the other curricula domains, but the message seems similar – ensuring the appropriate knowing-that skills to then move to knowing-how and knowing-with (from surface to deep to transfer). This can happen within a single lesson, but too often, the debate is phrased about exposing students to all (or lots of) the content before their later years of schooling and moving to the relational and the thinking like a mathematician, a historian, or service worker. The balance, the proportion, and the presence of both knowing-that and knowing-how/with within each lesson or series of lessons appear to motivate students, and there are differential teaching methods that maximize the learning depending on the nature of the output. If only there were far fewer content specifications in our curriculum, we could also go deeper.

It is the intentional teaching of the skills of the subject that needs to be planned, deliberate, and explicit. Successful reading requires the development of decoding skills, the development of vocabulary and comprehension, and the learning of specific strategies and processes – this is most evident in the younger years but present right through all levels of schooling. Successful math requires developing patterning skills, developing ordination, developing relationships, and learning and using basic facts and concepts. There can be much success in teaching mathematics as problem-solving from the earliest age (involving both the content and deeper skills) and then ensuring students learn appropriate skills to complete these problems. This can be more powerful than teaching basic facts and introducing problem-solving later. It is not attending to one or the other (skills than understanding) but attending to both – at the right time, with the right teaching methods, with the right levels and depth of integration.

In all cases, it is more powerful to be concerned about connecting the lesson with prior learning and understanding than attend to the oft-made claims of relating the lessons to the real or some future world. Hence, an excellent diagnosis of what the student brings to the lesson (their content, skills, conceptions, and misconceptions) is critical to success – ensuring the success criteria of the lesson is too hard or too easy is then based on this diagnosis. It is more than mere exposure or time on task, and evaluating progress to success criteria may mean some teachers need to spend less time on some concepts. Hence, questions about efficiencies as well as effectiveness are warranted. This can be difficult in classes with assigned timetables. What do you do with those who reach success well before others; and the COVID pandemic distance teaching has shown many successful ways to allow these efficient students to move to other topics or gain more depth.

It is a major mistake, and indeed can be the killer of a child’s later schooling, not to achieve mastery before introducing more challenging tasks. A common theme is the Matthew effect – if by age 8, students do not acquire sufficient content, skills, and relational ideas; and do not acquire the coat hanger or bigger ideas to interpret incoming new information; then the gaps between the haves and have-nots seemed to increase. Programs like Reading Recovery ensure the students gain sufficient skills in reading before moving to read to learn, and this requires more exposure for these students and dedicated teaching skills to lead to mastery efficiently. If only there were more recovery programs in math and other domains.

The more recent research on developing the many possible learning progressions is exciting. Yes, the detail can be overwhelming. There is certainly not one progression (a fault within many systems), and developing GPS-like systems to aid teachers and students to understand better the progressions, the best-next-teaching focus given the particular progression, and the continual recognition of the learning gains by the students are more likely to motivate the student to the next milestone/success criteria (Cawsey et al., 2019). The research by Petit et al. (2015) on progressions on fractions shows the enormity of this task, but it is powerful work, and much more is needed across the many topics in the various curricula. We then need more research on competing models of curricula relative to success in progress to achievement. One of the healthy findings from the science studies is the comparative studies of the various curricula, and many more comparative studies like this are needed in all domains.

Let me be clear – I am not a fan of the continual desire to retweak the curricula. I am often asked to be on various curricula review groups but resist unless there is a written stipulation that the revised curricula must include substantially less than the current version. Hence, I have only been on one review group over the past decade but withdrew when the new minister said, “Students need to know more, not less.” If you put all the various systems curricula in any one subject on the table, the most amazing finding is that they are so different. There is no such thing as essential knowledge. There is no such thing as perfect scope and sequence. There is no such thing as one world agreed curricula. Curricula is, and should be, a most contested topic, the essence of democratic debate, and the core of what is taught and valued in schools.

Reflections

Gardiner commented that coverage was the enemy of depth. Some claim that teachers teach that which aligns most closely to high-stakes tests. Does this resonate with your experience? Does teaching to the test equal that which is most valuable? What the test is, is an excellent representation of what you value and intended to cover. Can you devise such a valuable test prior to developing the lesson plans, teaching, and critiquing the impact on students?

Read the paragraph on p. 280 of the “Mathematics instructional processes” section, in which intentional or deliberate teaching is highlighted as having high effect sizes. List the elements outlined and used to discuss and explore future targets and priorities for staff development.

Complete an assessment of the Big Five skills of all your students (regardless of their grade level) as to their reading proficiency. Then with colleagues and reading resource specialists, devise a program for these students to enhance these skills.

After reading the sections on mathematics, discuss the implications of the conclusion that teaching math as problem-solving from the earliest age and then ensuring appropriate skills to complete these problems is more powerful than teaching basic facts then later introducing problem-solving. What are the issues involved in word problems for young children, and how can we overcome those challenges?

Together, draw out the key takeaways from the conclusions. The message seems to be that these themes are the ways forward to ensure successful teaching and learning, regardless of the curriculum on offer – thence evaluate the current ways teachers and leaders decide on curricula focus, scope, and progressions, and ensure there are opportunities for depth in some areas of each curriculum.

CHAPTER 11 Teaching with intent

DOI: 10.4324/9781003380542-11

The intentional alignment of goals, learning strategies, success criteria, and feedback

The main claim is that teaching needs to be intentional – and this means intentional alignment among the various components of lessons: success criteria, feedback, learning strategies, teaching methods, activities, and assessments. The focus is on the defense, value, and critical nature of the success criteria, with an appropriate balance of knowing-that, knowing-how, and knowing-with such that students do not misunderstand the intent (e.g., by overfocusing on the content or the deeper aspects of the lesson) if this is not aligned with the intent of the teacher. Of course, students are not only bound in their learning by what the teacher says, does, cares about, or intends. But they need not be confused about the intent of the teacher. Teacher clarity is paramount, even when they take digressions to interesting asides.

Thus, when preparing lessons, the choice of teaching methods requires a deep understanding of the following:

Curriculum, learning progressions, knowledge of what success looks like (in the time available), and learning intentions.

Cognitive task analysis involves successfully analyzing the steps required to complete a cognitive task. It involves knowing the surface, deep, and transfer knowledge, and the learning strategies required for success at each level.

Where students are, where they have come from, and their trajectories of learning.

Provision of teaching interventions and feedback to reduce the learning gap from where the students commence to the desired success criteria.

An evaluation strategy (including assessment) for monitoring your implementation and impact on students during and at the end of the lesson(s). Feedback to teachers’ rules.

These five processes are fundamental to the choice of teaching strategies. The intent of the lesson(s) should align with the criteria of success of the lesson(s) and also with the assessment and feedback to the teacher and to the student during and at the end of the lesson(s). Biggs (1999) argued that this meant there were three core notions underpinning intentional alignment: identifying the desired outcomes, deliberately aligning the planned learning activities and the desired success criteria, and ensuring that the assessment tasks will tell us if the actual outcomes match those that are intended or desired.

This is reminiscent of backward design. Wiggins and McTighe (2005) noted that teachers typically design a lesson using “forward design,” choosing the content and the activities, and then attempting to draw connections to the learning goals. Too often, such classrooms are dominated by many activities, and so often, the students see the purpose of learning as completing activities or doing the tasks, and success is completing the tasks on time. Decisions about assessment in forward design typically occur after the teaching and mirror the content of what has been taught, particularly whether the students have paid attention to the content.

In contrast, backward design has teachers consider the learning goals first. These learning goals embody the knowledge and skills that instructors want their students to have learned when they finish the series of lessons. Once the learning goals have been established, the second stage involves considering the assessment, and the intent is to be clear to the teacher (and students) what is valued in terms of the content, the deeper relations between ideas and conceptual understandings, and the desired skills to transfer to other tasks and contexts. The third stage is designing the teaching methods, activities, and resources to move students from where they start to the desired and transparent success criteria. This is the essence of teaching with intent. It asks teachers to act with intent – “action under development” (Russell, 2018):

Being intentional about the choice of learning intentions, success criteria, and evaluative methods, and seeking evaluative feedback about the effectiveness and efficiency of teaching methods and their implementation during and at the end of the lessons.

Being intentional about the process of learning, which can be considered to include developing knowledge, the deeper relational and conceptual understanding, and the skills of transfer.

Being intentional about the optimal learning strategies to teach and use in the lessons.

Being intentional about the choice of teaching methods, and acknowledging that there are very few teaching methods that are optimal at all the three parts of the learning cycle: knowing-that (surface), knowing-how (deep), and knowing-with (transfer). This requires teachers to have skills in differentiating teaching methods depending on where their students are in the learning cycle.

Being intentional does not mean using a teaching method when it is not having the intended impact; not making students do activities for the sake of doing them.

Being intentional means seeing the impact of teaching by students being taught to think aloud about their thinking and learning, and by students understanding the success criteria and their progression toward them.

This chapter outlines the evidence on learning intentions and goals, success criteria, skills in analyzing the learning strategies and cognitive requirements of tasks, and the power of feedback (including from assessments) – the core of the Intent to Teach model.

The model includes informing the students of the learning intentions and criteria for success of the learning task. These two phases are less about whether the students desire to attain the learning target (which is more about motivation) but more about whether they understand the nature of the learning tasks and what it means to succeed. When a student is aware of what it means to be successful before undertaking the task, this awareness leads to more goal-directed behaviors and a better understanding of the why and value of the teaching and feedback. In addition, students who can articulate or are taught these success criteria are more likely to be strategic in their choice of learning strategies, more likely to enjoy the thrill of success in learning, and more likely to reinvest in attaining the goals of the lesson(s).

A prediction from the Intent to Teach model is that students need to be taught how to gain an overall picture of what is to be learned (via learning intentions), understand the success criteria for the lessons to come, and be somewhat clear at the outset about what it means to master and be successful in learning from the lessons (Table 11.1). Then their subsequent learning is maximized. The overall effect across the 31 meta-analyses from success criteria is 0.54, with the greatest effects relating to providing students with the skills of planning and prediction, having intentions to implement goals, setting standards for self-judgments, and having the appropriate difficulty of goals. This allows students to see the whole or the gestalt of what is targeted to learn before starting the lessons. It thus provides a coat hanger on which surface-level knowledge can be organized.

Table 11.1 Summary statistics for learning intentions, success criteria, and depth of complexity

Learning intentions
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Clear goal intentions
7
504
97,824
613
0.44
0.04
4

Advance organizers
12
935
87,015
2,291
0.41
0.04
4

Planning and prediction
3
157
14,450
178
0.83
0.08
3

Learning hierarchies approach
1
24
2,209
24
0.19
0.00
1

Concept mapping
12
1,262
115,467
1,429
0.62
0.08
5

Success criteria
2
163
15,002
163
0.64
0.08
2

Appropriately challenging goals
6
375
36,955
473
0.60
0.06
4

Goal commitment
3
103
9,907
112
0.44
0.08
2

Mastery learning
17
814
76,638
667
0.67
0.10
4

Worked examples
2
83
5,257
179
0.47
0.04
2

Depth of complexity

Task complexity
3
331
213,627
413
0.62
0.08
4

Cognitive task analysis
2
27
2,485
95
1.09
0.11
2

Total/average
70
4,778
676,835.79
6,637
0.58
0.04
3

Learning intentions

Learning intentions (LI) establish what students intend to learn and can be presented in content, language, or social learning intentions. Many sources elaborate on how to optimally design (and success criteria, SC); this book is not the place for this discussion (see Clarke, 2021). The question here is more related to the evidence for the power of LI and SC – and a key finding is that in many instances, both are needed; indeed, LI without SC can move the students back to seeing learning as little more than doing, and without SC, they will have little to no concept of what the teachers consider to be successful in this learning (other than completion and compliance). For example, if an LI is to learn to write essays, this leaves the student wondering what a good essay looks like; hence, SC can provide this crucial information for them.

The essence of learning intentions is goal-directed behaviors. *Locke and Latham (1990) provided compelling evidence, including many meta-analyses showing how critical goals are for enhancing performance. Goals regulate actions, and they explain the nature of the link between the past and the future. A major finding of their book is that achievement is enhanced to the degree that students and teachers set challenging rather than do-your-best goals relative to the students’ present competencies and goals that students are committed to attaining. A major claim here is that it also makes a major difference if teachers are transparent up front to students about the intentions of their lessons and communicate or, even better, construct with the students the notions of what success looks like. By understanding success or when good is good enough, teachers can enhance students’ commitment to goals.

Many related influences emphasize learning intentions and success criteria, such as advance organizers, mastery learning, and learning hierarchies.

Advance organizers

In the 1960–1990s, there was much attention on behavioral objectives and advance organizers:

broadly defined as bridges from the reader’s previous knowledge to whatever is to be learned; they are supposed to be more abstract and inclusive than the more specific material to be learned, and to provide a means or organizing the new material.

(*Stone, 1983, p. 194)

There are 12 meta-analyses with an average effect of 0.42. The effects drop when the advance organizers are too specific, explicitly focused on the actual task and not at a higher level, or ignore challenges. The effects are higher when notions of success are included and shared with the students. *Kozlow (1978), for example, found that behavioral objectives were more effective when they involved comparisons to some standards of performance, becoming rather like success criteria rather than expository.

Planning and prediction

The average effect of planning and predicting from four meta-analyses is 0.83. These planning strategies include developing learning intentions and allocating resources, with specific attention to deciding upon the amount of time to spend on activities, choosing the optimal order of teaching the content and concepts, choosing the nature and indicators of progression to success, and making predictions about which cognitive and learning strategies to employ.

Care is needed about progressions. There is no such notion as a progression for all students, as there can be multiple progressions to success criteria and the need to allow for different times and ways for students to attain these success criteria (as noted earlier). Cawsey et al. (2019) likened progressions to GPS – there is a route that most will take from New York to Los Angeles, but if some go up through Canada and others down through the South, they should not be punished, seen as deviant or lesser, or be forced to drive to the main route. When learning, some love to go on side tracks, discover the beauty of the local scenery, and are more likely to recall the incidents of discovering, being lost, and feeling the joy of getting to their destination with many great stories.

Learning hierarchies

Learning hierarchies determine a hierarchal ordering of the skills to be learned that will then support later learning. The average effect size from one meta-analysis is 0.19 (*Horon & Lynn, 1980). This hints at the folly of developing learning progressions for all students as if all students follow the same series of steps in their learning. Care is also needed about too much speed along the road of progression, which can slightly hinder developing fluency and consolidation of learning (*Jackson & Suethanapornkul, 2013). As part of the planning, teachers need to be concerned about accuracy, complexity, and fluency; work at setting appropriate levels of challenge based on previous mastery; share the journey destination (success criteria); and teach students to evaluate their progression as well as enjoy the local scenery, knowing that failure and false avenues are normal.

Concept mapping

An effective way to explain learning intentions is via concept mapping, which involves developing graphical representations of the conceptual structure of the content to be learned. Such maps often identify the material to be learned with indicators of priorities and higher-order concepts. The average effect from 11 meta-analyses is 0.62.

One of the key success factors is to involve the students in developing the maps (*Nesbit & Adesope, 2006), not necessarily at the very beginning of a series of lessons but after they have started to learn some of the subject matter vocabularies and been exposed to the main ideas (*Kang, 2002). During such co-development sessions, there can be discussions about capturing the main ideas in what is to be learned, identifying themes, and noting interrelationships – particularly for the learners who do not have these organizing and synthesizing skills. They are more effective with those students least likely to know the relationship between lower- and higher-order notions; that is, with lower rather than higher-ability or highly verbal students (*Horton et al., 1993; *Nesbit & Adesope, 2006; *Vásquez & Caraballo, 1993). *Nesbit and Adesope (2006) noted that there is a lower cognitive load “by arranging nodes in two-dimensional space to represent relatedness, consolidating all references to a concept in a single symbol, and explicitly labeling links to identify relationships” (p. 434). Concept maps provide a coat hanger for students to understand the big ideas underlying the many content details (Karpicke & Blunt, 2011).

Success criteria

The purpose of the success criteria, or we are successful when, is to make students understand what the teacher uses as the destination or notion of mastery. They outline the criteria for judging their work, indicating when it is good enough, demonstrating to the students that the teacher is clear, and sharing the criteria for determining if the learning intentions have been successfully achieved. Too often, students may know the learning intention but do not know how the teacher will judge the students’ performance or know when or whether they have been successful (often leading to asking, “is this on the test?”). Then not surprisingly, there is an overfocus on the doing, completing, and seeing the learning over when the work is handed in. By providing success criteria, students can be assisted in monitoring their progress toward these judgments and standards.

Both learning intentions and success criteria are needed to maximize the impact on achievement. The bases of many claims about the value of student self-assessment, self-evaluation, self-monitoring, and self-learning are that students have a reasonable understanding of where they are at, where they are going, what it would look like when they get there, and where they will go to next: that is, they have learning intentions and success criteria. Martin (2006) argued that one method to assist students in setting task-specific and situation-specific goals was using the notion of personal bests. Task-specific goals provide students with clear information about what they are trying to achieve in the immediate future (both in terms of specificity and degree of challenge), and situation-specific goals provide students with the reason they want to achieve a particular outcome (to beat one’s previous level of achievement on that goal). Feedback relative to such success criteria is often more effective than when it is relative to other students’ performance (Zhao, 2022). Martin found that setting personal bests had high positive relationships with educational aspirations, school enjoyment, class participation, and persistence on the task. The most salient features of the personal bests were the specificity and degree of challenge of the goals and that the goals were seen to relate to self-improvement. Personal bests combined the best features of mastery and performance goals, as personal bests “primarily reflect a mastery orientation because it is self-referenced and self-improvement based and yet holds a slice of performance orientation because the student competes with his or her own previous performance” (Martin, 2006, p. 816).

While students welcome understanding what success looks (DeLuca et al., 2018), a dilemma is that students may interpret success criteria primarily in terms of knowing lots (no matter how brilliant the teacher develops and explains them to the students). This has led Hattie and Clarke (2019) to recommend, where appropriate, two success criteria – one for the closed/surface (fact, content, knowing-that) and one for the open/deeper (relation, transfer, knowing-how/with). For example, in a writing task, there can be knowing-that criteria specific to developing knowledge and skills (e.g., using correct punctuation, the role of adjectives) and others for deeper relational and knowing-how attributes (e.g., writing characterization of a character in a play, writing an explanation text).

Appropriately challenging goals

In Visible Learning, evidence showed that more difficult goals were more powerful than do-your-best goals. The performance of the students with the most challenging goals are over 250% higher than the performance of subjects with the easiest goals (*Wood et al., 1987). Therefore, we should ban doing your best, particularly as sometimes your best is not good enough, and such a platitude can reduce the commitment and skill to take on challenging tasks. The scenario is that effective teachers set appropriately challenging goals, communicate them to students, and then structure situations so that students can reach these goals. If teachers can encourage students to share a commitment to these challenging goals and if they provide feedback on how to be successful in learning as one is working to achieve the goals, then goals are more likely to be attained.

“Challenge” is a relative term – it depends upon a student’s current performance and understanding and on the success criteria deriving from the learning intention. The challenge should not be so difficult that the goal is seen as unattainable, given the student’s level of self-efficacy or confidence; instead, teachers and students must be able to see a pathway to attaining the challenging goal – a pathway that can include strategies for understanding the goal or intention, implementation plans to achieve it, and preferably, a commitment to achieving the goal.

Lomas et al. (2017) noted that many game designs are optimized to make them “not too hard, not too easy.” They randomly assigned players to easy, moderate, and difficult conditions, and allowed other groups of players to choose their difficulty freely. Moderately difficult levels were most motivating when self-selected, and too much or too little novelty reduced intrinsic motivation. They concluded that if difficulty decreases while novelty and suspense increase motivation, then an implication is to make easy, interesting games that are “not too easy, not too hard, not too boring” – The Goldilocks principle of challenge.

Chess players most enjoyed games with a 20% chance of winning; they like hard games but not too hard (Abuhamdeh & Csikszentmihalyi, 2012). In computer games, Lomas et al. (2017) noticed the opposite. The players claimed the optimal difficulty level to maximize intrinsic motivation seemed “as easy as possible.” They determined three design features to maximize success and motivation: player choice, novelty, and suspense. As games become more difficult, they can introduce novelty, and the “appeal of challenge is more determined by optimal novelty than optimal difficulty.” The message starts with minimizing difficulty and provides moderate degrees of novelty/challenge. Players generally like to feel successful and competent when playing something new. “Task repetition causes fatigue and should be accompanied by a regular drip of novelty. After a time, difficulty itself can provide this source of novelty. Both low and high-performance players benefit from a feeling of suspense during a close game. In general, challenge appears to be fun because it is interesting. Keeping games easy and interesting may be more important than balancing difficulty” (p. 1037). For this reason, our maxim “not too hard, not too easy” might be restated as “not too hard, not too boring.”

If the challenge remains too easy, this can be demotivating, and there is no novelty in continually getting it correct. Nuthall (2007) showed that students already know 40–50% of everything taught in classes. Berry (2023) found that teachers’ beliefs about engagement were more a function of doing the work regardless of standards of challenge or performance. Perhaps a more common issue in classes is that the work is neither challenging nor novel but too repetitive, sometimes too much novelty and not enough challenge, and sometimes too focused on doing and completing with little sense of a ladder of challenge.

Wilson et al. (2019) claimed there was an 85% rule for optimal learning. They showed that the learning rate is maximized when the difficulty is adjusted to keep learning accuracy around 85%. If learning is fast, then mixing in more difficult examples may be best, and such learning should involve spaced practice (over time and not all at once), supporting the importance of novelty. The message is that error rates of about 20% are tolerated, provided we scaffold the learning from easier to harder, quickly finding the sweet 80–85% success rate; and then the flow or balance between anxiety and boredom can be attained. To accomplish this, however, there needs to be a view that errors (the 15%) are opportunities to learn, not sources of embarrassment.

Goal commitment

A commitment to the success criteria is not as critical as making them appropriately challenging, but it helps. Without commitment, there can be little to no motivation to expend effort, and the teacher needs to rely on compliance, coercion, or external rewards to engender effort. In contrast, too high a commitment can lead to stress and anxiety.

There are many factors involved in committing to goals, such as conscientiousness, need for achievement, mastery goal orientation, preference for challenge, and influence of peers (Carroll et al., 2009). Peers influence goal commitment through pressure, modeling, and competition. The commitment can be moderated by whether the goal is publicly set, is a group goal, or is seen as too easy, too hard, or too boring (Epton et al., 2017). Public commitment to goals has a greater effect than private commitment. The bottom line is that students’ commitment helps in the investment of learning, thus the transparency, understanding, level of challenge, and acceptance of the success criteria become even more important.

Mastery learning

The claim underlying mastery learning is that all children can learn when provided with clear explanations of what it means to master the material being taught. In the mastery model, a level of mastery is set (e.g., 80% pass in the assessment developed at the outset before teaching commences), this level of mastery is then held constant, and time and ways of progress should be allowed to vary. The student does not proceed to new material until the mastery level is attained (as in many video games). Additional teaching and opportunities to learn are provided to those not reaching mastery.

The meta-analyses are quite dated, possibly indicating the loss of interest in implementing mastery learning – although many newer computer teaching methods often have mastery as a core part of their method. The effects are high in elementary, high school, and college (*Guskey & Gates, 1986; *Kulik et al., 1990) and particularly strong for lower-ability students (d = 0.96) (*Kulik et al., 1990).

Mastery teaching does increase the amount of instructional time required by, on average, 25% (because in nonmastery teaching, teachers move on even if students have yet to master). *Slavin (1987) argued that the more positive effects might be due to the extra time students spend reaching mastery than regular classroom instructional time – which perhaps points to the importance of extra time for many students. Slavin concluded that what matters most is the quality of teaching, follow-up, and materials; the extra time needed; and the quality and amount of corrective feedback.

Worked examples

Another form for demonstrating to students what success looks like and, thus, what the goal could be for their own learning is to provide worked examples. Worked examples typically consist of a problem statement and the appropriate steps to the solution. The defense for providing worked examples is that they reduce the cognitive load for students by concentrating on the processes that lead to the correct answer and not just providing a solution (which may or may not be correct). A typical example of worked examples consists of an introductory phase (exposure to the example), an acquisition or training phase, and a test phase (assessing the learning). Although there may be slight deviations, most studies follow this pattern, including a pretest or introduction of a delayed acquisition or test phase. The difference in average effects in the two meta-analyses is quite different. *Crissman used 62 studies, mainly from the 1980s to early 2000s, with high school and above in the sciences with a higher 0.57, and *Wittwer and Renkl (2010) used 18 studies in the sciences from 2,000+ with a minimal 0.16 effect.

The effects were higher for lower-ability students and twice as high for those with no previous knowledge or ability in a domain compared to those with prior knowledge – which hints at using them at the beginning of the session. Both reviews found high effects for fading (0.60), which is the omission of specific steps in the problem to ease the transition from worked examples to problem-solving. *Wittwer and Renkl did find that instructional explanations added to worked examples had a significant and positive effect on the acquisition of conceptual (0.36), strategic (0.31), and situational (0.37) knowledge. The value of worked examples might be constrained, therefore, to a method of demonstrating success criteria but nothing more. This initial exposure followed by the gradual fading of solution steps is more likely to lead to self-directed problem-solving by the learner.

Cognitive complexity and cognitive task analysis

Two aspects of success criteria and learning intentions are the complexity of the learning desired across the lessons and an analysis of cognitive demands in the tasks assigned to students. Cognitive task analysis typically involves teachers identifying effective task performance, the strategic decisions and procedures, the necessary content knowledge, how ideas to complete the task are related to each other (such as the hierarchical relationships among tasks or content), how learners reason and choose learning strategies as they solve problems, what students’ need to pay attention to and what they need to minimize when solving the task, and the potential errors and misconceptions that can disrupt learning the task. It is more than simply observing, analyzing behaviors, or reporting on what we think are the cognitive attributes of the task. But it is a deep understanding of the why and the how as learners make sense of things. Task analysis is often developed utilizing direct observations of students learning, interviewing novice and expert students or listening to their think alouds about how they think and process tasks, having a deep content analysis of existing documents (manuals, curriculum, progressions), doing process tracing capturing learning during task performance, doing conceptual mapping, having a deep knowledge of how we learn, and using prior studies of the progressions involved in successfully completing a task (and not completing it to ascertain the errors and false paths some students may take). A tall ask.

For both the surface (content, facts, ideas) and deeper (relational, conceptual, transfer), there can be varying levels of cognitive complexity. If this is not transparent to the teacher or student, then there is a major gap in the teaching process and possible confusion for the student. Further, intentional alignment is not only among the learning intentions, success criteria, and feedback at the varying levels of cognitive complexity but also on the optimal learning strategies to be invoked in the learning (see Chapter 11).

Thus, in preparing learning intentions, success criteria, and feedback, there also needs to be attention to a task analysis of the level of cognitive complexity desired from the lesson and in the student learning and choice of learning strategies (the latter outlined in the next chapter). Such task analysis involves identifying the cognitive skills or mental demands needed to perform a task proficiently.

For example, Shute et al. (2020) devised a sophisticated cognitive complexity model for teachers. Their DNA (decompose, network, assess) computer-based model does the following:

extracts and organizes knowledge and skills from subject matter experts, and thus can provide the basis for instruction, and SMART (Student Modeling Approach for Responsive Tutoring) uses the resulting structured curriculum elements as the basis for assessment, cognitive diagnosis, and instruction. Thus, DNA relates to the “what” to teach, and SMART addresses the “when” and “how” to teach it.

This process identifies the symbolic knowledge (the knowing-that), procedural skills (the problem-solving), and conceptual knowledge, (knowing-how).

The two meta-analyses on cognitive tasks have high effects, but the focus is on university students or adults. *Tofel-Grehl and Feldon (2013) reviewed cognitive task analysis in many contexts: medical, school, government, military, and industry. Lessons based on cognitive task analysis differed from noncognitive task analysis courses (d = 0.87) with larger effects in the military (1.44), industry (1.06), medical (0.73), and academic settings (0.67), and lower in government nonmilitary settings (−0.17). The effects were high for procedural knowledge (0.93) and declarative knowledge outcomes (0.83). *Lee (2004) specifically investigated cognitive task analysis within education settings but located only nine studies, all with university and adults but no school-age students. The overall effect was 1.70 between pre- and post-measures, and the effects double in university compared to other adult settings and similarly high for novices, intermediate, and expert students.

This method needs more curriculum mapping, sequencing, and lesson preparation investigation. The central message is the need to include deeper analysis of the procedures, the thinking and learning strategies, and the modeling of what, when, and how to teach declarative and procedural knowledge. This can make the learning intentions and success criteria much richer and more focused on the learning processes, and assist with feedback about which aspects of the cognitive thinking are in need of refined and enhanced teaching.

Feedback

A major theme from VL1 was the power of feedback. After word counts for the educators (“teachers,” “schools,” “students”) and common words (“the,” “a,” “of”), the next most common word in VL1 after “reading” (n = 299) was “feedback” (251 mentions). The research on feedback has been greatly accelerating since *Kluger and de Nisi’s 1996 meta-analysis (for a review, see Hattie et al., 2016). Some countries have made declarations about the year of feedback (I wonder what they do the following year, drop it?), and many articles and books have been written about the topic. Sadly, many ignore the massive variation in the effects of feedback. Too many overly focus on giving feedback and increasing the quantity of feedback, and forget the critical importance of teaching students and teachers to listen, receive, interpret, and engage in actionable feedback. There are many resources on the power and variability of feedback (Hattie & Clarke, 2019), and this section overviews the major claims (Table 11.2).

Table 11.2 Summary statistics for feedback variables

Feedback
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Feedback
9
1,005
61,164
3,117
0.51
0.08
4

Feedback (self)
2
103
9,480
103
0.14
0.08
2

Feedback (from students)
12
300
26,732
429
0.47
0.08
4

Feedback (reinforcement and cues)
3
108
21,053
211
1.01
0.08
3

Feedback (tasks and processes)
11
161
14,552
222
0.63
0.12
3

Feedback (from tests)
3
72
6,627
90
0.41
0.03
2

Feedback (comments vs. grades)
3
27
2,485
103
0.19
0.05
2

Feedback (technology)
2
62
6,291
92
0.55
0.08
2

Feedback (timing)
2
134
29,352
134
0.89
0.08
3

Productive failure (errors)
3
89
8,165
213
0.39
0.10
3

Peer- and self-grading
8
179
18,290
233
0.54
0.16
3

Formative evaluation
6
256
138,747
905
0.40
0.06
4

Alternative assessment methods
4
78
17,496
88
0.67
0.09
3

Peer assessment
2
91
8,375
178
0.41
0.09
2

Frequent testing
3
191
17,579
386
0.39
0.05
3

Effects of testing
7
728
7,081,731
2,168
0.59
0.03
5

Questioning
9
270
22,590
330
0.49
0.10
3

Classroom discussion
1
42
3,866
42
0.82
0.00
1

Total/average
90
3896
7,495,477
9,044
0.54
0.06
3

In an earlier database, all effect sizes within a meta-analysis were included (not just from the overall average but from all moderators), and this led to an average of 0.79 (Hattie & Timperley, 2007). However, the dependencies created by using multiple effects within a study led to revising the database to include only one (the overall mean effect size). Based on all possible moderators, the average was 0.79, and based only on the overall mean, it was 0.53 (showing the problems of dependencies within studies, and I thank Kiendl Korbinian from the University of Siegen for noting this). This overall effect of 0.53 is similar to the overall effect from the mean of 24 meta-analyses on feedback (0.48) (*Wisniewski et al., 2020).

Wisniewski et al. (2020) also highlighted the major variability in the effects of feedback. The effects of feedback were higher on cognitive (0.51) compared to motivational (0.33) outcomes and higher for high-information feedback (0.99), corrective feedback (0.46), and reinforcement or punishment feedback (0.24). The effects were high for teacher to student (0.47) and student to teacher (0.35) but much higher among students (0.85). Feedback is more effective the more information it contains, particularly about where-to-next (Hattie & Clarke, 2019). It does not matter if the feedback is delivered orally or in writing. Wisniewski et al.’s conclusion was that “feedback, on average, is powerful, but some feedback is more powerful” (p. 14).

To understand this variability, we began by building a model based on three feedback questions and three levels of the nature of feedback. Feedback is the answer to three questions: Where am I going? How am I going? And where to next? And feedback can be focused on the task, process, or self-regulation phase of learning. There have been other models (see Carless & Boud, 2018; Lipnevich et al., 2016; Shute, 2008; Wiliam, 2010), and others have investigated and critiqued our model (Ekecrantz, 2015). Brooks et al. (2019) confirmed that the preponderance of feedback in classes is at the task level, and the least amount of feedback addresses where-to-next. For many students, if there is no where-to-next, their reaction is that they did not receive any feedback (sometimes despite extensive comments, corrections, and criticisms). Brooks et al. developed a matrix that helps explain the Hattie and Timperley model (Figure 11.1).

Figure 11.1 Feedback matrix of the three questions and levels of processing feedback.

The task, process, and self-regulation overlap with the knowing-that (surface), knowing-how (deep), and knowing-with (transfer), and feedback can have differential effects depending on where the student is working along this learning continuum. For example, feedback about content is more powerful when the class focuses more on the surface or knowing-that aspects of the topic. Correct/incorrect, reteaching and building confidence in the needed content are important in this case. As students move from a focus on the content to the deeper conceptual phase of learning, different forms of feedback are required. Students then need feedback about the relations between ideas and error detection with the teacher. When we want students to take more agency about the next steps in their learning, feedback needs to promote student self-regulation, planning, and evaluative skills. The timing of feedback can also vary along this continuum. It needs to be more immediate in the content phase but more powerful if delayed in the self-regulation phase.

Feedback thrives on errors in conditions where students perceive low rather than high levels of threat to self-esteem, but praise tends to dilute the effect of feedback (as students focus on the praise about them or their effort and not the information about improvement or the where-to-next in the task).

The preponderance of the research has focused on the quantity and quality of teacher feedback. There are too few studies and no meta focused on how students hear, understand, and receive actionable feedback. Mandouit (2020) interviewed students about their reactions to simulated feedback examples. The students were most welcoming of where-to-next feedback, which led to four specific questions: Where have I done well? Where do I need to improve? How can I improve? And what can I do next time? The students also conversed much about the affective aspects of feedback and were clear that their motivational reactions tempered how they received and interpreted feedback. Too often, the intended teacher feedback was not heard, and even when it was heard, it was not understood. When students understood feedback, they valued it most when it led to actions about the next steps, and then they wanted help to move to these next steps.

Student rating of the quality of teaching

All the meta-analyses, but one, on the relation of the quality of teaching to learning come from student ratings of teachers by college and university students. While common in the tertiary setting, the feedback provided to teachers rarely leads to improvements in their teaching or the effectiveness of their courses.

Student ratings have been hotly contested, although most studies show that they are reliable, trustworthy, and valid (Marsh et al., 2019). Some have argued that they are merely popularity contests. In a meta-analysis of studies on the influence of an instructor’s personality on student ratings of instruction, *Abrami et al. (1982) found that instructor expressiveness substantially affected student ratings but had a small effect on student achievement. In contrast, lecture content had a considerable impact on student achievement but a negligible effect on student ratings.

*Cohen (1981) found an average correlation of r = 0.43 between overall teacher rating and student achievement. Students were reasonably accurate in evaluating their own progress in the course (r = 0.47). The relations were highest for perceived teaching skill and knowledge of the subject (r = 0.50), planning and organizing the course (r = 0.47), rapport with students (r = 0.31), and feedback (r = 0.31). Given the value of student evaluations as an index of teaching and their own learning, It is, therefore, discouraging to note that teachers do not seem to learn so much from this important source of information. *Cohen (1980, 1981) found that feedback from student ratings has a medium contribution to the improvement of teaching at the college level (d = 0.38). The effects were amplified when feedback was extended through such processes as consultation and feedback booklets (see also Hampton & Reiser, 2004).

The lack of student evaluations in elementary and high schools should be a significant concern. The stakes are too high to depend on beliefs that quality is high or that the students are too immature to have meaningful judgments about the effects of teachers on their learning. A key is not whether teachers are excellent, or even seen to be excellent by colleagues, but whether they are excellent as seen by students – the students sit in the classes, know whether the teacher sees learning through their eyes, know the quality of the relationships, and have views as to whether they are progressing or not. In addition, the visibility of learning from the students’ perspective needs to be known by teachers to understand better what learning looks and feels like for the students. Finally, the quality of student evaluation instruments is critical, although the meta-analysis shows little difference in the findings, regardless of the student evaluation questionnaire used.

Marsh et al. (2019) showed that there is much overlap between university and high school students valuing similar attributes of teachers. High school students most highly discriminated between the effective and less effective teachers on learning/value, enthusiasm, exams/grading/assignments, group interaction, individual interaction, breadth, organizations/clarity, planning, workload/difficulty, but they added five new scales that were relevant to the secondary school setting: relevance, choice, cognitive activation, classroom management, and technology. This is an emerging and exciting new field, and we have much to learn to interpret the student evaluations and show that this leads to improvement.

There is a recent book-length discussion about using student evaluations in schools (Rollett et al., 2021) based on a well-sourced model of student feedback that leads to an instrument that is the focus of many chapters in the book. Their meta-analysis (*Rohl, 2021) of longitudinal effects of receiving feedback from high school students shows the power of providing supportive feedback to teachers, interpretation of this feedback, and consequential programs of professional development tied to the scales (d = 0.21). The book also notes that most teachers have positive attitudes toward receiving such feedback (see Gobel et al., chapter 11).

One of the major themes of this sequel is that teachers need to hear about their impact – and it seems somewhat ironic that teachers will listen to feedback from external adult observers who come into their classes to conduct fleeting observations of their teaching and rarely can view the teacher as students do – but rarely systematically listen to their students. Yes, they listen to student questions and comments, they listen to the information in assignments and assessments, and they listen to students who volunteer to answer questions and engage in dialogue, but they miss many who aim to be silent, are not engaged with the teacher, and maybe are confused about the lesson.

Seeking and providing feedback: from technology

Three stories. First: A class of 10-year-olds was being taught by an anthropomorphic-looking robot. It was impressive, the students were most engaged, and there was palpable excitement as they interacted with the robot. At the end of the class, they were asked – what did you like about the robot teacher? Their comments focused on the lack of relationships – it did not know I was the naughty student, it did not treat me differently because I have special needs, I could ask it to explain many times without getting angry, and so on. If, as Nuthall (2007) claims, we all need three to five exposure to ideas to be successful in learning, bring on the robots. If we are to avoid low expectations or expectations that degrade the student involvement and experience, then bring on the robots.

Second: When developing the e-asTTle assessment model for the New Zealand government (https://e-asttle.tki.org.nz), we experimented with computer-based scoring of essays. At that time (early 2000s), they were okay, but the expense was prohibitive. But we had purchased a license to explore, so I approached my faculty, suggesting that we permit students to enter their assignments into the scoring program and receive the feedback, as this would augment teaching to improve their understanding. The reaction was swift and loud: this will never happen as this is cheating.

Third: Turnitin Feedback Studio is one of many computer-based systems that provide students feedback via automated and teacher-generated options. A survey examining over 2,000 students’ perceptions of instructor feedback found that students more strongly prefer “suggestions for improvement” rather than “praise or discouragement” (Turnitin, 2014). In a subsequent survey on students’ perceptions of feedback, Turnitin (2015) found that students consider instructor feedback to be just as critical for their learning as doing homework, studying, and listening to lectures. From the 1,155 responses, a majority of students (78%) reported that receiving and using teacher feedback is very or extremely important for learning. Therefore, any automated system has to provide improvement feedback at least and augment or improve on teacher feedback.

We randomly selected 3,204 high school and university student assignments that were submitted, received feedback, and were resubmitted for final grading in Turnitin (Hattie et al., 2021b). There was an average of 14 comments per essay, with comments about where-to-next, need of support, referencing, unclear comments, grammar, and praise. The improvement was substantial, d = 0.70, between first and final grade. The best predictors were the frequency of where-to-next comments, the most negative was the presence of praise comments (because there is so little where-to-next information in praise), and improvement was not related to the number or length of comments. More is not necessarily better, but providing feedback on how to improve (where to next) mattered most. As noted earlier, students prefer feedback that assists them in knowing where to learn next and how to attain this where-to-next status, and it seems some technology solutions may maximize this form of feedback (Brooks et al., 2019; Mandouit, 2020). No matter whether the feedback was more general or quite specific, where-to-next feedback seemed to be heard and actioned by the students. However, it is critical to note that where-am-I-going feedback and how-am-I-going feedback can help inform where-to-next feedback.

There have been two meta-analyses of the effect of computer provision of feedback. *Van der Kliel et al. (2015) reported that elaborated feedback was more effective (0.61) than knowledge of results (0.32) or knowledge of correct response (0.05). They noted that the majority of the elaborated feedback has lowest impact at the task level (−0.06), but metacognitive information or questions in combination with knowledge of results or knowledge of correct responses was most powerful. They concluded that elaborated feedback in the form of subtle guidance might generally be more effective than highly specific guidance. These results are in line with claims by Narciss and Huth (2004) that *not providing knowledge of correct responses leads to deep processing and subsequently better learning” (p. 28). They found no effect of timing – immediate or delayed, although *Kulik and Kulik (1988) found that immediate was more effective than delayed. There was a caution: delayed can be more powerful if knowledge of results or correct responses is offered immediately, allowing time for any wrong information to fade. It may be that elaborated responses are then better delayed.

Seeking and providing feedback: questioning

Freire (2018) aimed to create a society “in which a minority does not exploit the majority: to create a society where, for example, asking questions would be a common, daily activity” (pp. 61–62). Questions from students can be powerful feedback to teachers about their impact and for students to engage in the wonder and the depth, aiding them in consolidating deeper learning and clarifying their learning.

However, it is also noted that teachers dominate the class regarding questions, and questions are the second most dominant teaching method after teacher talk (Cotton, 1989). The estimates of questions per day vary between 100 and 350 (Brualdi, 1998; Clinton & Dawson, 2018), and the responses from the teacher questions to the students are typically some form of recalling facts, judgment, or correction; primarily reinforcing in nature; affirming; restating; and consolidating student responses (often requiring less than a three-word answer). In over 80% of the teacher questions, the teachers already know the answer (Shomoossi, 2004); most of the questions are closed with low cognitive demands on students (see Asay & Orgill, 2010; Erdogan & Campbell, 2008; Nystrand et al., 2001); and the most common form of interaction is still the IRE cycle: initiate a question, get a response, and evaluate the response. The reason for so much questioning relates to the mind frames about teaching and learning held by many teachers – that is, their role is to impart knowledge and information about a subject, and student learning is the acquisition of this information through processes of repetition, memorization, and recall: hence the need for much questioning to check that they have recalled this information.

But not all questions are the same. Boaler and Brodie (2004), for example, examined mathematics classes in terms of nine categories of questions: gathering questions requiring an immediate answer, questions inserting terminology, questions exploring points, probing questions getting students to explain thinking, questions generating discussion, question linking and relating among ideas, questions extending thinking, orienting questions that help students to focus on key ideas, and questions establishing context. Despite this richness, more than 95% of the questions asked by teachers were of the first form. Nystrand et al. (1998) have argued that the two most powerful questions were uptake questions, whereby teachers validate particular student ideas by incorporating their responses into subsequent questions, and authentic questions – questions asked to obtain valued information or questions without prespecified answers, not simply to see what students know and don’t know.

Authentic questions, like uptake, also contribute to coherence. By asking authentic questions, teachers elicit students’ ideas, opinions, and feelings, making students’ prior knowledge and values available as a context for processing new information. (Thus, they) … contribute to the coherence of instruction by enlarging the network of available meanings in the class.

(p. 7)

Nystrand et al. (2001) also noted that the rates of student questions were lower in low-track classes (along with more off-task behavior and less completing of assignments). In these classes, discourse tends toward monologue and fewer uptake or authentic student questions – a vicious cycle. They showed that there needs to be more authentic questions in these classes focused on the substance of the material students were studying. *Samson et al. (1987) found that factual prequestions can facilitate learning provided they are directly related to the texts or materials to be learned (and have a negative effect when the questions asked are unrelated to the text material to be learned, *Hamaker, 1986; *Redfield & Rousseau, 1981).

One of the downsides of questioning whole classes is the low participation by many students in the question answering. *Randolph (2007) found that response cards were powerful in increasing participation (by 50%) and were preferred (82%) to hand-raising. Response cards are cards, signs, or items that all students in the class simultaneously hold up to display their responses to questions or problems. Students using response cards, on average, performed d = 1.08 higher on quizzes and d = 0.38 higher on tests than students in the hand-raising condition.

Seeking and providing feedback: a focus on errors

Feedback thrives on errors, challenge demands errors, learning is about error detection, error repair is a teaching focus, and failure needs to be a learner’s best friend. Errors can serve as important feedback information, indicating where the student’s thinking and knowledge are not adequately developed. But students and teachers often shun errors, usually in the name of not negatively impacting a student’s self-esteem; peers can be nasty, brutish, and short to those showing they do not know. When students make errors in classroom discussion, it is usually quickly corrected (by the teacher or by the teacher asking a peer), and many students soon learn that it is better to look like you know and hope you do not get asked. Thus, avoiding and withdrawing are successful tactics to maintain their sense of themselves as learners.

From 24 studies, *Keith and Frese (2008) found d = 0.44 for teaching that encouraged compared to teaching that did not encourage errors during learning. The effect was much higher when the clarity of the feedback was high (0.56) compared to low (0.19) after the students had been exposed to the material (0.56) rather than in the earlier phases of first exposure (−0.15) and when in the knowing-how/with (0.80) compared to the knowing-that phase (0.20).

Many learning models explicitly include attention to errors and failure. For example, Piaget (1952) discussed cognitive disequilibrium, which occurs when learners encounter a situation contrary to their current mental model (such misconceptions have been studied particularly in science learning), and then students are challenged or instructed until they either assimilate those differences into their mental model or modify their mental model according to the new information. Productive failure invites students to solve, usually ill-structured or complex, problems before instruction to evaluate their disequilibrium; try and invent multiple solutions; and realize what they need to learn from the subsequent instruction (*Sinha & Kapur, 2021).

*Darabi et al. (2018) reported d = 0.43 from studies of productive failure, and they concluded that compared to other strategies, the failure-based strategy showed a positive and significant result. *Sinha and Kapur (2021) used 53 studies of productive failure teaching leading to an overall effect of 0.36, with effects increasing to the extent the program had fidelity with the four major principles: activation and differentiation of prior knowledge in relation to the targeted concepts, attention to critical conceptual features, explanation and elaboration of these features, and organization and assembly of the critical conceptual features into the targeted concepts. Hence, starting to teach a new concept with problem-solving so that students can experience disequilibrium and learn more about what they do not know may make them more focused on the value of the subsequent instruction. The bottom line is that understanding, detecting, and wallowing in errors and failure can be critical to learning.

Seeking and providing feedback: classroom discussion

In the earlier work on classroom discussion, Flanders (1965) observed the two-thirds principle: that for about two-thirds of most school lessons, somebody is talking; that about two-thirds of this talking is done by the teacher; and that two-thirds of teacher talk is Direct Instruction. The initiation-response-evaluation (teacher asks a question, a student responds, the teacher evaluates) dominates (Nystrand et al., 2001). Class discussion is rare (less than 15 seconds a day in English and about 30 seconds in social studies classes), and more than 60% had no discussion at all, and another 15% had no more than 2 minutes per day (Nystrand et al., 1998). These findings were mirrored in the MET study (Kane et al., 2013). In this study of 3,000 classes, 60% of the classes did not have a single classroom discussion over three months.

Goodwin et al. (2020) analyzed the classroom talk of 19,944 fourth and fifth grades and their 745 language arts teachers. They noted four dominant modes: teacher explaining, teacher questioning, teacher encouraging student talk, and teacher summarizing learning. The one factor that did not relate to student learning was increasing the proportion of student talk. This was because too much student talk was about their perceptions of quantity, not quality, and was less on critical thinking. Student talk was particularly a concern when it involved summarizing learning because such summaries were about the teacher’s (not students’ beliefs) about the what (not why) of the learning, and too much was tied to knowledge acquisition than critical thinking.

Classroom discussions (not IRE types) can be a way for teachers to hear feedback and for students to engage in a dialogue of learning, hearing alternative interpretations, and becoming more engaged in the learning process (Alexander, 2020; Mercer et al., 2019). Classroom dialogue is, thus, dynamic, relational, and mediated by experiences with others. Specjal (2022) reviewed many approaches to creating this dialogue, such as the following:

Collaborative reasoning (Anderson et al., 1998), which aims to encourage students to use reasoned discourse as a means for choosing among alternative perspectives on an issue, invites students to adopt a position on the issue and generate reasons that support their position.

Questioning the author (Beck et al., 2020), which asks students to examine the author’s position as an expert, to read against the text, and to presume that difficulties they encounter when reading challenging texts are not necessarily attributable to their own inadequacies.

Book club (Raphael & McMahon 1998) comprises four elements: reading, writing, engaging with this reading and writing in small-group discussion (book clubs), and sharing with the class.

Computer-based knowledge forum, developed by Scardamalia and Bereiter (2006), invites students to collectively investigate problems by adding individual notes, graphs, build-ons, and rise-above-it notes.

The Scratch community bulletin board (Roque et al., 2016) allows students to receive feedback from others. The Web of Inquiry system supports students in generating theories, constructing hypotheses, and conducting and analyzing their investigations.

Many approaches effectively promoted students’ comprehension, especially those categorized as more efferent in nature. That is, those interventions in which discussion gives prominence to reading to acquire and retrieve particular information. This stance focuses on “the ideas, information, directions, conclusions to be retained, used, or acted on after the reading event” (Rosenblatt, 1978, p. 27). Most methods were effective at increasing student talk and decreasing teacher talk, but these increases in student talk did not necessarily result in concomitant increases in student comprehension. Increasing talk is not enough; a particular kind of talk is necessary to promote comprehension. Very few approaches effectively increased literal or inferential comprehension and critical thinking and reasoning about text. They argued that talk is a means and not an end.

It is one thing to get students to talk to each other during literacy instruction but quite another to ensure that such engagement translates into significant learning. Simply putting students into groups and encouraging them to talk is not enough to enhance comprehension and learning; it is but a step in the process.

Specjal (2022) noted that what teachers and students do before, during, and after discussions matter. The before or priming stage requires at least a basic understanding of the text before participating in a discussion. During discussions, teachers should aim to promote meaningful interactions by challenging student responses, scaffolding, and demonstrating norms of discussion. Teachers can also gradually release responsibility to the students so they can co-construct and inter-think understandings of the text. This discussion requires students to externalize their cognitive processes (think aloud), and teachers can listen to how students are processing the text to help inform them of their next teaching moves. After the discussion is the time for summative feedback regarding the student’s discourse practices and understandings about, around, and with the text. With the COVID pandemic experience, these student-based talk opportunities have blossomed (White & Collins, 2021).

Seeking and providing feedback: self- and peer-grading

There are many advantages of self-grading; as it requires students to make judgments about the work metacognitively, it can increase deeper understanding of the success criteria and sense of progress toward success, make expectations about performance more transparent and explicit, decrease any cynicism about grading, and provide another opportunity to apply what they have learned. *Sanchez et al. (2017) reported that students who engaged in self-grading (0.34) and those who engaged in peer-grading (0.29) performed better on subsequent tests than did students who did not. Both self- and peer-grading correlated high (r = 0.67) with teacher grades. The number of times they were graded made little difference, whether they were trained in grading (e.g., by the use of practice or examples) or used rubrics. Active engagement in the grading process results in beneficial effects for student learning.

Seeking and providing feedback: formative evaluation

There have been many claims for the power of formative evaluation, but sadly, many miss the power and value of these methods – more often by confusing formative and summative with testing. The terms “formative” and “summative” were introduced by Michael Scriven 50 years ago (Scriven, 1967, 1991), and his distinction between these terms was more related to interpretations and time – as illustrated by Bob Stake’s maxim: When the cook tastes the soup, it is formative; when the guests taste the soup, it is summative. Bloom (1969) transferred these terms into formative and summative assessment. This complicated the meaning and led to lesser impact, as many began to see the major focus was testing. Take any test, and it can be interpreted formatively (during the learning) or summatively (at the end phase of the learning) – it is not that the test is formative or summative; it is the nature of the evaluation. There are many other ways, besides testing, that enable teachers to make interpretations of learning quantity and quality during or at the end of a series of lessons.

Too often, testing is used as a motivator for students. If their major motive is framed around what is in the test, then there is the belief that learning is knowing lots, and no wonder external motivators (such as testing) may have short but few longer-term gains. There have been many who have noted that much classroom assessment adds little to the learning process. Black and Wiliam (1998), for example, reviewed 578 publications relating to the role of assessment in learning and concluded that classroom assessment is as follows:

typically encourages superficial and rote learning, concentrating on recall of isolated details, usually items of knowledge which pupils soon forget, … teachers do not generally review the assessment questions that they use and do not discuss them critically with peers, so there is little reflection on what is being assessed.

(p. 17)

At that time, we argued that Black and Wiliam needed to go further and should have argued that effective learning and assessment occurs not by being concerned about the testing itself but by integrating the assessment into the cycle of challenging goals (e.g., student receptivity to the challenges) and feedback (Hattie & Jaeger, 1998). This cycle requires teachers who are thinking, reflective, enthusiastic, passionate, and knowledgeable about their impact on students, and who are open to feedback from their students about their impact and thence improve their teaching. It certainly would be more helpful to use the terms “formative” and “summative” interpretations to remove it from testing (as this is but one method of feedback).

The four meta-analyses on formative methods vary in their focus. *Fuchs and Fuchs (1986) examined the teachers’ effects of systematic formative evaluation and found that this technique increased achievement for students with a mild learning disability (d = 0.70). The formative evaluations were effective across student age, treatment duration, frequency of measurement, and special-needs status. When teachers were required to use data- and evidence-based models, the effect sizes were higher than when data were evaluated by teacher judgment. In addition, when the data was graphed, effect sizes were higher than when data was simply recorded.

*Kingston and Nash (2011) were highly critical of the quality of the research on the efficacy of formative assessment. They, thus, ended up with only 13 studies and a 0.25 effect. *Kim (2005) demonstrated that students improved in their learning after introducing performance formative assessments (0.39), and these effects were larger the longer the assessment had been implemented. *Lee et al. (2018) reviewed 32 studies that conducted formative assessments to improve student learning in K–12 classrooms and found d = 0.28. It seems a major limitation of most of this work is that formative and summative have been too closely tied to the testing movement. While there are gains, it is likely to be much higher for many other methods of formative and summative evaluation or formative student activity. Such thinking, formatively and summatively, is a form of evaluation thinking.

Seeking and providing feedback: frequency of testing

The effects show that testing is most worthwhile when there is at least one semester between high-stakes tests, when tests include conceptual and problem-solving questions rather than only factual, and when there is feedback. Otherwise, the effects drop considerably (e.g., *Gocman, 2003; feedback, 0.62; no feedback, 0.30).

Seeking and providing feedback: effects of testing

Most meta-analyses showed high positive effects from testing. For example, *Phelps (2019) used 149 studies on test frequency and feedback, and reported an effect size of 0.84. He found higher effects for testing at the end of a series of lessons than during the teaching, where students need to construct answers rather than select (e.g., multiple-choice), and when feedback was combined with follow-up testing or higher-stakes testing. This latter, he suggested, was probably because of the motivating influence of high-stakes testing, when students are more likely to listen to feedback and are more likely to act on it (but it may then have fewer longer-term effects). *Fuchs and Fuchs (1986) found these effects were higher when examinees were tested by familiar than unfamiliar examiners (hinting at a stereotype effect; see earlier).

Seeking and providing feedback: alternative testing methods

There is support for more comprehensive methods of testing, such as accumulating portfolios. Both *Gozuyesil and Tanriseven (2017, d = 1.01) and *Basol and Erbay (2017, d = 0.83) supported the use of portfolios, and there were no differences if they were paper-based or e-portfolios. Progress monitoring involves more frequent, shorter assessments checking for student understanding. Prenkert (2015) found a small effect (0.17), but the effect was higher in math (0.54) than in reading (0.14) – probably reflecting the emphasis on the quick probes to check for knowing-that, more commonly the focus on math assessment.

Conclusions

There are high effects when there is the intentional alignment of the curriculum content, learning intentions, success criteria, assessment and feedback, and a deep understanding of the nature of the cognitive engagement of the lessons. The request is for the optimal proportion of more content and deeper thinking. The aim is to intentionally align these in terms of the learning intentions, success criteria, and nature of feedback and assessment across the lessons. This is the essence of a deliberate intent to teach. At all times, teachers need the evaluative mind frame to choose their teaching methods, monitor the fidelity of the student’s progression to success, and seek feedback from students about the efficiency and effectiveness of the teaching on the impact on their learning. The fundamentals of intent to teach involves judiciously choosing not only the coverage of the curriculum, finding the optimal balance of content and depth, being attentive to the cognitive and learning strategies that may need to be taught, honing on the Goldilocks principle of challenge, seeing assessment as feedback to the teacher about their impact (about what, for whom, and how much), and celebrating the small and longer-term successes as students move toward the criteria of success.

There are high effects when there is intentional alignment of the curriculum content, learning intentions, success criteria, assessment and feedback, and a deep understanding of the nature of the cognitive complexity of the lessons. When there is misalignment, students tend to believe learning is about knowing lots and recalling for assessment purposes. There should not be a mismatch between the surface and deeper or knowing-that and knowing-how foci and the lessons’ activities, feedback, assessments, and outcomes.

The argument is not that one depth of learning (surface or deep) is more critical than the other but that the optimal proportion or balance of knowing-that and knowing-how/with across the lessons should be ensured. Indeed, it is unreasonable to expect students to relate, extend, develop deeper conceptual understanding, and transfer their learning unless they have ideas to relate, extend, problem-solve, and so on. Identifying the thinking, the learning strategies, and the ways of knowing are critical aspects of planning and then teaching. It is more than content; it is more than deeper thinking; it is intentionally aligning these in terms of the learning intentions, success criteria, and nature of feedback and assessment across the lessons. This is the essence of the Intent to Teach model.

Using backward design methods, starting with the success criteria, doing an excellent diagnosis of where students are currently performing, and then creating lessons while considering the knowing-that and knowing-how/with aspects of the learning while aligning the assessment and feedback qualities are likely to lead to the highest effectiveness and efficiencies in teaching and learning. This is what is meant by the Intent to Teach model of lesson design, implementation, and evaluation.

It is the alignment, not just the lesson design; not only in the implementation but also in the clarity of students’ understanding of the knowing-that and knowing-how/with learning intentions and success criteria. It is then planning and being aware of the numerous learning progressions, taking into account where the students start (re their skill, will, and thrill) and the need for varying time and ways to attain success. It is intentionally considering the cognitive strategies, the possible misconceptions, and the learning strategies that may need to be taught. Then it is intentionally filling this gap between where they are and where they need to be that is the focus of the teaching, feedback, and assessment. The choice of teaching methods must consider these criteria, especially the level of cognitive processing needed for success in each task or activity. As will be shown in Chapter 13, there are teaching methods that optimize the knowing-that and the knowing-how/with of the lesson goals.

A critical decision is when and how long to engage in a focus on surface, deep, or transfer activities and learning. Thence, when to stop teaching more content and move to relating, extending, and deepening this content, using methods such as inquiry-based teaching. Similarly, when to stop inquiry teaching and add more content. Teig et al. (2018) showed that there might be an optimal limit on the impact and number of inquiry activities, and the effects on students then begin to decrease. This may help explain Oliver et al.’s (2021) findings across six countries using inquiry-based science methods. Students who reported experiencing high frequencies of inquiry strategies in their classrooms consistently evidenced lower levels of scientific literacy. Maybe what is needed is a readiness measure to evaluate students’ preparedness to move from surface to deep and back (see Ngoc et al., 2020).

At all times, teachers need an evaluative mind frame to choose these methods, monitor the fidelity of the progression to success for each student, and seek feedback from students about the efficiency and effectiveness of the teaching on the impact on their learning. As will be shown in Chapter 13, the various teaching methods can be loosely classified relative to maximizing the knowing-that or knowing-how/with. Each can be used across both, but in many cases, the effectiveness reduces if the intent of the teaching method is not aligned with the knowing aspects of the success criteria. In our own VL+ work, we will not employ a person who has a fervent disposition to a particular method (e.g., “I am a DI person,” “I am a problem-based advocate”), as that is not always the method that maximizes learning. When a preferred method does not work, it is incumbent on the teacher to choose an alternative method that has the desired impact on the student. Paraphrasing a quote attributed to Einstein, insanity is using the same teaching method over and over again and expecting different results.

The fundamentals of intent to teach involve the following:

Judiciously choosing the focus of the lesson based on the curriculum.

Finding the optimal balance of content and depth (“The greatest enemy of understanding is coverage – I can’t repeat that often enough”; Gardner, 1993, p. 24). Add in cognitive capacity.

Being attentive to the learning strategies that may need to be taught.

Honing on the Goldilocks principle of challenge.

Choosing activities that enhance the content and depth, and allow for the cognitive complexity desired to be developed.

Seeing assessment as feedback to the teacher about their impact (about what, for whom, and how much).

Choosing teaching methods appropriate to the cognitive complexity and success criteria of the lessons, varying for knowing-that and knowing-how/with.

Informing, celebrating the short and longer-term successes as students move toward the criteria of success.

The next chapter will dive deeper into the optimal learning strategies, and Chapter 12 investigates explicitly various teaching methods, and through the message is the judicious choice of learning strategies relevant to the tasks being taught and the optimal teaching methods to maximize the chance of all students attaining the worthwhile success criteria.

Reflections

There are many models for lesson planning. For example, Hunter (2004) has seven phases (objective, hook, input and modeling, guided practice, independent practice, understanding, and closure), and Pollock and Tolone (2020) have five (set a goal, access prior knowledge, introduce new knowledge, apply thinking skills to this new knowledge, review initial goal). Considering the claims in this chapter, devise a school-wide lesson planning template, then check its effectiveness and efficiency to enhance student understanding of the purpose of the learning and the consequential student learning.

Review a recent set of lessons concerning the success criteria. For example, were the goals appropriately challenging, did all the students make mastery (and what strategies did you use if not), and were the goals both surface and deep/transfer related?

Make comments and grades on an assignment, hand them back, wait a day (so not just short-term memory), then ask the students to make notes about the feedback you provided. From this activity, you can see the effectiveness of your feedback in terms of what the students heard, understood, and could action.

Use Figure 11.1 to create feedback to students’ responses, aiming the feedback at the level where the students are currently working (or next level up, where appropriate) and relative to the three feedback questions.

Use the four feedback questions to review our current practice of students receiving either verbal or written feedback, from us or from each other. Do they answer the following: Where have I done well? Where do I need to improve? How can I improve? What can I do next time?

From a video or peer observation, count the student questions about their work in your class. Then work together to increase student questions, help-seeking, peer engagement. At the end of the lesson, invite students to write at least one question they still have at the foot of their work before it is handed in.

CHAPTER 12 Learning strategies

DOI: 10.4324/9781003380542-12

The day-to-day focus of schools has always included learning – how to know, how to know more efficiently, and how to know more effectively. However, learning also, as Biesta (2013) reminds us, is always about something, and (as stated many times throughout this book) the focus here is achievement, but there are many other abouts, such as social and emotional aspects, respect for self and others, and development of a sense of cultural development (the German notion of bildung; see Hattie & Larsen, 2020).

From a teacher’s perspective, key decisions relate to what students are now ready to learn, how their learning can be enabled, and how to increase the how-to-learn proficiencies of students. Of course, learning and achievement are not dichotomous; they are related – as it is through learning we come to achieve. The question in this chapter relates to identifying the most effective strategies for learning.

In our search, Greg Donoghue and I identified over 400 learning strategies (Hattie & Donoghue, 2016, 2018). Many were relabeled versions of others, some were minor modifications of others, but there remained many contenders that claimed to be powerful learning strategies. As we discovered, some strategies work well at certain times in the learning cycle but not in others. Thus, we developed a model of learning to provide a basis for interpreting the evidence from our meta-synthesis. The argument is that learning strategies can most effectively enhance performance when they assist the learner with the tasks required by the student’s present phases of learning, be it knowing-that, knowing-how, or knowing-with (cf., Pressley et al., 1989).

The model describes the learning of a whole topic, not an individual lesson. It includes three inputs that are also the three outcomes (the skill, will, and thrill); three phases of the learning process (surface, deep, and transfer), with surface and deep learning each comprising an acquisition phase and a consolidation phase; and all within an environment for the learning (Figure 12.1). The proposal is that various learning strategies are differentially effective depending on the degree to which the students are aware of the criteria of success, on the phases of the learning process in which the strategies are used, and on whether the student is acquiring or consolidating their understanding.

Figure 12.1 A model of the breadth and depth of learning.

The skill, the will, and the thrill

The skill

The first component describes the prior achievement the student brings to the task. As Ausubel (1968, p. 1) claimed “if I had to reduce all of educational psychology to just one principle, I would say this – The most important single factor influencing learning is what the learner already knows. Ascertain this and teach him accordingly.” Other influences related to the skills include their working memory, beliefs, as well as their encouragement and expectations from the student’s cultural background and home. Learning engages a multiplicity of cultural repertoires of practice, ways of knowing, and capacities to connect (Nasir et al., 2021).

There are 50 meta-analyses that have investigated the relationship among prior ability and 5 between prior achievement and subsequent achievement, and not surprisingly, these relations are high (Table 12.1). Moreover, the effects of prior achievement are lowest in the early years and highest from high school to university. One of the purposes of school, however, is to identify those students who are underperforming relative to their abilities and, thus, not to merely accept prior achievement as destiny.

Table 12.1 Summary statistics for the skill variables

Skills
No. meta
No. as studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Prior ability and achievement
50
6,825
1,867,089
21,977
0.65
0.05
3

Working memory strength
4
437
58,287
4,107
0.63
0.08
4

Executive functioning
2
314
73,552
314
0.62
0.05
3

Total/average
56
7,576
1,998,928
26,398
0.63
0.03
3

The other important skill is working memory – which relates to the amount of information that can be retained in short-term working memory when engaged in processing, learning, comprehension, problem-solving, or goal-directed thinking (Cowan, 2012). Working memory is strongly related to a person’s ability to reason with novel information (i.e., general fluid intelligence (Ackerman, 2005).

The will

Dispositions are more habits of mind or tendencies to respond to situations in specific ways. For example, Claxton (2013) claimed that the mind frame of a “powerful learner” is based on the four major dispositions: resilience or emotional strength, resourcefulness or cognitive capabilities, reflection or strategic awareness, and relating or social sophistication. These dispositions involve the proficiency to edit, select, adapt, and respond to the environment in a recurrent, characteristic manner.

There are 94 meta-analyses related to the dispositions of learning, and the average effect size is 0.42 (SE = 0.06; Table 12.2). The effect of self-control (0.66) and self-efficacy is highest (d = 0.64), followed by emotional intelligence (0.50) and positive self-concept (0.51). These influences relate to the student’s concept of self as a learner and their sensitivities to themselves and others. A key task is to develop their conception as learners, which is why our VL+ implementation program spends much effort on helping students identify the attributes of good learners (so many start by thinking a good learner is one that comes to class well prepared, sits up straight, and watches the teacher work), whereas we aim to teach dispositions of courage, failure as a learner’s best friend, and skills at taking on challenges.

Table 12.2 Summary statistics for the will variables

Will
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Self-efficacy
12
640
1,313,310
648
0.64
0.03
5

Emotional intelligence
11
567
137,458
1,845
0.50
0.11
5

Self-control
4
126
90,405
157
0.66
0.06
3

Positive self-concept
13
846
634,598
3,557
0.51
0.03
5

Concentration-persistence-engagement
10
550
354,511
1,202
0.41
0.12
5

Reducing anxiety
22
1,551
1,698,986
4,983
0.40
0.04
5

Reducing boredom
4
89
51,128
139
0.46
0.13
3

Grit
3
267
356,004
782
0.35
0.02
4

Reducing depression
3
96
59,008
113
0.30
0.08
3

Mindfulness
7
228
20,208
244
0.26
0.13
3

Growth vs. fixed mindsets
5
325
867,175
565
0.19
0.03
4

Total/average
94
5,285
5,582,791
14,235
0.42
0.06
4

Influences informing a concept as a learner include having positive attitudes to the content (0.46); reducing anxiety (0.40), which is often related to setting tasks with too challenging success criteria; and reducing boredom (which is often associated with setting too easy success criteria). It is worth noting the major foci in the anxiety and stress literature in the 1980s moved from a preoccupation with understanding levels of stress to providing coping strategies – and these strategies were powerful mediators in whether people coped or not (Lazarus & Folkman, 1984). Similarly, in learning, it is less the levels of anxiety and stress, and more the development of coping strategies to deal with the anxiety and stress. These coping strategies include the following:

Learning to cope with error and making mistakes (Frydenberg, 2010) and so reducing self-blame; and being taught to effectively regulate blame and other negative emotions (Galla & Wood, 2012).

Increasing self-efficacy, which relates to developing the students’ conviction in their own competence to attain desired outcomes (Bandura, 1993).

Increasing social support and help-seeking, reducing self-blame, and learning to cope with error and making mistakes (Frydenberg, 2010).

Increasing coping strategies to deal with anxiety and promoting confidence to tackle difficult and challenging learning tasks free up essential cognitive resources required for academic work.

There has been much discussion about students having growth – or incremental – mindsets (human attributes are malleable, not fixed) rather than fixed mindsets (attributes are fixed and invariant) (Dweck, 2006). However, the evidence (d = 0.19) shows how difficult it is to change to growth mindsets, how many of the current programs are divorced from regular classwork, and that more attention is needed to transfer any growth learning to challenging school tasks. Growth refers to the skills and dispositions to take on challenges; hence, this aligns with the highest influences of all dispositions.

The thrill

The thrill relates to the motivation for learning: there can be a thrill in learning, but for many students, it can also be dull, uninviting, and boring. Having a performance approach motivation (learning to merely pass tests or for short-term gains) is ineffective, and having a mastery approach is a little more effective (Table 12.3). A possible reason why mastery motivation approaches are not successful is that too often, the outcomes of tasks and assessments are at the surface level, and having mastery goals with no strategic sense of when to use them can be counterproductive (most students do not have sufficient time to master all asked of them). A reminder (from earlier) is that the original intent of mastery learning was that it was a means to an end in that the core knowing-that notions needed to be mastered before moving to deeper knowing, so maybe a more apt reason why mastery approach is not that high is that there are too few opportunities to use the material mastered to move to knowing-how and knowing-with. The achieving or strategic approach involves knowing when to use performance and when to master approaches; that is, when to be surface and when to be deep. This is the most effective form of motivation.

Table 12.3 Summary statistics for the thrill variables

Thrill
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Deep motivation and approach
3
165
15,186
170
0.58
0.08
2

Achieving motivation and approach
4
252
23,194
257
0.44
0.02
3

Mastery goals
8
655
133,634
898
0.20
0.09
4

Performance goals
9
625
56,513
450
0.04
0.03
4

Surface motivation and approach
4
447
132,854
452
0.00
0.08
4

Total/average
28
2,144
361,379.98
2,227
0.25
0.03
3

Teachers could develop and model the strategic approach by providing two success criteria (one about the knowing-that and one about the knowing-how/with), ensuring the lesson activities are clear as to goals, and ensuring feedback and assessment tasks are also aligned (perhaps providing two tasks, one for knowing-that and one for knowing-how). Teaching students that success can then be attributed to their effort, and investment can help cement this power of goal setting.

Physical environment

Many books and internet sites on study skills claim that it is important to attend to various features of the environment, such having as a quiet room, having no music or television, having high levels of social support, giving students control over their learning, allowing students to study at preferred times of the day, and ensuring sufficient sleep and exercise.

Despite the inordinate attention, particularly by parents, on structuring the environment as a precondition for effective study, such effects are generally relatively small (Table 12.4). For example, it seems to make no difference if there is background music (provided it is not a burden on working memory), a sense of control over learning, a time of day to study, a degree of social support, or the use of exercise. Given that most students receive sufficient sleep and exercise, it is perhaps not surprising that these are low effects; of course, extreme sleep deprivation may have marked effects.

Table 12.4 Summary statistics for the physical variables

Physical
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Time management
3
356
111,668
443
0.47
0.04
4

Environmental structuring
2
10
920
10
0.44
0.00
1

Exercise
4
279
182,106
1,939
0.16
0.00
1

Time of day to study
2
50
31,229
50
0.18
0.08
2

Background music
2
79
6,417
79
0.08
0.07
2

Student control over learning
6
226
19,671
383
0.03
0.05
3

Sleep
4
107
75,483
107
0.02
0.03
3

Total/average
23
1,107
427,495
3,011
0.20
0.03
2

The three phases of learning: knowing-that (surface), knowing-how (deep), and knowing-with (transfer)

This cognitive complexity model was introduced in Chapter 3 and highlights the importance of both surface or knowing-that and deep or knowing-how/with learning, and does not privilege one over the other but rather insists that all three levels are critical. Although the model does seem to imply an order, it must be noted that these are fuzzy distinctions (surface and deep learning can be accomplished simultaneously), but it is useful to separate them to identify the most effective learning strategies. More often than not, a student must have sufficient surface knowledge before moving to deep learning and then to the transfer of these understandings. As Entwistle (1976) noted, “The verb ‘to learn’ takes the accusative”; that is, it only makes sense to analyze learning concerning the subject or content area and the particular piece of work toward which the learning is directed and also the context within which the learning takes place (cf., Biesta, 2013). The key debate, therefore, is whether the learning is directed content that is meaningful to the student, as this will directly affect student dispositions, in particular, a student’s motivation to learn and willingness to reinvest in their learning.

To recall the SOLO model, the first two levels (one, then many ideas or perspectives) refer to developing surface knowing or knowing-that, and the latter two levels (relate and extend) refer to developing deeper knowing and procedural learning or knowing-how (see Petty, 2016, for examples in practice). Surface learning refers to studying without much reflecting on either purpose or strategy, learning many ideas from more than one perspective without necessarily relating them, and memorizing facts and procedures routinely. Deep learning refers to seeking meaning, relating and extending ideas, looking for patterns and underlying principles, checking evidence and relating it to conclusions, examining arguments cautiously and critically, and becoming actively interested in course content.

Our model also distinguishes between first acquiring knowledge and then consolidating it. During the acquisition phase, information from a teacher or instructional materials is attended to by the student, and this is taken into short-term memory. During the consolidation phase, a learner needs to actively process and rehearse the material, increasing the likelihood of moving that knowledge to longer-term memory. Finally, there can be a retrieval process at both phases, which involves transferring the knowing and understanding from long-term memory back into short-term working memory.

Acquiring surface learning

Strategies include organizing, summarizing, record keeping, highlighting, underlining, note-taking, peer explaining, and using mnemonics to help students master the surface knowledge (Table 12.5). These strategies can be deliberately taught and may be the only set of strategies that can be taught irrespective of the content. However, it may be that for some of these strategies, the impact is likely to be higher if they are taught within each content domain, as some of the skills (such as highlighting, note-taking, and summarizing) may require specific ideas germane to the content being studied.

Table 12.5 Summary statistics for acquiring surface knowledge

Acquiring surface
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Integrating with prior knowledge
1
10
920
12
0.93
0.00
1

Mnemonics
4
166
6,763
191
0.65
0.14
3

Outlining and summarizing
11
886
117,120
1,094
0.63
0.07
5

Modality effect
2
78
6,811
125
0.55
0.06
2

Rereading
2
159
5,763
159
0.50
0.06
2

Imagery
2
147
2,043
194
0.55
0.06
3

Note-taking
7
191
20,650
343
0.33
0.06
3

Underlining and highlighting
3
128
8,697
226
0.42
0.11
3

Working memory training
12
390
26,586
2,804
0.37
0.05
4

Total/average
44
2,155
195,353
5,148
0.55
0.07
3

Reducing cognitive load when teaching and learning is a key part of cognitive load theory developed by Sweller and colleagues (2008, 2010). It is important to optimize intrinsic (effort associated with the topic) and germane load (effort put into creating a permanent store of knowledge), and reduce extrinsic load (the way the information or tasks are presented to the learner) on the student. We have so much working memory capacity (e.g., most of us can hold four to seven ideas or facts in our heads at the one time), and a major step is to reduce distractions from the task or problem that we want the students to work on – such as reducing cartoons, unnecessary information, complications in text unrelated to the problem. The teacher must also be attentive to hints of overload (confusion) and underload (boredom). We can optimize intrinsic load by preteaching, providing appropriate vocabulary, segmenting lower levels of the task before more complete or complex tasks, teaching the needed cognitive complexity (see section on cognitive task analysis, Chapter 11), and forward chaining (progressively building a student’ skills in the order they will need them in the end product) (see Lovell, 2020, for an excellent elaboration of these ideas and methods).

Consolidating surface learning

Once a student has begun to develop surface knowing, it is then important to encode it in a manner such that it can be retrieved at later appropriate moments. This encoding involves two groups of learning strategies: the first develops storage strength (the degree to which a memory is durably established or well learned), and the second develops strategies that develop retrieval strength (the degree to which a memory is accessible at a given point in time). Encoding strategies are aimed to develop both but particularly emphasizes on developing retrieval strength. Both groups of strategies invoke an investment in learning, and this involves the tendency to seek out, engage in, enjoy, and continuously pursue opportunities for effortful cognitive activity. Although some may not enjoy this phase, it involves a willingness to practice, be curious, and explore again, and a willingness to tolerate ambiguity and uncertainty during this investment phase. In turn, this requires sufficient metacognition and a calibrated sense of progress toward the desired learning outcomes. Strategies include practice testing, spaced vs. mass practice, teaching of test-taking, interleaved practice, rehearsal, maximizing of effort, help-seeking, time on task, learning of how to receive feedback, and deliberate practice (i.e., practicing with the help of an expert or receiving feedback during practice). Deliberate practice involves specific attention to overcoming a weakness or break new ground or practice with feedback and not merely overpracticing (which could lead to overlearning wrong, poor, or weak habits and knowing).

The investment of effort and deliberate practice is critical in this consolidation phase, as are the abilities to listen, seek, and interpret the feedback that is provided (Table 12.6). During this consolidation phase, the task is to review and practice (or overlearn) the material. Such investment is more valuable if spaced over time rather than massed. Rehearsal and memorization are valuable – but note that memorization is not so worthwhile during the acquisition phase. The difficult task is to make this investment in learning worthwhile, adjust the rehearsal as it progresses in light of high levels of feedback, and not engage in mindless drill and practice. These strategies relating to consolidating learning are heavily dependent on the student’s proficiency to invest time on task wisely, practice and learn from this practice, and overlearn such that the learning is more readily available in working memory for a deeper understanding.

Table 12.6 Summary statistics for consolidating surface knowledge

Consolidating surface
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Rehearsal and memorization
4
151
13,898
151
0.71
0.00
2

Help-seeking
3
99
14,136
99
0.73
0.11
2

Deliberate practice
3
161
13,689
316
0.49
0.08
3

Effort management
1
15
1,381
15
0.77
0.00
1

Spaced vs. mass practice
5
510
183,962
1,115
0.59
0.05
4

Giving/receiving feedback
9
1,005
61,164
3,117
0.51
0.08
4

Interleaved practice
3
180
11,003
374
0.46
0.07
3

Time on task
12
375
68,974
522
0.46
0.10
4

Practice testing
5
744
224,457
902
0.49
0.04
5

Teaching test-taking and coaching
12
328
96,865
477
0.24
0.09
4

Total/average
57
3,568
689,527
7,088
0.55
0.05
3

Acquiring deep learning

Students who have high levels of awareness, control, or strategic choice of multiple strategies are often referred to as being self-regulated or having high metacognition levels. Thus, self-regulated students become like teachers, as they have a repertoire of strategies to apply when their current strategy is not working, and they have a clear conception of what success on the task looks like. More technically, Pintrich (2000) described self-regulation as “an active, intentional process whereby learners set goals for their learning and then attempt to monitor, regulate and control their cognition, motivation, and behavior, guided and constrained by their goals and the contextual features in the environment.” These students know the what, where, who, when, and why of learning, and the how, when, and why to use which learning strategies. In addition, they know what to do when they do not know what to do. Self-regulation strategies include elaboration and organization; strategy monitoring; concept mapping; and metacognitive strategies, such as self-assessment, help-seeking, self-regulation, and elaborative interrogation (the skill of asking, “Why is this so?”).

Nearly all the strategies at this phase are powerful in enhancing learning (Table 12.7). The ability to elaborate and organize, monitor the uses of the learning strategies, and have a variety of metacognitive strategies are the critical determinants of success in this phase of learning. A major purpose is for the student to deliberately activate prior knowledge and then make relations and extensions beyond what they have learned at the surface phase.

Table 12.7 Summary statistics for acquiring deep knowing

Acquiring deep
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Elaboration and organization
1
50
4,602
50
0.75
0.00
1

Self-judgment and reflection
2
77
6,980
77
0.69
0.08
2

Elaborative interrogation
2
304
6,740
304
0.59
0.05
3

Meta-cognitive strategies
18
863
1,313,625
2,175
0.52
0.11
5

Strategy monitoring
2
235
21,629
235
0.54
0.08
2

Self-regulation
20
1,621
427,838
3,401
0.51
0.08
5

Total/average
45
3,150
1,781,413.6
6,242
0.60
0.04
3

Consolidating deep learning

Once a student has acquired surface and deep learning of a subtopic or concept to the extent that it becomes part of their repertoire of skills and strategies, we may claim that they have automatized the learning. In many senses, this automatization becomes an idea or chunk of learning in its own right. Then the cycle continues as proximate chunks are learned from surface idea to deeper knowing; then that chunk becomes a surface idea and so on (Pegg & Tall, 2010). There is a series of learning strategies that develop the learner’s proficiency to consolidate deeper thinking and to be more strategic about learning. These include self-questioning, self-monitoring, self-explanation, self-verbalizing the steps in a problem, seeking of help from peers and peer tutoring, collaborative learning, evaluation and reflection, problem-solving, and critical thinking techniques.

At this phase of learning a topic, the power of working with others is most apparent (Table 12.8). This involves skills in seeking help from others, listening to others in the discussion, and developing strategies to speak the language of learning. The seeking of help, for example, is more powerful toward the beginning rather than end of the learning, especially in the deep consolidating phase, and when the help is autonomy-oriented (e.g., seeking help) than oriented to providing complete solutions (Koo et al., 2022). Through such listening and speaking about their learning, students and teachers realize what they do deeply know, what they do not know, and where they are struggling to find relations and extensions. An important strategy is when students become teachers of others and learn from peers, as this involves high levels of regulating, monitoring, anticipating, and listening to their impact on the learner.

Table 12.8 Summary statistics for consolidating deep knowing

Consolidating deep
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Seeking help from peers
2
42
3,866
42
0.68
0.13
1

Help seeking skills
3
99
14,136
99
0.73
0.11
2

Classroom discussion
1
42
3,866
42
0.82
0.00
1

Evaluation and reflection
1
54
4,970
54
0.75
0.09
1

Self consequences
1
75
6,903
75
0.70
0.08
2

Problem-solving teaching
11
639
59,305
1,313
0.61
0.08
5

Self-monitoring
3
203
16,858
203
0.50
0.08
3

Self-explanation
5
120
10,868
186
0.55
0.10
3

Self-verbalization and self-questioning
7
360
22,218
1,397
0.58
0.08
4

Cooperative learning
37
1,376
144,534
2,258
0.53
0.08
5

Becoming a teacher (peer tutoring)
18
890
78,515
1,624
0.66
0.06
5

Collaborative
12
955
91,734
1,082
0.45
0.07
4

Total/average
101
4,855
457,771
8,375
0.62
0.08
3

There has been much research confirming that teaching help-seeking strategies are successful, but how this strategy works in classrooms is more complex. Teachers have to welcome students seeking help, and there needs to be knowledgeable others (e.g., peers) from whom to seek the help. Too often, students left in unsupported environments can seek and gain incorrect help and not know the help is incorrect. Ryan and Shin (2011) also distinguished between adaptive help-seeking (seeking help from others, such as an explanation, a hint, or an example, that would further the learning and promote independent problem-solving in the future) and expedient help-seeking (seeking help that expedites task completion, such as help that provides the answer and is not focused on learning). They showed that adaptive help-seeking from peers declines and expedient help-seeking increases during early adolescence. Further, increases in expedient help-seeking were associated with declines in achievement, but changes in adaptive help-seeking were unrelated to achievement. The key is for teachers to teach adaptive help-seeking, ensure the help is dependable and correct, and see this as more of a student than a teacher skill. Help-seeking needs to be welcomed before it can have an effect.

Transfer

There are skills involved in transferring knowledge and understanding from one situation to a new situation. Indeed, some have considered that successful transfer could be thought of as synonymous with learning. There are many distinctions relating to transfer: near and far transfer, low and high transfer, transfer to new situations and problem-solving transfer, and positive and negative transfer. Transfer is a dynamic, not static, process that requires learners to actively choose and evaluate strategies, consider resources and surface information, and when available, receive or seek feedback to enhance these adaptive skills. Reciprocal teaching is one program specifically aiming to teach these skills; for example, Scardamalia and Bereiter (2006) have developed programs in the teaching of transfer in writing, where students are taught to identify goals, improve and elaborate existing ideas, strive for idea cohesion, present their ideas to groups, and think aloud about how they might proceed.

Similarly, Schoenfeld (2010) outlined a problem-solving approach to mathematics that involves the transfer of skills and knowledge from one situation to another. Marton (2014) argued that transfer occurs when the learner learns strategies that apply in a certain situation such that they are enabled to do the same thing in another situation when they realize that the second situation resembles (or is perceived to resemble) the first situation. He claimed that not only sameness, similarity, or identity might connect situations to each other, but also small differences might also connect them. Learning how to detect such differences is critical for the transfer of learning. As Heraclitus claimed, no two experiences are identical; you do not step into the same river twice.

*Sala and Gobet (2016) specifically investigate the effects of near and far transfer on working memory after training programs, on students with learning disability (near = 0.26, far = 0.06), and on those not so classified (near = 0.49, far = 0.08). They concluded that schools should focus on domain-specific knowledge rather than general and allegedly transferable skills. Other metas on transfer in schools led to a different conclusion – there is an emerging literature that transfer can be intentionally taught. Feuerstein (1980) guides the student through steps of a process successfully, helping where necessary; then asks, “How did you do that?”; gets a sound response; then asks, “Where else could you use that process?”; and if they cannot think of a response, suggests and asks for another. When they can think of a relevant transfer, ask, “Why does it transfer?”

*Marzano et al. (2000) found high effects on transfer from teaching students to identify similarities and differences (1.32). The transfer model promoted by Marton (2014) seems to be supported in that a key in teaching for transfer involves understanding the patterns, similarities, and differences in the transfer before applying the strategies to new tasks. Marton argued that transfer occurs when students learn strategies that apply in a certain situation such that they are enabled to do the same thing in another situation to the degree that they realize how the second situation does (or does not) resemble the first situation. Learning to detect differences and similarities is the key that leads to the transfer of learning.

But it is the right proportions of teaching knowing-that, knowing-how, and knowing-with that matter. This is illustrated using the Stern et al. (2021) “ACT Learning Transfer” model. The acquire phase is intentionally teaching the knowledge and skills – the organizing concepts that we then wish them to use in transfer. The connect phase is finding relations between the ideas and deeper conceptual understanding. The transfer phase involves teaching students “to look for similarities and differences to evaluate new information or situations and assimilate this into our existing network of thoughts and experiences” (p. 21). It involves using pattern-seeking, generalizing from a specific instance to a broader rule, using this knowledge to unlock the new situation, moving beyond being stuck in specific instances, and priming students to make connections, explore relationships, and innovate. “The more we can coach students to apply general patterns and conceptual insights to dissimilar, real-world contexts, the more flexible and creative their problem-solving.”

Rayner et al. (2013) investigated transfer in math and looked at various teaching transfer methods (d = 0.80 from teaching transfer in math). Schema activation instruction outperformed the other conceptual categories of instructional interventions. We need more such meta-analyses on the effects of the strategies and the circumstances when they are most likely to impact positively.

Conclusions

The efficiency of the strategies depends on fully understanding the skills, wills, and thrills that student brings to the class.

The aim is to enhance the will (e.g., the confidence and willingness to reinvest in more and deeper learning), the thrill (e.g., the emotions associated with successful learning and the curiosity and the willingness to explore what one does not know), and the skills (e.g., the content and the deeper understanding).

Timing and intention matter.

A critical skill is teaching students the skills of detecting the nature of success of the learning desired and knowing where they are in the learning cycle so they better optimize the choice of teaching and learning strategies – this is the essence of self-regulation.

This differential effectiveness of learning strategies helps explain why generic study skills programs have low transfer into various disciplines, and instead, they need to be taught in the context of the subject domain.

There is an important distinction between being first exposed to learning and consolidating this learning.

Transfer is a major outcome of learning and is more likely to occur if students are taught how to detect similarities and differences between one situation and a new situation before they try to transfer their learning to the new situation. These skills can be taught.

Preparation for future learning involves opportunities to try our hunches in different contexts, receive feedback, engage in productive failure, and learn to revise our knowing based on feedback.

There needs to be an emphasis on knowledge-building rather than knowledge-telling and systematic inquiry based on theory-building and disconfirmation rather than simply following processes for how to find some result.

A major claim of this model is that the efficiency of the strategies depends on fully understanding what each student brings to the class and the nature and timing of the learning for a given topic:

If the success criteria require the retention of accurate detail (surface learning), then the surface learning strategies will be more effective than higher-level strategies.

However, if the intention is to help students understand the context (deeper learning) to apply it in a new context (transfer), then knowing-how strategies are needed.

Students need multiple strategies depending on where they are in the learning cycle (Figure 12.1), particularly when their first choice does not make the difference. Teachers may need to be more explicit in how their learning intentions and success criteria relate to the nature of cognitive complexity (from surface to deep to transfer), as this often indicates the optimal learning strategies.

The student walks into each class with prior skills, dispositions, and motivations; these are the core to any lesson’s outcomes. There is an intertwining role of skill, will, and thrill – as both inputs and outputs of learning. The aim, thus, is to enhance the will (e.g., the confidence and willingness to reinvest in more and deeper learning), the thrill (e.g., the emotions associated with successful learning, the curiosity, and the willingness to explore what one does not know), and the skills (e.g., the content, learning skills, and the deeper understanding).

Timing and intention matter. Identifying the top ten learning strategies is impossible, as success depends on whether the teacher’s intent is knowing-that or knowing-how and whether the material is first exposed or consolidated. It also depends on the intentional alignment of the success criteria, feedback, teaching methods, and assessment.

If this alignment is not made explicit by the teacher, then a critical skill is to teach students the skills of detecting the nature of the learning desired and where they are in the learning cycle so they better optimize the choice of learning strategies – this is the essence of self-regulation. So often, they turn to the assessment tasks to begin to understand what level of complexity is required or recall the nature of the feedback on previous assessments to understand what the teacher values.

In summary, to acquire surface learning, it is worthwhile knowing how to summarize, outline, and relate the learning to prior achievement; and then consolidate this learning by engaging in deliberate practice, rehearsing over time, and learning how to seek and receive feedback to modify this effort. Deep understanding requires planning, evaluating, and learning to monitor the use of one’s learning strategies; consolidating deep understanding calls on the strategy of self-talk, self-evaluation, and self-questioning, and the seeking of help from peers. Such consolidation requires the learner to think aloud, learn the language of thinking, know how to seek help, self-question, and work through the consequences of the next steps in learning. Transferring learning to new situations involves detecting similarities and differences between the old and the new problem or situations.

We have called this the intentional teaching and learning model. This differential effectiveness of learning strategies helps explain why generic study skills programs have low transfer into various disciplines. Instead, they need to be taught in the context of the subject domain. Wiliam (2014, p. 2), for example, noted as follows:

mathematics proof, critical thinking might involve ensuring that each step follows from the previous one (e.g., by checking that there has not been a division by zero). In reading a historical account, critical thinking might involve considering the author of the account, the potential biases and limitations that the author may be bringing to the account, and what other knowledge the reader has about the events being described. The important point here is that although there is some commonality between the processes in mathematics and history, they are not the same. Developing a capacity for critical thinking in history does not make one better at critical thinking in mathematics. For all of the apparent similarities, critical thinking in history and mathematics are different, and they are developed in different ways.

There is an important distinction between being first exposed to some learning and consolidating this learning. This distinction is far from novel. Shuell (1990) distinguished among initial, intermediate, and final phases of learning. In the initial phase, the students can encounter a “large array of facts and information that are more-or-less isolated conceptually … there appears to be little more than a wasteland with few landmarks to guide the traveler on their journey towards understanding and mastery.” Students can use existing schema to make sense of this new information or be guided to have a more appropriate schema (and thus, experience early stages of concept learning and relation among ideas); otherwise, the information may remain as isolated facts or be linked erroneously to previous understandings. In the intermediate phase, the learner begins to see similarities and relationships among these seemingly conceptually isolated pieces of information. “The fog continues to lift but has not completely burnt off.” During the final phase, the knowledge structure becomes well-integrated and functions more autonomously, and the emphasis is more on performance or exhibiting the outcome of learning.

Transfer is a major outcome of learning and is more likely to occur if students are taught how to detect similarities and differences between one situation and a new situation before they try transferring their learning to the new situation. These skills can be taught. Too many (particularly struggling) students over-rehearse a few learning strategies (e.g., copying and highlighting) and apply them regardless of the demands of new tasks. Further, this ability to notice similarities and differences over content is quite different for novices and experts. We do not simply learn from experience but also from mistakes.

Preparation for future learning involves opportunities to try our hunches in different contexts, receive feedback, engage in productive failure, and learn to revise our knowing based on feedback. The aim is to solve problems more efficiently and to let go of previously acquired knowledge in light of more sophisticated understandings – and this can have emotional consequences: failure to change strategies in new situations has been described as the tyranny of success. It is not always productive for students to try the same thing that worked last time. Hence, there may need to be an emphasis on the Popper knowledge-testing (i.e., seeking evidence that you are wrong), leading to knowledge-building, rather than simply on knowledge-telling. This requires systematic inquiry based on testing knowledge and disconfirmation leading to theory building, rather than merely following processes to find some result.

Reflections

When we reviewed 20,000 hours of classroom transcripts for examples of (a) a teacher thinking aloud showing their strategies of learning when solving a problem, (b) a teacher asking a student to think aloud about their strategies of learning, or (c) a teacher providing a different learning strategy when the student’s solution did not work – we found zero. So what would the results be in your classroom or school?

Share strategies for finding out students’ prior knowledge/current understanding before launching into a new lesson.

Nuthall (2007) found that students need three to five exposures to new concepts, regardless of prior ability. So in reviews of your lessons via video or audio – how many opportunities for learning do you provide?

With colleagues, devise five to seven learning attributes you wish for your students. Devise a progression, teaching resources, and measures for each. Check for student understanding of this learning language throughout the school.

High student self-efficacy is crucial for effective learning. Identify potential barriers to such confidence in your school – such as privileging academic success over progress to achievement, ability grouping and other comparative systems, low expectations, and students’ beliefs about their potential progress in lessons.

Effective discussion involves students being activated as learning resources for each other. Read the section on “Consolidating deep learning” and discuss how students can be organized to explain to or coach each other when needed so that those with deeper understanding can teach others, and how to teach students to think aloud, critique, and engage in peer discussions. Discuss how often students should change who they sit next to, to optimize their opportunities to benefit from a range of different social and cognitive learning experiences.

Read paragraphs four to six of the conclusion, then create lists for the elements of each learning stage: surface learning acquisition, deep understanding acquisition, and transfer to new situations. Discuss how these can be used as a guide for teachers’ planning and how students can be taught to know and use these learning strategies.

CHAPTER 13 Teaching strategies

DOI: 10.4324/9781003380542-13

I have often (over) claimed that it is more than just about how we teach but about the impact of our teaching. This adage is not meant to underestimate the importance of decisions about teaching but to ask for a greater focus on the impact of teaching. The intent is not just to cover the curriculum, make the tasks enjoyable and real world, and engage all students in the doing of the tasks. The main intent is to maximize the impact on the learning lives of the students: their skill, will, and thrill of becoming educated. The fitness-for-purpose arguments explored in the previous chapter and the learning intentions arguments in Chapter 11 inform the optimal interpretations about the choice of teaching methods. Reiterating Kenny Rogers, there is a time to play ’em and a time to hold ’em.

There are so many teaching methods, and most teachers have large toolboxes of methods. There are also advocates of specific methods, usually proclaiming it worked for them so why not for others. Some schools proudly proclaim they are problem based or deep learning schools as if this is the only, the best, or necessarily the optimal way to teach and learn. As will be seen, it depends when you use various methods, and surely, if the method does not have the desired impact, we should change it to another. But the message is not to be eclectic and trial many different teaching methods in the hope of impact.

There are over 16 million hits on “teaching methods” in Google. Many professional learning sessions debate and promote particular teaching methods and numerous books and articles on the topic. We are asked to teach like champions (Lemov, 2012), teach like your hair’s on fire (Esquith, 2007), teach like Finland (Sahlberg, 2021), and teach like a pirate (Burgess, 2012). I want you to teach to maximize your impact on the learning lives of all students.

There are few taxonomies of teaching methods. Marzano (2016) identified 332 teaching methods, which he classified into Direct Instruction lessons (presenting new content), practicing and deepening lessons (developing skills, strategies, and processes fluently and accurately, where teaching assists in connecting student understanding of the topic with previously learned content and in facilitating the practice of essential skills), and knowledge application lessons (teaching that encourage students to move beyond the content and begin generating their own claims and conclusions). This is somewhat akin to moving from knowing-that, knowing-how, and knowing-with (surface to deep to transfer). But when we classified 332 teaching methods in Marzano’s Compendium as to which best fit with the learning model (from Chapter 12), only 32 were directly classifiable (see Figure 13.1). Most cross boundaries, and while this is not necessarily a problem (learning is rarely compartmental or linear), it does introduce another confusion in the intentional alignment of success, teaching, and learning for the students.

Figure 13.1 Teaching methods aligned to the fi ve parts of the VL model.

Beck (1998) surveyed 25 teacher education textbooks and noted that “it seems rather surprising that so little attention has been given to establishing a more uniform and common classification system” (p. 37). As a result, he developed five clusters of strategies (and also three more presentation methods via performance, technology, and grouping; Figure 13.2).

Figure 13.2 Beck’s fi ve clusters of teaching strategies.

The claim is that we may need to choose our teaching methods for the current topic depending on whether the lesson’s focus is on the knowing-that, knowing-how, or knowing-with phase. This necessitates multiple methods as so few current teaching methods succeed across all five levels of learning. Therefore, we must be intentional about the right choice for the right students at the right moment. Note, the use of intent to teach implies a deliberate, purposeful, and considered decision to use a method with a clear eye to what students bring to the class, the criteria of success, and the intentional alignment of the choice of teaching methods to the assessments, success criteria, and learning processes (as described in Chapters 11 & 12).

The various teaching methods investigated via meta-analysis have been loosely grouped into one of two parts of the learning model (knowing-that, and knowing-how and knowing-with), but as noted, many can be used at all levels – the intent is more critical than the method.

Methods aiming for surface or knowing-that

Table 13.1 Summary statistics for teaching knowing-that

Teaching for knowing-that
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Direct Instruction
8
3,052
563,150
10,392
0.56
0.08
5

Explicit teaching strategies
21
5,976
1,584,597
13,333
0.63
0.09
5

Cognitive behavioral programs
1
5
460
5
0.29
0.00
1

Lectures
3
273
27,296
614
−0.26
0.08
4

Co-/team teaching
8
327
32,933
494
0.21
0.12
3

Response to intervention
8
180
21,907
689
0.73
0.11
4

Adjunct aids
15
408
121,824
790
0.64
0.09
4

Immediacy
1
16
5,437
16
0.16
0.00
1

Background music
2
79
6,417
79
0.08
0.07
2

Humor
1
20
4,801
21
0.04
0.00
1

Total/average
68
10,336
2,368,822.3
26,433
0.31
0.06
3

Direct Instruction

When VL1 was first published, the developers of Direct Instruction (DI) emailed a nice message about how pleased they were to see support for their model. I replied but suggested that given that the method often gets bad press and is often woefully misunderstood, maybe a name change would help. There were no further emails; I presumed this idea was not welcomed. But Direct Instruction, when implemented with fidelity, is most effective (Hornby & Greaves, 2022). It must not be confused with didactic teaching, scripted lessons, or an overemphasis on surface knowing. There are also many variants, such as explicit instruction. However, many of these advocates are dismayed to be associated with Direct Instruction and often are among the greatest critics, thus muddying the waters even more.

Effective DI involves seven major steps:

Before the lesson is prepared, the teacher should have a clear idea of the learning intentions. What, specifically, should the student be able to do, understand, and care about due to the teaching?

The teacher needs to know what success criteria of performance are expected, when, and what students will be held accountable for from the lesson/activity. Likewise, the students need to be informed about the standards of performance.

There is a need to build commitment and engagement in the learning task. In the terminology of Direct Instruction, this is sometimes called a hook to grab the student’s attention (although some argue that the hook can distract and is better after the students have mastered the content and skills). The aim is to put students into a receptive frame of mind, to focus student attention on the lesson, and to share the learning intentions.

There are guides to how the teacher should present the lesson – including notions such as inputting, modeling, and checking for understanding. Input refers to providing the information needed for students to gain knowledge or skill through lectures, film, tapes, video, pictures, and so on. Modeling is where the teacher shows students examples of what is expected as an end product of their work. The critical aspects of modeling are explained through labeling, categorizing, and comparing to exemplars of what is desired. Finally, checking for understanding involves monitoring whether students have got it before proceeding. It is essential that students practice doing it right, so the teacher must know that students understand before they start to practice. If there is any doubt that the class has not understood, the concept or skill should be retaught before practice begins.

There is the notion of guided practice. This involves an opportunity for each student to demonstrate their grasp of new learning by working through an activity or exercise under the teacher’s direct supervision. The teacher moves around the room to determine the mastery level and provide feedback and individual remediation as needed.

There is the closure part of the lesson. Closure involves those actions or statements by a teacher designed to bring a lesson presentation to an appropriate conclusion; the part wherein students are helped to bring things together in their own minds to make sense of what has just been taught. “Any questions? No. OK, let’s move on” is not closure. Closure is used to cue students to the fact that they have arrived at an important point in the lesson or the end of a lesson; to help organize student learning; to help form a coherent picture; to consolidate, eliminate confusion and frustration, and so on; and to reinforce the major points to be learned. Thus, closure involves reviewing and clarifying the key points of a lesson, tying them together into a coherent whole, and ensuring the student will apply them by ensuring they become part of the student’s conceptual network.

There is independent practice. Once students have mastered the content or skill, it is time to provide reinforcement practice. It is provided on a repeating schedule so that the learning is not forgotten. It may be homework or group or individual work in class. It is important to note that this practice can provide for decontextualization: enough different contexts so that the skill or concept may be applied to any relevant situation and not only the context in which it was originally learned. For example, if the lesson is about inference from reading a passage about dinosaurs, the practice should be about inference from reading about another topic, such as whales. The advocates of Direct Instruction argue that the failure to do this seventh step is responsible for most students’ failure to apply something learned.

In a nutshell, the teacher decides the learning intentions and success criteria, makes them transparent to the students, demonstrates them by modeling, and evaluates if they understand by checking for understanding; student practice is first guided and then becomes more independent; and the teacher retells what they have told students by tying it all together with closure.

DI is especially effective at surface-level knowledge, and while it is also effective at deeper levels, it is perhaps harder to implement with fidelity. The effects are similar for the more low-level word attack (d = 0.64) and also for high-level comprehension (d = 0.54), and similar for elementary and high school students (*Adams & Engelmann, 1996; *White, 1988).

Explicit teaching strategies

This is a mixed bag of methods, and many meta-analyses specifically focus on students with learning difficulties or second language learners. The major message is that the more explicit the instruction, the more effective. *Norris and Ortega (2000) specifically asked whether an explicit or an implicit approach was more effective for short-term L2 instruction for second language learners. Across 49 studies, 70% of the instructional treatments involved explicit techniques, while only 30% involved implicit techniques. The explicit methods had higher effects (1.22) than the implicit (0.69) methods. One of the reasons for the superiority of explicit methods was that implicit methods were typically operationalized in relatively restricted ways. In contrast, explicit treatments often involve combinations of several instructional components. Similarly, *Spada and Tomita (2010) found higher effects for explicit (d = 0.75) than implicit (0.35) instruction with English as a second language students and that explicit instruction effects increased most on delayed post-tests (1.0 vs. 0.55).

There has been a healthy debate about explicit vs. implicit or unguided methods of instruction. For example, Sweller et al. (2007) argued that unguided or minimally guided learning is significantly less effective and less efficient than guidance specifically designed to support the cognitive processing necessary for learning. Such unguided learning is based on the notion that “people learn best in an unguided or minimally guided environment where they are primarily asked to mimic the problem-solving activities of experts or learn and discover collaboratively with others” (p. 80). Such methods include problem-based learning, experiential learning, discovery learning, inquiry learning, and constructivist learning.

advocates of this approach imply that instructional guidance that provides or embeds learning strategies in instruction interferes with the natural processes by which learners draw on their unique, prior experience and learning styles to construct new, situated knowledge that will achieve their goals.

(p. 76)

Instead, the cited meta-studies outline the case for more direct or explicit instruction. With unguided methods, students often become lost, become frustrated, and have false starts, and their confusion can lead to misconceptions. In particular, many have used Sweller’s cognitive load model to argue that the unguided exploration of a highly complex environment may generate a heavy working memory load that is detrimental to learning. This unguided method can generate a much larger cognitive load and lead to poorer learning. Note an important codicil: “Only when learners have sufficiently high prior knowledge that provides ‘internal’ guidance does the advantage of guidance begin to reduce” (2007, p. 82; cf., Mihalca & Mengelkamp, 2020).

The major message is that there may be a time during the instructional cycle for the teacher to release responsibility from more explicit teaching when (a) gradually, the students have sufficient knowing-that knowledge to move to relating ideas and making deeper conceptual links; (b) the task asked of them is more open-ended, is more problem-based, and requires multiple learning strategies to resolve; (c) there is opportunity to interact, critique, and evaluate solutions with peers; and d) there is a safety net for students to seek more guided instruction by teachers. For example, class discussion can be unguided for a short time, but the teacher may need to intervene to prompt discovery and correct misconceptions should they arise.

The major reason for introducing the notion of intent to teach is to avoid these claims about one teaching method vs. another (e.g., explicit vs. unguided, DI vs. PBL). Instead, it highlights the importance of choosing a method that is intentionally aligned with the success criteria of the lesson – particularly when focused on knowing-that, knowing-how, or knowing-with outcomes. Many have used explicit instruction, but it, too, has suffered the bipolarity wars, and I have aimed for a term (intent to teach) that implies that the choice of teaching method needs to be aligned with the nature of the learning task (see Chapter 11).

Cognitive-behavioral programs

*Karch et al. (2013) investigated cognitive training programs to improve the basic functions that underpin cognition, such as attention, memory, and executive functions. They found a positive but low effect (0.29) on enhancing academic achievement. In addition, they noted that such training is but one element of a comprehensive program.

Lecturing

Flaig et al. (2019) noted the many advantages of lectures (nearly all with tertiary students), such as that they are cost efficiency, have a high degree of teacher control, and are much more straightforward to prepare primarily because they have one flow, fewer interruptions, and are less responsiveness to what is happening in students thinking during the lecture. The evidence, however, is that lectures are not that different from other teaching methods for acquiring surface knowledge but much less effective for deeper understanding, such as knowing-how. Using 67 studies from 8,336 tertiary students, they found zero effects for declarative knowledge (−0.04), and thus, lectures are not effective; but even lower effects for procedural knowledge (−0.53), noncognitive learning outcomes (−0.23), and student satisfaction (−0.17). They concluded that lectures are as good as other methods for introductory classes when the goal is declarative knowledge but not so good for classes with deeper learning goals.

Co-teaching/team teaching

Co-teaching involves two teachers working together in a single physical space to deliver instruction. There are many variants: one teaching, one assisting, station teaching, parallel teaching, alternate teaching, and team teaching. The typical claims in favor of team teaching include that it considers the strength of both teachers; it spurs creativity, as teachers are forced to plan together and can spark off each other; and it allows for more individual attention to students (Robinson & Schaible, 1995).

*Willett et al. (1983) included team teaching in their meta-analysis of effects in science and did not find much support (d = 0.06). *Murawski and Swanson (2001) investigated co-teaching with regular and special education teachers of mainstreamed students. They only found six articles with d = 0.31. *King-Sears et al. (2021), from 26 studies, reported d = 0.47 from co-taught students compared to students in special education classrooms. There were lower effects in elementary schools (0.25) compared to middle schools (0.56) and high schools (0.52), and higher in language arts (0.60) than in mathematics (0.42). *Khoury (2014) used 20 studies with d = 0.44, with similar effects in elementary (0.28) and secondary (0.19), and in language arts (0.28) and math/science (0.28); and higher effects in mainstream (0.66) than in resources rooms (0.44). Losinski et al. (2019) noted that there were few studies, and too many were of low quality, and they reported a zero effect (d = −0.01).

One of the major limitations is the nature of co-teaching implementation – whether it is one teacher teaching at a time, one teaching and one acting like a one-on-one with special needs, or two teachers working in tandem.

Response to intervention

Response to intervention (RTI) is a multitiered approach for providing individualized instructional services and interventions at increasing intensity levels to students who experience difficulties in progressing. Tier 1 involves an intervention to the whole class with careful monitoring of student progress such that those not enhancing their learning, those falling behind, or those with special needs receive additional or differentiated instruction by the regular teacher, with assistance, if needed, by a multidisciplinary teaching of intervention specialists, counselors, or specialized educators. Tier 2 involves more intense individualized or small-group approaches that supplement core instruction, which is provided in combination with the Tier 1 regular class interventions. If the tracking data shows no meaningful acceleration, then Tier 3 is where more individualized intense instruction is provided. Success for Tier 2 and 3 is when the student returns to Tier 1 conditions and makes similar progress to peers.

With few exceptions, the RTI research has mainly focused on the efficacy of each component in the model but not on the value of the RTI process as an integrated whole. There is much variability in the overall effects, thus begging for more control over the nature of the interventions. *Burns and Symington (2007), using 57 studies, found a large effect of 1.15 on student outcomes, with much greater effects when the team included university (1.32) compared to field non-university-based teams (0.54). After omitting some very high outliers, *Burns et al. (2005) found an effect of 0.96 on student outcomes and 1.53 in reducing referrals to and subsequent new placements in special education. They found the opposite of *Burns and Symington: The effect was higher when RTI was implemented within schools and systems (1.73) than implemented by university faculty for research (0.47). Torres (2016) found higher effects for Tier 1 interventions (1.32) than for Tier 2 (0.67), and there were insufficient studies to analyze Tier 3. The effects were highest on reading fluency (1.54) and phonological awareness and phonics (1.66), and lower on reading comprehension (0.54). *Tran et al. (2011) also found higher effects on the fundamental skills (word identification, 1.53; naming speed, 0.74; phonological awareness, 0.82; vocabulary, 1.19; and higher on comprehension, 1.43). They also noted the variability in the average number of weeks for intervention (19 weeks), with an average session time of an hour and 32 sessions.

Many have identified problems with the model, and it is quickly being replaced with a multitiered support system (MTSS). Some have estimated that about one in five students need support beyond what they have received in their classrooms; these students are not necessarily the students behind in their learning. In addition, the RTI model does not always well serve students with behavior or mental health concerns; although disruptive students can be too quickly assigned to Tier 3 to get them out of the classroom (Hettleman, 2019), some personify the problem as the student, whereas it may be the teacher’s effectiveness; there can be too much emphasis on specialists undertaking the assessment and Tier 3 interventions and leaving little behind in the interventions by the regular teacher; and the model can be expensive and not scalable to improve how teachers work with these students.

The MTSS process focuses on providing instruction and intervention with the classroom teacher. MTSS is more general education and less assessment oriented. It is framed in a culture of collaboration with teachers to enhance student learning than have specialists working in class or take students out for more intensive remediation. MTSS deliberately addresses academic, social, behavior, attendance, and emotional issues, and involves curriculum design, positive interventions and support, teacher learning and collaboration, and family involvement. In MTSS, professional learning communities are responsible for examining student data to make decisions about who needs Tier 2 and 3 interventions, are engaged in problem-solving protocols for instructional decision-making, evaluates the effects of these interventions (quality of implementation and student progress), and thus, puts primacy of working with teachers to work with students (Sugai & Horner, 2009). MTSS follows the 5D model of implementation (Chapter 4): great diagnosing, designing evidence-based interventions, attending to delivery implementation, doubling back to check fidelity and impact, and doubling up to then scale up the solutions (particularly within the regular class).

The three tiers in MTSS are more about the support the teacher and student receive, and not focused so much on the students themselves – students are not classified as Tier 1, 2, or 3, but receiving Tier 1, 2, and 3 support. Within Tier 1, the universal support to all students is reviewed and improved, and the resources needed for teachers to be successful are a focus. Within Tier 2, resources can be provided to support teachers in working with groups of students who need more opportunities and extra assistance to benefit from the core programs in the class. Finally, within Tier 3, there can be more intensive individualized support plans, but they still aim to return and work with students in the regular class.

*Shepley and Grisham-Brown (2019) investigated the effects of MTSS, using 14 studies with preschool students, leading to an overall effect of 0.31. There were higher effects for engaging (0.59), enhancing social skills (0.55), reducing challenging behavior (0.51), listening comprehension (0.50), and rhyming (0.40); and lowest for print knowledge (0.24), expressive language (0.16), receptive language (0.05), and phonological awareness (0.05). They concluded that MTSS effectively improved social-emotional outcomes in preschool-aged children when significant time for professional development and one-on-one coaching is provided to teachers.

Adjunct aids

Adjunct aids typically involve pictures, diagrams, and others inserted in textbooks to support learning. It seems that it is not so much the presence of adjunct aids that enhances achievement but how and where they are used in the texts and the student’s level of sophistication when using adjunct aids. Adjunct aids act as signals highlighting key information (Mayer, 2019), with *Richter et al. (2015) reporting d = 0.35 in favor of aids that signal compared to nonsignal multimedia learning material. The effects were higher for students with a low to medium than a higher level of domain-specific prior knowledge. Adjunct aids can assist learning when they function to attract and direct attention and highlight main ideas and comprehension, and when the text helps readers see details in the pictures. *Höffler et al. (2007) found that animations were superior to static pictures (d = 0.46), but it made a difference whether the animation was for decorative purposes (d = 0.29) or representational purposes (d = 0.89). That is, the animation should be central to the concept being learned.

Teacher immediacy

The immediacy of teacher responses to the students shows them that teachers listen and respond. “The applications of immediacy to educational settings introduced the idea that a teacher, through certain cues, could reduce the perceived distance between instructor and learners and thereby influence certain classroom outcomes, especially student learning” (*Allen et al., 2006, p. 22). Students perceive this immediacy as an acknowledgment of the teacher’s engagement; it reduces the perceived distance between instructor and learners, is seen as rewarding to the student, and increases their enthusiasm or commitment to the learning task (Christophel & Gorham, 1995). The effects of teacher immediacy were much stronger on affective learning, such as attitudes to the teacher, course, or engagement in the learning experience (d = 1.15) than on achievement (d = 0.16).

From these results and the correlation between affective and achievement learning, Allen et al. (2006) concluded as follows:

teacher immediacy behaviors predict or cause a level of affective learning. In turn, the level of affective learning predicts or causes the level of cognitive thinking…. (the) teacher creates a motivational or affective outcome that subsequently contributes to the generation of a cognitive outcome.

(p. 26)

They suggested that the teacher’s immediacy also provided a source of feedback by the teacher about their interest, care, and involvement in the student’s learning.

Background music

It costs banks, airports, and shopping centers to play music, and they bear this cost because calm music is likely to invite shoppers to stay, browse, relax, feel comfortable, and thus, buy more (Garlin & Owen, 2006). If music is in the background, it can also serve as white noise and is unlikely to affect learning; but if it is intrusive or distracting, it can interfere with learning. *Kämpfe et al. (2010) reported zero effects of music on reading (r = 0.01), and the effect was not influenced by the music’s loudness or tempo.

The Mozart effect relates to increased IQ and achievement when listening to Mozart, with effects supposedly starting in the crib (also noted in Chapter 10 on “Reading: music-based reading programs”). *Hetland (2000) used 36 studies with d = 0.46 for playing music on spatial tasks, and Pietschnig et al. (2010) used 39 studies with d = 0.37 for any music compared to no music and d = 0.15 for listening specifically to Mozart. Looks promising. However, they did note that studies published from the labs of the two major claimants for the Mozart effect were three times higher than from other sources, and when these were removed, the effects went to zero, and they concluded, “there is only little support for a specific Mozart effect.”

Humor

Each week, Jim Knight and I separately send our caption to the New Yorker cartoon competition – we have never won, although twice, the winning caption was the same as one of ours. Does this participation in humor increase our achievement, no; is it fun, yes; have we learned a lot about the psychology of jokes, yes; and it is a great distraction that then charges the batteries to return to our other work.

*Martin et al. (2006) reviewed 20 studies, with an r = 0.02 between classes including humor and classes not including humor. Not enough to justify their conclusion that the effects on learning are moderate and positive; they are not. It matters not whether there is humor or not. But we will not stop having fun inventing captions.

Methods aiming for deep

There is no lack of argument for the allure of deep learning. Quinn et al. (2019), for example, start with the equity hypothesis: “Deep learning is good for all but is especially effective for those most disconnected from schooling” (p. 5). And this deep learning is the process of acquiring six global competencies: character, citizenship, collaboration, communication, creativity, and critical thinking. Note the absence of the content, which needs to be included. Marton (2014), one of the first major advocates for the notion of deep learning, built his model atop the learning of precious knowledge, and he considered deep learning involved a profound transformation or knowledge construction. Deep learning includes relating ideas, transforming knowledge, constructing new ways of thinking, making connections, doing ongoing evaluation, self-evaluating, dealing with cognitive conflict, rethinking the problem or questions, and doing conceptual change (see Bowden et al., 2014, for elaborations and connections to previous researchers). Who would not want these attributes? Throughout this book, these attributes are considered the knowing-how and knowing-with of learning, built upon the precious knowledge or knowing-that (Table 13.2).

Table 13.2 Summary statistics for teaching knowing-how

Teaching for knowing-how/with
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Problem-based learning
27
991
159,856
1,590
0.45
0.11
5

Problem-solving teaching
11
639
59,305
1,313
0.61
0.08
5

Flipped classrooms
48
2,492
302,850
3,303
0.56
0.12
5

Inquiry-based teaching
13
450
51,761
650
0.50
0.11
4

Inductive teaching
3
171
8,600
171
0.60
0.05
3

Reciprocal teaching
2
38
2,150
53
0.74
0.08
2

Discovery-based teaching
2
193
17,763
394
0.27
0.08
2

Constructivist teaching
4
154
14,745
241
0.92
0.08
3

Scaffolding and situated learning
7
274
31,362
681
0.52
0.10
4

Collaborative learning
12
955
91,734
1,082
0.45
0.07
4

Cooperative learning
37
1,376
144,534
2,258
0.53
0.08
5

Cooperative vs. competitive learning
8
1,031
100,018
960
0.58
0.11
4

Cooperative vs. individualistic learning
5
959
108,659
906
0.62
0.07
4

Competitive vs. individualistic learning
4
831
76,484
203
0.27
0.23
3

Philosophy in schools
3
38
4,860
38
0.54
0.04
2

Out-of-class communication
1
11
7,113
11
0.54
0.08
2

Mentoring
5
175
27,812
671
0.18
0.05
3

Jigsaw method
1
37
3,405
37
1.20
0.00
1

Total/average
193
10,815
1,213,011
14,562
0.56
0.08
3

There are now many advocates and models of deep learning. For example, Fullan and Langworthy (2014) have developed new pedagogies based on building learning partnerships between and among students and teachers, aimed at deep learning goals and enabled by pervasive digital access. The aim is to enhance deep learning, which involves discovering and mastering existing knowledge and then creating and using new knowledge in the world such that students become leaders of their own learning and able to define and pursue their own learning goals using the resources, tools, and connections that digital access enables. The intent is to “drive out of our schools the boredom and alienation of students and teachers – an incredible waste when there is so much to learn.”

Problem-based learning

There have been 27 meta-analyses on problem-based learning (PBL), with effects ranging from −0.30 to 1.15. This is a major spread. Eight of the ten metas with average effects above 0.50 are from developing countries (Turkey, Malaysia, Iran, Turkey, Taiwan, Korea, and China), and the average from the seven metas based on university students (usually in medicine) is 0.18, and pre-2015, the average was 0.26 (and post = 0.57, mainly because of the metas from developing countries). What is remarkable is that there is limited to no information on the nature of implementation of PBL – as if there were but one method. The variety of implementations of PBL is likely a major reason for the major spread of the effect sizes from the meta-analyses.

Another critical moderator is when PBL is introduced. *Albanese and Mitchell (1993) noted that the effects of PBL were much higher in the fourth year compared to the first year of medical programs. They argued that lack of experience and lack of essential surface knowledge lead students to make more errors in their knowledge base, add irrelevant material to their explanations, and engage in backward reasoning (from the unknown to the givens). In contrast, those with sufficient content knowledge were more engaged in forward-reasoning and, thus, more able to benefit from PBL. Walker and Leary (2009) also noted that students without adequate content base tended to engage in far more backward-driven reasoning, resulting in more errors during problem-solving. These errors may persist even after the educational intervention is complete. Problem-based learning works more successfully when students have sufficient content knowledge to make connections. For example, *Gijbels et al. (2005) used 40 studies and reported an average effect from PBL of −0.04 for an understanding of concepts, 0.75 for an understanding of the principles that link concepts, and 0.40 for conditions and procedures for application, and concluded that PBL performs better at the second and third levels of the knowledge structure – that is, after students have sufficient content or surface knowledge.

*Sinha and Kapur (2021) noted the success of exposing students to explicit failure-driven experiences before problem-solving activities; giving students tasks to detect what they do not know to heighten their attention, they are more likely to be receptive to learning this material. This also informs teachers on what needs to be taught. They argued that this experience nudged students to realize the extent to which their activated knowledge was (ir)relevant for solving the problem, made them focus more on learning a better correct procedure, understand which solutions work and which do not, triggered curiosity to find these optimal solutions and processes, and enhanced their deeper conceptual gains from the subsequent problem-solving.

PBL requires sound knowing-that learning, so it is important that problem-based teaching is introduced optimally in the learning cycle. For knowing-that knowledge, problem-based learning can have limited and even negative effects, whereas for deeper learning, when students already have the knowing-that knowledge, problem-based learning can have important positive effects. This should not be surprising, as problem-based learning places more emphasis on meaning and understanding than on reproduction, acquisition, or surface-level knowledge.

For example, *Vernon and Blake (1993) found that the more traditional instructional methods were more effective in raising achievement than problem-based learning (d = −0.18). *Dochy et al. (2003) found an overall negative effect for problem-based learning compared to a conventional learning environment on knowledge (d = −0.78). Still, they noted that problem-based learning positively affected skills (d = 0.66). Students taught using problem-based learning had less knowledge but had better recall of their knowledge. This is probably because, in problem-based learning, knowledge is more often elaborated, and consequentially, the students had a better recall of this knowledge. Similarly, *Gijbels et al. (2005) found zero effects from problem-based learning on the learning of concepts (d = −0.04) but positive effects on application (d = 0.40) and principles (d = 0.75). They concluded that “PBL had the most positive effects when the focal constructs being assessed were at the level of understanding the principles that link concepts, the second level of the knowledge structure” (*Gijbels et al., 2005, p. 45). It is the application and principles underlying the knowledge rather than the concepts or knowledge that are most influenced by problem-based learning. The application of knowledge, not the development of knowledge, is the heart of the success of problem-based learning. *Smith (2003) also found that effects from problem-based learning were higher in self-directed learning (d = 0.54), and *Newman (2004) found negative effects of problem-based learning on the “accumulation of facts.” There is a limit to what we can reasonably expect students to discover, and it may require teaching students not only sufficient prior content but also skills as to how to be engaged in PBL.

There needs to be more control in meta-analyses of teaching strategies about the nature of the intervention and when implemented in the instructional cycle. There also needs to be caution of strategy messiahs – who preach for one method over others (regardless of whether it is PBL or any of the methods outlined in this chapter) – there is a right place, time, and focus for teaching strategies. Kenny Rogers was right.

Problem-solving teaching

Problem-solving involves the act of defining or determining the cause of the problem; identifying, prioritizing, and selecting alternatives for a solution; or using multiple perspectives to uncover the issues related to a particular problem, designing an intervention plan, and then evaluating the outcome. These methods can be used as part of any teaching method and are not formal methods, like problem-based teaching. The overall effects are high. *Liao and Chang (2016) reviewed problem-posing, problem generation, problem writing, and problem strategies, and found d = 0.34 on affective learning outcomes and 0.58 on academic achievement. *Zheng et al. (2011) investigated math-word problem-solving interventions for children with math disabilities, with an overall effect of 0.78. The successful strategies included strategy cues, sequencing, task reduction, advance organizers, questioning, elaboration, and skill modeling. The interventions were more effective when not combined with teaching computational skills.

*Mellinger (1991) examined studies on the development of cognitive flexibility in problem-solving, leading to high effects on verbal flexibility (d = 0.81) and figural flexibility (d = 0.40). *Curbelo (1984) found similar high effects on problem-solving in mathematics. *Hembree (1992) found significant direct links between problem-solving and various basic performance measures in basic mathematics. The teacher characteristic with the most positive effect on students’ performance was specialist training in the heuristic methods (d = 0.71), including Pólya’s (1971) four phases of (1) understanding the problem, (2) obtaining a plan of the solution, (3) carrying out the plan, and (4) examining the solution obtained. *Taconis et al. (2001) reviewed 22 science studies and reported the highest correlations of problem-solving with individual work (r = 0.29), where there was an emphasis on schema construction (0.29), when guidelines and criteria were provided (0.23), and in the presence of immediate feedback (0.22), and much lower where there was an emphasis on knowing-that declarative knowledge (0.02). They concluded that problem-solving teaching was valuable when taught within the context of the subject knowledge (and not in general problem-solving courses). When augmented with studying worked examples or concept maps, they all stimulate the development of the knowledge base and thinking skills.

Flipped classrooms

Flipped learning is an instructional method that has gained substantial interest and traction among educators and policy makers worldwide, especially in higher education. Compared to the traditional method of lecture and practice, flipped learning moves the face-to-face lecture component out-of-class time, typically as a set of online lectures accompanied by formative assessment quizzes to be completed by the student before coming to class. Then class time can be better used for problem-solving and active learning. One of the first to use the term, Bergmann and Sams (2012) defined “flipped learning” as a teaching method in which that “traditionally done in class is now done at home and that which is traditionally done as homework is now completed in class” (p. 13). By getting students to learn basic content online and before class, instructors can make use of the freed-up lecture time to clarify students’ understandings of the concept and design classroom instruction with active learning strategies that allow students to engage deeply with the targeted concept. Like most teaching methods, some advocates list many benefits, but over the many meta-analyses, there is, like with PBL, remarkable variation.

There are now 48 meta-analyses based on 2,492 articles (although our best estimate is that about 30% are common articles in any two meta-analyses), about 60,000–80,000 students have an average effect of 0.56, but the averages range from 0.19 to 1.19 – which is enormous. This variance should be concerning and indeed suggests any average effect is of little value and certainly begs for a more detailed understanding of this very large variance. This is one topic we decided to investigate the nature of implementation and outcomes, and not imagine that there was a homogenous notion of what is meant by flipping learning (see Kapur et al., 2022).

The meta-analyses with the largest effect sizes all came from non-Western countries and the lowest came from Western countries. Shorter (< 4–6 weeks) had higher effects than longer implementations, and there are hints that any positive effects may be more due to the extra time students spend in the teaching and tasks. Most notable were the myriad of implementation methods in both the flipped and regular classroom. There was also a surprising lack of diagnoses and formative assessment after the flipping leading that led to modifications to the in-class sessions.

We coded the nature of the implementation of the pre- and in-class sessions to see the effects of the nature of implementation. We located 174 articles on the effects of flipped learning, leading to an overall effect of 0.37 (SE = 0.025). We also saw the effects were more than doubled in the developing (d = 0.81) compared to developed countries (d = 0.40); shorter (< 1 month, d = 0.52) outperformed longer programs (d = 0.34); there were no differences for the presence or not of formative assessment or feedback from the flipped section; but the major moderator of the achievement effect related to the nature of the intervention – for both the flipped and the in-class sections.

Again, what was striking is that many of the class sessions were repeats and overviews of what students were asked to do in the flipped session (e.g., prior students were asked to review PowerPoints of the lectures that were then delivered in class). We classified both prior and class sections. Categories for coding implementation included the following: “Students’ active involvement and responding” included group work, student presentations, class discussions, role-plays, and use of clickers. “Problem-solving” included completing problem sets and problem-based learning. “Student involvement but more structured and content related” included case studies, Socratic questioning, and debates. “Assessments” included modifying lectures on the basis of assessment tasks, pre-class quizzes, in-class quizzes, and class quizzes at the end of the session.

The greatest effect for flipped learning is when the in-class includes a lecture or involves labs and demonstrations. This hints that the major advantage of flipped learning is the doubling or extended exposure to the lecturer’s interpretation of the knowledge and understanding, and not necessarily any active involvement by the students. Indeed, with more active involvement in class, the effects are not that different between flipping and traditional, and when students are involved in gaining more knowledge and understanding through active learning in the in-class sessions, the effects are reversed in favor of traditional classes. The use of assessment before commencing the class after the flipped activities actually decreased the advantages of flipping (mainly because the assessment was not then used to modify the in-class instruction). A major message is that the effects of flipped classrooms depend on the nature of the implementation.

This teaching method is another example of the need to code the nature, length, and time of interventions, and not presume that a particular teaching strategy is uniformly implemented. Gene Glass (2019) also noted that in education, unlike medicine, we can rarely control the dosage, fidelity, and quality of intervention. In medicine, the implementation could be an “intravenous injection of 10 mg of Nortriptyline,” and this is uniform and well-defined, “whereas even interventions that carry the same label in education are subject to substantial variation from place to place, or time to time” (p. 4). Exposing students to flipped learning, to paraphrase Glass, can take many different forms, some of which are effective and some of which are not.

If, however, an intervention attains consistently attained high effects, despite being implemented in many different variants, different dosages, across all ages and content domains in many countries, there may a stronger case for saying the intervention is replicable (unlike in many sciences where they have more control over the intervention). Hence, the nature of replication in the social sciences is somewhat different. We desire replication of high effects across many conditions and, thus, can then tolerate modifications to local circumstances.

It appears that providing pre-class activities followed by an in-class traditional lecture is more powerful than providing pre-class activities followed by in-class active student engagement. (This is contrary to many of the avowed claims of flipped learning, that the active learning aimed to move from the surface to the deep; but the evidence does not support this claim.) It is the increase in exposure, the additional time, the practice, or the repetition effect that allow flipped learning to make an increase in student learning. It is also likely the assignments or assessments could have a major influence on how students go about learning despite the teaching method – if the assessments value knowing-that knowledge, then deeper methods are more likely to have minimal effect. Students, especially tertiary students, can be most strategic in working out what is desired, despite any messages about the importance of deeper thinking.

This leads, again, to the important claim about intentional alignment in Chapter 11. For example, if the assessment tasks address lower-level surface activities, then the backwash from testing is much more powerful than any claims to the contrary by the teacher. Students with deep learning motives and strategies will perform poorly if the assessment is based on narrow and low cognitive level goals (Lai & Biggs, 1994). Thus, good teaching needs to address all the parts of the educational experience – the curriculum goals, teaching methods, class experiences, and particularly, assessment tasks and grading.

Biggs’s constructive alignment notion asks teachers to be clear about what they want their students to learn and how they would manifest that learning in terms of “performances of understanding.” While acknowledging Biggs’s notion, I chose not to use constructivism because of the inevitable misinterpretations with the long-vexed debate about constructivism. Hence, intentional alignment. The students need to be exposed to knowledge and understanding relative to these goals, placed in situations that are judged likely to elicit the required learnings, and the assessment and in-class activities need to be aligned with the criteria of success. Otherwise, there is safety in resorting to knowing lots, repeating what has been said, and privileging surface-level knowledge.

This notion of intentional alignment was not so present in many of the studies on flipped learning. Instead, the focus was more on engaging students in repetitive activities – the same in the flipped was repeated in the class, usually via asking students to prereview videos of classes, prereview the PowerPoints then used in class, or listen to a teacher repeat material already exposed to the students. There is no reason to claim these are not worthwhile activities, but it does not seem in the spirit of the advocates of flipped learning, which typically make claims about the power of flipped learning to deepen understanding. Assuring that the pre, the class, the assignments, the assessments, and the grading are aligned with such claims seems necessary if flipped learning, or any form of teaching method, is to be appropriately evaluated.

Inquiry-based teaching

Inquiry-based teaching is the art of developing challenging situations in which students are asked to observe and question phenomena, pose explanations of what they observe, devise and conduct experiments in which data are collected to support or contradict their theories, analyze data, draw conclusions from experimental data, design and build models, or do any combination of these. Such learning situations are aimed to be open-ended in that they do not aim to achieve a single right answer for a particular question being addressed but rather involve students more in the process of observing, posing questions, engaging in experimentation or exploration, and learning to analyze and reason. The eight meta-analysis have average effects ranging from 0.17 to 1.03, with an average of 0.48.

Once again, better controls over the nature of what inquiry means when it is introduced in the learning cycle and the effects on surface and deep learning may help unravel the large variance around the effects of inquiry teaching. For example, Shymansky et al. (1990) reported greater effects of inquiry teaching on process (d = 0.40) than on content (d = 0.26); *Smith (1996) found larger effects from inquiry methods in critical thinking skills (d = 1.02) than in achievement (d = 0.40) and less in laboratory skills (d = 0.24) and process skills (d = 0.18); *Sweitzer and Anderson (1983) found effects twice as large on processes than on content; *Bredderman (1983) reported higher effects on science process (d = 0.52) than the effects on science content (d = 0.16); and *Lazonder and Harmsen (2016) reviewed 72 studies comparing inquiry to expository instruction approaches, with an overall effect of 0.78 for improving inquiry skills and 0.37 for improving domain knowledge. Inquiry learning seems more successful for improving inquiry skills, but maybe other methods are needed to also add more positive effects on content and surface knowledge.

Inductive teaching

Induction teaching involves moving from the specific to the general, while deduction begins with the general and ends with the specific. *Lott (1983) investigated whether this order effect matters, and d = 0.06 suggests it makes no difference which order is used – general to specific or vice versa. This was across many outcomes, such as knowledge, application, process, transfer, comprehension, and problem-solving.

*Klauer and Phye (2008; *Klauer 2014) were more interested in inductive reasoning to teach detecting generalizations, rules, or regularities. The overall effect (d = 0.60) is quite high, showing positive effects of teaching these skills and supporting that specifically teaching the skills of making comparisons and contrasts can be enhancing, but they noted that there comes a point after students have acquired inductive reasoning when there needs to be greater knowledge and understanding to capitalize on these methods more fully.

Reciprocal teaching

Reciprocal teaching was devised as an instructional process to teach students cognitive strategies that aim to improve learning outcomes (initially in reading comprehension). The emphasis is on teachers enabling their students to learn and use cognitive strategies, such as summarizing, questioning, clarifying, and predicting, and these are “supported through dialogue between teacher and students as they attempt to gain meaning from text” (*Rosenshine & Meister, 1994, p. 479). Each student takes a turn at being the teacher, and often, the teacher and students take turns leading a dialogue concerning sections of a text. Students check their own understanding of the material they have encountered by generating questions and summarizing. Expert scaffolding is essential for cognitive development, as students move from spectator to performer after repeated modeling by adults. The aim, therefore, is to help students actively bring meaning to the written word and assist them to learn to monitor their own learning and thinking.

The effect size from both meta-analyses is a very high d = 0.74, and both studies found that this high effect was evident regardless of who delivered the intervention, with classroom teachers being able to implement reciprocal teaching with the same level of effect as produced by the study authors. *Rosenshine and Meister (1994) reported no differences in results by grade level, number of sessions, size of the instructional group, number of cognitive strategies taught, or whether the investigator or the teacher did the training. The effects were greater when the comprehension assessments were experimenter-developed (d = 0.88) than when using standardized tests (d = 0.32), although both short-answer tests and tests asking students to summarize passages gave similar results. The effects were highest when there was explicit teaching of cognitive strategies before beginning reciprocal teaching dialogue, showing the importance of modeling and practice as well as giving instruction in the use of the strategies close to the time students used them. The explicit teaching of cognitive strategies and deliberate practice with content when using these strategies make major differences.

Discovery-based teaching

The argument is that students learn best by interacting with materials, eliciting explanations, discovering patterns, exploring phenomena, conducting simulations, and finding target information of conceptual understanding independently – that is, through discovery. *Alfieri et al. (2011) found that explicit instruction was more successful than unassisted discovery learning (0.38), and what they termed “enhanced discovery” (feedback, worked examples, scaffolding, and elicited explanations) was more favorable than discovery learning (0.30). It is teaching students when they misconstrue, to know steps to success and to understand how to construct knowledge and understanding, that matters.

Constructivist teaching

Constructivism is a form of knowing and not a form of teaching, and it is important not to confuse constructing conceptual knowledge with the current focus on constructivism (Bereiter, 2002). Constructing conceptual knowledge involves considering learning from the learner’s viewpoint, starting from the premise that all learners are active, appreciating that what they learn is socially constructed, and understanding that learners need to create or recreate knowledge of themselves (Phillips, 1995). If this is the meaning of constructivism from a learner perspective, then the more direct and active methods of teaching appear to be optimum for achieving this type of learning.

The three meta-analyses, all based on Turkish research, reported very high effects for constructivist teaching. *Erisen and Gunay (2015) used a very wide definition of “constructivist teaching” – those that “establishes a dynamic link between the human brain and its environment.” The average effect from 27 studies was 1.10, with some effects so large to be questionable (e.g., 3.2, 5.0, and half > 1.5). From the titles of the selected articles, it seems constructivism casts a wide net: bottom-up learning, collaborative learning environments, computer-aided learning, teacher training, and reflective thinking practices. *Ayaz and Sekerci (2015) found 53 Turkish studies (none overlapped with *Erisen and Gunay), with an overall effect of 0.93. They claimed constructivism establishes a link between the new information and the information that exists in individuals, students add their own comments and test their ideas, and teachers guide and help learners to reach the information and to construct it. Their methods included drama, realistic math, concept mapping, and problem-based learning. *Semerci and Batdi (2015) used 28 articles (only 4 in common with the previous 2 metas) with an overall d = 0.92 and defined “constructivism” as learners constructing knowledge through real-life experiences as a cognitive activity, rather than simply learning from abstract concepts.

These metas included a similarly wide range of methods, and it would have been more meaningful to investigate each of the many methods rather than group them under the constructivist title. Given many of the methods, there seems more support for the intention of teaching on how to construct knowledge and how to make relations among ideas than any inference of students discovering themselves. As Bereiter (2002) showed, oftentimes, the most powerful way to enable students to construct knowledge is to deliberately lead them to the construction and certainly not wait till they discover themselves.

Scaffolding and situated learning

The zone of proximal development (Vygotsky, 2012) involves teachers and students recognizing a gap between what a learner accomplishes alone and what can be accomplished with assistance of an expert. Thus, the learner needs scaffolding from experts to reduce this gap, but the extent and specificity of this help need to reduce over time – hence, the notion of the teacher’s gradual release of responsibility such that when reaching toward the success criteria, the scaffolding fades or stops as students are finally able to complete a task or resolve a problem without any support or assistance from external aids or assistance. At high levels of scaffolding, there can be Direct Instruction, provision of worked examples, and modeling of thinking aloud, but then the skill of the teacher is to fade this support, which thus involves careful evaluation of where students are relative to success (doing it themselves), the student’s skills to identify and work through errors and missteps, and the self-regulation skills to become their own teachers.

Belland (2014) noted the complexities in scaffolding strategies that can make it hard for teachers in busy classrooms to be aware of the nature and extent of scaffolding for each student. Hence, the use of computer technology to provide scaffolding, detect growth in learning, and know how to fade the level of scaffolds. The overall effect was d = 0.53, and this is similar to many of the intelligent learning systems reviewed later (e.g., *Ma et al., 2014, d = 0.43; *VanLehn, 2001, d = 0.76; *Gerard et al., 2015, d = 0.34; *Wang & Mao, 2019, d = 0.49; *Kulik & Fletcher, 2016, d = 0.66). *Belland (2014) found these effects were similar across levels of schooling, across all students regardless of prior achievement, and across many forms of outcome (concept, principles, and application); and the particular effects of fading (releasing teacher responsibility) was high (d = 0.62). The conclusion is that computer-based scaffolding is highly efficacious.

Fisher and Frey (2021) have explored this notion of teacher’s gradual release of responsibility and note that effective teaching can follow a progression in which teachers gradually do less of the work and students gradually assume responsibility for their learning. This entails a deliberate intent to teach the student the skills of taking responsibility for the learning, and they particularly note the importance of students working and learning with peers in their gradually taking on the responsibility for the learning.

Collaborative and cooperative learning

Learning is often portrayed as a social activity. Vygotsky (2012) argued that interactive social communication and activity are the basis for a child’s successful cognitive development. But in many classes, students sit in groups but work alone.

Deming (2017) traced the employment rates in the United States from 1980 onwards for those with science and math skills (but this would likely apply to many other domains). He divided individuals with math or science skills into four groups – above- and below-average content skills, and above- and below-average social skills (Figure 13.3). Occupations with higher content knowledge and higher social skill requirements have grown robustly throughout the wage distribution, and no surprise, those demanding low content and social skills are declining. What is fascinating is that higher social skills and lower achievement outperform higher achievement and lower social skills. Jobs with high social skill requirements have experienced greater relative growth (although employment and wage growth have been strongest in jobs that require high levels of both cognitive and social skills). The top attributes in job advertisements included oral and written communication, collaboration, and problem-solving (Rios et al., 2020). These social intelligence skills were requested regardless of also requesting a bachelor or graduate degree and were demanded twice as much for lower qualifications (e.g., associate diploma).

Figure 13.3 Changes in US employment share by skills required.

Source: Deming (2017)

This shows the critical importance of developing skills to work in and contribute to groups, learning to be team players, and being translators of domain knowledge to others. Employers want team players, translators, communicators, and those with high social sensitivity, along with the previously valued knowledge. Therefore, we educa tors need to have a higher focus on teaching students to collaborate.

There are several reasons that collaborative learning benefits achievement. For example, students working in groups can be introduced to new ideas that conflict with their own understanding. This can lead them to seek new information to clarify the conflict or to attempt to explain and justify their own position. In addition, students working together can generate new approaches to solving problems that none of them knew before working together. Individuals can then adopt these approaches to use in future problem-solving. Further, students also benefit by giving and receiving help. Giving help requires the giver to clarify and reorganize their understanding, helping them to understand the material better, fill in gaps in the receiver’s understanding, and help them clarify misconceptions.

But this does not mean we need more group work; but we need to intentionally teach more skills and confidence to work in groups, on tasks that require multiple interpretations, and where the feedback and rewards are a function of the individual contributions to the group as well as the group’s performance. Working in groups can have considerable drawbacks for learning, particularly when students do not know how to work together and lack exposure to good models and instruction. The status of individuals within a group can make some students consistent leaders and others always followers. Some are dominant; some cannot pause, seek, or see the views of others as worthwhile; some cannot engage in turn-taking; and there can be social loafing (which is rarely welcomed in groups). The person whose ideas are respected may not always be the person with the best understanding of the problem to be solved.

*Johnson and Johnson (2009) identified five key elements that are important in cooperative learning:

Positive interdependence exists when one student’s success positively influences the chances of other students’ successes.

Individual accountability involves two components: each student is responsible for his or her own learning; and each student is responsible for helping the other group members learn.

Promotive interactions occur when individuals encourage and facilitate each other’s efforts to accomplish the goal of the group.

Effective collaboration includes giving and receiving elaborate explanations with a focus on encouraging understanding.

Knowledge-building discourse is fostered in democratic environments where the teacher is no longer the arbiter of truth.

There is much overlap between collaborative and cooperative teaching methods and, thus, little surprise that the average effects are quite similar. The effects were highest for tertiary, then high schools and preschools, compared to elementary students (*Caper et al., 2015), and higher for students with special needs (1.18) compared to other peers (0.85) (*Jung, 2006) and across all ability groups (*Gu, 2013; *Romero, 2009); but cooperative teaching methods had small effects (about 0.15) on students’ attitude to the subject (*Kyndt, 2013; *Caper, 2015).

The core question is how to make collaborative groups work successfully. Teaching specific cooperative learning strategies (such as instructing students to reach consensus; *Tenenbaum et al., 2019) leads to better outcomes than when students were generally encouraged to work together (*Lou at al., 2001). Effects are greater on process outcomes (0.58) and knowledge outcomes (0.53), and lower on affective outcomes (0.38; *Jeong et al., 2019; see also *Chen et al., 2018). There is a need to provide scaffolding (d = 0.34), such as assigned roles and metacognitive prompts that remind students about steps in the process; teach turn-taking; and provide reminders to generate explanations and to reflect on their work. Scaffolding is particularly valuable for complex problems (0.48) compared to surface and content problems (0.22) (*Williams, 2009). There are problems and tasks more appropriate to group work, such as tasks requiring multiple interpretations and evaluative decision-making (rather than a focus on content development). Groups are less effective at developing content or knowing-that knowledge and more powerful for relational or knowing-how development.

The effects of collaborative groups were similar across face-to-face collaboration (0.51), synchronous collaboration (0.51), and asynchronous collaboration (0.50) (Jeong et al., 2019), and there were no differential findings relating to age, gender composition, or area of learning (*Tenenbaum et al., 2019). Studies conducted between one month and one semester (i.e., < 1 semester) reported the largest effect size (0.52) (*Chen et al., 2018). There is a right time, right place, and right set of tasks to teach groups skills and confidence.

Cooperative vs. competitive vs. individualistic learning

One of the major contributions by the Johnsons is their specificity of the control groups. Rather than compare cooperative or collaborative vs. business as usual, they compared cooperative or collaborative vs. individualistic or competitive learning situations. In all comparisons, cooperative methods beat competitive and beat individualistic learning – pointing again to the power of peers in the learning equation: cooperative vs. individualistic learning (d = 0.55), cooperative vs. competitive learning (d = 0.53), and competitive vs. individualistic learning (d = 0.24). It is worth noting, however, that these methods need not conflict. Using international comparisons, New Zealand comes out top on cooperativeness in schools and is also top in competitiveness. Again, what counts is considering when and where to use a strategy, and deliberately teaching and valuing the skills needed to learn in different contexts.

Philosophy for Children

Philosophy for Children is an educational program in K–12 setting that engage students in group dialogues focused on philosophical issues. It typically involves dialogues promoted by a reading or video and attention to concepts, such as truth, fairness, or equity. The aim is to increase student questioning, construction of arguments, and engagement in reasoned discussion. *Yan et al. (2018) found no differences in age and duration of the program (from 5 to 40 hours+) but higher effects in Asian (0.69) than Westernized countries (0.39), and the largest effects were for reasoning (1.06) rather than general achievement (0.40) and reading comprehension (0.28). As *García-Moriyón et al. (2005) concluded, “children can do philosophy and this practice helps them to develop their high order thinking skills” (p. 17), and in a similar vein, *Trickey and Topping (2007) concluded that Philosophy for Children can led to significantly gains “in measurable terms both academically and socially” (p. 375).

Out-of-class communication

The classroom is not the only forum for learning, especially at the tertiary level. These out-of-class experiences refer to the “structured and unstructured activities or conditions that are not directly part of an institution’s formal, course-related, instructional processes” (Pascarella & Terenzini, 1991, p. 150). This includes voluntary involvement in campus organizations to interpersonal conversations with academics (homework is dealt with later). *Goldman et al. (2016) found 11 studies with a correlation of 0.26 (d = 0.54) with cognitive learning and concluded that out-of-class communication is a valuable component of the collegiate experience.

Mentoring

Mentoring is a form of peer tutoring, although it usually involves older persons (often students or adults) providing academic or social assistance or both to younger people. Mentoring usually involves little, if any, teaching and is more an apprentice model based on social and role model experiences. Hansford and Ehrich (2006) noted the major claimed benefits of mentoring related to collegiality, sharing of ideas, a reappraisal of beliefs and values, personal satisfaction, support, and understanding. The major problems are the lack of time; personality mismatch; lack of training; mentor not understanding the mentee or program goals or expectations; and mentor being out-of-touch, defensive, or untrusting.

Mentoring has a close to zero effect on performance outcomes (d = 0.12), attitudes (satisfaction, d = 0.06), school attitudes (d = 0.19), or motivation and involvement (d = 0.11) (*Eby et al., 2008). *Wood and Mayo-Wilson (2012) concluded that the mentoring programs included in their review did not reliably improve any of the included outcomes. From only six studies, the effect on achievement was −0.01 and, similarly, close to zero for attendance, self-esteem, and behavior. *DuBois et al. (2002) also found low effects (d = 0.18) on achievement, and these low effects were not related to the setting in which mentoring activities took place, the compensation of mentors, the gender or race composition of the mentee or mentor group, or the frequency of contact or the length of the relationship between mentors and youth. The effects were higher when conducted by teachers (0.26) than others (0.09) and when there was pretraining for mentors (0.22 vs. 0.11), pointing to the importance of skills of teaching for mentors.

Mentoring may do no harm, but it seems of slight advantage to any factor studied. Once again, unless the mentors have the teaching and relationship skills, they are unlikely to add much value.

Jigsaw method

The jigsaw method was invented to teach art and to help reduce racial cliques in integrated schools. Of all the teaching methods that have been studied using meta-analysis, it has the highest effect size. *Batdi (2014) reviewed 37 Turkish studies on the jigsaw method, with d = 1.16. Just as in a jigsaw puzzle, each student’s part is essential for the completion and full understanding of the final product (see www.jigsaw.org/overview.html). The jigsaw method ensures that all students have the knowledge, the subject matter vocabulary, and the main ideas, before venturing to link their ideas more deeply and be able to then transfer knowledge to new problems or issues – a rarity among most teaching methods. There are many variants, but one common implementation is provided in Figure 13.4.

Figure 13.4 Outline of the jigsaw teaching method aligned with the VL model.

The fundamental principles in these steps are to maximize turn-taking, to have no one be dominant, and to come to an agreement and not necessarily a consensus. All participants must have the opportunity to be heard and argue their case; participants may ask questions, challenge others on their points, and ask for clarifications; but participants must listen to other points of view. Thus, individual responsibility and interdependence are woven into the structure of jigsaw, so there is an increased probability of having more depth in some aspects, building ideas and content, and then working with peers to see relations and transfer ideas. Step 1 is akin to the surface acquiring, step 2 to the surface consolidation, step 3 to the beginnings of deep acquiring, and steps 4 and 5 to deep consolidation and, potentially, transfer.

Not only does the jigsaw increase and deepen knowledge and understanding, but it also reduces students’ prejudice (Aronson, 2002; Levy et al., 2017), increases intrinsic motivation and feelings of competence (Hänze & Berger, 2007), and stimulates a mix of cooperation, competition, and individualistic efforts and both epistemic (beliefs about knowledge and knowing) and relational regulation (Nolan et al., 2018; Roseth et al., 2019).

Conclusions

When choosing a teaching method, the major message is to consider the intentional alignment with the success criteria – to focus on the knowing-that, knowing-how, or knowing-with of the lessons, and with assignments and assessments.

There is a healthy debate on whether exposure to deeper notions could well precede learning the content, but it is fascinating that the most successful teaching methods at least cover all five parts of the learning model systematically.

It is concerning that lessons are often created around activities and success is more in completing these activities.

A major message for meta-analysts is to be more systematic in coding the nature and degree of implementation. There are many ways to interpret and implement various methods, there are many parts of the learning cycle that can affect the success of a method, and there can be a critical adaptation that may lead to success or, worse, neutralize the benefits of a particular method.

There is support for a more differentiated choice of teaching methods relative to the learning progressions.

The art of teaching is knowing what levels of challenge each student is prepared to make, reducing the inequities in the class by using student peers as part of the teaching (per the jigsaw method), and having the evaluative thinking skills to make decisions about optimal teaching methods and knowing when best to implement them.

The major message when choosing a teaching method is to consider the intentional alignment with the success criteria; the focus on the knowing-that, knowing-how, or knowing-with of the lessons (surface, deep, transfer); and the other factors outlined earlier in the intent to teach model (Chapter 11). To these, it may be worth adding how to decide on teaching methods that optimize peer learning and teaching in ways that capitalize on the divergent strengths and differences in the class, create social cooperation, and (as noted in Chapter 7) maximize the students’ collective efficacy.

We can somewhat classify teaching methods relative to the learning model, but there is no claim that this is how all students learn all the time. Indeed, there is a healthy debate about whether exposure to deeper notions could well precede learning the content, but it is fascinating that the most successful teaching methods at least cover all five parts of the learning model systematically.

It does seem surprising that most teaching methods are neutral on how they relate to a concept of learning (e.g., its depth of complexity), and it is concerning that, too often, lessons are created around activities and success is seen as completing these activities. Perhaps we need more comparative studies of teaching methods relative to the features within the intentional alignment of the intent to teach model. This might lead to new teaching methods. For example, think-pair-share could become know-think-pair-share; Direct Instruction and reciprocal teaching could be combined to become DIRT; summarizing and outlining could be added to problem-based teaching; and so on. These more all-embracing methods require high levels of teacher evaluative thinking to decide the following:

When the optimal time is to use the parts of the methods.

When to stop and move to another method.

How to keep a constant eye on the impact of the teaching methods on the learning of the students to continue, stop, or replace.

A major message for meta-analysts is to be more systematic in coding the nature and degree of implementation (and the original studies may need to explain their implementation more fully). The salutary claims for flipped learning show the importance of such coding for meta-analysis, as there are many ways to interpret and implement these methods, there are many parts of the learning cycle that can affect the success of a method, and there can be many critical adaptations that may lead to success or, worse, can lead to neutralizing the benefits of a particular method.

From our meta-analysis of flipped learning, for example, we developed an F4 model that combines the successes of flipped learning and illustrates that problem-solving could be part of the initial exposure, provided teachers evaluate what the students do not know so teaching can be optimized:

Fail – allowing students something they cannot do or understand and so provide opportunities for the instructor and the student to diagnose, check, and understand what was and was not understood.

Flip – pre-exposure to the knowing-that ideas in the upcoming class (as simple as providing a video of the class).

Fix – a class where these misconceptions are explored, with an opportunity to reengage in learning the ideas.

Feed – feedback to the students and instructor about levels of understanding and where-to-next directions.

The nature of flipping, thus, could be more oriented around productive failure. That is, when learning a new concept, instead of first viewing an online lecture (or similar), students start with a preparatory problem-solving activity designed to activate their knowledge about what they are going to learn in the lecture. It is by assisting the students to orient toward what they do not and need to know that subsequent learning is maximized. Only after the fail stage do students proceed to view the content, the surface knowledge, the vocabulary (e.g., by watching online lectures in order to learn the targeted concepts). They can then move to the third (fix) step where they convene for in-class activities to consolidate what they have generated, compare and contrast student-generated and canonical solutions, attend to the critical concept features, relearn and extend their knowledge and understanding, and observe how these features are organized and assembled. Finally, the feed stage is where the students and the teachers learn what has been learned, who has accomplished this learning, and the magnitude or strength of this learning. The emphasis here is for teachers to seek feedback about how they can improve their next teaching, not only after the flipped phase (with its emphasis on what the students do not know) but also at the end of the flipped and in-class phases.

More attention is needed to move away from seeing lessons in terms of a series of activities. As Nuthall (2007) noted, so often, the lessons have a four-part structure: an instruction phase, an activity phase, a reporting phase, and a wrap-up phase. This structure is supported by Berry’s (2023) finding that success in learning is seen as each student engaging in the doing and, preferable, finishing the activities. Instead, the meta-synthesis findings support a more strategically sequenced choice of teaching methods aimed at stepping through the knowing-that, knowing-how, and knowing-with learning progression (Figure 13.1). Teaching should provide students multiple ways and multiple attempts, each with variable durations, to engage in the learning required to meet the success criteria. It involves not only teaching the precious knowledge, the deeper conceptual ideas, and the near and far transfer to other tasks, but also teaching the optimal learning strategies, being aware of the complexity of the cognitive demands, and creating an environment of high trust for errors, not knowing, and misunderstandings to be seen by students as welcome and legitimate parts of the lesson. The art of teaching is knowing what levels of challenge each student is able to meet, reducing the inequities in the class by using student peers as part of the teaching (per the jigsaw method). The teacher needs the evaluative thinking skills to make decisions about what the optimal teaching methods are and when best to implement them.

Reflections

Direct Instruction is often misinterpreted as simply lecturing at the front. After reading the seven stages of DI on pages 360–361 review your current practice for using this method. Share examples of each stage for colleagues and with students. The hook, for instance, can be a prior knowledge whole-class question that student pairs then discuss to reveal their current understanding and tune them into the learning within the lesson.

The application of knowledge, not the development of knowledge, is the heart of the success of problem-based learning. Share examples of how problems given before enough knowledge is gained have been largely ineffective. When planning a lesson, determine a readiness measure to ensure students have sufficient knowledge before moving to the problem-focused part of the lesson.

Reciprocal teaching (0.74) is highly effective for reading comprehension and other subjects in developing key cognitive strategies (predicting, summarizing, clarifying, questioning). High school teachers, implement this method as part of your class to ensure students have sufficient depth of knowing.

The jigsaw strategy (page 387) has many positive outcomes. Use it first as the basis of a staff meeting in which new knowledge is to be shared; then experiment with students and report back findings to the staff meeting about your implementation and its impact on students.

Too often, lessons are created around activities, and success is more in terms of completing these activities. When clear learning intentions are the focus for planning and teaching (What do I want them to learn/get out of this lesson?), activity completion becomes less important than the learning. Review the alignment among your recent learning intentions, activities, and consequential impact on students.

Invite two students to work together as learning detectives for a day. Ask them to report back to the class the evidence of where they heard learning discussions, evidence of learning in action, and examples of learning.

Debate this notion: We should not hold an instructional strategy in higher esteem than students’ learning.

CHAPTER 14 Implementations using technologies

DOI: 10.4324/9781003380542-14

The world of computers became more complex after 2009 – no longer are there computer rooms, everyone seems to have an integrated device in their pocket (or thumb ready), the iPhone was first released in 2007 and the iPad in 2009, there is now higher bandwidth, and technology is not only ever-present but harder to place into categories (as in VL1, 2009). In VL1, I used the term “CAI” (computer-assisted instruction), and a number have politely asked what CAI is, which was the familiar but no longer used phrase pre-IT. Now, apps are ever-present such that the machinery is no longer the focus and the notion of the computer ubiquitous, but this does not stop the advocates who promote technology as the next revolution. As Dylan Wiliam has commented, technology is the revolution still coming after 50 years. There are now over 307 meta-analyses, about 17,000 studies, over 3.3 million students, 28 types of IT influences, with an overall average of 0.34 – and this average has barely changed since the first meta-analysis in 1977 (Figure 14.1).

Figure 14.1 Effect size by year of publication.

An analysis of the much hyped one-laptop-per-child is a sobering tale of the technology sell – which is so often based on charisma, social imaginaries, and technological utopias (Ames, 2019). Promoted by luminaires at MIT, the aim was to drop $100 laptops in countries least able to afford more expensive machines (they actually started at $185), schools were depicted as the problems to children’s learning growth, there was no need for teachers, and the promise was to turn on the natural wonder of children to construct knowledge that would unleash their potential and “eliminate poverty, create peace, and work on the environment” (Negroponte, cited in Ames, 2019, p. 14). Similar lofty claims have been made about so many tech revolutions (e.g., MOOCs and, before this, television). The realities turned out much different, with very few countries adopting the $185 laptops and few children using them, and they were no great solutions. Many schools in Western countries have cupboards filled with laptops, many still in their shrink-wraps. *Zheng (2016) reported d = 0.16 from the use of one-on-one laptops.

The defining feature of the more general meta-analysis on computers is their consistency in overall mean – hovering between about 0.3 and 0.5 (Table 14.1). Yes, the devil is in the details, but asking about the impact of technology should remain less fascinating than asking about the effects when apps and IT are integrated with teaching. Harder, but more interesting. Once again, it is the principle of intentional alignment that matters most.

Table 14.1 Summary statistics for technology variables

Technology
No. metas
No. studies
Total no.
No. effects
Weighted ES
SE
Robust ness

Information and computer technology
63
4,280
576,861
7,292
0.51
0.08
5

Technology with elementary students
6
264
24,298
664
0.44
0.05
3

Technology with high school students
9
680
72,131
760
0.30
0.05
4

Technology with college students
15
1,962
226,621
1,786
0.34
0.08
5

Distance or online education
28
1,499
4,182,905
2,440
0.25
0.07
5

Webinars
1
15
591
36
0.33
0.09
1

One-on-one laptops
1
10
920
10
0.16
0.04
1

FaceTime and social media
3
72
123,728
72
−0.12
0.05
2

Technology in mathematics
25
1,118
309,212
2,216
0.37
0.07
5

Technology in science
6
391
39,048
567
0.18
0.08
3

Technology in reading/literacy
17
736
180,647
1,547
0.26
0.09
4

Technology in writing
4
76
5,840
81
0.41
0.10
2

Technology in other subjects
3
96
7,838
103
0.58
0.08
2

Technology with learning needs students
5
125
16,390
152
0.60
0.05
3

Technology in small groups
3
193
19,877
616
0.17
0.08
3

Intelligent tutoring systems
5
283
30,520
288
0.52
0.03
4

Use of PowerPoint
1
12
1,104
16
0.26
0.00
1

Smart boards
2
72
6,627
72
0.89
0.09
2

Online and digital tools
12
489
106,437
1,440
0.36
0.10
4

Virtual reality
20
699
67,054
721
0.57
0.11
4

Robotics
4
87
8,287
87
0.57
0.13
2

Interactive video/ multimedia
9
432
42,444
3,987
0.54
0.08
4

Clickers
3
132
33,366
236
0.21
0.15
3

Simulations
14
700
62,162
1,087
0.53
0.09
4

Gaming
43
1,985
262,920
2,955
0.41
0.14

Web-based learning
7
391
52,188
544
0.40
0.25
4

Mobile/touch devices/ tablets
20
1,003
115,058
2,267
0.59
0.11
5

Presence of mobile phones
3
93
297,746
95
−0.27
0.08
3

Screen time
1
30
480,479
30
−0.29
0.19
2

Audio/visual methods
9
487
44,435
359
0.29
0.07
3

Programmed instruction
8
493
45,375
391
0.24
0.08
3

Total/average
307
16,920
7,180,188
29,962
0.34
0.08
3

The right questions relate to why this technology revolution has not had the major impact that has been so promised. Cuban (2003) asked this why not question and concluded that technologies will never be used in any transformative sense until we change our teaching methods. At best, we will use technology to consume more facts and knowledge, and we will use the internet instead of the encyclopedia, PowerPoint and Word instead of slate and paper, and iPads to complete practice tasks online instead of on worksheets. As before technology, if we prioritize knowledge consumption, then technology can help students become more efficient but not necessarily more effective. It will only be when we move from using technology as a newer form of knowledge consumption to seeing technology as an aid to teaching for enhanced knowledge production that there may be an effect.

Second-order meta-syntheses of technology

There have been at least three second-order meta-analyses of this voluminous literature. Higgins et al. (2012) reviewed 45 meta-analyses published between 1990 and 2012, and noted that the typical overall effect size of the impact of technology on learning is between 0.3 and 0.4. They concluded that it is not whether technology is used (or not) that makes the difference but how well the technology is used to support teaching and learning. They noted higher effects when technology was used collaboratively in pairs or small groups, for short-focused interventions, and for tutorial and remedial consolidation work; and higher effects in math and science than in literacy.

Tamim et al. (2011) reviewed 37 meta-analyses (with < 25% overlap in studies across the meta-analyses) with an average of d = 0.31. Better designed studies had slightly lower effects (0.27) compared to weaker or moderate designed studies (0.36), and there were high effects in secondary (0.45) than elementary (0.27) and higher education (0.27). Young et al. (2017), in a meta-synthesis of 19 meta-analyses, reported similar effects relating to three major uses of technology: completing tasks formally conducted by hand, which allowed teachers to assign more complex and involved learning activities (d = 0.47); practicing skills and developing instrumental understanding (d = 0.42); and developing conceptual understanding via, for example, virtual manipulatives, modeling, and applications (d = 0.36).

Major principles relating to the use of technology

Throughout this chapter, ten major messages from the 300+ metas are outlined, as it seems not as meaningful to explore again (as in VL1) the specific technologies. There are four critical underpinnings to the success of using technology: the power of social media, the use to enable feedback to teachers, the focus on learning, and the major benefits for consolidating learning. These update the principles in VL 2009, which still apply: computers are used effectively (a) when there is a diversity of teaching strategies; (b) when there is a pretraining in the use of computers as a teaching and learning tool; (c) when there are multiple learning opportunities (e.g., deliberative practice, increasing time on task); (d) when the student, not teacher, is in control of learning, then peer learning is optimized; and (e) when feedback is optimized.

The start is to acknowledge that the use of technology is effective for all students. Across all the 260+ meta-analyses, there is more similarity in the effects across all students than there are moderation effects of student background. For example, the effects are similar across preschool (2 metas, d = 0.54), elementary (20 metas, d = 0.44), high school (9 metas, d = 0.30), and college students (16 metas, d = 0.45); in math (33 metas, d = 0.41), science (11 metas, d = 0.33), language (6 meta, d = 0.55), reading/literacy (21 metas, d = 0.35), and writing (3 metas, d = 0.42); and with students with learning needs (6 metas, d = 0.49). The effects were larger, however, in the Global South (e.g., Iran, Nigeria, India, Saudi, Turkey, Cyprus, n = 27, d = 0.61) than in the Western and more well-resourced countries (n = 130, d = 0.41). Perhaps this asks about the comparison group – more traditional and teacher dominated in these developing countries compared to more varied teaching methods in Western countries.

Social media: the use of mobiles and social media can be a distraction, but social media has much promise

The presence of mobile phones, Facebook, social media platforms, and fun game apps in the classroom have a negative effect. Who knew! But the negative effects are not strong enough to warrant overreaction. Teaching students to control their distracted use of iPhones, games, and so on during learning time may be more worthwhile than outright bans, particularly, given the potentials offered by the use of social media apps (without the negative features of Facebook, WhatsApp, etc.).

*Kates et al. (2018) reviewed experimental and cross-sectional studies where cell phones were being used for non-educational purposes. They found an overall correlation of r = −0.16 among achievement with the number of text messages or phone calls sent/received in a given period (r = −0.12), with a measure of mobile phone addiction (r = −0.12) or with time spent using a mobile phone (r = −0.15). The correlation was similar for K–12 (d = 0.12) and high school students (d = −0.17), and across country regions (Middle East, −0.11; Asia, −0.13; and US, −0.20). *Adelantado-Renau et al. (2019) were interested in the consequences of excessive screen time (computer use, internet surfing, mobile phone use, television viewing, video game playing). From 58 studies, the overall effect on academic performance was r = −0.15 (language, −0.18; math, −0.25) and similar for video game playing (−0.15) and television (−0.19). *Marker et al. (2018) found r = 0.07 from time on sites such as Facebook and Twitter and achievement, and −0.03 to time spent studying for schools using these sites. In a similar meta-analysis, *Liu et al. (2007) reported r = 0.08, and *Huang (2018), r = −0.07 – overall, the effects are close to zero. These close-to-zero relations were not moderated by time spent on these sites (r = −0.10) and visiting frequency (r = −0.01), and the correlations were similar for males and females.

This is not to imply that mobile technologies are not potentially worthwhile media to deliver instruction. *Fabian et al. (2016) noted that many of the use of these technologies replicated the more traditional computer-based learning activities, with similar findings. They found an overall d = 0.30 and greater improvement in low-compared to high-skills groups. Sung et al. (2016) found d = 0.52 compared to instruction without mobile devices or compared to desktop computer use. The effects were higher the more the app on the mobile related to learning (d = 0.63) when used in the classroom (0.50) compared to other settings; for self-directed study (0.31) compared to lectures (0.10), discovery and exploration (0.12), problem-solving (0.09), podcasting (0.01), and project-based learning (0.01); across all levels of schooling; and particularly, in language arts (0.35) and science (0.23). It is less the device or medium for delivering the learning and more the quality and intentional alignment of the app to the success criteria of the lessons.

*Davies (2012) has led fascinating research observing students developing critical thinking skills. When using social media tools (e.g., Edmodo, akin to Facebook), students were more engaged and were less shut down by teachers. Instead, the online discussion forum allowed students to keep conversations active. They were likely to talk about what they did not know and discuss their errors and misconceptions with peers and teachers via social media tools – even when the teachers were standing beside them. Students are familiar (for good and sometimes for bad) with social media as a forum for discussion. As well, teachers noted that some students who did not normally contribute to classroom discussions contributed online, and students were less concerned about what they looked like in front of their peers and were much more likely to move from surface to deeper interactions (cf., *Blok et al., 2002).

Social media: the use of computers is more effective when peer learning is optimized

Working in pairs is much more effective than when computers are used alone or in larger groups. *Lou et al. (2001) reported higher effects for pairs than individuals or more than two in a group and a higher frequency of positive peer interactions (d = 0.33), higher frequency of using appropriate learning or task strategies (d = 0.50), more perseverance on tasks (d = 0.48), and more students succeeded (d = 0.28) than those learning individually when using technology. *Gordon-Holliday (1991) found effects were larger for learning in pairs (d = 0.54) compared to alone (d = 0.25); and *Kuchler (1998) reported d = 0.69 for pairs and d = 0.29 for individuals (see also Liao, 2007). Students learning individually requested more help from the teacher (d = 0.67) and accomplished tasks faster than those working in groups (d = 0.16). Students in pairs attempted a greater number of tasks (d = 0.15), used more learning strategies (d = 0.36), and had a more positive attitude toward small-group learning d = 0.54), but there was little difference in attitude toward instruction (d = 0.07), and they needed more task completion time in groups than when alone (d = −0.21; *Lou et al., 2001). These results show that when learning using technology, it is important for discussions to be emphasized and for each student to work with a peer to articulate, explain, and understand a variety of possible hypotheses and solutions. Peers in pairs seem powerful.

Teachers are major users of social media

Cuban (2001) has outlined reasons why technology has been “oversold and under-used” and has tracked “how little school and classroom practice changed in the direction (the high-tech advocates) sought, the anemic results in student achievement, and uncertainties in getting the right jobs after graduation, given the claims accompanying these new technologies” (Cuban, 2018, p. 1). One of the fascinating findings is that teachers are major users of computers – but more for their personal and administrative use, although they find it more difficult to see how computers can be related to their particular conceptions of teaching (Cuban, 2001). When many of today’s teachers were students in schools, computers were not as common, and many were then taught in teachers’ colleges by lecturers who were even more distanced from the use of computers in their teaching and learning. For too many teachers, teaching using computer resources is not part of their grammar of schooling. *Abrami et al. (2006) noted that many teachers “are still on the threshold of understanding how to design courses to maximize the potentials of technology” (p. 32). Hence, there needs to be some pretraining in using computers as a teaching and learning tool for that use to be effective. Such pretraining is now most often built into apps (who reads manuals upfront anymore?). But teachers are big social media users.

The Australian Institute for Teachers and School Leadership (AITSL) realized the potential to connect teachers using digital channels early and has successfully leveraged technologies over the past ten years to support quality teaching and school leadership across Australia. AITSL’s My Induction app, for example, is designed as a mentor in the pocket for beginning teachers and has had over 18,000 downloads since its launch in 2017 (www.aitsl.edu.au/teach/improve-practice/start-your-career/my-induction-app). The app has advice and scenarios to help beginning teachers familiarize themselves with teaching, a well-being tracker, and the ability for newer teachers to submit questions for response by experienced teachers. A second web-based app, My Teaching Advice, allows any teacher in their first five years of teaching to log in and have a live chat with a more experienced teacher to troubleshoot problems of practice and get advice. For teachers in rural or remote settings or in other contexts where it can be hard to access the expertise of others (e.g., early childhood settings), this app removes those barriers through technology.

As part of this, AITSL has also focused on social media and has almost 120,000 followers across Twitter, LinkedIn, and Facebook. This gives AITSL an unrivaled direct connection to teachers across Australia. One Tuesday night, I answered teachers’ questions (#HattieChat). It trended at number 2 nationally (with a reach of 100,000). A private Facebook group (Teachers Supporting Teachers), launched during remote teaching in May 2020, has almost 10,000 teachers from across Australia, posing questions and sharing advice to help them improve. Most teachers use technology daily with their students, which translates to their professional lives where they actively connect and share, whether it be through an app, Twitter chat, or Facebook group. This may be the greatest benefit of technologies in schools.

The use of technology for teaching via distance learning can enhance learning

There have been 17 meta-analyses on distance learning with d = 0.05, and 5 more recent metas favoring distance education with d < 0.30 (ave d = 0.59). The preponderance of the evidence shows little difference in the achievement of students regardless of the medium – distance or in-class. However, there are attributes of distance learning that can increase the impact, such as the use of blended learning with greater use of student-to-student interactions. For example, *Bernard et al. (2009) showed when students are given the means to communicate and interact with one another (0.49) or interact online with content (0.46), the effects on achievement are higher than when they interact or watch teachers work online (0.32). *Means et al. (2009) similarly found that the effects were higher for peer-to-peer collaboration (0.25) compared to independent, active online learning (0.05). Further, as noted earlier, students working on their own are not as effective as when working with peers and even more effective when paired up with an active online expert (e.g., a teacher who has thought through the intentional alignment of the use of the app, etc., with the other parts of the teaching and learning program).

*Roberts (2002) found higher effects for online learning when students engaged in independent study as part of a sequence of instruction (0.85) and teaching using behavioral (d = 0.81, more resource content focused with limited student interactions) and constructivist methods (d = 0.70, more instructor-designed student-student, and student-teacher interaction). In addition, the effects were highest when there were instructor-led interactions (d = 1.01) than where there was no provision for collaboration (d = 0.81) and when it was required but did not involve the instructor (d = 0.45), and lowest when it was offered but not required (d = 0.28). This aligns with their finding that the effects were higher where there was formative feedback provided (d = 0.87).

Few people or content moderators are affecting this conclusion of no difference. For example, *Cavanaugh (2001) reported no moderation relating to academic content, grade level, type of school, frequency of the distance learning experience, timing of instruction, instructor preparation and experience in distance education, and the setting of the students (see also *Bernard et al., 2009; *Lou et al., 2006; *Zhao et al., 2005).

Imagine an experiment. We close all in-class schooling and force all students to go online from home, then bring them back into class, repeat banning them to their homes, and repeat. Of course, you would never get ethics permission for such an experiment. But this is what has/is occurring during the COVID pandemic. Teachers have found it challenging to replicate the old grammar of schooling online (e.g., lots of teacher talk, surveillance of doing the work, content-focused IRE questions). Instead, teachers had to teach students to use self-regulation skills and the skills for working in peer groups without teacher surveillance, and they had to gradually release responsibility (Hattie, 2021).

The more recent use of apps is underdeveloped concerning impact on learning

Given technology is almost everywhere, the growth, use, and effectiveness of apps need to become more the focus of research, but the meta-analyses literature is not very mature in evaluating apps. Like the early days of computers, there is still too much app advocacy and attestation. What is needed is evidence of the impact on students’ motivation, affect, and achievement progress. There are hints as to the value of some apps: dynamic geometry software (*Chan & Leung, 2014, d = 1.02), captioned video (*Perez et al., 2013, d = 0.99), vocabulary video game (*Thompson, 2020, d = 0.70), Kahoot! (*Yu, 2021, d = 0.65), computer-based graphics (*Gordon-Holliday, 1991, d = 0.26), 3D virtual environments (*Avci et al., 2019, d = 0.25), and interactive story-books (*Takacs, 2016, d = 0.16).

There was a flurry of interesting audience response system apps. Clickers are instant response systems, allowing teachers to ask questions and have students respond, immediately see the range of responses, and thence, react by focusing on errors, moving on when there is mastery, or attending to specific student misunderstandings. From 28 studies, *Chien et al. (2016) reported an effect of 0.49 on immediate post-tests and 0.34 on delayed post-tests. The effects were similar regardless of the length of their use in classes but much larger (d = 1.09) when clicker responses were followed by inviting discussions among peers and across the class to articulate their thought processes and then revealing the correct answers. Chein et al. argued that it was the feedback-intervention effect that accounted for the superiority of clicker-integrated instruction. *Castillo-Manzano et al. (2016) found 0.29 from 51 studies with non-university students, which was twice the effect (0.48) compared to university students (0.22), and the effects in the applied sciences (0.38) were larger than in the pure sciences (0.05) – which they suggested was because students are more likely to be amenable to active learning. *Hunsu et al. (2016) used 53 studies but a very low overall effect (d = 0.05) on cognitive (retention, 0.00; knowledge transfer, 0.03; final achievement, 0.21) and 0.23 on noncognitive (e.g., attendance, 0.21; engagement, 0.19; self-efficacy, 0.86). After some worthwhile comparisons, they concluded that it was the asking of questions that may explain the effect, as the nonclicker classes did not differ when there was a “similar question-driven pedagogy.” Anthis (2011), for example, found similar effects when students used clickers or when they were asked to raise their hands in class. The effect of classes with similar levels of questions, with or without click-ers, was 0.00. Overall, increasing the opportunities for asking questions, providing feedback, and sharing thinking aloud is most influential, and clickers can enable these to occur with a positive impact.

Learning: the possibilities of gaming and simulations

I can become absorbed in video games, and my all-time favorite is Angry Birds. I like the sense of challenge, the clarity of what success looks like, the emotion when I succeed, and the fact that winning means more learning and playing. But I have no delusions that playing Angry Birds increases my achievement in any worthwhile domain. *Ferguson (2015) found that such games have a −0.01 effect on improving achievement and equally low effects on aggressive (0.06) or prosocial behavior (0.04). Much has been made of the fun, engaging, and competitive aspects of gaming and simulations. There are 39 metas on gaming and simulations with an overall d = 0.43, and the retention effect of gaming over nongaming is also reasonable (d = 0.36) (*Wouters & Oostendorp, 2013).

An important implication of the findings is that these games may be most worthwhile to teach consolidation of learning, as they have a much more enjoyable way to drill and practice than mere repetition. The skill then relates to when to stop the consolidation and use the learned skills in deeper learning – which is unlikely to come from the same game/resource that led to successful consolidation. It is couching the deliberate practice in a meaningful and engaging game where students have to interact and negotiate meaning. A typical finding is the value of such deliberate practice with students with special needs, but the claim about gaming for consolidation applies to all learners (*Chiu et al., 2012). *Merchant et al. (2014) showed that the effects for procedural tasks (0.62) or opportunities to practice (0.59) were higher than for declarative tasks (0.47), and both were higher than for knowledge-based learning (0.00). *Sitz-mann (2011) also found the effects on procedural tasks (0.37) higher than for declarative tasks (0.28).

Gaming and simulations can assist in critical thinking and the development of alternative learning strategies, as they often require thinking, decision-making, and weighing options about alternative solutions; they demand flexibility in using different strategies to solve dilemmas; and they value trial-and-error thinking with immediate feedback. *Mao et al. (2022), for example, noted that simulations often involve “complex problems with imperfect information let students try different strategies to solve them, and then give feedback to help them evaluate their information use and decisions, which improves their critical thinking” (p. 1683). Similarly, *Bediou et al. (2018) noted that many games and simulations require strategy shifts (task-switching, d = 0.55), as the same strategy does not always work. Thus, players need to try others (cf., *Chernikova et al., 2020).

Another reason for the power of gaming and simulation is that they promote goal-seeing by the learners, they can lead to reputation enhancement among peers, and they provide immediate feedback on performance. It helps when the game is a narrative or story than not (*Mao et al., 2021; *Sailer & Homner, 2020). The effects are also higher when the game is part of and supplemented with other instructional methods (*Sitzmann, 2011, 0.51 vs. −0.12 when standalone; *Wouters et al., 2013, 0.41 vs. 0.20 when standalone).

The effects of gaming are high at all levels of schooling (*Bai, 2020; *Lamb, 2018; *Tsai & Tsai, 2020) and higher when there is less collaboration (*Chernikova et al., 2020; *Clark et al., 2014; *Duan, 2017; *Merchant et al., 2014; *Yildririm & Sen, 2019); for affective and motivational outcomes (*Lamb et al., 2018; *Sailer & Homner, 2020; *Wouters et al., 2013); and across most disciplines (*Bai et al., 2020), although some found higher effects in the non-STEM domains (*Osenbach, 2011; *Yildirim & Sun, 2019). Shorter bursts are more effective than longer interventions (*Bai et al., 2020; *Merchant et al., 2014).

Simulations typically involve the use of a model or game (such as role-playing, decision-making) with an aim to engage students in learning (although some games are not engaging or fun). Many simulations are not competitive but do aim to mimic real-world problems. *VanSickle (1986) found small effects (d = 0.12) for recall of knowledge of facts, concepts, and generalizations, and d = 0.18 for retention over time. He concluded that these findings show that simulating and gaming have a small positive effect over alternative instructional techniques, although somewhat larger when compared with lectures only (d = 0.32). *Dekkers and Donatti (1981) found slightly higher effects for achievement (d = 0.33) and similar effects for retention (d = 0.15) but much higher attitude effects (d = 0.64). *McKenna (1991) found a similar effect (d = 0.38) and reported no differences over age groups, but simulations were more effective with lower- than higher-ability students. *Lee (1990), however, found simulation and gaming had higher effect sizes in achievement when used with students in higher grade levels. As did *Dekkers and Donatti, McKenna found that shorter (up to one week) interventions were more effective than longer interactions. *Remmer and Jernsted (1982) examined the effectiveness of simulation games in high school and college level instruction. The effects on achievement were small, leading them to conclude that the use of simulation games on achievement and retention was not more effective than conventional instruction. *Armstrong (1991) found an overall effect (d = 0.29) between computer-based simulations and traditional instruction, and the effects were similar for low-level thinking, high-level thinking, and retention outcomes.

*LeJeune (2002) used simulations in science that model real-world phenomena or duplicate traditional laboratory activities. He divided achievement outcomes into surface (d = 0.34) and deeper outcomes (d = 0.38), and found no effects on attitudes (d = − 0.03) or on retention at least two weeks later (d = 0.19). The effects in colleges (d = 0.49) were much greater than in K–12 (d = 0.14). The surface outcomes were greater when taught to confirm what had been taught (d = 0.44) compared to allowing the students to explore during their learning (d = 0.27), but there were no differences when the outcomes were deeper thinking (d = 0.35 vs. d = 0.41). He concluded that these simulations improved low-level achievement, such as the ability to learn scientific facts, comprehend scientific processes, and apply that knowledge to everyday phenomena; and deeper outcomes, such as problem-solving ability and other high-level thinking skills.

There is excitement about virtual reality and interactive video uses of technology

Engaging students in virtual reality and interactive uses of video is commonplace in the gaming world, and this sets high expectations for using these methods in learning activities. The earlier implementations had low effects. *Blanchard et al. (1999) used meta-analysis over ten implementations of a multimedia curriculum based on video games. They found a very low overall effect, both in mathematics (d = 0.13) and language arts (d = 0.18), and in high (d = 0.23) and low (d = 0.16) quality implementations of the multimedia method. *Liao (1999) looked at the effects of hypermedia vs. traditional instruction on students’ achievement and found positive effects for hypermedia over traditional instruction. The effects were greater when regular class teachers rather than specialist teachers were used, in elementary school compared to high school, and when used to supplement rather than substitute regular instruction. Liao noted that hypermedia might be more effective when used to supplement traditional learning.

But the last 20 years have seen much higher effects, as the technology has improved. *Baker and Dwyer (2000) explored the instructional effects of visualization compared to no visualization in interactive video (d = 0.71) and argued that visual presentations can convey the essence of the message to be learned. **Clark and Angert (1980) carried out a meta-analytic study on pictorial effectiveness, which focused on the use of static iconic visuals in instructional materials. Four major variables were investigated: illustrations, pacing, grade level, and achievement. Five physical attributes of illustrations were also investigated: production, shading, context, embellishment, and chroma. Illustrated materials were more effective than verbal descriptions (particularly with high school students), and color illustrations were more effective than black and white.

*Garzón (2019) reported 0.64 from augmented reality, with similar high effects at primary (0.65), lower secondary (0.60), upper secondary school (0.70), and undergraduate (0.62). The effects were highest in the natural sciences (0.69), social science (0.71), health (0.81), and arts and humanities (0.96); and lower in information and communication technology (0.36) and education (0.27). *Zhao et al. (2020) specifically investigated virtual reality in university anatomy classes with d = 0.53 on learning gains and 0.77 on greater satisfaction and interest in learning. *Thompson and von Gillern (2020) investigated language acquisition for ESL students, d = 0.70, with higher effects for college students (0.83) than high school (0.52) and K–4 students (0.57). Finally, *van Daal and Sandvik (2012) investigated multimedia interventions on alphabetic knowledge (0.64), phonological awareness (0.75), rapid automatic naming (0.21), concepts of print (0.86), vocabulary (0.68), word reading (0.53), spelling (1.11), comprehension (0.52), and syntax (0.66).

*Merchant et al. (2014) reported 0.51 from games, 0.44 from virtual worlds, and 0.41 from simulations. The effects were higher for declarative tasks (0.68), then procedural tasks (0.25), and zero for skills-based tasks (0.00). Elaborative explanations were more effective than visual clues for declarative tasks, and knowledge of correct response was more effective than visual clues for procedural tasks. The effects were higher when students worked alone in these worlds (0.72, alone; 0.01, collaboratively). They argued that students need the skills base before engaging with these gaming and virtual reality, and that there are many other more efficient ways to gain the skills before engaging with the virtual worlds.

*Avci et al. (2019) reviewed studies using three-dimensional (3D) virtual worlds (d = 0.33) and augmented reality technologies (d = 0.46). *Ozdemir et al. (2018) reviewed augmented reality interventions, d = 0.52, with no differences related to curricula (natural sciences, 0.56; and social sciences, 0.41); higher in undergraduate (0.84) and high school (0.62) than elementary (0.30); and higher when delivered via mobile devices (0.69) and tablets (0.67), compared to webcam-based devises (0.16) They argued the major explanation for the effects was that AR enables students to practice what they are learning in an entertaining environment. Once again, this hints at the high value of these interventions to assist in consolidating learning and engaging students in deliberate practice. *Ozdemir et al. (2018) concluded that augmented reality helps construct students’ knowledge to clarify the relations among theoretical concepts or principles.

Consolidation: the power of technology when used to assist students in pacing and mastery, and when focused on where-to-next

One early example of technology making learning more efficient is the influence of word processors. When using these packages, students tend to write much more than when asked to write on paper, and the quality of writing is enhanced especially for the weaker writers (*Bangert-Drowns, 1993). This more is not more low-quality writing, as the quality of writing and length were highly positively related to the use of word processing: students are more likely to make revisions, write more, and make fewer errors (*Goldberg et al., 2003; *Schramm, 1989). *Torgerson and Elbourne (2002) found that, on average, students who used computers when learning to write were not only more engaged and motivated in their writing but produced work that was of greater length and higher quality than students learning to write on paper (d = 0.40).

There are now many intelligent tutoring systems (ITS). These systems typically perform tutoring functions by presenting information to be learned, asking questions, assigning learning tasks, providing feedback or hints, answering questions posed by students, or offering prompts to provoke cognitive, motivational, or metacognitive change. These systems then compute inferences from student responses to adapt various tutoring functions and offer feedback, offer hints, deal with common misconceptions, notice the ways students solve problems, and can recommend next learning moves based on matching trajectories of the particular student with students with similar trajectories in their databases to recommend the optimal next strategies.

*Ma et al. (2014) reviewed 107 studies and found that ITS was associated with greater achievement in comparison with teacher-led, large-group instruction (g = 0.42); non-ITS computer-based instruction (g = 0.57); and textbooks or workbooks (g = 0.35). The effects were similar whether the ITS was used as the principal means of instruction (0.38), a supplement to teacher-led instruction (0.33), separate in-class activities (0.47), supplementary after-class instruction (0.43), or homework (0.45). The ITS based on Bayesian network modeling (0.54) and explanation and misconception tailoring (0.34) were most effective. The effects were similar across grade levels (elementary, 0.31; middle school, 0.41; high school, 0.40; tertiary, 0.43) and across many subjects. The effects were similar on transfer (0.44) or retention tasks (0.35) and on both procedural (0.39) and declarative tasks (0.37), but higher when both (0.65).

*Gerard et al. (2015) reported d = 0.34 and concluded that instruction with automated guidance added much value over typical classroom instruction. There was no significant difference among the different typical instructional conditions (worksheets with no teacher guidance, 0.44; regular teacher-facilitated instruction, 0.37; same computer program with non-adaptive guidance or teacher-generated, 0.25). They found that students with low prior knowledge (0.59) had much greater effects than those with high prior knowledge (−0.14). They noted that when the automated guidance was available upon the student’s request, low-performing students took greater advantage of its help features (e.g., more frequent requests for detailed hints) than did high-performing students; and this guidance encouraged students to reconsider their ideas, provided hints, and aimed to improve the students’ self-monitoring strategies (and did not merely provide the correct answer). The students with high prior knowledge were more likely to ignore the automated guidance. The effects were similar regardless of grade level, curricula domain, duration of the intervention, or type of scoring technology.

*VanLehn (2011) found similar high effects for intelligent tutoring systems (d = 0.76) and human tutoring (0.79), but both were much higher than regular instruction with no tutoring (0.30). His explanation for the success of the intelligent tutoring system was that it gives immediate feedback on individual steps and as follows:

students can examine the first incorrect step and know that one of the inferences that led to it must be wrong. As long as the tutoring system ensures that there is only a little reasoning required for each step, then compared to answer-based tutoring, students should find it much easier to find and fix the inference that caused a step to be flagged as incorrect. Moreover, step-based tutoring systems usually give hints to make it even easier for students to self-repair their knowledge. Thus, facilitating self-repair provides one explanation for the observed interaction plateau if we assume that debugging the reasoning behind an incorrect step during step-based tutoring is not much more difficult for students than debugging the reasoning behind an incorrect utterance to a human tutor.

(p. 211)

Consolidation: the use of technology is more effective when there are multiple opportunities for learning (e.g., deliberative practice, increased time on task)

There are many ways whereby computers can assist during the consolidation phase of learning – as many apps can provide students with multiple but varied learning opportunities and deliberate practice. Such practice need not be deadly, and a critical skill of many apps is to make deliberative practice engaging and worthwhile. Luik (2007) classified 145 attributes of computer drills into six categories: learner motivation, learner pacing, presentation of information, characteristics of questions, characteristics of replying, and feedback. The key attributes that led to the highest effects included learner pacing, not losing sight of the learning goal, and the immediate announcement of correctness or otherwise of the answer to the drill. Note that most computer games include much engaging drill and practice with increasing levels of challenge that usually is mastered by overlearning or undertaking high degrees of drill and practice. Also note that it is the understanding of prior learning (a student’s past score or level), many opportunities to practice and learn, much feedback, and clear and challenging success criteria that are the key fundamentals to making deliberate practice engaging and worthwhile to students (cf., *Burns & Bozeman, 1981). Perhaps teachers should pause and wonder why their traditional teaching is less effective than many computer drill-and-practice apps at consolidating learning. A major saving for teachers is to intentionally include consolidation apps in their design of teaching.

Consolidation: the use of technology is more effective when feedback is optimized

Feedback feeds on error, and a high-trust environment is needed for many students to acknowledge their lack of understanding. *Timmerman and Kruepke (2006) found that explanations (d = 0.66) and remediation (d = 0.73) are much more effective than just provision of the correct answer (d = −0.11, see also *Cohen & Dacanay, 1994).

A further advantage of computers is that they respond to the student despite being male or female, Black or white, slow or fast. Teachers claim expertise in their flexibility in anticipating students’ reactions and deciding when and to whom to provide feedback, but given the low levels of feedback in most classrooms, it is clear that this flexibility means many students miss out. Computer feedback is potentially less threatening to students and can occur in a more programmed manner (*Blok et al., 2002). Praise often dilutes the effects of feedback as we recall the praise and can reduce the interpretation of the corrections and the request to go deeper, do more, and revise our work (see Hattie & Clarke, 2021). Computers, as inanimate objects, have no need to be liked, bond, or create memories of relationships with students. Thus, feedback is more likely to be heard, understood, and actionable without moderating relationships, liking or not.

As noted earlier, there are many types of feedback, and feedback is optimized when there are appropriate, challenging tasks (thus, inducing error). *Timmerman and Kruepke (2006) found that explanations (d = 0.66) and remediation (d = 0.73) are much more effective than just provision of the correct answer (d = − 0.11, see also *Cohen & Dacanay, 1994). *Lou et al. (2001) found that effects were more positive when tasks were challenging (d = 0.13) than when moderately challenging (d = − 0.34) or not challenging (d = − 0.57). There seems to be low value in asking students to engage in computer-assisted instruction activities unless there is some challenge.

The meta-analysis by *Gillingham and Guthrie (1987) provided the highest average of all computer-assisted instruction studies, but it was based on only 13 studies. Nevertheless, they established three critical principles included: the teacher needs to use computer-assisted instruction to manage the attention and motivation of the learner, the teacher needs to use computer-assisted instruction to present new subject matter content and learning strategies to the learner, and the teacher needs to use computer-assisted instruction to guide the practice and active involvement of learners.

Conclusions

It is time to think of technology like curricula, books, and other resources, and focus more on the implementation effects when including technologies along with teaching methods, the when and how within the learning cycle, and the dosage and fidelity of various apps.

There are exciting developments in technology to open up to teachers the nature of their delivery and impact within classrooms, and provide automatically coded diagnoses of the teaching in real time.

There are exciting developments in technology to help teachers realize the students’ sense of learning, engagement, and emotions during the learning.

There are exciting possibilities in technology for teachers to assist in consolidating learning; diagnosing enablers and barriers to student learning; using social media to create learning communities (for students and teachers) and better allow for expressing not knowing, misconceptions, and errors; tracking progress over time; and developing intelligent tutoring systems.

If there is to be a third edition of VL, there would not be a chapter on technology influences, as it is time to think of technology like curricula, books, and other resources, and focus more on the implementation effects when including technologies and teaching methods, the when and how within the learning cycle, the dosage and fidelity of various apps, and so on. Given the COVID pandemic, teachers use aspects of technology as wallpaper to deliver their lessons, and many have learned that this medium is not conducive to the old grammar of talking, asking content questions using the IRE format (see p. 328), and making students do tasks under their surveillance.

The message throughout this chapter is not that technology has not made a difference. It has (as that is what an effect of 0.3–0.5 signifies). There are not many classes across developed countries where technology is not ever-present, and if their impact were measured in terms of efficiencies, the effects would be much higher. It is the smart use of technologies to consolidate learning, engage students in peer-to-peer-to-teaching discussions, think aloud, and assist in pacing and feedback that offers the most promise.

Teachers are major users of technology, and many digital tools have aided teachers in keeping a lesson’s momentum on course and avoiding distractions. However, there is still a constancy in how teachers set goals, organize, and execute lessons: these have not been replaced by digital tools (Cuban, 2018). To paraphrase Schön (2017) – schools fight like mad to stay the same. Even under the current grammar of schooling, IT still has promise. But it is not IT; it is ITT – It’s the Teaching. Or more correctly, it is the teaching as well, as it needs to be highly effective; and with apologies to JFK, ask not what your teaching can do with IT, ask what IT can do for your teaching. The technology is now almost immaterial – it is what we use it for, how we use it, and the quality of this use. The principles of teaching developed so far throughout this book are similarly applicable when technology is involved, although as we will see, there are some major assets that technology can bring to make this quality teaching more efficient and effective.

Bringing together the power of technology in diagnosing, tutoring, and providing feedback is coming together with exciting possibilities. For example, Sinclair et al. (2021) used advances in machine learning and artificial intelligence to diagnose reading comprehension learning. They identified 260 linguistic features extracted from children’s speech and writing, used the simple view of reading (Gough & Tunmer, 1986), and aimed to identify the top lexical and syntactic feature predictors for the oral- and text-elicited data sets. Such models can help teachers provide more specific interventions (e.g., relating to developing syntax, lexical, affect) and help evaluate the advances students are making. They are more likely to branch when mastery is reached or no progress is being made.

Another major set of advances is helping teachers see their classes through the eyes of others. For example, Clinton and Dawson (2018) developed an app, VisibleClassroom, that records and transcribes the lesson and provides 16–20 automatically coded diagnoses in real time. From their database of 15,000 lessons, teacher talk, on average, 89% of the time, asks 130 questions, most requiring less than three-word answer, and over 90% of the interaction is about the content, the facts, the knowing-that. Moreover, above-average students prefer more teacher talk and focus on facts, as that is the game in which they are winners; it is students below average that want the teacher to shut up and listen to how they are processing, provide optimal learning strategies, and hear how they are working through their work so that teachers can diagnose and offer alternative ways to learn.

One of the major implications of the COVID pandemic distance learning is that nearly every teacher now is a user of technology. As noted earlier, the powers of social media can be most positive, and there needs more consideration in developing design principles for incorporating these positive aspects into in-class lessons (e.g., see Boelens et al., 2017). The successes outlined throughout this chapter point to exciting possibilities, such as improving resources/apps for teachers to assist in consolidating learning, diagnosing enablers and barriers to student learning, using social media to create learning communities (for students and teachers), tracking progress over time, and using intelligent tutoring systems.

Reflections

Consider the many ways teachers and leaders use social media to enhance professional development, to communicate about teaching and learning, to listen to podcasts about education, and to find resources for teaching. Create a top hits list of sites in your staffroom and regularly update this list.

During the COVID pandemic distance learning, educators became digital natives. Ask your students what they preferred during this time and how you could implement many of the more beneficial aspects of distance learning in your school. Consider at least one day a month where all students must work online alone and with others (this can be undertaken within the school rather than home).

Explore the use of social media apps to allow students to talk with you and their peers about what they do not know, what they are struggling with, and where they consider they need to relearn.

CHAPTER 15 Whole-school and out-of-school influences

DOI: 10.4324/9781003380542-15

So much of a young person’s life is spent not at school. Between 5 and 18, young people spend 5.5 hours × 200 days a year = 15,400 hours in school and, over the same time, are awake (assume 9 hours sleep) 48,545 hours – so 68% of their awake time is not in school. So much of their physical experiences, emotional development, social interactions, leisure activities, and family time impact their learning, as do their skills to recharge their batteries for learning at school. While it would be fascinating to have meta-analyses on the effects of these activities, the focus here is on school-related activities expected to be completed outside the school.

This chapter reviews two significant themes: school-wide initiatives and outof-school influences. A major reason for many, both forms of interventions are to reduce the achievement gaps among groups of students or increase the impact on students with specific learning needs (Table 15.1).

Table 15.1 Summary statistics for whole-school and out-of-school variables

Whole-school and out-of-school
No. metas
No. studies
Total No.
No. effects
Weighted ES
SE
Robust ness

Whole-school improvement programs
2
50
41,709,233
707
0.29
0.01
3

Whole-school reduction of achievement gap
1
30
2,761
30
0.11
0.00
1

Interventions for students with learning needs
5
424
47,146
2,878
0.74
0.03
4

Corporal punishment
1
20
977,367
47
−0.22
0.02
2

Special college programs
2
93
8,560
93
0.20
0.04
2

Service learning
8
157
21,152
118
0.53
0.08
3

Home-school programs
1
14
1,289
14
0.16
0.00
1

After-school programs
8
309
535,374
940
0.24
0.09
4

COVID
3
40
3,589
62
0.09
0.08

Extracurricular activities
11
250
247,965
1,145
0.10
0.05
4

Homework
9
265
337,096
434
0.29
0.06
4

Total/average
48
1,612
43,887,942
6,406
0.24
0.04
3

Whole-school improvement programs

Since VL1 was published, the VL+ school-wide model has been developed and implemented in over 10,000 schools worldwide. After ten years, we took stock of the evaluation evidence from these experiences. The engagement within systems or schools commences with completing a School Capability Assessment where schools provide evidence related to 12–18 dimensions and rate their impact using a traffic light system (green for regular practice in this school, to red for not evident in this school). This helps establish both a baseline and a focus for deciding which parts of the VL+ model to implement (e.g., feedback, mind frames, collective efficacy, assessment capable students). The baseline also becomes core to the ongoing evaluation of impact. From this assessment, a program logic model is developed with a particular focus on shorter-, medium-, and long-term outcomes, and then the 5D Building to Impact model of implementation is introduced (Hamilton et al., 2022; see Chapter 4).

Clinton and Clarke (2020) completed a Decade of Impact report based on data from 11,000+ school leaders who completed the School Capability Assessment, 29,000+ teachers who completed the mind frames survey, 13,000+ who completed the feedback survey before and after the implementation, as well as 47 in-depth case studies. An important part of the model is moving the focus from improving teaching to the impact of this teaching, from collective efficacy to collective efficacy about the impact on students, and from teaching to learning. We have been struck by the lack of implementation models in schools with little attention to discovery, dosage, fidelity, adaptation, quality, or formative evaluation of implementations.

Most teachers and school leaders (80–95%) claimed that the VL+ program improved teacher practice, student engagement, and student achievement. The effect sizes from pre- to post-program include the following: the program developed a shared language of learning across the schools (0.83), students became more assessment capable (0.79), there was a greater familiarity with effective feedback (0.58), and there were gains in reading (0.55) and mathematics (0.57). In addition, in 91% of the case study schools, there was convincing evidence of an increase in student engagement in lessons. From classroom observation studies, there were increases in the number of classes (pre- to post-implementation), where students asked clarifying questions about the learning intentions (39% to 55%), students worked in groups and had significant conversations with each other (19% to 32%), students could state that they were learning and talking to other students, the teacher discussed learning strategies with students (31% to 63%), students supported their peers’ learning (37% to 72%), students sought feedback and regarded errors as opportunities (39% to 81%), and students became more aware of their learning steps (30% to 52%).

The evaluation pointed to the need for strong buy-in and engagement by the school or system leaders, the importance of ascertaining readiness for implementation, the critical roles of the consultants, the value of a well-articulated implementation and evaluation model, the importance of developing a shared language of learning, the collective efficacy focus on impact, and the attention to the many barriers and enablers relating to resourcing, leadership, and mindset.

In one of the more ambitious meta-analyses, *Borman et al. (2003) reviewed the achievement effects of comprehensive school reform programs. They noted that many comprehensive programs were being scaled up at an unprecedented rate and that these programs were serving millions of students and being implemented in many school districts. Such comprehensive reform appeals to many superintendents and school officials as a systematic answer to the issue of improving teaching. They noted that effect sizes from studies undertaken by the developers of the programs were larger, and there were no differences in effects relating to socioeconomic background of the school, there were very little difference by subject, and the effects became most apparent after the fifth year of implementation. They considered the programs with the strongest systematic evidence of effectiveness to be Direct Instruction (d = 0.21), Comer’s School Development program (d = 0.18), and Success for All (d = 0.18). Other programs (with more than ten effects) are listed in Table 15.2.

Table 15.2 Summary of effects for comprehensive teaching reforms

Program
No. studies
No. effects
d
SE
Age
Focus

Roots and Wings
6
14
0.38
0.04
K-6
students

High Schools That Work
45
64
0.30
0.01
9-12
curriculum

Microsociety
3
32
0.29
0.03
K-8
students

Modern Red Schoolhouse
6
23
0.26
0.03
K-12
curriculum

Onward to Excellence II
4
13
0.25
0.03
K-12
curriculum

American's Choice
2
27
0.22
0.02
K-12
standards

The Learning Network
3
38
0.22
0.02
K-8
teaching

Direct Instruction
49
182
0.21
0.02
K-8
students

Expeditionary Learning Outward Bound Students
6
40
0.19
0.03
K-12

Success for All
42
173
0.18
0.01
K-8
students

School Development Program
10
25
0.15
0.03
K-12
community

Centre for Effective Schools
1
26
0.13
0.01
K-12
students

Accelerated Schools
6
50
0.09
0.02
K-8
students

Edison
5
209
0.06
0.01
K-12
school

Co-nect
5
42
0.04
0.02
K-12
curriculum

Community Learning Centers
5
17
0.03
0.03
K-8
curriculum

Core Knowledge
6
58
0.03
0.02
K-8
curriculum

High/Scope
4
23
−0.02
0.04
K-3
curriculum

Source: Borman et al. (2003)

The critical components of these programs were ongoing professional development, measurable goals and benchmarks for student learning, a faculty vote of the model’s acceptance and buy-in, the use of specific and innovative curricular materials, and instructional practices designed to improve teaching and student learning.

Given many schools and systems implemented more than one intervention and many interventions are introduced before great diagnosis, the low level of the effects of school reform is perhaps not surprising. Moreover, there is often incremental growth of new programs and interventions in schools. Our greatest barrier in VL+ is when the principal, superintendent, director-general, or minister changes, as then many successful programs are abandoned for the new broom. These new folk searches for failure to justify introducing their new broom or solution from their kit of interventions in their toolbox. However, scaling up success starts from the opposite assumption: reliably identifying success and scaling this up. Unfortunately, there are no meta-analyses on scaling success, and indeed, there is very few literature on this topic (but see Clarke & Dede, 2009; Cohen & Mehta, 2017; Elmore, 1996) or evaluation of the cost-benefit analyses of the many reforms and whole-school interventions (see Levin & Belfield, 2015).

Reducing the achievement gap

A common cause for reform is reducing the achievement gap, usually invoking interventions to improve minority students’ performance. These gaps have been present for eons and will be with us forever – as there are wide variations of students when they begin schooling. Hanushek et al. (2019) linked assessments from NAEP, TIMSS, and PISA in the US from 1954 to 2001; showed the remarkable consistency in the gaps between the top and bottom quartiles of SES in elementary school; showed that the gaps have increased substantially at high school (by d = 0.30). They then noted the long-term failure of major educational policies to reduce SES gaps, suggesting a need to reconsider standard approaches to mitigating disparities. Their suggestion is more focus on the teaching expertise.

Gaps are not my favorite metaphor, as there will always be variance in achievement, so it is not possible to close the gaps, and any policy so aiming to do must fail. Further, the gap more often refers to the gap at the left-hand side of the distribution and, thus, ignores the gap above the mean (so at least the metaphor should be closing the gaps), and many minority students are achieving above the average – too often, the metaphor ignores that these students should also be accelerated and stretched. For example, Hattie (2009) showed the curves of performance for Māori and Pakeha students in reading, and this shows the two gaps – we need to move all students’ achievement from left to right. It is more marching from left to right than closing the gap (Figure 15.1). We need more interventions for those minority students who are above the average but who may not be stretched to their full potential.

Figure 15.1 Gaps in reading for Māori, Pasifi ka, Pakeha, and Asian students from grades 4–12 in New Zealand.

*Jeynes (2015) located 30 studies of interventions to bridge the achievement gap for Latinx and African American students, with d = 0.11 (0.02 in elementary and 0.13 in high schools). The factors most related to reductions in the gap were family interventions (0.22), religious faith background (0.35), and culture of the school (0.16), although not related where classroom structures and high expectations by teachers. In VL1, I noted the success of the Te Kotahitanga intervention for Māori students in New Zealand (Bishop et al., 2009). Some aims were to stop doing deficit thinking, engaging in a discourse of disadvantage, or personifying the issue as the students’, the family’s, the home’s, or the culture’s problem – instead, it is about the success of teachers in making signifi-cant impacts on the learning lives of these students – with the benefit that if teachers have this mindset for Māori students, all students are winners (but the converse is often not the case). The model is based on developing the expertise of teachers and is closely aligned with the VL claims throughout this book. After the implementation of the program, the Māori students commented that their teachers were focusing more on teaching and less on primarily caring; they felt more respected as learners, knew their teachers had high expectations of them, and had suppressed their anger and frustration at the way they were related to and interacted with by most of their teachers. The effect sizes after introducing the program in mathematics was 0.76 compared to 0.52 for those students not in the program.

Comprehension interventions for learning disabled students

It would be possible to have a whole book on the effects of the various interventions for students with learning difficulties, and indeed, Mitchell and Sutherland (2020) and *Swanson (1999, 2000) have provided such books. Swanson et al. summarized the research based on group and single-subject designs, a common design when working with these students. For the group design studies, they located 180 studies with a mean effect size of d = 0.50, and the more successful interventions included meta-cognitive (d = 0.98), attribution (d = 0.79), and programs aimed at specific skills – word recognition (d = 0.71), reading comprehension (d = 0.82), spelling (d = 0.54), memory/recall (d = 0.81), mathematics (d = 0.58), writing (d = 0.84), vocabulary (d = 0.79), attitude/self-concept (d = 0.68), general reading (d = 0.60), phonics (d = 0.70), creativity (d = 0.84), social skills (d = 0.46), and language (d = 0.54). The effects were high for the 85 single-subject designs (d = 0.90), with high effects in most curricula areas. They concluded that a combined Direct Instruction and strategy instruction model was an “effective procedure for remediating learning disabilities” (Swanson, 1999, p. 218). These two approaches are somewhat independent, hence the importance of using both to maximize the effect on achievement. The important instructional components included “attention to sequencing, drill-repetition-practice, segmenting information into parts or units for later synthesis, controlling task difficulty through prompts and cues, making use of technology, systematically modeling problem-solving steps, and making use of small interactive groups” (p. 218). They also noted the much higher effects from the bottom-up approach to teaching reading that emphasizes accurate word recognition, decoding, and letter awareness, compared to the top-down approach where reading is viewed as dependent on the reader’s cognitive and language abilities (including familiarity with the topic of discourse). More importantly, the Direct Instruction and strategy training models were superior to both the bottom-up and top-down models.

A major review of instructional methods to enhance various learning strategies among learning disabled students was also published by *Swanson (1999a).

He found that teaching the 20+ identified strategies by themselves (d = 0.72) or by Direct Instruction without an emphasis on strategies (d = 0.72) was very effective but even more so when strategy training was combined with methods of Direct Instruction (d = 0.84). The strategies with the greatest impact on the achievement outcomes included controlling for difficulty or processing demands of the tasks (scaffolding), directed response and questioning (Socratic teaching, directing students to ask questions), sequencing (breaking down the task, step-by-step prompts), drill-repetition-practice-review, segmentation, and strategy cueing. These effects were stronger in reading (d = 0.82) than in mathematics (d = 0.58). *Swanson (2001) investigated programs to enhance higher-order processing for adolescents with learning disabilities. Programs that included extended deliberate practice yielded larger outcomes, with the strongest instructional components relating to extended practice. The highest effects were in areas of meta-cognition (e.g., planning, self-questioning, interviews of strategy behaviors) and understanding text (e.g., inferential comprehension, thematic understanding, content knowledge). The hardest area to change was related to learned attributions (e.g., self-efficacy and effort). Similar high gains were shown by *O’Neal (1985) with cerebral palsy students (d = 0.81); *Forness and Kavale (1993), with learning disabled students (0.71); and *Xin and Jitendra (1999), with students with mathematics learning difficulties (0.89) and the focus of special education supplement programs (*Beese, 2008, d = 0.48).

Out-of-school programs

Service learning

Service learning aims to integrate community service with in-school teaching and learning. *Pazmino (2018) included students learning Spanish in a university course and meeting weekly with a Spanish community group to engage in social activities, engage in conversations, and run short courses for these community members. The advanced students gained d = 0.72 compared to the control group (0.08), but there was little difference for more novice studies (0.26 vs. 0.23), hinting that maybe service learning comes after sufficient skill development. *Celio et al. (2011) reported d = 0.43 from service learning on achievement (and similar in K–12 and tertiary), 0.58 for academic motivation and attitudes, 0.28 for improvement in attitude to learning, and 0.30 for personal outcomes (e.g., volunteer motivations, well-being). *White (2001) reported 0.87 for academic progress, 0.51 for self-concept, and 0.59 for social and personal growth. *Yorio and Ye (2012) found 0.53 for cognitive development, 0.37 for understanding social issues, and 0.37 for self-concept/personal insight. *Warren (2012) reported 0.32 with similar effects on self-reported gains (0.37) and from exams and assignments (0.31).

Service learning has much promise, but more research is needed on when in a program to introduce it and what the role is of prior learning and sufficient content knowledge, and there needs to be a closer investigation of the various implementation possibilities.

Home-school programs

*Penuel et al. (2002) were interested in using technology to develop home-school connections in student learning. In particular, they looked at the use of laptops, programs desired effects using discrete educational software for use at home and school, and desktop programs. The effects of these programs on reading were small (d = 0.10), slightly higher for mathematics (d = 0.18), and highest for writing (d = 0.34); there was not a lot of evidence showing enhanced home-school communication or increased parental involvement that affected students’ learning.

*Daucourt et al. (2021) reviewed home math programs. She was particularly interested in the home math environment, which includes all math-related activities, attitudes, expectations, resources, and interactions between parents and children in the home. From 51 studies, the correlation was 0.14 (d = 0.28), and the effects were similar for preschool and kindergarten (r = 0.15), elementary (0.13), and middle school students (0.17). The effects of various home involvements were similar for direct mathematical activities (r = 0.13), parent math attitudes and/or beliefs (r = 0.07), math expectations (r = 0.24), and parent math talk (r = 0.09).

The COVID pandemic distance learning has led to much more focus on home-school connection programs, which will be a focus of much research over the next few years.

*Zierer (2021) has completed the first meta-analysis of seven European studies based on over 5 million students (cf., *Storey & Zhang, 2021, −0.15; *König & Frey, 2022, −0.18). The average effects were as follows: primary school, −0.17; high school, −0.10; secondary mathematics, −0.17; and secondary native language, −0.11. These effects reflect the loss of trajectory and should not be interpreted as learning loss. They compared the growth students made in previous years to the growth in the COVID pandemic year, and there was a slight reduction in improvement due to distance learning/school closures. These effects are mirrored in the OECD systematic review: overall, −0.10; mathematics, −0.12; and reading, −0.14 (Hammerstein et al., 2021). Webber (2021) used the data from over 5 million students across New Zealand to compare the gains made in 2020 to each of the previous ten years. There were slight drops in reading (−0.05), mathematics (−0.03), and writing (−0.30). There were no differential effects for Māori, Pasifika, or students across schools from different socioeconomic backgrounds. I wrote a summary of seven such studies on achievement and seven on the social-emotional effects, and the title summed up the message “An ode to expertise” (Hattie, 2021). Given the massive disruption and the many equity and access issues, it seems remarkable that the overall effects are so small, and while much more nuanced studies are needed about who did and did not flourish or otherwise, there is a need to recognize the expertise of educators to minimize the disruption and enable learning to continue. We need much more debate about what did and did not work well, the focus during the COVID pandemic, and the rebound when schools recommenced in-class.

After-school programs

When my boys started school, I received pressure to join the school board. But not being a fan that parents (most without expertise in running a school) should dominate these boards, I declined and decided to undertake a voluntary task at the school outside my expertise. As a result, I became the accountant for the after-school care center. I now appreciate accountants’ skills and acknowledge the expertise of those who run these after-school programs.

Compared to many in-school teaching activities, the effects are relatively low but seem a worthwhile addition. *Scott-Little et al. (2002) found much lower effects (0.18) with similarly low effects across reading, math, writing, and science. *Peters et al. (2014) found positive effects on achievement (0.17) and attendance (0.09), and closer to zero effects on emotional-behavioral outcomes (0.04). *Durlak et al. (2010) were primarily interested in the effects of after-school programs on personal and social skills, such as self-concept (0.34), school bonding (0.14), and positive social behaviors (0.18), but also included effects on achievement (0.17), school grades (0.12), and school attendance (0.10). Although, among the reasons for success is a focus on promoting skill development, coding for the qualities of implementation, and improving teacher expertise of the staff, *Crawford (2011) found much higher effects (0.40) and similar in math (0.42) and reading (0.38). She identified a theory of change to attain these higher effects, including great diagnosis of the after-school help needed (using reliable diagnostic tests) and review of current in-class data on these students as to what they can and cannot do (yet), and it is important to ensure that experienced teachers are engaged in the teaching in these programs.

Extracurricular activities

Extracurricular activities include sport, music, performance, community-based activities, tutoring, and academic pursuits. The effects are typically tiny and close to zero, and of course, there are more worthwhile outcomes often intended from these activities other than achievement. Also, the direction of the effect is likely that those doing reasonably well in schools are more likely to choose academic extracurricular activities. *Lewis (2003), for example, concluded from d = 0.47 that “this means that on academic performance indicators, the average group of students participating in any or all kinds of extracurricular activities had scores that were about one-half standard deviation above the average group of non-participants” (p. 68). The effects of engagement in prosocial activities (e.g., scouting, volunteering) are 0.25, sports are 0.10, and student employment or vocational activity participation are 0.01.

There are also many academic tutoring programs, often aimed at helping those struggling at school or serving the needs of parents wanting boosts for their children. *Lauer et al. (2006) found small gains from out-of-school curriculum-related courses, with similar effects on reading (d = 0.05) and mathematics (d = 0.09). Similarly, *Gellin (2003) found tiny effects (d = 0.08); *Kidron (2014), d = 0.04; *Shanshan and Xiangdong (2015), d = 0.06; and Marshall (1997) was specifically interested in whether the provision of extracurricular activities made a difference to students from lower socioeconomic background, and given the average effects was 0.04, the answer must be no. *Chappell et al. (2011) specifically reviewed out-of-school tutoring for students eligible for free or reduced-price lunch, and the effects were trivial (0.04) in math and reading. It did not matter whether the provider was for-profit or nonprofit, whether the tutoring was face-to-face or online, whether the provider was newly established or preexisting, and whether the sessions were one-on-one or smaller groups. They argued that the quality of the tutoring instructor mattered, but even these differences were tiny.

It was impossible to find a meta-analysis on the effects of students working after school. However, Marsh and Kleitman (2005) found work employment had a positive effect if the intention was to save funds for further education but negative if not. Further, working more than 17 hours in grade 8 and more than 24 hours per week in grade 12 led to greater unemployment levels following high school.

Homework

Homework involves “tasks assigned to students by school teachers that are meant to be carried out during non-school hours” (*Cooper, 1989, p. 7). The overall effect is small in elementary and higher in high schools. This has led some to abandon homework in elementary schools, but this may not be the best interpretation, especially when many parents judge the effectiveness of schools by the presence or amount of homework. The aim could instead be to improve the effect of homework.

Cooper (1989) argued that the effects of homework are twice as large for high as for junior high school and twice as large again for junior high school as for elementary students. Bas et al. (2017) also found the effects in high school, 0.48, greater than in elementary schools, 0.15. Furthermore, a shorter-duration is more effective than a longer-duration homework. *Bas et al. (2017) reported 0.44 for short duration compared to 0.24 for longer duration, and *Cooper et al. (2006) estimated a correlation of near zero (d = −0.04) between time spent on homework and achievement (see also Trautwein et al., 2002).

The nature of the homework also makes a difference: homework that entails the deliberate practice of something already taught has higher effects than homework requiring studying new or higher-order ideas. *Fan et al. (2017) specifically investigated math and science homework and, based on 41 studies, found an overall d = 0.44, with similar effects in math (0.40) and science (0.52). Fan et al.’s explanation for their larger effect was that much math homework is revision and deliberate practice. High school teachers are more likely to assign homework related to deliberate practice, and students have learned to study alone more effectively. When in doubt, elementary students need expertise and often have less effective study habits. The effects are negative (−0.19) where there is parent surveillance of homework (Boonk et al., 2018; Clinton & Hattie, 2013; *Fernández-Alons et al., 2022; Muhlenbruck et al., 1999). *Cooper et al. (2001, p. 197) noted that “parent support for autonomous student behavior showed a positive relationship to achievement, whereas direct instructional involvement showed a negative relationship.” Further, Trautwein et al. (2002) warned against homework that undermined a student’s motivation and led to the student internalizing incorrect routines. Instead, they found support for short, frequent homework that the teachers closely monitored.

The message is to keep homework short, focus on opportunities to practice what they have already been taught deliberately, not require parental surveillance, ensure it is valued as part of the school program (e.g., marking homework illustrates this value), and note that homework is schoolwork completed at home (Cooper, 1994; Horsley & Walker, 2012). Thus, if there are issues, it is a school problem to resolve.

Conclusions

Teacher collective efficacy requires deliberate and much intent by school leaders to create the time, the right focus on impact, and a high trust for collective efficacy to occur in schools.

Too many schools seem to be peopled by independent contractors who love to teach their way; talk about kids, curriculum, assessment, and almost anything but talk about their impact from their teaching; share their notions of high expectation for all; and share problems of practice and impact.

We need more attention to scaling up success from whole-school programs and comparative studies of system reforms, and knowing when to stop a low-impact program.

All students need to gain at least a year’s growth for a year’s input, and this should not differ by race, gender, or any other way we group and label students.

We need to see our impact first through the impact on those most needing greatest growth and scale from this, whereas so often, we see the effects on the brightest and scale to all – which does not work.

Perhaps schools could learn from the COVID pandemic distance learning experiences to involve caregivers and students more and differently in learning. For example, the nature of homework could be further developed such that it moves to opportunities to practice that already taught, and engagement in service learning may be more worthwhile as a family-child learning opportunity.

In Chapter 9, the power of teacher collective efficacy was noted. This requires deliberate and much intent by school leaders to create the time, the right focus on impact, and a high trust for collective efficacy to occur in schools. Too many schools, however, seem to be peopled by independent contractors who love to teach their way; talk about kids, curriculum, and assessment much more than talk about their impact from their teaching; share their notions of high expectation for all; and share problems of practice and impact. Whole-school improvement models can be difficult, diffuse in their implementation, and ignore the high levels of variance within the school. But a model where students get great teachers by chance than by design should not be random. We know how hard it is to engage in whole-school models, and our VL+ program has taught us the importance of having great discovery, establishing readiness, ensuring intentional alignment of interventions with these discoveries, attending closely to the dosage and fidelity of implementation, continuously evaluating and scaling up, and smelling the roses and sharing the joys when the impact is enhanced. There needs to be much more research and meta-analysis on models of implementation, the effectiveness of evaluating models, and identification of the barriers that enable whole-school improvement. We need more attention to scaling up success from whole-school programs and comparative studies (e.g., *Borman et al., 2003) and knowing when to stop a low-impact program.

We need to think differently about reducing the achievement gap. There will always be gaps, so a policy to reduce them must fail. While we can reduce the gaps somewhat, care is needed as there are multiple gaps, and we should be as concerned about those minority students above the average who are not thriving and learning as their peers. All students need at least a year’s growth for a year’s input, and this should not differ by race, gender, or any other way we group and label students. The most successful program I have witnessed, the Te Kotahitanga intervention for Māori students in New Zealand, focused on teacher expectations, focused on teacher’s interactions with students as seen by the students, and showed that what works best for Māori students worked for all students but what worked for most did not work for Māori students (Bishop et al., 2009). We need to see our impact first through the impact on those that most needed the greatest growth and scale from this, whereas so often, we see the effects on the brightest and scale to all – which does not work. The evidence from whole-school programs on students with special needs reinforces these claims, and moving so that all students have learning interventions reduces the labeling, maximizes the chance of growth, and advances all students. We have much to learn from effective teaching of students with special needs for all students. As Bishop, Swanson, Mitchell, and others have shown, deliberate intentions to teach to maximize impact, evaluate our impact on all students, work collectively with colleagues (resource teachers et al.), and intentionally align teaching with the intent matter most for these students – when they feel welcomed into the environment of learning.

There are high levels of variance in the amount of time a child spends in school. A PISA study (Charbonnier & Truong, 2014) noted five countries with 4,000–4,500 hours a year (Hungary, Estonia, Russia, Korea, Czech Republic) and five with 9,000–10,000 hours (Australia, Ireland, the Netherlands, Spain, Luxembourg). The PISA math mean for the countries with the longest hours were 30 score points below those with the shortest hours. More of the same is not necessarily desirable. Lengthening the school day or year ignores the possible importance of out-of-school effects. It diminishes attention to the efficiencies and effectiveness of the impact of teachers during school hours. Most of these studies are correlational, but Lopez-Agudo and Marcenaro-Gutierrez (2022) used a more sophisticated analysis and came to the same conclusion – the amount of instruction time does not seem to be positively associated with students’ academic performance. They suggested that students may be receiving excessive unproductive instruction time, entailing a waste of resources.

The greatest travesty of the worldwide experience with the COVID pandemic will be learning nothing about what worked well and what did not work, for whom, and about what – and rushing back to the old normal. The experience asked for more connection with home and caregivers, demanded that students needed to work often with their peers without the teacher being ever-present, and allowed many not to experience the pressures of in-person contact with others. It asked many parents to understand the language of learning, noting that getting it right was not always the aim, that failure and errors are opportunities, and that teaching requires higher levels of expertise than many possessed. They moved from overseeing homework to listening and working with their children and young adults on many tasks. Perhaps schools could learn from these experiences to involve caregivers and students more and differently in learning. The nature of homework could be further developed such that it moves to opportunities to practice that which has already been taught, and engagement in service learning may be more worthwhile as a family-child learning opportunity.

Reflections

Discuss the 5D Building to Impact Visible Learning model. Use it to oversee the implementation of some of the claims in this sequel.

The section on “Comprehensive interventions for learning disabled students” reveals that the highest effect size is from meta-cognitive interventions. Students’ own low self-efficacy and lack of understanding about how to learn are often the roots of low attainment. Revisit previous reflection questions about student self-efficacy and review your current strategies, systems, and relationships for all students.

Read the final paragraph on homework and discuss the implications. For example, in elementary, are we keeping the homework short and focusing on deliberate practice and reading? In middle or high school, are we asking students to engage in new material or, as highlighted here, focusing more effectively on practicing what has already been taught?

After reading the final paragraph of the conclusion, discuss to what extent we have capitalized on the positive factors described as a result of the COVID pandemic? Ask your teachers and students to create a positives and negatives list of their experiences during the COVID pandemic distance learning and in the regular class, and use these findings to discuss improvements to your school.

CHAPTER 16 Conclusions

DOI: 10.4324/9781003380542-16

My life changed 14 years ago from a rather sedate but enjoyable life as a psycho-metrician working in academia. Since 2008, VL has taken over. VL permitted me to access the highest offices in the land, to have a team working in 10,000+ schools throughout the world, to have so many teachers and school leaders sending their stories and case studies, to write 30+ books relating to VL with great colleagues, and to have the greatest honor of all as an academic and writer – I now have the world’s best critics. You could spend your life as an academic, and no one could care; and that so many have taken the time and intellect to contest, explore, and critique my work is the greatest academic reward.

I recall the early days of this work when colleagues like Michael Scriven kept asking – but so what? You have a great data set, but what does it mean? This search for the underlying interpretation and meaning was also the core of my claims as a psychometrician (we have great data, but so what; we have great score reports but are they interpreted correctly and with consequences for improving learning experiences and outcomes, O’Leary et al., 2014). Michael sat on my shoulder when I was writing every page of VL1 for many reasons – one being his contention that just because something correlated with outcomes did not make it a part of excellence (Scriven, 1994). Many high-impact teachers do not use the jigsaw method, do not work in well-resourced schools with small class sizes, do not have great principals, do not have master’s degrees, do not use technology or flip their lessons, and do not use Direct Instruction. These factors are correlates, hence the search for the underlying reasons for the choices made to enhance student achievement. It is not the correlates; it is that which explains the underlying story of the correlates.

Over the years since, I have continued to learn from the increasing number of meta-analyses, from the evidence of implementing VL+ in schools, and from working in policy with governments and systems, and at all times, I am considering, evaluating, and enhancing the underlying VL story. This conclusion looks back to the VL1 conclusions from 14 years ago and reiterates and reemphasizes what I still hold to be the case. It highlights the additional conclusions from this sequel and then uses these claims to look ahead to the next decade (Table 16.1).

Table 16.1 Looking back 14 years, looking at what I have learned 14 years later, and moving forward

14 years ago
Today
Moving forward

Teachers have the greatest in-school impact on students
Clarity about purpose
Opening up of classrooms

The personal nature of learning
Importance of mind frames
More research evidence on lesson plans

The remarkable diversity within schools and classrooms
Role of evaluative thinking
Use and value of social media

The need to consider the ideas as well as the relations among ideas and the transfer of ideas
Intent to teach and intentional alignment
Continual reminder that we are dealing with children, not mini-adults

Evidence can make the difference
The poverty of models of implementation
Systems need catching up to evidence and evaluative thinking

Develop an explanatory story based on discriminating those effect above and below the average of all
Importance of student assessment capabilities
Scaling up and de-implementation

The power and diversity of feedback
Teacher and student collective efficacy The integral nature of the social and emotional attributes as core to learning The effectiveness of various learning strategies
The dominant VL claim remains - know thy impact

Conclusions from 14 years ago

Six major issues were raised in the conclusion chapter of VL1 (2009). First was the finding that teachers had the greatest in-school impact on students. The effects were highest when teachers were directive (I now call this intentional to move from any inference that “directive” means to talk the most, do the most work in the class, and control the learning) and caring, are actively engaged in the passion of teaching and learning, are aware of what each and every student brings to the lesson, are clear to students about the learning intentions and success criteria, and create a climate in the class that is fair and welcoming to come and learn.

Second was the reminder of the personal nature of learning. Teaching is what teachers think and do, whereas for students, their focus is on learning. Maybe we need to teach less, learn more. This begs questions about how teachers consider concepts and models of learning, how can and do we know students’ learning strategies and processes of learning, and how to teach students ways of learning as well as understand their reactions and their consequences of learning. Yes, learning is always about something. It is hard to imagine successful learning without successful teaching (teachers are not meant to stand on the sidelines primarily), there is a (loose to tight) relation between teaching and learning, but the balance of our discussions tends to be weighed to the teaching debates than to the notions of learning. There is a need to rebalance this equation.

The third conclusion noted the remarkable diversity within classrooms. Not only diversity in the attributes students bring to the class but also the diversity of experiencing within classrooms. This diversity is the norm, hence the plea for educators to be continually looking for rival alternative explanations for their impact on each of their students – searching as to whether there are more efficient and effective ways to teach each student to accelerate, to be motivated to want to learn, to work alone and with others, and to experience the joy of learning. It is seeking what is not working, seeking the students where the teaching is not working, and seeking evidence of a lack of sufficient growth. And then acting. These three questions are more often asked by high- than low-impact teachers.

Fourth, the need to consider the ideas as well as the relations among ideas and the transfer of ideas. In VL1, I used the SOLO notions of surface, deep, and transfer. John Biggs (personal communication) reminded me he did not use the notion of surface as the idea and facts part of his SOLO model. The use of surface has led some to confuse surface with shallow. While surface does mean facts and content to many, throughout this sequel, I have moved to knowing-that to indicate this level of knowing – what is the that that we need to know then to move to the knowing-how as we relate the ideas (the deep), and how do we then use these ideas with other ideas, especially in new or novel situations (the transfer). It is the case that the dominant focus in schools is knowing-that, with students being assigned activities to learn facts; facts are the focus of assessment; and feedback so often encourages more facts. The current grammar of schooling mired in facts is entrenched, supported by the above-average students (who are winners in this game), reinforced by the tests, promoted by politicians, and reinvented under many guises. My high school motto was “knowledge is power” but so is relating knowledge, using knowledge, and using wisdom and beauty. Further, not knowing can be the essence of curiosity, the forerunner of asking difficult and important questions that interrogate ingrained customs. We need to question those who arrogate power to themselves as the guardians of knowledge as if it were out there waiting to be absorbed.

The COVID pandemic showed that when students are not under surveillance or captured in classrooms but asked to work alone, then many attributes of the grammar of schooling break down. On Zoom, teachers could not talk 89% of the time, ask 150 questions a day requiring less than a 3-word answer, continually engage in surveillance to ensure students are doing the work, and maintain responsibility all day for the flow of teaching. Suddenly, there needed to be attention to how students learn, how they work alone and in groups, how they need to learn to seek help, how failure and struggle are highlighted and core to successful learning, and how students want to move beyond the facts to see relations among them, use ideas in new contexts, and enjoy the joy of the aha moment of making relations. Success was more a function of teachers gradually (and sometimes quickly) releasing responsibility for control and teaching to the students; teachers had to teach students self-regulation and collective team skills; teachers had to listen more than talk, engage in triage to prioritize, make coming to the Zoom sessions inviting and welcoming, teach students that struggle and failure were necessary parts of learning and that seeking help was a great skill, and regard efficiency as important as effectiveness (as some students completed the learning quickly at home compared to regular classrooms). So can teachers change? Absolutely, yes – they did. Sadly, as we come back to regular classrooms, they also change back to the old grammar of schooling. There is a need for robust discussions about what worked well and what did not under distance learning and stop much of the current grammar of classrooms and instead develop and evaluate a newer grammar of schooling. What an opportunity.

The fifth conclusion questioned whether evidence could make a difference. If we reflect back 15–25 years ago, the use of research evidence in schools and systems was a rarity; nowadays, it is commonplace. Research evidence now sits alongside evidence from within the school (from the teachers noticing, the tests, and the artifacts of student work), and there is a focus on interpreting and triangulating these multiple forms of evidence. As I noted in 2009, “What a phenomenal change.” This interpretation of research has continued, and now, educators are more sophisticated in their critique, evaluation, and use of research. But there is still much to do. Neeleman (2019) noted that there was still a primacy for teacher judgments based on their perceptions of their impact rather than triangulating with alternative perspectives of their impact and the use of research evidence. In her study, rather than being motivated by the explicit ambition to improve cognitive student achievement or by research evidence, Dutch secondary school leaders predominantly find that their school intervention decisions are motivated by their personal beliefs about the pedagogical task of education. I also noted the findings of Alessi (1988) who reviewed more than 5,000 children referred to school psychologists because they were failing at school. Not one psychologist located the problem as due to a poor instructional program, poor school practices, a poor teacher, or something to do with school. The problems were claimed to be related to the home and located within the student. In our VL+ work, one of the more difficult tasks is to get teachers to take credit for successful learning in their classes (they ascribe the success to what students do, not what teachers do and think). We need more debates about theories of action and, particularly, about the attribution of success to teachers and then the understanding of how to scale this success to all students (Elmore, 1996). We need more evaluation and critique of our theories of change and action – and subjecting these explanations to critical evaluation by peers, by students, and by seeking plausible alternative explanations – to best help make next best teaching moves.

A related sixth conclusion was a reminder about the purpose of the book. VL was not about collecting hundreds of influence or a plethora of effect sizes, but the aim was to develop an explanatory story based on discriminating those effect above and below the average of all influences. I have continued challenging others to provide alterative explanatory stories and refined my explanation in Chapter 3 and Chapter 4. In addition, a few years back, I released all the data in a free website (Metax™) not only to keep up with the many meta-analyses published in VL1 but to challenge and enable colleagues to create alternative explanations.

What have I learned 14 years later?

The model I proposed in VL1 has been subject to disconfirmation every time a new meta-analysis is published. It has survived with modifications and perhaps some alternate emphases. Also, when introducing the model into schools, it is seriously subjected to critique – educators are not passive recipients of research claims and adopters of new ideas simply because they are new or research-based, and one of the more powerful consequences of the VL+ program is that it recognizes the excellence that already is all around us in schools and systems. There are nine major moves in prioritizing the VL message.

The first is more clarity about purpose. Steen Lepper Larsen convinced me that more attention was needed to the purposes of schooling and a greater acknowledgment of the social system in which schools operate (Hattie & Larsen, 2020). The evidence within this book primarily comes from more resourced countries, although these societies have much variance in their social structures, beliefs, values, and purposes. For all, the purpose is for gaining at least a year’s growth for a year’s input, having clarity of what this growth looks like, engendering students to want to invest in this learning, knowing how to learn alone and in teams (as educators and students), and knowing how to teach students to become their own teachers. For school leaders, a major purpose is to lead school-system-wide improvement processes to maximize impact of all on students and make schools inviting places to come and learn. For teachers, a major purpose is to see their impact through eyes of students, teach students to become their own teachers, and become students of their own teaching. For parents, the purpose is to become their child’s first learner. For students, a major purpose is to learn to become their own teachers and want to reinvest in school learning.

The second is a deeper dive beyond the big ideas from VL1: teachers working together to evaluate their impact, having high expectations for all students, being transparent in converting these high expectations through the success criteria, teaching students to take on the challenge to aim high (using the Goldilocks principle of not too hard, not too easy, and not too boring), creating fair and high-trust classrooms wherein errors and mistakes are seen as opportunities to learn, continually seeking feedback about their impact, and having relentless focus on learning. Note that the big ideas are less related to students, curricula, structure, years of schooling, and assessments – and more to how educators think. Hence the investigation of these mind frames or ways of thinking. It is more about how we think about what we do than focusing on what we do. It is moving beyond debates about teaching to the impact of that teaching; it is critically evaluating our beliefs and actively seeking feedback about our impact. The mind frames for all are based on welcoming challenge, having high expectations, building confidence, building a language of learning, and creating an inviting climate. They involve developing respect for others, knowing and disrupting biases, supporting identities, and enabling belonging.

The third discovery goes one step deeper – these mind frames all pertain to evaluative thinking. This book is dedicated to Michael Scriven, the grandfather of the discipline of evaluation. Working with Michael is a major privilege, as you hear him thinking aloud, hear his critique, and hear his relentless pursuit for philosophically underpinnings. Over many discussions, he taught me about the ways of thinking evaluatively – the reasoning, the logic, the ethics, the principles of thinking, the critically of equity and making value claims. In recent years, others have labeled his pioneering work “evaluative thinking.” High-impact educators think this way, and over the next years, I hope that others explore the nature of this evaluative thinking, continuing the work of Michael (Scriven, 2003, 2007; Berliner, 2008; Kahneman, 2011; and Ritchhart and Church, 2020). How do we undertake research to better understand, highlight, and teach these ways of thinking? The core notions of evaluative thinking include critical thinking, problem-solving, interpreting evidence, reasoning, continually checking for unintended consequences, being aware and searching for biases, being able to stand in the shoes of others and see and understand their perspectives, and knowing the critical equity issues of making value judgments to optimally decide on next steps for action or consideration.

It would be so much easier if the fourth finding were the identification of the top teaching and learning methods. Alas, there is a major moderator – the cognitive complexity of the ideas in the lesson. Hence, the importance of the intent to teach and intentional alignment across decisions about the knowing-that, knowing-how, and knowing-with aspects of the curricula. Thus, the importance of cognitive task analysis, optimal teaching interventions, and the assessment or evaluation strategies. Adopting these notions avoids the frustrating and nonproductive debates about explicit vs. guided, Direct Instruction vs. problem-based methods, and so on. The influence with the greatest number of studies (N = 5,976) is explicit teaching strategies, but deeper perusal of these studies shows the wide breadth of these strategies such that sometimes, it is hard to understand what is meant by explicit and what is not. As well, the use of explicit is sometimes code for focusing on rich knowledge or knowing-that and the (false) assumption that you cannot be explicit in teaching for knowing-how or deeper outcomes. Another confusion is that explicit does not necessarily mean teacher dominance, excessive teacher talk, and the ignoring of the reactions of the students.

Moreover, the research on learning strategies (Chapter 12) undermines the belief that the teaching and learning strategies for the knowing-that are necessarily optimal for the knowing-how or knowing-with (and vice versa). Teachers gradually releasing responsibility and enabling students to learn to be their own teachers can require explicit teaching methods. It is time to drop the “explicit” term and the debates about explicit vs. guided.

The notions of the intent to teach and intentional alignment are deeply related to clarity, and the claim is for educators to intentionally align their purposes, success criteria, feedback, activities, learning and teaching strategies, and assessment. There are major implications for lesson design, reinforcement for backward design, and much emphasis on being intentional about the desired levels of complexity of the content (knowing-that, knowing-how, and knowing-with). It does seem surprising that most teaching methods are silent on the relation to this complexity.

My years of working with Corwin implementing VL+ programs have highlighted the fifth emphasis, and that is the poverty of models of implementation in our schools and systems. I recall Michael Fullan’s summary of Victorian education (where I live) – that it had the best policies in the world. Just not implemented. I have studied, talked to, and admire Michael Barber’s model, and I once said to him that deliverology was stunningly successful, but it was too complicated (Barber et al., 2010). He replied – but implementation is complicated. Despite a richness of resources on being a principal and the corpus of research on teachers implementing in the classroom, there is a major mismatch between developing and deciding policies and their implementation. Our own synthesis from other disciplines, as well as within education (e.g., Fullan, 2007; Reeves, 2015), led to a five-phase model based on discover, design, deliver, double-back, and double-up. Further, more research is needed on understanding de-implementation (how to stop, substitute, replace ineffective and inefficiencies in teaching and learning), as there is a tendency to add to schools and a false belief by some that a busy teacher is a good teacher and a good school has so many programs.

Sixth, I was surprised at the power of clarity and (wrongly) had not seen it as so critical (Fisher et al., 2018), but I was delighted with the importance of student assessment capabilities. That is teaching students how to interpret and action the results and comments from assessments, know their current level of understanding and progress, have confidence to take on new challenges, select tools to guide their learning, seek feedback and recognize that errors are opportunities to learn, and recognize and esteem their learning and teach others. These attributes are often termed “self-regulation,” but by terming them as part of assessment, capable students anchor the skills and dispositions that teachers can intentionally develop.

Also, when I saw the graph (see p. 382) and research by Deming (2017) about the desire of employers (now) for graduates who would work in teams, translate their knowledge, and see problems from the perspective of others, it crystallized probably the greatest change from my own school days: the importance of being taught to work in teams. At the same time, I had been battling with the implications of teacher collective efficacy and suggested to the team we explore student collective efficacy to then help translate to teachers (Hattie et al., 2021a). It is not simply putting students (or teachers) in groups and devising activities that involve sharing. Instead, it is deliberately teaching the skills of working in groups (e.g., turn-taking), the confidence and skills to contribute to the group, and the confidence that the group can come up with better outcome than any individual. It requires lessons and tasks that require multiple interpretations and evaluative judgments, and success criteria and assessment of the individual and group contributions.

Seventh, the integral nature of the social and emotional attributes as core to learning. The COVID pandemic amplified this connection. The development of knowledge, skills, and understanding is completely entwined with advancing students’ sense of who they are and their sense of self, belonging, identity, culture, and well-being. We need care, however, that we do not now expect teachers to become counsellors, and the interest needs to be about the social and emotional aspects of learning. One way that SEL remains difficult is that, unlike achievement, there is not necessarily a cumulative increasing set of competencies. Students can change their concepts of self and dispositions, such as resilience and confidence, to take on challenges within a day, across years, and among contexts. For example, West et al. (2020) followed about 400,000 students across three years and found that growth mindset, self-efficacy, self-management, and social awareness did not increase monotonically over time. Self-management increased in later elementary grades before tapering off in high school; self-efficacy declines in middle school; growth mindset starts to increase during early adolescent, and social awareness declines across grades, especially so in middle school years. Is it reasonable to hold teachers accountable for developing SEL when it bounces around so much? It is also worth noting that Fullan (2021) has argued that academic obsessions are not a primary intrinsic motivation for most students (even for those who seem to succeed in this game). Instead, he highlights purpose, meaning, belongingness, control, and making a contribution. Houghton et al. (2022) showed that the major reason, by far, students wanted to return to in-class teaching during the COVID pandemic was their desire to be with friends – not that they wanted to learn what teachers were teaching them. I am delighted there are now syntheses of meta-analysis in motivation and self-concept, and more is needed about the intrinsic relation between social and emotional learning and their relation to achievement. It is not either-or; it is when and how.

Eighth is the power of feedback. As noted in Chapter 11, I have undertaken many studies to better understand the power and variability of feedback and have moved my interest from not only the giving, frequency, and timing but to how feedback is heard, understood, and actioned. Once again, this highlights the importance of educators to know their impact when they provide feedback to become more effective listeners of how students interpret and use their feedback. This move also highlights a shift to a fascination with listening. As well as being a mark of respect to demonstrate that you have listened to another, the skills of listening can lead to testing understandings, seeing how others react to your interpretation of what they said, and allowing others to amend or reject your interpretations of what they mean. Active listening is the essence of relationships, core to working with others, the starting point of effective feedback, and a skill we need to teach all educators and students.

Ninth, when we won the bid for establishing a Science of Learning Centre, we ventured deeply into the effectiveness of various learning strategies. The plea is for much greater attention to how students learn, for the development and use of models of learning, and for methods to teach students optimal learning strategies as part of lessons.

Moving ahead

It is unlikely that there will be another edition of VL after this sequel. The world of publishing has fast moved online, e-book sales are increasing, and knowledge and meta-analysis continue their steady growth. There is already an increasing number of second-order meta-analyses, so perhaps a Wikipedia version of the influences may capture the vastness of the research, provided there is attention and contestation of the big ideas underlying these studies. However, maybe we will go the other way and make the raw data from studies more readily available, and the excitement of adding evidence from within the classrooms could provide another layer of richness.

The first leap forward relates to bringing together the technology to assist teachers to know their impact and open up classrooms. As noted earlier, seven years ago, my partner, Janet Clinton, led a project called VisibleClassroom. VisibleClassroom is an app for teachers to record their teaching, and immediately a transcript and a dashboard are available for them to review – as if they now see and hear themselves as a student experienced them. There is a provision where the teacher talk can be transmitted to the students’ iPad immediately, so they have a second chance to hear what the teacher has said (and also can listen to teachers talking to other groups). The dashboard displays many metrics common to well-established observational tools (such as Danielson, 2012; Marzano, 2007), such as the presence of clear goals, learning intentions, success criteria, whether there is linkage of new knowledge with old, clarity of instructions, repetition of comments, degree of prompting, nature and amount of feedback, the number of questions asked, proportion of deep- and surface-level talk and questions, the speed of their talk, and the balance of teacher and student talk.

One study (Clinton & Dawson, 2018) showed that the greatest improvements in practice were in deepening pupil understanding and introducing and explaining new concepts, as teachers claimed that the dashboard helped them see this was too often missing in their current dialogue. Another study (Sanders et al., 2021) noted that the provision of the script via tablets led to students having greater focus and less disruption, and kept those who otherwise would have been struggling to become engaged in the lesson. The teachers spoke highly of the effects of the dashboard; were definitely more aware with what they were saying, how they were saying it, and how much they were talking; and modified their behavior in response to the dashboard.

Since then, other advances have been made, and it is possible to imagine a rich source of evidence of what occurs in classrooms to be added to the current evidence bases (Lugini, 2020; Kelly et al., 2018). Liu and Cohen (2021) analyzed 1,000 transcriptions of 4th–5th grade classes (no. students = 13,370). Teachers talked 85% of the time, their language was more analytical than social (narrative or interpersonal), and they spent 10% on classroom management routines. Teacher-centered instruction and classroom management routines negatively predicted achievement value-added scores, and student-teacher interactive instruction (content understanding, quality of feedback, instructional dialogue) positively predicted achievement value-added scores.

The use of more advanced technology (robotics, virtual learning) in classrooms could decrease the dominance of teacher talk time. For example, with a robot, a student can have a more balanced dialogue; be the instigator of actions and queries; be less emotively involved, leading to less anxiety and more positive attitudes to learning; allow more risk-taking with their thinking; generate ideas and solve problems; and have multiple opportunities to review and retry, with no penalties or perception they are not smart; and the robot’s dignity is not threatened (Newton & Newton, 2019).

With these technologies, it is possible to relate the happenings in classes (who does the talking, about what, with whom); the focus of discussion (the knowing-that, knowing-how, knowing-with); the role of error, emotion, and excitement in learning; the emotional loading of discussions; and much more to multiple processes and outcomes of classes. As well, information from robotic and virtual reality, social media, emotion bands, eye gaze, head pose, and facial expressions can add to the richness of understanding engagement, interest, and involvement (Goldberg et al., 2021). Of course, we can be overwhelmed with data, so rich score reporting should be the starting point to deciding on optimal data to help inform teachers and students as to the next best teaching and learning move. Yes, it will raise ethical questions and it will lead to an overdose of data, but initially, the richness for researchers could lead to more exciting explanations of the learning lives of classrooms. For example, Wang et al. (2013) used an automated speech recognition tool to clarify interaction patterns between teachers and students to help monitor students’ participation in classes.

A second leap would be to ask for more research on lesson plans. Every night, millions of teacher hours are spent devising lessons, activities, and assignments for the next day, mainly uninformed by evidence other than professional judgment. Santelises and Dabrowski (2015) analyzed the lesson assignments of 92 ELA teachers, and only 5% were in the high range of alignment, where the text provided had high alignment to the assignment, cognitive challenge, motivation, and engagement. About 50% of the assignment tasks were not connected to the text; 38% were aligned with grade-appropriate standards; 4% pushed students thinking to higher levels; it was rare for there to be academic discussions; 18% required no writing; 17% asked for one to two sentences, 27% for multiple short responses, and only 9% required multiple paragraphs; and 86% limited student thinking to recall, reproduction, or basic applications of content. Such poverty can explain the boredom, the confusion, and the lower motivation to engage in the knowing-that and knowing-how of the discipline – instead, students are asked to work out what is required and what is in the teacher’s mind, and just do the work. Building an evidence base of lesson planning and optimal links to the learning progressions would be a most valuable exercise.

A third leap forward is the use and value of social media. I noted in Chapter 14 the potential power of social media and consider this communication system (already used extensively already by students) was too often shunned in classrooms. When I was 7, we had a pencil test – if I passed, I was allowed to advance to fountain pens. The evil was the new Biros, which were considered the bane of great writing. Ditto when television came to town, and similarly for so many innovations that are portrayed as the Death Star for learning. Students are willing to use social media to talk about what they know, what they don’t know, where they are struggling and where they need help (Davies, 2016) – even when the teacher is standing next to them. We need more investigation of how to optimally use these resources to advance hearing how students are thinking and evaluating learning decisions.

A fourth leap forward is to continually remind us that we are dealing with children, not mini-adults. We are not preparing them for their future – they will create their futures. We are teaching them precious knowledge, ways of thinking accepted by our society, and ways of being critics and the conscience of the knowing and understanding and enjoy being a child. The European notion of bildung, or the development of character, is critical to our role as educators. As I have noted many times, I do not exclude achievement from this development, as knowing-that, knowing-how, and knowing-with about precious knowledge are the sources of advancement of students and our society. Getting the right balance of the component parts is the key.

A fifth leap relates to systems. Throughout this sequel, there has been too little comment on the place and powers of systems in which schools are embedded. Some of the studies with the largest samples of students, however, where included via whole-school reforms. Within our VL+ work, there are so many exemplary schools where we know students are making more than a year’s growth annually, but scaling this up going from school to school is hard work. We have worked with all schools across a system and can document those major improvements by identifying the pockets of success, forming a collation around them, and inviting others to join. Too often, the improvement in schools to adopt a whole-school mission to enhance the learning, enjoyment, and outcomes for students stops when the principal leaves (or across systems when the superintendent, director general, or minister leaves). Systems still depend on a model of independent contractors, where pockets are permitted to prosper but system-wide embedded enhancements protect and advance a return to the old status quo.

Sixth, we need more attention at every level of scaling up success. Our discipline lacks a depth of research, discussion, and understanding of scaling up successful interventions. The top-down promotion of policies on schools can be ever-changing and not responsive to success already in the system. The reliance on system, leader, and teacher autonomy mitigates against scaling, and the reluctance to stop or de-implement the current orthodoxy has been shown to be difficult (but note what occurred during the COVID pandemic distance schooling). We are pretty good at finding problems and fixing them but far less successful at reliably identifying excellence and scaling it up.

Scaling up involves understanding and sometimes challenging the often very strong theories of teaching and leadership that educators have. It requires having a burning platform and well-articulated and convincing story (e.g., the VL research and story is the basis of VL+); reliably identifying success; building a coalition around this success; developing an inclusive language focusing on improvement (and not necessarily change); appreciating the skills, emotions, and intentions of educators; and promoting, sustaining, and feeding success. The burning platform must offer solutions to problems that educators have and want to solve and meet the felt needs by those who would implement them (Cohen & Mehta, 2017). Beware of educators with solutions – mainly because their solutions are rarely based on a diagnosis of issues that the solutions are intending to remedy, and they often commence by critiquing the very people they then wish to implement their solutions.

The seventh leap is reifying the move from debates about teaching to a relentless focus on the impact of teaching. The dominant VL claim is to know thy impact, which feeds the motivation why most of us become educators and goes to the core of schooling (Elmore, 1996). It speaks to the parents who ask and need to trust the schools to deliver on this impact, hones the focus of policy makers and politicians to resource and own their impact, and commits to students that we can and will have an impact on them – all intentionally positive. It starts from recognizing the success already present in a school and system, and it aims for a unity of commitment to a common theme (enhancing one’s impact) at all levels (politicians, policy makers, principals, teachers, parents, and students). An aim is to make the VL thinking more routine and normal in this school and system, and it can become the essence of the educators and students’ theories of teaching and learning. The model has a strong model of implementation and can adjust and allow for diversity and variance; and it can learn and become stronger through critique of the model, of the implementation, and of the consequences on educators and students. It is self-sustaining if for no other reason than it is built on a robust evidence base and seeks further evidence of its impact. We still have much to learn, as the autonomy of the next principal, superintendent, or politician can reverse any positive effects upon appointment.

The eighth leap is to take more care about the questions we ask about our work. Oftentimes, research is more powerful not when it leads to great findings but when it changes the questions we need to ask. One of the more powerful outcomes of research is to modify or change the questions. Over my VL journey, there have even many changes in the questions I am now asking (Table 16.2).

Table 16.2 Changing the core questions about teaching and learning

Asking less
To

About teaching
The impact of teaching

What works
What works best

Influence of the individual
Power of the collective

Influence of educator’s experience
Influence of educator’s expertise

How students engage in doing the work
How students think, know, and solve

Impact of talking
Power of listening

How to label to explain why a student cannot
Diagnosis as the first step to implementation

Labeling to be seen as different
Learning interventions for all

Grouping to reduce variance
Welcoming many progressions and times to succeed

About nature, frequency, and timing of feedback
How students hear, understand, and action feedback

Impact most students
All means all

Adjust learning to the learner
Using the power of acceleration for all

Coverage of curriculum
Cover less and go deeper

Surface or deep
Right proportions of surface and deep

The ninth leap is the plea that many have asked over the past millennium – should we advance more of the same (it does work for many, we have honed the model for 150 years, we know it), or is there better way to educate our students? The OECD (2021) has long advanced developing scenarios for the future of schooling. I have researched innovative learning environments and seen stunning learning primarily because the multiple teachers in these classrooms become each other’s best critics and evaluators; their shared notion of teaching is infectious in translating to a shared notion of learning; it forces teacher to yield command, control, talk, and focus primarily on facts; but the percentage of these ILE schools are within the 5–10% that Cuban noted from his history of teaching. That is, 85% of schools look somewhat similar to classrooms 150 years ago (but now, there is less likely to be a piano), 10–15% are much more efficient, and 5–10% are markedly different. The COVID pandemic showed that there is an alternate way instead of the teacher being in front of the class conducting the orchestra all the time. Many students flourished under ILE and the COVID pandemic system, and they are not always the same students who flourish in the regular in-class.

The COVID pandemic also exposed the soft underbelly of teacher supply and retention, and the meltdown of regular schools may be more a function of supply and demand. As more teachers are engaging in gig thinking, wanting shorter-term employment, wanting to work for three to four days a week, and asking for more flexibility than the basic structure of entering the profession, working 20+ years to become a principal may break down. Teachers are starting in the profession at a later age, often after one to two career directions already and often preferring to stay in the classroom, and most of all, they want a collaborative profession where expertise is valued.

The tenth leap relates to research and meta-analyses. I continually recall the conclusion to Gene Glass’s (1976) initial meta-analysis article – if we had the raw data from studies, we would not need meta-analysis. Along with the claims earlier about collecting richer classroom data, with more access to the raw data, then more sophisticated analysis and interpretative methods can be used to make sense of these multiplicity of data. By having access to such rich data, with the underlying research question top of mind, we can more fully understand variation in the impact across the multiplicity of attributes of students. Variation is the norm, and indeed, Stephen Gould (2013) claimed variation is nature’s only irreducible essence. Variation is the hard reality, not a set of imperfect measures for a central tendency.

What is needed is for more professional learning that leads to educators weighing up often competing research claims, and it would be most worthwhile to have more meta-analyses that make comparisons across interventions, investigate the differential effects in various contexts, elaborate on the control group or business as usual, be more specific about costs relative to benefits, code for the nature of implementation of influences, specifically look for unintended consequences and side effects (Zhao, 2018), and explore efficiency and affective outcomes alongside achievement effectiveness. Ultimately, however, I continue to see research advances in terms of Popperian falsification. The core question for researchers and educators remains – what evidence would I accept that I was wrong? As Cronbach (1980, p. 103) said “A proposition deserves some degree of trust only when it has survived serious attempts to falsify it.”

Not only because I am married to a professor of evaluation and that I dedicate this book to the grandfather of evaluation, Michael Scriven, but the core notions of evaluation, evaluative thinking, and what is it we value needs to be ever-present in the class, school, and system. It is rare to find evaluation methods built into interventions. More often, evaluation comes after implementation and, at best then, can make summative claims but with little evaluation of implementation processes, and hardly ever are there comparisons of an intervention with business as normal. Edovald and Nevill (2021) found that only 38% of 1,017 education trials identified between 1980 and 2016 included a process evaluation component (see also Connolly et al., 2018). Comparative trials are critical, and evaluation and evaluative thinking are the core of our business.

The seventeenth-century educational thinker Comenius was quite vivid in his description of non-evaluative thinking (Smith, 2017, p. 16):

Translating this vivid comic imagery of quarrelsome academics vomiting and defecating their undigested knowledge and forcing poisons down students’ thoughts into more prosaic terms, we see Comenius leveling several accusations against the teaching profession: Too much learning is from books that offer a distant substitute from reality. Too many teachers set themselves up as guises which lacking their own inner store of wisdom. Too much teaching is the violent imposition on students of content that will not actually contribute to their flourishing. Too many of the disciplines are thickets of confusing options each at war with the rest.

The eleventh and final leap is the question that has been core to educators from time zero. How to advance both the equity issues that are the basis for educating in the first place? In any school system, it is a major mistake is to assume outcomes = achievement. There is so much more to growing up in schools: there are many skills, wills, and thrills of learning, living, communicating, developing character, and learning to work and understand others. Unfortunately, there are still too many vicious inequalities in our schools, some embedded in the name of enhancing achievement (ability grouping, school composition manipulations), and equity issues abound and need contesting. Piketty (2022) has surveyed the economic structures across the world in a series of books and claimed “the diffusion of knowledge has always been the central enabling real equality” (p. 176). I note the percentage of the world that were literate and numerate (say to Level I in PISA) that in 1900, it was about 15%, and in 2000, it was 85%. Surely, the most remarkable feature of the twentieth century is the impact of teachers. But we still have miles to travel. Piketty noted that in the US, parents’ income predicts almost perfectly the child’s chances of going to a university, and further, the more affluent parents have greater access to the highly specialized curriculum in the richer universities. Such social selection is not reserved to the US.

Concluding comment

Like all education endeavors, Visible Learning has a life course. This book embodies my ideal of continuous learning, of being challenged and critiqued, and of conjectures and refutations, and it invites educators to align their thinking, actions, and impact with robust evidence.

Appendix A: References to related publications germane to topics throughout this sequel

(Note: There are 86 translations across 14 books into 29 languages – Arabic, Azerbaijani, Brazilian Portuguese, Mandarin Czech, Danish, Dutch, Dutch (Flemish), Estonian, French, German, Hebrew, Italian, Japanese, Korean, Lithuanian, Norwegian, Polish, Portuguese, Romanian, Russian, Serbian, Slovenian, Spanish, Swedish, and Turkish.)

Overviews

Hamilton, A., & Hattie J.A.C. (2022). The Lean Education Manifesto: Designing More Cost-Effective Initiatives that Generate Stronger Outcomes for Developing Nations. Corwin.

Hamilton, A., Reeves, D., Clinton, J.M., & Hattie, J.A.C. (2022). Implementation: The 5D model. Corwin.

Hattie, J.A.C. (2009). Visible learning: A synthesis of 800+ meta-analyses on achievement. Routledge.

Hattie, J.A.C. (2012). Visible learning for teachers. Maximizing impact on achievement. Routledge.

Hattie, J.A.C. (2015). The applicability of Visible Learning to Higher Education. Scholarship of Teaching and Learning in Psychology, 1(1), 79–91.

Hattie, J.A.C. (2022). John A. C. Hattie: The Pioneer and Developer of Visible Learning in Education and Science of Learning. (pp. 149–180). In Bembenutty, H., Contemporary pioneers in teaching and learning. IAP Publishers.

Hattie, J.A.C., & Anderman, E. (2013). Handbook on Student Achievement. Routledge.

Hattie, J.A.C., & Anderman, E. (2020). Visible Learning: Guide to Student Achievement – Schools edition. Routledge.

Hattie, J.A.C., Bustamante, V., Almarode, J., Fisher, D., & Frey, N. (2021). Great teaching by design: From intention to implementation in the Visible Learning Classroom. Corwin.

Hattie, J.A.C., & Larsen, S. (2020). The purposes of education: In conversation. Routledge.

Hattie, J.A.C., Masters, D., & Birch, K. (2016). Visible Learning into Action. Routledge.

Hattie, J.A.C., & Toscano, M. (2018). John Hattie on interpretation, the story of research, and the necessity of falsifiability: In dialogue with Maurizio Toscano. in J. Quay, J., Bleazby, S., Stolz. M., Toscano, & R Webster (Eds.). Theory and Philosophy in Education Research: Methodological Dialogues. Routledge.

Hattie, J.A.C., & Zierer, K. (2019). Visible learning: Auf den punkt gebracht. Schneider, Verlag, Hohengehren GmbH. Baltmannsweiler, Germany.

Hattie, J.A.C., & Zierer, Z. (2020). Visible Learning Insights. Routledge.

Meta-analyses and syntheses

Allen, K., Kern, M. L., Vella-Brodrick, D., Hattie, J., & Waters, L. (2018). What schools need to know about fostering school belonging: A meta-analysis. Educational Psychology Review, 30(1), 1–34.

Donoghue, G., & Hattie, J.A.C. (2021). Learning strategies: A meta-analysis of Dunlosky et al. (2013a). Frontiers of Education. 31 March 2021

Hattie, J.A. (1987). Identifying the salient facets of a model of student learning: A synthesis of meta-analyses. International Journal of Educational Research, 11, 187–212.

Hattie, J.A. (1992). Towards a model of schooling: A synthesis of meta-analyses. Australian Journal of Education, 36, 5–13.

Hattie, J.A., Biggs, J., & Purdie, N. (1996). Effects of learning skills intervention on student learning: A meta-analysis. Review of Research in Education, 66, 99–136.

Hattie, J.A.C., Rogers, H.J., & Swaminathan, H. (2011). The role of meta-analysis in educational research. In M. Peters (Ed.), Contextualising Educational Research. Sage: Beverly Hills, CA. https://link.springer.com/chapter/10.1007/978-94-007-6809-3_26

Purdie, N., & Hattie, J.A. (1999). The relationship between study skills and learning outcomes: A meta-analysis. Australian Journal of Education, 43, 72–86.

Rowe, L.I., Hattie, J.A.C., & Hester, R. (2021). G versus c: Comparing Individual and collective intelligence across two meta-analyses. Cognitive Research: Principles and Implications. https://cognitiveresearchjournal.springeropen.com/articles/10.1186/s41235-021-00285-2

Teachers

Clinton, J.M., & Hattie, J.A.C. (2014). Teachers as evaluators – An empowerment evaluation approach. In Fetterman, D., Kaftarian, S., & Wandersman, A. (Eds.). Empowerment Evaluation: Knowledge and Tools for Self-assessment, Evaluation Capacity Building, and Accountability. Sage.

Clinton, J.M., Hattie, J.A.C., & Nawab, D. (2018). The good teacher: Our best teachers are inspired, influential and passionate. In M. Harring (Ed.) Handbook for School Pedagogics. (pp. 880–888). Munster, Germany, Waxmann.

Donohoo, J., O’Leary, T., & Hattie, J.A.C. (2020), The design and validation of the enabling conditions for collective teacher efficacy scale (EC-CTES), Journal of Professional Capital and Community, 5(2), 147–166.

Dunn, R., & Hattie, J.A.C. (2021) Leading by design to develop teacher expertise. Corwin.

Hattie, J.A.C., Anderson, M., Clinton, J., & Rickards, F. (2015). Developing an evidence base model for the effects of teacher education programs on teacher candidates. In T.O. Seng & W.C. Liu (Eds.). Teacher Effectiveness: Capacity building in a complex learning era. Cengage Learning Asia.

Rickards, F., Hattie, J.A.C., & Reid, C. (2021). The turning point: Growing expertise, evaluative thinking, and the future of the teaching profession. Routledge.

Rowe, L.I., & Hattie, J.A.C. (2022). Research through the eyes of teachers. In C. Overson, C. Hakala, L. Kordonowyi, & V. Benassi (Eds.), In their own words: What scholars want you to know about why and how to apply the science of learning in your academic setting. APA Publications.

School leaders

Bendiksen, L., Robinson, V.M.J., & Hattie J.A.C. (2012). Principal Instructional Leadership – what is it and what is its relationship to secondary school performance? SET: Research information for teachers, 1, 2–8. https://search.informit.org/doi/abs/10.3316/informit.505725715123816

Donohoo, J., Hattie, J.A.C., & Eells, R. (2018). The power of collective efficacy. Educational Leadership, 75(6), 41–44.

Hamilton, A., Wiliam, D., & Hattie, J.A.C. (2023). Room for Impact: The 9P De-Implementation Playbook for Educators. Corwin.

Hattie, J.A.C., & Clinton, J.M. (2011). School leaders as evaluators. In Activate: A leader’s guide to people, practices and processes. (pp. 93–118). Englewood, CO: The Leadership and learning Centre.

Hattie, J.A.C., Donohoo, J., & deWitt, P. (2021). Understanding impact to foster collective efficacy. Principal, 24(2), 14–17.

Ingvarson, L., & Hattie, J.A.C. (Eds). (2008). Assessing teachers for professional certification: The first decade of the National Board for Professional Teaching Standards. Advances in Program Evaluation Series #11, Elsevier (pp. 610).

Moir, S., Hattie, J.A.C., & Jansen, C. (2014). Teacher perspectives of ‘effective’ leadership in schools Australian Education Leader, 36 (4), 36–41.

Reeves, D., & Hattie, J.A.C. (2012). Editors. Activate: A leader’s guide to people, practices and processes. The Leadership and Learning Centre.

Systems

Cawsey, A.M., Hattie, J.A.C., & Masters, G. (2019). Growth to achievement: On-demand resources for teachers. Commonwealth Department of Education Report.

Hattie, J.A.C., (2015). What doesn’t work in education: The Politics of distraction. Open Ideas at Pearsons. www.pearson.com/hattie/distractions.html Politics of Distraction.

Hattie, J.A.C., (2015). What does work in education: The Politics of Collaborative action. Open Ideas at Pearsons. www.pearson.com/hattie/distractions.html

Mind frames

Hattie, J., & Smith, R. (Eds.). (2020). 10 Mindframes for Leaders: The Visible Learning Approach to School Success. Corwin.

Hattie, J.A.C., & Hattie, K.R. (2022). Visible learning for parents. Routledge.

Hattie, J.A.C., & Zierer, K. (2018). 10 Mindframes for Visible Learning. Routledge.

Learning

Bolton, S., & Hattie, J.A.C. (2018). Cognitive and brain development: Executive function, Piaget, and the prefrontal cortex. Archives of Psychology, 1(3), 1–16.

Clinton, J.M., & Hattie, J.A.C. (2021). Cognitive complexity of evaluator competencies. Evaluation and Program Planning, 89, 1–8.

Hattie, J.A.C. (2017, June 28). Misinterpreting the growth mindset: Why we’re doings students a disservice. Finding Common Ground, Education Week https://blogs.edweek.org/edweek/finding_common_ground/2017/06/misinterpreting_the_growth_mindset_why_were_doing_students_a_disservice.html

Hattie, J.A.C. (2019). Learning Strategies in the Intentional alignment of Tertiary Teaching. Psychology of Education Review, 43(1), 20–22.

Hattie, J.A.C., & Donoghue, G. (2016). Learning strategies: A synthesis and conceptual model. Nature: Science of Learning. 1. doi:10.1038/npjscilearn.2016.13.

Hattie, J.A.C., & Donoghue, G.M. (2018). A model of learning: Optimizing the effectiveness of learning strategies. In K. Illeris (Ed.), Contemporary theories of learning. 2nd. Ed., (pp. 97–113). Routledge, Oxon. (Also 15 Aktuelle Laringsteorier Forfattern og Samfundslitteratyr, 2019).

Hattie, J.A.C., & Nugent, A. (2021). The ‘science of learning’: Birth or Renaissance. In A. Carroll, R. Cunnington, & A. Nugent (2021). Learning under the lens: Applying findings from the Science of Learning to the classroom. Routledge.

Hattie, J.A.C., Hoddis, F., & Kang, S. (2020). Theories of motivation: Integration and ways forward. Contemporary Educational Psychology, 61, 101865.

Hattie, J.A.C., & Timperley, H (2007). The power of feedback. Review of Educational Research, 77(1), 81–112.

Hattie, J.A.C., & Wollenschläger, M. (2014). A conceptualization of feedback. In H. Ditton & A. Muller (Eds)., Feedback und Ruckmeldungen. (pp. 135–150), Munster Germany: Waxmann.

Hattie, J.A.C., & Yates, G. (2014). Visible Learning and the Science of How we Learn. Routledge.

Hawthorne, B., Vella-Broderick, D., & Hattie, J.A.C. (2019). Well-being as a cognitive load reducing agent: A review of the literature. Frontiers in Education, 4:121.

Hodis, F., Hattie, J.A.C., & Hodis, M. (2017). Investigating student motivation at the confluence of multiple effectiveness strivings: A study of promotion, prevention, locomotion, assessment, and their interrelationships. Personality and Individual Differences, 109, 181–191.

Purdie, N., & Hattie, J.A.C. (2002). Assessing students’ conceptions of learning. Australian Journal of Educational and Developmental Psychology, 2, 17–32.

Class size

Hattie, J.A.C. (2007). The paradox of reducing class size and improved learning outcomes. International Journal of Education, 42, 387–425.

Hattie, J.A.C. (2013). Class size. In J. Hattie, & E. Anderson (Eds.), International Guide to Student Achievement (1 ed., pp. 131–133). US (New York): Routledge.

Hattie, J.A.C. (2016). The right question in the debates about class size: Why is the (positive) effect so small? Blatchford, P., Chan, K.W., Galton, M., Lai, K.C., & Lee, J.C.L. (Eds)., Class Size: Eastern and Western Perspectives. (pp. 105–118). Routledge.

Feedback

Brooks, C., Burton, R., & Hattie, J.A.C. (2021). Feedback for learning. In K-A, Allen, A. Reupert, & L. Oades (Eds). Building Better Schools with Evidence-based Policy: adaptable policy for teachers and school leaders. (pp. 65–70). Routledge.

Brooks, C., Burton, R., Van der Kleij, F., Ablaza, C., Carroll, A., Hattie, J., & Neill, S. (2021). Teachers activating learners: The effects of a student-centred feedback approach on writing. Teaching and Teacher Education. Teaching and Teacher Education, 105, 103387.

Brooks, C., Burton, R., Van der Kleij, F., Carroll, A., Hattie, J.A.C. (2021). Towards student-centred feedback practices: Evaluating the impact of a professional learning intervention in primary schools. Assessment in Education: Principles, Policy and Practice.

Brooks, C., Burton, R., van der Kleij, F., Carroll, A., Olave, K., & Hattie, J.A.C. (2020). From fixing the work to improving the learner: An initial evaluation of a professional learning intervention using a new student-centred feedback model. Studies in Educational Evaluation, 68, 1–12.

Brooks, C., Carroll, A., Gillies, R., & Hattie, J.A.C. (2019). A matrix of feedback for learning. The Australasian Journal of Teacher Education. 44(4), 14–21.

Brooks, C., Huang, Y., Hattie, J., Carroll, A., & Burton, R. (2019) What is my next step? School students’ perceptions of feedback. Frontiers in Education, 4, 96–106.

Gan, M.J.S., & Hattie, J.A.C. (2014). Prompting secondary students’ Use of Criteria, Feedback Specificity and Feedback Levels during an Investigative task. Instructional Science, 42(6), 861–878. https://link.springer.com/content/pdf/10.1007/s11251-014-9319-4.pdf

Harks, B., Rakoczy, K., Hattie, J., Besser, M., & Klieme, E. (2013). The effects of feedback on achievement, interest, and self-evaluation: The role of feedback’s perceived usefulness. Educational Psychology, 24(4), 269–290. DOI: 10.1080/01443410.2013.785384

Hattie, J., Crivelli, J., Van Gompel, K., West-Smith, P., & Wike, K. (2021). Feedback That Leads to Improvement in Student Essays: Testing the Hypothesis that “Where to Next” Feedback is Most Powerful. Frontiers of Education. 6:645758. doi: 10.3389/feduc.2021.645758

Hattie, J.A.C., & Clarke, S. (2019) Visible Learning: Feedback. Routledge.

Hattie, J.A.C., Gan. M., & Brooks, C. (2016). Instruction based on feedback. In R. Mayer & P. Alexander (Eds.), Handbook of Research on Learning and Instruction. (2nd Ed., pp. 249–271). Routledge.

Hattie, J.A.C., & Yates, G.C.R. (2014). Using feedback to promote learning. In V.A. Benassi, C.E. Overson, & C.M. Hakala (Eds.). Applying science of learning in education: Infusing psychological science into the curriculum. pp. 45–58. American Psychological Association.

Wisniewski, B., Zierer, Z., & Hattie, J.A.C. (2020). The power of feedback revisited: A meta-analysis of educational feedback research Journal: Frontiers in Psychology, 10, 3087.

Wollenschläger, M., Hattie, J.A.C., Machts, N., Möller, J., & Harms, U. (2016). What makes rubrics effective in teacher-feedback? Transparency of learning goals is not enough. Contemporary Education Psychology, 44, 1–11.

Assessment

Brown, G.T., O’Leary, T.M., & Hattie, J.A.C. (2019). Effective reporting for formative assessment: The asTTle case example. In D. Zapata-Rivera (Ed.). Score Reporting: Research and Applications. Routledge.

Clarke, S., Timperley, H., & Hattie, J.A. (2003). Assessing formative assessment. Hodder Moa Beckett, New Zealand.

Fisher, D., Frey, N., Hattie, J.A.C., & Flories, K.T. (2019). Learner’s guide: Becoming an assessment-capable Visible Learner, Grades 3–5. Corwin

Fisher, D., Frey, N., Hattie, J.A.C., & Flories, K.T. (2019). Teacher’s guide: Becoming an assessment-capable Visible Learner: Level I: Grades 6–12. Corwin

Frey, N., Hattie, J.A.C., & Fisher, D. (2018). Developing Assessment Capable Learners. Corwin.

Frey, N., Hattie, J.A.C., & Fisher, D. (2019). Learner’s Notebook: Developing Assessment Capable Learners. Level 1: Grades 6–10. Corwin.

Frey, N., Hattie, J.A.C., & Fisher, D. (2019). Learner’s Notebook: Developing Assessment Capable Learners. Level 2: Grades 3–5. Corwin.

Frey, N., Hattie, J.A.C., & Fisher, D. (2019). Learner’s Notebook: Developing Assessment Capable Learners. Level 3: Grades K–2. Corwin.

Hattie, J.A.C., & Leeson, H. (2013). Future directions in assessment and testing in education and psychology. In K. Geisenger & B. Bracken (Eds.), APA Handbook of Testing and Assessment in Psychology. Vol. 3 (pp. 591–622). APA Press.

Students

Carroll, A., Houghton, S., Durkin, K., & Hattie, J.A.C. (2009). Adolescent Reputations and Risk: Developmental trajectories to delinquency. Springer.

Hattie, J.A.C., Clarke, S., Fisher, D., & Frey, N. (2021) Collective student efficacy. Corwin.

Pfost, M., Hattie, J.A.C., Dorfler, T., & Artelt, C. (2014). Individual Differences in Reading Development: A Review of 25 Years of Empirical Research on Matthew Effects in Reading, Review of Educational Research, 84(2), 203–244.

Wardman, J., & Hattie, J. (2012). Administrators’ perceptions of full-year acceleration at high school. Australasian Journal of Gifted Education, 21(1), 32–41.

Wardman, J., & Hattie, J.A.C. (2018). What works better than the rest? The impact of various curricula provisions for gifted learners. In B. Wallace, D.A. Sisk, & J. Senior (Eds). The Sage Handbook of gifted and talented education. (pp. 321–334). Sage.

Mathematics

Almarode, J., Fisher, D., Assof, J., Moore, S.D., Hattie, J.A.C., & Frey, N. (2019). Teaching Mathematics in the Visible Learning Classroom High School, Grades 6–8. Corwin

Almarode, J., Fisher, D., Assof, J., Moore, S.D., Hattie, J.A.C., & Frey, N. (2019). Teaching Mathematics in the Visible Learning Classroom High School, Grades High School. Corwin

Almarode, J., Fisher, D., Thunder, K., Hattie, J.A.C., & Frey, N. (2020). Teaching Mathematics in the Visible Learning Classroom High School, Grades K–2. Corwin

Almarode, J., Fisher, D., Thunder, K., Moore, S.D., Hattie, J.A.C., & Frey, N. (2019). Teaching Mathematics in the Visible Learning Classroom High School, Grades 3–5. Corwin

Hattie, J.A.C., & Fisher, D., Frey, N., Gojak, L.M., Moore, S.D., & Mellman, W. (2017). Visible Learning for Mathematics, Grades K–12: What works best to optimize student learning. Corwin.

Hattie, J.A.C., & Fisher, D., Frey, N., Gojak, L.M., Moore, S.D., & Mellman, W. (2019). Visible Learning for Mathematics, High School Classroom Comparison. Corwin.

English language learning

Fisher, D., Frey, N., Hattie, J., & Amador-Valerio, O. (2019). Visible learning for English Language Learners. Corwin

Fisher, D., Frey, N., & Hattie, J.A.C. (2017). Teaching Literacy in the Visible Learning Classroom: K–5 classroom companion. Vol I. Corwin.

Fisher, D., Frey, N., & Hattie, J.A.C. (2017). Teaching Literacy in the Visible Learning Classroom: 6–12 classroom companion. Vol II. Corwin.

Fisher, D., Frey, N., & Hattie, J.A.C. (2016). Visible Learning for Literacy, Grades K–12: Implementing the practices that work best to accelerate student learning. Corwin. Winner Corwin Book of the Year 2016.

Science

Almarode, J.T., Fisher, D., Frey, N., & Hattie, J.A.C. (2018). Visible learning in Science, Grades K–12: What works best to optimize student learning. Corwin.

Fisher, D., Frey, N., Almarode, J., & Hattie, J.A.C. (2017). Teaching Science in the Visible Learning Classroom: K–5 classroom companion. Vol I. Corwin.

Social studies

Hattie, J.A.C., Stern, J., Fisher, D., & Frey, N. (2020). Visible Learning for Social Studies, Grades K–12: designing student learning for conceptual understanding. Corwin.

Music

McPherson, G.E., & Hattie, J.A.C. (2021). High impact teaching mindframes. In G.E. McPherson (Ed.), The Oxford Handbook of Music Performance, Vol. 1. Oxford University Press.

McPherson, G.E., & Hattie, J.A.C. (2021). High impact teaching mindframes. In G.E. McPherson (Ed.), The Oxford Handbook of Music Performance, Vol. 1. (pp. 123–150). Oxford University Press.

McPherson, G.E., Blackwell, J., & Hattie, J. (2022). Feedback in Music Performance Teaching. Frontiers in Psychology, 13, www.frontiersin.org/article/10.3389/fpsyg.2022.891025, DOI=10.3389/fpsyg.2022.891025

Higher education

Hattie, J.A.C. (2011). Which Strategies Best Enhance Teaching and Learning in Higher Education? In D. Mashek & E. Hammer (Ed.), Empirical research in teaching and learning: Contributions from social psychology. (pp. 130–142). Wiley-Blackwell.

Hattie, J.A.C. (2015). The applicability of Visible Learning to Higher Education. Scholarship of Teaching and Learning in Psychology, 1(1), 79–91.

Distance and COVID teaching

Hattie, J. A. C. (2021). An Ode to expertise. Corwin. https://protect-au.mimecast.com/s/43HsCXLKNwFX9YWqYT62nlB?domain=f.hubspotusercontent10.net

Fisher, D., Frey, N., & Hattie. J.A.C. (2021). The distance learning playbook Grades K–12: instruction: Teaching for engagement and impact in any setting.

Fisher, D., Frey, N., Almarode, J., Hattie. J., Wiseman, R. (2021). The distance learning playbook for parents: Teaching for engagement and impact in any setting. Corwin.

Fisher, D., Frey, N., Almarode, J.T., & Hattie, J.A.C. (2021). The distance learning playbook for college and university instruction. Corwin

Fisher, D., Frey, N., Bustamante, V. J., & Hattie. J.A.C. (2021). Assessment playbook for Distance Blended Learning. Corwin.

Fisher, D., Frey, N., Smith, D., & Hattie, J.A.C. (2021). Leading the Rebound: 20+ must-dos to restart teaching and learning. Corwin.

Fisher, D., Frey, N., Smith, D., & Hattie, J.A.C. (2021). Rebound. Grades K–12: A playbook for rebuilding agency, accelerating learning recovery, and rethinking schools. Corwin.

Development of Visible Learning

Fraser, B., Walberg, H., Welch, W., & Hattie, J.A. (1987). Syntheses of educational productivity research. Monograph in International Journal of Educational Research Series. Pergamon.

Hattie, J.A. (1992). Towards a model of schooling: A synthesis of meta-analyses. Australian Journal of Education, 36, 5–13.

Hattie, J.A. (1993). Measuring the effects of schooling. SET, 2, 1–4.

Hattie, J.A.C. (2010). On being a ‘critic and conscience of society’: The role of the education academic in public debates. New Zealand Journal of Educational Studies, 45(1), 85–96.

Hattie, J.A.C. (2017). Educators are not uncritical believes of a cult figure. School Leadership and Management, 27(4), 427–430.

Hattie, J.A.C. (2022). An Interview with John A. C. Hattie: The Pioneer and Developer of Visible Learning in Education and Science of Learning. In Bembenutty, H., Contemporary pioneers in teaching and learning. IAP Publishers.

Kraft, M., & Hattie, J.A.C. (2021) Interpreting Education Research for Policy and Practice: A Conversation. Educational Leadership. 78(8), 18–24.

Gold papers (www.visiblelearning.com/content/gold-papers)

Hamilton, A., & Hattie, J.A.C. (2021). Getting to G.O.L.D.: The visible Learning approach to unleashing education improvement.

Hamilton, A., & Hattie, J.A.C. (2021). Not all that glitters is gold. Can education technology finally deliver?

Hattie, J.A.C., & Hamilton, A. (2019). Real GOLD vs. Fool’s GOLD: The Visible Learning methodology for finding what works best in education.

Hattie, J.A.C., & Hamilton, A. (2020). As good as Gold? Why we focus on the wrong drivers in education.

Appendix B: Chapter headings from Handbook on Student Achievement (Hattie & Anderman, 2013)

Bolded chapters were updated in Visible Learning: Guide to Student Achievement – Schools edition (Hattie & Anderman, 2020).

Section 1. Understanding Achievement Eric M. Anderman & John Hattie

1.1 Defining Student Achievement R. Guskey

1.2 Academic Achievement: An Elementary School Perspective Alan Bates, Rena Shiffl et, & Miranda Lin

1.3 Academic Achievement: An Adolescent Perspective R. Trent Haines & Christian E. Mueller

1.4 Adult Education and Achievement M. Cecil Smith

1.5 Academic Achievement: A Higher Education Perspective Terrell L. Strayhorn

1.6 Developmental Education for Adults and Academic Achievement Joshua D. Hawley & Shu Chen Chiang

Section 2. Influences from the Student Mimi Bong

2.1 Entry to School Collette Tayler

2.2 Piagetian Approaches Philip Adey & Michael Shayer

2.3 Entry to Tertiary Education Emer Smyth

2.4 Physical Activity Janet Clinton

2.5 Gender Influences Judith Gill

2.6 Engagement and Opportunity to Learn Phillip L. Ackerman

2.7 Behavioral Engagement in Learning Jennifer Fredricks

2.8 Goal Setting and Academic Achievement Dominique Morisano & Edwin A. Locke

2.9 Self-Reported Grades and GPA Marcus Credé & Nathan R. Kuncel

2.10 Conceptual Change Stella Vosniadou & Panagiotis Tsoumakis

2.11 Social Motivation and Academic Motivation Tim Urdan

2.12 Attitudes and Dispositions Robert D. Renaud

2.13 Personality Influences Meera Komarraju

2.14 Academic Self-Concept Herbert W. Marsh & Marjorie Seaton

2.15 Self-Efficacy Mimi Bong

2.16 Motivation Dale H. Schunk and Carol A. Mullen

2.17 Friendship in School Annemaree Carroll, Stephen Houghton, and Sasha Lynn

2.18 Indigenous and Other Minoritized Students Russell Bishop

2.19 Low Academic Success David A. Bergin

2.20 Learning Difficulties in School R. Allan Allday & Mitchell L. Yell

Section 3. Influences from the Student Mimi Bong

3.1 Resident and Nonresident Fathers William Jeynes

3.2 Home Environment Burkhard Gniewosz & Jacquelynne S. Eccles

3.3 Socioeconomic Status and Student Achievement Jeanne Brooks-Gunn & Erin Bamgarner

3.4 Welfare Policies Lisa A. Gennetian & Pamela A. Morris

3.5 Family–School Partnerships and Academic Achievement Andrew J. Martin

3.6 Parent Involvement in Learning Wendy S. Grolnick, Jacquelyn N. Raftery, & Elizabeth S. Flamm

3.7 Maternal Employment and Achievement Rachel G. Lucas-Thompson & Wendy A. Goldberg

3.8 Television and Academic Achievement Andrew J. Martin

Section 4. Influences from the School Catherine Bradshaw

4.1 Charter Schools and Academic Achievement Ann Allen

4.2 Ability Grouping Ed Baines

4.3 Evaluating and Improving Student–Teacher Interactions Anne H. Cash & Bridget K. Hamre

4.4 Mixed-Grade Elementary-School Classes and Student Achievement Linley Cornish

4.5 School-Based Mental Health Erin Dowdy, Matthew P. Quirk, & Jenna K. Chin

4.6 Achievement in Faith-Based Schools L. Mickey Fenzel

4.7 Class Size John Hattie

4.8 Financing Schools Eric A. Hanushek

4.9 Influences of School Layout and Design on Student Achievement C. Kenneth Tanner

4.10 Grade Retention Shane R. Jimerson & Jacqueline A. Brown

4.11 Inclusive Education Geoff Lindsay

4.12 School-Wide Positive Behavior Interventions and Supports and Academic Achievement Kent McIntosh, Sophie V. Ty, Robert H. Horner, & George Sugai

4.13 School Connectedness Clea McNeely

4.14 Teacher Mentoring, Coaching, and Consultation E lise T. Pas & Daniel S. Newman

4.15 The Link between Student Mobility and Academics Bess Rose & Catherine P. Bradshaw

4.16 Service-Learning Shelley H. Billig

4.17 Single-Sex Schools and Academic Achievement Shirley L. Yu & Isabel Rodriguez-Hejazi

4.18 Summer School and Student Achievement in the United States Jordan D. Matsudaira

4.19 Within Class Grouping: Arguments, Practices, and Research Evidence Yiping Lou

4.20 Special Education and Academic Achievement Benjamin Zablotsky & Michael S. Rosenberg

4.21 Social and Emotional Learning and Academic Achievement Jessika Zmuda & Catherine P. Bradshaw

4.22 Middle School Transitions Eric M. Anderman

Section 5. Influences from the Classroom Julianne Turner

5.1 Classroom Instructional Contexts Debra K. Meyer

5.2 Academic Motivation and Achievement in Classrooms Lynley H. Anderman

5.3 Elementary Classroom Management Inge R. Poole & Carolyn M. Evertson

5.4 Emotion and Achievement in the Classroom Thomas Goetz & Nathan C. Hall

5.5 Secondary Classroom Management Anne Gregory & Jennifer R. Jones

5.6 Homework and Academic Achievement Jianzhong Xu

5.7 The Role of Formative Assessment in Student Achievement Aryn C. Karpinski & Jerome V. D’Agostino

5.8 Peer Influences in Elementary School Gary W. Ladd

5.9 Acceleration for All Henry M. Levin & Pilar Soler

5.10 Ability Grouping Janet Ward Schofi eld

5.11 Collaboration in the Classroom Noreen M. Webb

Section 6. Influences from the Teacher Anita Woolfolk

6.1 Teacher–Student Relationships Heather A. Davis

6.2 National Board for Professional Teaching Standards Mary E. Dilworth

6.3 Classroom Management and Student Achievement Jerry Freiberg

6.4 Fostering Student Creativity in the Era of High-Stakes Testing Bree Frick

6.5 Nontraditional Teacher Preparation Belinda G. Gimbert

6.6 Quality of Teaching Laura Goe

6.7 Methods for Studying Teacher and Teaching Effectiveness Alexander Gröschner, Tina Seidel, & Richard J. Shavelson

6.8 Teachers’ Expectations Lee Jussim

6.9 Teacher Enthusiasm and Student Learning Melanie Keller, Knut Neumann, & Hans E. Fischer

6.10 Teachers’ Cultural and Professional Identities and Student Outcomes Revathy Kumar & Linda Alvarado

6.11 Teacher Intelligence: What Is It and Why Do We Care? Andrew J. McEachin & Dominic J. Brewer

6.12 Pedagogical Content Knowledge Julie Gess-Newsome

6.13 Teacher Beliefs about Teaching and Learning: The Role of Idea-Oriented Pedagogy Richard Prawat

6.14 School Reform Linda Valli & Carla Finkelstein

6.15 Teacher Efficacy John A. Ross

6.16 Teachers’ Epistemological Beliefs and Achievement Gregory Schraw, Joanne Brownlee, & Lori Olafson

6.17 Teacher Motivation and Student Achievement Outcomes H. M. G. Watt & P. W. Richardson

6.18 The Relation of Teacher Characteristics to Student Achievement Xin Ma

Section 7. Influences from the Curriculum Rayne Sperling

7.1 Values Education Programs Terence Lovat

7.2 Activity-Based Learning Strategies Kira J. Carbonneau & Scott C. Marley

7.3 Bilingual Education Programs and Student Achievement Jill Fitzgerald & Jackie Eunjung Relyea-Kim

7.4 Intelligent Tutors – Strategy Types Bonnie J. F. Meyer

7.5 Creativity and Creativity Programs Heather L. Hammond, Lauren E. Skidmore, Amanda Wilcox-Herzog, & James C. Kaufman

7.6 Outdoor Education Justin Dillon

7.7 Role of Discussion in Reading Comprehension Ian A. G. Wilkinson & Kathryn Nelson

7.8 The Impact of Calculators on Student Achievement in the K–12 Mathematics Classroom Aimee J. Ellington

7.9 Second Language Vocabulary Yongqi Gu

7.10 Language Teaching Curricula Eli Hinkel

7.11 Measurement of History Achievement in the United States and Europe Mark Smith, Joel Breakstone, Sam Wineburg

7.12 Reading: Phonics Instruction William E. Tunmer & Alison W. Arrow

7.13 Repeated Reading William J. Therrien & Sarah J. Watt

7.14 Reading: Sentence Combining: Grammar Programs Bruce Saddler & Nicole Bak

7.15 Extracurricular Activities in Secondary Schools Boaz Shulruf & Grace Ying Wang

7.16 Improving Academic Achievement with Social Skills Frank M. Gresham, Michael J. Vance, & Jeffrey Chenier

7.17 Visual Perception Programs Barbara Hanna Wasik, Adrienne N. Villagomez, Sheena Berry, & Sandra B. Evarrs

7.18 Reading: Vocabulary Programs A. Wilson, R. Jesson, & S. McNaughton

7.19 Achievement in Adolescent Health Education Megan Sanders, Rashea Hamilton, & Eric M. Anderman

7.20 Writing Achievement Mark Torrance & Raquel Fidalgo

7.21 Reading: Comprehension Programs Janice F. Almasi & Barbara Martin Palmer

7.22 Response to Intervention: The Sum is Greater than Its Parts Paul J. Riccomini & Gregory W. Smith

7.23 Successful Mathematics Achievement is Attainable Patti Brosnan, Aaron Schmidlin, & Melva R. Gran

Section 8. Influences from Teaching Strategies Christine Rubie-Davies

8.1 Goal Orientation Andrew J. Martin

8.2 Goal Setting and Personal Best Goals Andrew J. Martin

8.3 Keller’s Personalized System of Instruction Eric J. Fox

8.4 Concept Mapping Joseph D. Novak

8.5 Direct Instruction Gregory Arief D. Liem & Andrew J. Martin

8.6 Reciprocal Teaching Annemarie Palincsar

8.7 Cooperative, Competitive, and Individualistic Learning Environments David W. Johnson & Roger T. Johnson

8.8 Peer Tutoring School-Age Children Dilara Deniz Can and Marika Ginsburg-Block

8.9 Problem Solving R. Taconi

8.10 Problem-Based Learning David Gijbels, Piet Van den Bossche, & Sofie Loyens

8.11 The Search for the Key for Individualised Instruction Catherine Scott

8.12 Instructional Simulations Jennifer J. Vogel-Walcutt, Naomi Malone, & Sae Schatz

8.13 Programmed Instruction Deborah V. Svoboda, Andrea L. Jones, & Kimberly van Vulpen, Donna Harrington

8.14 Multimedia Learning Richard E. Mayer

8.15 Technology-Supported Learning and Academic Achievement Peter Reimann & Anindito Aditomo

8.16 Feedback Helen Timperley

8.17 Individualized Instruction Hersh C. Waxman, Beverly L. Alford, & Danielle B. Brown

8.18 Worked Examples Paul Ayres & John Sweller

8.19 Spaced and Massed Practice Dominic A. Simon

8.20 Questioning Scotty D. Craig

8.21 Effects of Testing Jaekyung Lee & Young-Sun Lee

8.22 Metacognitive Strategies Linda Baker

8.23 Mentoring Brian Hansford & Lisa Catherine Ehrich

8.24 Teacher Immediacy Ann Bainbridge Frymier

8.25 Teaching Assistants Peter Blatchford & Rob Webster

8.26 Time on Task Tamara van Gog

8.27 Study Skills Dale H. Schunk & Carol A. Mullen

8.28 Matching Style of Learning Steve Higgins

8.29 Two Types of Perceived Control Over Learning: Perceived Efficacy and Perceived Autonomy. Nir Madjar & Avi Assor

8.30 Distance Education Yong Zhao & Jing Lei

8.31 Home School Programs Andrea Clements

8.32 Evidence Based Reading Comprehension Programs for Students with Learning Disabilities H. Lee Swanson & Michael Orosco

Section 9. Influences from an International Perspective Julian G. Elliott

9.1 Some Challenges to Educational Achievement in the Russian Federation since the End of the Soviet Union Julian G. Elliott

9.2 Large-Scale Assessments of Achievement in Canada1 Kadriye Ercikan, Maria Elena Oliveri, & Debra Sandilands

9.3 Student Achievement in Israel: The Challenges of Ethnic and Religious Diversity Yariv Fenige

9.4 Academic Achievement in Finland Jennifer Chung & Michael Crossley

9.5 Ghana David Peterson del Mar

9.6 Academic Achievement in South Africa Anil Kanjee

9.7 Educational Assessment and Educational Achievement in South America Jorge Manzi & David D. Preiss

9.8 Changing Definitions of Student Learning and Achievement in Post-conflict Nepal Shabnam Koirala-Azad

9.9 A Historical Perspective on Educational and Academic Achievement in Nigeria Charles Okonkwo & Richard Tabulawa

9.10 International Large-Scale Assessment Studies of Student Achievement Petra Stanat & Oliver Lüdtke

9.11 Academic Achievement in Singapore Pak Tee Ng

9.12 Academic Achievement in South Korea Jongho Shin

9.13 An Overview of Student Achievement and Related Factors in Taiwan Jeng Liu

References

Abuhamdeh, S., & Csikszentmihalyi, M. (2012). The importance of challenge for the enjoyment of intrinsically motivated, goal-directed activities. Personality and Social Psychology Bulletin, 38(3), 317–330.

Ackerman, P. L. (2005). Ability determinants of individual differences in skilled performance. In R. Sternberg & J. Pretz (Eds.), Cognition and intelligence: Identifying the mechanisms of the mind (pp. 142–159). Cambridge University Press.

AITSL (2017). Spotlight: Induction of beginning teachers in Australia – What do early career teachers say? https://www.aitsl.edu.au/tools-resources/resource/spotlight-induction-of-beginning-teachers-in-australia---what-do-early-career-teachers-say

Alessi, G. (1988). Diagnosis diagnosed: A systemic reaction. Professional School Psychology, 3, 145–151.

Alexander, R. (2000). Culture and pedagogy: International comparisons in primary education. Blackwell.

Alexander, R. (2020). A dialogic teaching companion. Routledge.

Allen, K., Kern, M. L., Vella-Brodrick, D., Hattie, J., & Waters, L. (2018). What schools need to know about fostering school belonging: A meta-analysis. Educational Psychology Review, 30(1), 1–34.

Alsowat, H. H. (2020). Evidence-based practices of English language teaching: A meta-analysis of meta-analyses. English Language Teaching, 13(11), 75–93.

Alter, J., & Coggshall, J. G. (2009). Teaching as a clinical practice profession: Implications for teacher preparation and state policy. Issue Brief. National Comprehensive Center for Teacher Quality.

Ames, M. G. (2019). The charisma machine: The life, death, and legacy of one laptop per child. MIT Press.

Amrein, A. L., & Berliner, D. C. (2002). High-stakes testing & student learning [Electronic Version]. Education Policy Analysis Archives, 10, 18–19.

Anderman, E. M., & Anderman, L. H. (2020). Classroom motivation: Linking research to teacher practice. Routledge.

Anderson, L. W., & Krathwohl, D. R. (2001). A taxonomy for learning, teaching, and assessing: A revision of Bloom’s taxonomy of educational objectives. Longman.

Anderson, R. C., Chinn, C., Waggoner, M., & Nguyen, K. (1998). Intellectually stimulating story discussions. In J. Osborn & F. Lehr (Eds.), Literacy for all: Issues in teaching and learning (pp. 170–186). Guilford.

Anscombe, G. E. M. (2000). Intention. Harvard University Press.

Anthis, K. (2011). Is it the clicker, or is it the question? Untangling the effects of student response system use. Teaching of Psychology, 38(3), 189–193.

Aristotle. (350BC/1995). Politics books III and IV (R. Robinson, Ed.). Oxford: Clarendon Press.

Aronson, E. (2002). Building empathy, compassion, and achievement in the jigsaw classroom. In J. Aronson (Ed.), Improving academic achievement: Impact of psychological factors on education (pp. 209–225). Academic Press.

Artelt, C., & Gräsel, C. (2009). Diagnostische Kompetenz von Lehrkräften. Zeitschrift für pädagogische Psychologie, 23(34), 157–160.

Asay, L. D., & Orgill, M. (2010). Analysis of essential features of inquiry found in articles published in The Science Teacher, 1998–2007. Journal of Science Teacher Education, 21(1), 57–79.

Atteberry, A., & Bryk, A. S. (2011). Analyzing teacher participation in literacy coaching activities. The Elementary School Journal, 112(2), 356–382.

Atteberry, A., Loeb, S., & Wyckoff, J. (2015). Do First Impressions Matter? Predicting Early Career Teacher Effectiveness. AERA Open, 1(4). https://doi.org/10.1177/2332858415607834

Atteberry, A., & McEachin, A. (2021). School’s out: The role of summers in understanding achievement disparities. American Educational Research Journal, 58(2), 239–282.

Australian Institute for Teaching and School Leadership. (2022). Strategic plan: 2022–2026. AITSL, Melbourne.

Ausubel, D. P. (1968). Educational psychology: A cognitive view. Holt, Rinehart, and Winston.

Baddeley, A. (2017). Exploring working memory: Selected works of Alan Baddeley. Routledge.

Bailey, R. (2000). Education in the open society: Karl Popper and schooling. Routledge.

Baird, M. D., & Pane, J. F. (2019). Translating standardized effects of education programs into more interpretable metrics. Educational Researcher, 48(4), 217–228.

Ball, D. L., & Forzani, F. M. (2010). What does it take to make a teacher?. Phi Delta Kappan, 92(2), 8–12.

Ball, D. L., Thames, M. H., & Phelps, G. (2008). Content knowledge for teaching: What makes it special? Journal of Teacher Education, 59(5), 389–407.

Ballard, T., Neal, A., Farrell, S., Lloyd, E., Lim, J., & Heathcote, A. (2022). A general architecture for modeling the dynamics of goal-directed motivation and decision-making. Psychological Review, 129(1), 146–174.

Bandura, A. (1986). Social foundations of thought and action. Englewood Cliffs.

Bandura, A. (1993). Perceived self-efficacy in cognitive development and functioning. Educational Psychologist, 28(2), 117–148.

Barber, M., Moffit, A., & Kihn, P. (2010). Deliverology 101: A field guide for educational leaders. Corwin Press.

Baumeister, R. F., Vohs, K. D., & Tice, D. M. (2007). The strength model of self-control. Current Directions in Psychological Science, 16(6), 351–355.

Baumrind, D. (1968). Authoritarian vs. authoritative parental control. Adolescence, 3(11), 255–272.

Beck, C. R. (1998). A taxonomy for identifying, classifying, and interrelating teaching strategies. The Journal of General Education, 47(1), 37–62.

Beck, I. L., McKeown, M. G., & Sandora, C. A. (2020). Robust comprehension instruction with Questioning the author: 15 years smarter. Guilford Publications.

Belland, B. R. (2014). Scaffolding: Definition, current debates, and future directions. In J. M. Spector, M. D. Merrill, J. Elen, & M. J. Bishop (Eds.), Handbook of research on educational communications and technology (4th ed., pp. 505–518). Springer.

Bendikson, L., Robinson, V. M. J., & Hattie, J. A. (2011). Identifying the comparative academic performance of secondary schools. Journal of Educational Administration, 49, 433–449.

Bereiter, C. (2002). Education and mind in the knowledge age. Lawrence Erlbaum Associates.

Bergeron, P. J., & Rivard, L. (2017). How to engage in pseudoscience with real data: A criticism of John Hattie’s arguments in visible learning from the perspective of a statistician. McGill Journal of Education/Revue des sciences de l’éducation de McGill, 52(1), 237–246.

Bergmann, J., & Sams, A. (2012). Flip your classroom: Reach every student in every class every day. International Society for Technology in Education.

Berk, R. (2011). Evidence-based versus junk-based evaluation research: Some lessons from 35 years of the evaluation review. Evaluation Review, 35(3), 191–203.

Berkely, J., & Hattie, J. A. C. (in review). Making learning visible: Observable correlates of the Aha! Moment when moving from surface to deep thinking.

Berliner, D. C. (2008). The nature of expertise in teaching. In M. Cochran-Smith, S. Feiman-Nemser, D. J. McIntyre, & K. E. Demers (Eds.), Handbook of research on teacher education (3rd ed., pp. 808–823). Routledge.

Bernstein, B. O. (1971). On the classification and framing of educational knowledge. In M. Young (Ed.), Knowledge and control (pp. 47–69). Collier-Macmillan.

Bernstein, B. O., Lubinski, D., & Benbow, C. P. (2021). Academic acceleration in gifted youth and fruitless concerns regarding psychological well-being: A 35-year longitudinal study. Journal of Educational Psychology, 113(4), 830.

Berry, A. (2023). Reimaging student engagement: From disrupting to driving. Corwin.

Best, J. R., Miller, P. H., & Naglieri, J. A. (2011). Relations between executive function and academic achievement from ages 5 to 17 in a large, representative national sample. Learning and Individual Differences, 21(4), 327–336.

Biesta, G. J. J. (2012). Giving teaching back to education: Responding to the disappearance of the teacher. Phenomenology & Practice, 6(2), 35–49.

Biesta, G. J. J. (2013). The beautiful risk of education. Paradigm.

Biesta, G. J. J. (2020). Can the prevailing description of educational reality be considered complete? On the Parks-Eichmann paradox, spooky action at a distance and a missing dimension in the theory of education. Policy Futures in Education, 18(8), 1011–1025.

Biesta, G. J. J. cited in Seth, E. (2017). EEF toolkit ‘more akin to pig farming than science’. www.tes.com/magazine/archived/eef-toolkit-more-akin-pig-farming-science

Biggs, J. B. (1985). The role of metalearning in study processes. British Journal of Educational Psychology, 55(3), 185–212.

Biggs, J. B. (1999). What the student does: Teaching for enhanced learning. Higher Education Research and Development, 18(1), 57–75.

Biggs, J. B. (2003). Aligning teaching for constructing learning. Higher Education Academy, 1(4), 1–4.

Biggs, J. B., & Collis, K. F. (1982). Evaluating the quality of learning: The SOLO taxonomy (Structure of the Observed Learning Outcome). Academic Press.

Biggs, J. B., Tan, C., & Kennedy, G. (2023). Teaching for quality learning at university. McGraw-Hill Education.

Biller, A. M., Molenda, C., Zerbini, G., Roenneberg, T., & Winnebeck, E. C. (2022). Sleep improvements on days with later school starts persist after 1 year in a flexible start system. Scientific reports, 12(1), 1–14.

Bishop, R. (2019). Teaching to the North-East: Relationship-based learning in practice. New Zealand Council for Educational Research.

Bishop, R., Berryman, M., Cavanagh, T., & Teddy, L. (2009). Te Kotahitanga: Addressing educational disparities facing Māori students in New Zealand. Teaching and Teacher Education, 25(5), 734–742.

Black, P., & Wiliam, D. (1998). Assessment and classroom learning. Assessment in Education: Principles, Policy & Practice, 5(1), 7–74.

Blatchford, P., & Russell, A. (2020). Rethinking class size: The complex story of impact on teaching and learning. UCL Press.

Blatchford, P., Russell, A., & Webster, R. (2012). Reassessing the impact of teaching assistants: How research challenges practice and policy. Routledge.

Blewdin, M., & Baldwin, K. (2015). Visible learning at Brisbane Catholic education schools evaluation report 2015. Cognition.

Blömeke, S., Nilsen, T., & Scherer, R. (2021). School innovativeness is associated with enhanced teacher collaboration, innovative classroom practices, and job satisfaction. Journal of Educational Psychology, 113(8), 1645–1667.

Bloom, B. S. (1971). Handbook on formative and summative evaluation of student learning.

Bloom, B. S., Englehart, M. D., Furst, E. J., Hill, W. H., & Krathwohl, D. R. (1956). Taxonomy of educational objectives, handbook I: The cognitive domain. David McKay Co. McGraw-Hill.

Boaler, J., & Brodie, K. (2004, October). The importance, nature and impact of teacher questions. In Proceedings of the twenty-sixth annual meeting of the North American Chapter of the International Group for the Psychology of Mathematics Education (Vol. 2, pp. 774–782).

Boelens, R., De Wever, B., & Voet, M. (2017). Four key challenges to the design of blended learning: A systematic literature review. Educational Research Review, 22, 1–18.

Bolkan, S. (2017). Development and validation of the clarity indicators scale. Communication Education, 66(1), 19–36.

Bolton, S., & Hattie, J. A. C. (2018). Cognitive and brain development: Executive function, Piaget, and the prefrontal cortex. Archives of Psychology, 1(3), 1–16.

Boonk, L., Gijselaers, H. J., Ritzen, H., & Brand-Gruwel, S. (2018). A review of the relationship between parental involvement indicators and academic achievement. Educational Research Review, 24, 10–30.

Bowden, J., Marton, F., & Zull, J. (2014). The deep learning process and the construction of knowledge. In J. Hermida (Ed.), Facilitating deep learning: Pathways to success for university and college teachers (pp. 15–48). Apple Academic Press.

Braddock, J. H., II. (1990). Tracking: Implications for student race-ethnic subgroups (No. 1). Centre for Research on Effective Schooling for Disadvantaged Students, Johns Hopkins University.

Brandén, M., & Bygren, M. (2018). School choice and school segregation: Lessons from Sweden’s school voucher system. The IAS Working Paper Series, 1–40.

Braun, H. (2004). Reconsidering the impact of high-stakes testing. Education Policy Analysis Archives, 12, 1–45.

Brooks, C., Carroll, A., Gillies, R., & Hattie, J. A. C. (2019). A matrix of feedback for learning. The Australasian Journal of Teacher Education, 44(4), 14–21.

Broudy, H. S. (2017). Types of knowledge and purposes of education. In R. Anderson, R. J. Spiro, & W. E. Montague (Eds.), Schooling and the acquisition of knowledge (pp. 1–17). Routledge.

Brown, G. T. (2011). Self-regulation of assessment beliefs and attitudes: A review of the students’ conceptions of assessment inventory. Educational Psychology, 31(6), 731–748.

Brualdi, A. C. (1998). Classroom questions, ERIC/AE Digest No. EDO-TM-98-02.

Bull, R., Espy, K. A., & Wiebe, S. A. (2008). Short-term memory, working memory, and executive functioning in preschoolers: Longitudinal predictors of mathematical achievement at age 7 years. Developmental Neuropsychology, 33(3), 205–228.

Burgess, D. (2012). Teach like a pirate: Increase student engagement, boost your creativity, and transform your life as an educator. Dave Burgess Consulting.

Byers, T., Imms, W., & Hartnell-Young, E. (2018). Comparative analysis of the impact of traditional versus innovative learning environments on student attitudes and learning outcomes. Studies in Educational Evaluation, 58, 167–177.

Byers, T., Mahat, M., Liu, K., Knock, A., & Imms, W. (2018). Systematic review of the effects of learning environments on student learning outcomes. Innovative learning environments and teachers change. Melbourne: University of Melbourne, LEaRN. www.iletc.com.au/publications/reports

Campbell, D. T., & Stanley, J. C. (1966). Experimental and Quasi-experimental designs. Rand McNally.

Campbell, J., & Van Nieuwerburgh, C. (2018). The leader’s guide to coaching in schools: Creating conditions for effective learning. Corwin.

Carless, D., & Boud, D. (2018). The development of student feedback literacy: Enabling uptake of feedback. Assessment & Evaluation in Higher Education, 43(8), 1315–1325.

Caro, D. H., McDonald, J. T., & Willms, J. D. (2009). Socio-economic status and academic achievement trajectories from childhood to adolescence. Canadian Journal of Education/Revue canadienne de l’éducation, 32(3), 558–590.

Carroll, A., Houghton, S., Durkin, K., & Hattie, J. A. C. (2009). Adolescent reputations and risk: Developmental trajectories to delinquency. Springer.

Carvalho, R. G., & Abreu, C. C. (2018). Students’ characteristics and teachers’ estimates about their academic achievement. Paidéia (Ribeirão Preto), 28.

Carver, C. S., & Scheier, M. F. (2001). On the self-regulation of behavior. Cambridge University Press.

Castejón, A., & Zancajo, A. (2015). Educational differentiation policies and the performance of disadvantaged students across OECD countries. European Educational Research Journal, 14(3–4), 222–239.

Castles, A., Rastle, K., & Nation, K. (2018). Ending the reading wars: Reading acquisition from novice to expert. Psychological Science in the Public Interest, 19(1), 5–51.

Cawsey, A. M., Hattie, J. A. C., & Masters, G. (2019). Growth to achievement: On-demand resources for teachers. Commonwealth Department of Education Report. https://www.education.gov.au/quality-schools-package/resources/growth-achievement-demand-resources-teachers

Chapman, J. W., Greaney, K. T., Arrow, A. W., & Tunmer, W. E. (2018). Teachers’ use of phonics, knowledge of language constructs, and preferred word identification prompts in relation to beginning readers. Australian Journal of Learning Difficulties, 23(1), 87–104.

Charbonnier, E., & Truong, N. (2014). How much time do primary and lower secondary students spend in the classroom? Education Indicators in Focus, 22. Retrieved May 10, 2016. https://www.oecd.org/education/EAG2014-Indicator%20D1%20(eng).pdf

Chen, Q., Zhu, Y., & Chui, W. H. (2021). A meta-analysis on effects of parenting programs on bullying prevention. Trauma, Violence, & Abuse, 22(5), 1209–1220.

Chetty, R., Friedman, J. N., Hilger, N., Saez, E., Schanzenbach, D. W., & Yagan, D. (2011). How does your kindergarten classroom affect your earnings? Evidence from project STAR. The Quarterly Journal of Economics, 126(4), 1593–1660.

Christophel, D. M., & Gorham, J. (1995). A test-retest analysis of student motivation, teacher immediacy, and perceived sources of motivation and demotivation in college classes. Communication Education, 44(4), 292–306.

Clarke, D., Keitel, C., & Shimizu, Y. (2006). Mathematics classrooms in twelve countries: The insider’s perspective. BRILL.

Clarke, J., & Dede, C. (2009). Design for scalability: A case study of the River City curriculum. Journal of Science Education and Technology, 18(4), 353–365.

Clarke, S. (2021). Unlocking learning intentions and success criteria: Shifting from product to process across the disciplines. Corwin Press.

Claxton, G. (2013). What’s the point of school? Rediscovering the heart of education. Oneworld Publications.

Clay, M. M. (2000). Becoming literate: The construction of inner control.

Clinton, J. M., & Clarke, A. (2020). Visible learning +: A decade of impact: Prepared for Corwin. Heinemann. University of Melbourne. www.visiblelearning.com/resources

Clinton, J. M., & Dawson, G. (2018). Enfranchising the profession through evaluation: A story from Australia. Teachers and Teaching, 24(3), 312–327.

Clinton, J. C., & Hattie, J. A. C. (2013). New Zealand students’ perceptions of parental involvement: Relations with liking, efficacy and achievement. Asia Pacific Journal of Education, 33(3), 324–337.

Clinton, J. M., & Hattie, J. A. C. (2014). Teachers as evaluators – An empowerment evaluation approach. In D. Fetterman, S. Kaftarian, & A. Wandersman (Eds.), Empowerment evaluation: Knowledge and tools for self-assessment, evaluation capacity building, and accountability (pp. 86–111). Sage.

Clinton, J. M., & Hattie, J. A. C. (2021). Cognitive complexity of evaluator competencies. Evaluation and Program Planning, 89, 1–8.

Clinton, J. M., Hattie, J. A. C., & Dixon, R. (2005, June). When the language of schooling enters the home: Evaluation of the Flaxmere Project. Final report to Ministry of Education. https://thehub.swa.govt.nz/assets/documents/41339_Evaluation_Flaxmere_0.pdf

Clinton, J. M., Hattie, J. A. C., & Nawab, D. (2018). The good teacher: Our best teachers are inspired, influential and passionate. In M. Harring (Ed.), Handbook for school pedagogics (pp. 880–888). Waxmann.

Cloney, D., Cleveland, G., Hattie, J., & Tayler, C. (2016). Variations in the availability and quality of early childhood education and care by socioeconomic status of neighborhoods. Early Education and Development, 27(3), 384–401.

Cohen, D. K., & Mehta, J. D. (2017). Why reform sometimes succeeds: Understanding the conditions that produce reforms that last. American Educational Research Journal, 54(4), 644–690.

Cohn, C. M. G. (1984). Creativity training effectiveness: A research synthesis. Arizona State University.

Cohn, C. M. G. (1986). A research synthesis of creativity training effectiveness: Methodological issues [Paper presentation]. Annual meeting of the American Educational Research Association. San Francisco, CA.

Connolly, P., Keenan, C., & Urbanska, K. (2018). The trials of evidence-based practice in education: A systematic review of randomised controlled trials in education research 1980–2016. Educational Research, 60(3), 276–291.

Cooper, H. (1994). Homework research and policy: A review of the literature. Research/Practice, 2(2).

Corballis, M. C., Hattie, J., & Fletcher, R. (2008). Handedness and intellectual achievement: An even-handed look. Neuropsychologia, 46(1), 374–378.

Cosden, M., Zimmer, J., & Tuss, P. (1993). The impact of age, sex, and ethnicity on kindergarten entry and retention decisions. Educational Evaluation and Policy Analysis, 15(2), 209–222.

Cotton, K. (1989). Classroom questioning. School Improvement Research Series: Northwest Labs.

Cowan, N. (2012). Working memory capacity. Psychology Press.

Craven, R. G., Marsh, H. W., & Print, M. (2000). Gifted, streamed and mixed-ability programs for gifted students: Impact on self-concept, motivation, and achievement. Australian Journal of Education, 44(1), 51–75.

Cronbach, L. J. (1980). Validity on parole: How can we go straight? New Directions for Testing and Measurement, 5, 99–108.

Cuban, L. (2001). Oversold and underused: Computers in the classroom. Harvard University Press.

Cuban, L. (2003). Why is it so hard to get good schools? Teachers College Press.

Cuban, L. (2018). The flight of a butterfly or the path of a bullet? Using technology to transform teaching and learning. Harvard Education Press.

Da Costa, S., Páez, D., Sánchez, F., Garaigordobil, M., & Gondim, S. (2015). Personal factors of creativity: A second order meta-analysis. Revista de Psicología del Trabajo y de las Organizaciones, 31(3), 165–173.

Danielson, C. (2012). Observing classroom practice. Educational Leadership, 70(3), 32–37.

Darling-Hammond, L. (2000). Teacher quality and student achievement. Education Policy Analysis Archives, 8, 1.

Darling-Hammond, L., & Lieberman, A. (2013). Teacher education around the world. Routledge.

Das, J. P., Naglieri, J. A., & Kirby, J. R. (1994). Assessment of cognitive processes: The PASS theory of intelligence. Allyn & Bacon.

Datnow, A., & Park, V. (2018). Opening or closing doors for students? Equity and data use in schools. Journal of Educational Change, 19(2), 131–152.

Davies (2016). Investigating the use of talk in middle and secondary classrooms. Unpublished doctoral dissertation, University of Melbourne.

Deacon, S. H., & Sparks, E. (2015). Children’s spelling development: Theories and evidence. The Oxford Handbook of Reading, 311–325.

DeLuca, C., Chapman-Chin, A. E., LaPointe-McEwan, D., & Klinger, D. A. (2018). Student perspectives on assessment for learning. The Curriculum Journal, 29(1), 77–94.

Deming, D. J. (2014). Using school choice lotteries to test measures of school effectiveness. American Economic Review, 104(5), 406–411.

Deming, D. J. (2017). The growing importance of social skills in the labor market. The Quarterly Journal of Economics, 132(4), 1593–1640.

Deussen, T., Coskie, T., Robinson, L., & Autio, E. (2007). Coach can mean many things: Five categories of literacy coaches in Reading First. National Center for Education Evaluation and Regional Assistance, Institute of Education Sciences, US Department of Education.

Deutsch, J., Dumas, A., & Silber, J. (2013). Estimating an educational production function for five countries of Latin America on the basis of the PISA data. Economics of Education Review, 36, 245–262.

DeWitt, P. M. (2017). School climate: Leading with collective efficacy. Corwin Press.

Dinsmore, D. L., & Alexander, P. A. (2012). A critical discussion of deep and surface processing: What it means, how it is measured, the role of context, and model specification. Educational Psychology Review, 24(4), 499–567.

Dinsmore, D. L., & Alexander, P. A. (2016). A multidimensional investigation of deep-level and surface-level processing. The Journal of Experimental Education, 84(2), 213–244.

Donoghue, G. M., & Hattie, J. A. C. (2021). A meta-analysis of ten learning techniques. Frontiers of Education, 6, 581216. http://doi.org/10.3389/feduc.2021.581216

Donohoo, J., Hattie, J. A. C., & Eells, R. (2018). The power of collective efficacy. Educational Leadership, 75(6), 41–44.

Dörnyei, Z. (2001). New themes and approaches in second language motivation research. Annual Review of Applied Linguistics, 21, 43–59.

Dreyfus, H. L., & Dreyfus, S. E. (2005). Peripheral vision: Expertise in real world contexts. Organization Studies, 26(5), 779–792.

Duckworth, A. (2016). Grit: The power of passion and perseverance. Scribner.

Dunlosky, J., & Lipko, A. R. (2007). Metacomprehension: A brief history and how to improve its accuracy. Current Directions in Psychological Science, 16(4), 228–232.

Dupriez, V., Dumay, X., & Vause, A. (2008). How do school systems manage pupils’ heterogeneity? Comparative Education Review, 52(2), 245–273.

Duru-Bellat, M., & Mingat, A. (1997). La constitution de classes de niveau dans les collèges, les effets pervers d’un pratique à visée égalisatrice. Revue française de Sociologie, XXVI(4), 759–789.

Dweck, C. S. (2006). Mindset: The new psychology of success. Random House.

Dweck, C. S. (2008). Brainology: Transforming students’ motivation to learn. Independent School, 67(2), 110–119.

Dweck, C. S., & Yeager, D. S. (2019). Mindsets: A view from two eras. Perspectives on Psychological Science, 14(3), 481–496.

Eacott, S. (2017). School leadership and the cult of the guru: The neo-Taylorism of Hattie. School Leadership & Management, 37(4), 413–426.

Eccles, J. S., & Wigfield, A. (1997). Young adolescent development. In J. Irvin (Ed.), What current research says to the middle level practitioner (pp. 15–29). National Middle Schools Association.

Eccles, J. S., & Wigfield, A. (2020). From expectancy-value theory to situated expectancy-value theory: A developmental, social cognitive, and sociocultural perspective on motivation. Contemporary Educational Psychology, 61, 101859.

Edovald, T., & Nevill, C. (2021). Working out what works: The case of the Education Endowment Foundation in England. ECNU Review of Education, 4(1), 46–64.

Ekecrantz, S. (2015). Feedback and student learning? A critical review of research. Utbildning och Lärande/Education and Learning, 9(2), 15–34.

Elmore, R. (1996). Getting to scale with good educational practice. Harvard Educational Review, 66(1), 1–27.

Engel, S. (2011). Children’s need to know: Curiosity in schools. Harvard Educational Review, 81(4), 625–645.

Ennemoser, M., & Schneider, W. (2007). Relations of television viewing and reading: Findings from a 4-year longitudinal study. Journal of Educational Psychology, 99(2), 349.

Entwistle, N. J. (1976). The verb “to learn” takes the accusative. British Journal of Educational Psychology, 46, 1–3.

Epton, T., Currie, S., & Armitage, C. J. (2017). Unique effects of setting goals on behavior change: Systematic review and meta-analysis. Journal of Consulting and Clinical Psychology, 85(12), 1182.

Erdogan, I., & Campbell, T. (2008). Teacher questioning and interaction patterns in classrooms facilitated with differing levels of constructivist teaching practices. International Journal of Science Education, 30(14), 1891–1914.

Ericsson, K. A., & Kintsch, W. (1995). Long-term working memory. Psychological Review, 102(2), 211.

Esquith, R. (2007). Teach like your hair’s on fire: The methods and madness inside. Penguin.

Ferguson, C. J., Miguel, C. S., Kilburn Jr, J. C., & Sanchez, P. (2007). The effectiveness of school-based anti-bullying programs: A meta-analytic review. Criminal Justice Review, 32(4), 401–414.

Ferrer-Esteban, G. (2016). Trade-off between effectiveness and equity? An analysis of social sorting between classrooms and between schools. Comparative Education Review, 60(1), 151–183.

Feuerstein, R. (1980). Instrumental enrichment: An intervention program for cognitive modifiability. University Park Press.

Fiel, J. E., & Zhang, Y. (2019). With all deliberate speed: The reversal of court-ordered school desegregation, 1970–2013. American Journal of Sociology, 124(6), 1685–1719.

Finn, J. D., & Cox, D. (1992). Participation and withdrawal among fourth-grade pupils. American Educational Research Journal, 29(1), 141–162.

Fisher, A., & Scriven, M. (1997). Critical thinking its definition and assessment. Centre for Research in Critical Thinking.

Fisher, D., & Frey, N. (2021). Better learning through structured teaching: A framework for the gradual release of responsibility. ASCD.

Fisher, D., Frey, N., Amador, O., & Assof, J. (2018). The teacher clarity playbook: A hands-on guide to creating learning intentions and success criteria for organized, effective instruction. Corwin Press.

Fisher, E. P. (1992). The impact of play on development: A meta-analysis. Play & Culture.

Flanders, N. A. (1965). Teacher influence, pupil attitudes, and achievement: Ned A. Flanders (No. 12). US Department of Health, Education, and Welfare, Office of Education.

Fletcher, R., & Hattie, J. (2011). Intelligence and intelligence testing. Routledge.

Foer, J. (2012). Moonwalking with Einstein: The art and science of remembering everything. Penguin.

Fortuny, K., & Chaudry, A. (2011). A comprehensive review of immigrant access to health and human services. Urban Institute.

Fox, L. (2016). Essays on teacher preferences, teacher quality, and teacher expectations. Stanford University.

Fraguas, D., Díaz-Caneja, C. M., Ayora, M., Durán-Cutilla, M., Abregú-Crespo, R., Ezquiaga-Bravo, I., Martin-Babarro, J., & Arango, C. (2021). Assessment of school anti-bullying interventions: a meta-analysis of randomized clinical trials. JAMA pediatrics, 175(1), 44–55.

Fredricks, J. A., Filsecker, M., & Lawson, M. A. (2016). Student engagement, context, and adjustment: Addressing definitional, measurement, and methodological issues. Learning and Instruction, 43, 1–4.

Fredrickson, B. L. (2001). The role of positive emotions in positive psychology: The broaden-and-build theory of positive emotions. American Psychologist, 56, 218–226.

Freire, P. (2018). Pedagogy of the oppressed (50th Anniversary ed., M. Bergman Ramos, Trans.). Bloomsbury.

Frej, S., Janson, P., & Odalen, J. (2017). Evaluation of the education administration’s (utbildningsförvaltningens) visible learning education. Stockholm.

Frey, N., Hattie, J., & Fisher, D. (2018). Developing assessment-capable visible learners, grades K–12: Maximizing skill, will, and thrill. Corwin Press.

Friedkin, N. E., & Necochea, J. (1988). School system size and performance: A contingency perspective. Educational Evaluation and Policy Analysis, 10(3), 237–249.

Frost, R. O., & Marten, P. A. (1990). Perfectionism and evaluative threat. Cognitive Therapy and Research, 14(6), 559–572.

Frydenberg, E. (2010). Think positively!: A course for developing coping skills in adolescents. A&C Black.

Fullan, M. (2007). Leading in a culture of change. John Wiley & Sons.

Fullan, M. (2021). The right drivers for whole system success. Centre for Strategic Education.

Fullan, M., & Langworthy, M. (2014). A rich seam: How new pedagogies find deep learning. Pearson.

Gaffney, H., Ttofi, M. M., & Farrington, D. P. (2021). Effectiveness of school-based programs to reduce bullying perpetration and victimization: An updated systematic review and meta-analysis. Campbell Systematic Reviews, 17(2), e1143.

Galla, B. M., & Wood, J. J. (2012). Emotional self-efficacy moderates anxiety-related impairments in math performance in elementary school-age youth. Personality & Individual Differences, 52, 118–122.

Galton, M. J. (1995). Crisis in the primary classroom. D. Fulton Publishers.

Galton, M. J., & Willcocks, J. (Eds.). (1983). Moving from the primary classroom (Vol. 4). Routledge.

Gamoran, A. (2009). Tracking and inequality: New directions for research and practice. WCER working paper No. 2009–6. Wisconsin Center for Education Research (NJ1). Basic Books.

Gardner, H.W. (1993). Educating for understanding. The American School Board Journal, July, 20–24.

Garlin, F. V., & Owen, K. (2006). Setting the tone with the tune: A meta-analytic review of the effects of background music in retail settings. Journal of Business Research, 59(6), 755–764.

Gerber, J., & Wheeler, L. (2009). On being rejected: A meta-analysis of experimental research on rejection. Perspectives on Psychological Science, 4(5), 468–488.

Glass, G. V. (1976). Primary, secondary, and meta-analysis of research. Educational Researcher, 5(10), 3–8.

Glass, G. V. (2019, August). The promise of meta-analysis for our schools: A Q&A with NEPC. National Education Policy Center Newsletter, http://nepc.colorado.edu/publication/newsletter-glass

Glazerman, S., Mayer, D., & Decker, P. (2006). Alternative routes to teaching: The impacts of Teach for America on student achievement and other outcomes. Journal of Policy Analysis and Management: The Journal of the Association for Public Policy Analysis and Management, 25(1), 75–96.

Gleeson, J., Rickinson, M., Walsh, L., Salisbury, M., & Cirkony, C. (2022). Evidence-informed practice in Australian education. In The Emerald handbook of evidence-informed practice in education. Emerald Publishing Limited.

Goldberg, P., Sümer, Ö., Stürmer, K., Wagner, W., Göllner, R., Gerjets, P., Kasneci, E., & Trautwein, U. (2021). Attentive or not? Toward a machine learning approach to assessing students’ visible engagement in classroom instruction. Educational Psychology Review, 33(1), 27–49.

Goldring, E. B. (1990). Assessing the status of information on classroom organizational frameworks for gifted students. The Journal of Educational Research, 83(6), 313–327.

Goodwin, A. P., Petscher, Y., Jones, S., McFadden, S., Reynolds, D., & Lantos, T. (2020). The monster in the classroom: Assessing language to inform instruction. The Reading Teacher, 73(5), 603–616.

Gordon-Holliday, B. W. (1991). A quantitative analysis of the relationship between computer graphics and mathematics achievement and problem-solving [Unpublished doctoral dissertation]. University of Cincinnati.

Gore, J., Rickards, B., & Fray, L. (2022). From performative to professional accountability: Re-imagining “the field of judgment” through teacher professional development. Journal of Education Policy, 1–22.

Gottfried, M. A. (2013). The spillover effects of grade-retained classmates: Evidence from urban elementary schools. American Journal of Education, 119(3), 405–444.

Gough, P. B., & Tunmer, W. E. (1986). Decoding, reading, and reading disability. Remedial and Special Education, 7(1), 6–10.

Gould, S. J. (2013). The median isn’t the message. AMA Journal of Ethics, 15(1), 77–81.

Goos, M., Pipa, J., & Peixoto, F. (2021). Effectiveness of grade retention: A systematic review and meta-analysis. Educational Research Review, 34, 100401.

Graham, S. (2020). An attributional theory of motivation. Contemporary Educational Psychology, 61, 101861.

Graham, S., Harris, K. R., Mason, L., Fink-Chorzempa, B., Moran, S., & Saddler, B. (2018). How do primary grade teachers teach handwriting? A national survey. Reading and Writing, 21(1), 49–69.

Grasby, K. L., Little, C. W., Byrne, B., Coventry, W. L., Olson, R. K., Larsen, S., & Samuelsson, S. (2020). Estimating classroom-level influences on literacy and numeracy: A twin study. Journal of Educational Psychology, 112(6), 1154.

Gross, M. U. (2006). Exceptionally gifted children: Long-term outcomes of academic acceleration and nonacceleration. Journal for the Education of the Gifted, 29(4), 404–429.

Gundlach, H. A. D. (2022). Teacher turnover in schools and the profession: A meta-analytic review [Unpublished doctoral dissertation]. University of Melbourne. https://minerva-access.unimelb.edu.au/items/9092099e-41cb-4712-8085-3a5cbf918473

Gutiérrez-de-Rozas, B., López-Martín, E., & Carpintero Molina, E. (2022). Defining the profile of students with low academic achievement: A cross-country analysis through PISA 2018 data Frontiers in Education. OECD.

Hallinger, P., & Kovačević, J. (2019). A bibliometric review of research on educational administration: Science mapping the literature, 1960 to 2018. Review of Educational Research, 89(3), 335–369.

Hamilton, A., & Hattie, J. A. C. (2022). The lean education manifesto: Designing more cost-effective initiatives that generate stronger outcomes for developing nations. Corwin.

Hamilton, A., Reeves, D., Clinton, J. M., & Hattie, J. A. C. (2022). Building to impact: The 5D implementation playbook for educators. Corwin.

Hamilton, A., Wiliam, D., & Hattie, J. A. C. (2023). Room for Impact: The 9P De-Implementation Playbook for Educators. Corwin.

Hammerstein, S., König, C., Dreisörner, T., & Frey, A. (2021). Effects of COVID-19-related school closures on student achievement-a systematic review. Frontiers in Psychology, 4020.

Hampton, S. E., & Reiser, R. A. (2004). Effects of a theory-based feedback and consultation process on instruction and learning in college classrooms. Research in Higher Education.

Hansford, B., & Ehrich, L. C. (2006). The principalship: How significant is mentoring? Journal of Educational Administration, 44(1), 36–52.

Hanushek, E. A. (1989). The impact of differential expenditures on school performance. Educational Researcher, 18(4), 45–62.

Hanushek, E. A. (2013). Financing schools. In J. A. C. Hattie & E. Anderman (Eds.), International guide to student achievement (pp. 147–149). Routledge.

Hanushek, E. A., Peterson, P. E., Talpey, L. M., & Woessmann, L. (2019). The unwavering SES achievement gap: Trends in US student performance (No. w25648). National Bureau of Economic Research.

Hanushek, E. A., & Rivkin, S. G. (2006). Teacher quality. Handbook of the Economics of Education, 2, 1051–1078.

Hanushek, E. A., & Woessmann, L. (2006). Does educational tracking affect performance and inequality? Differences-in-differences evidence across countries. The Economic Journal, 116(510), C63–C76.

Hänze, M., & Berger, R. (2007). Cooperative learning, motivational effects, and student characteristics: An experimental study comparing cooperative learning and direct instruction in 12th grade physics classes. Learning and Instruction, 17(1), 29–41.

Hargreaves, A., & O’Connor, M. T. (2018). Collaborative professionalism: When teaching together means learning for all. Corwin Press.

Hargreaves, L., & Galton, M. (2002). Transfer from the primary classroom. Routledge.

Hart, B., & Risley, T. R. (2003). The early catastrophe: The 30 million word gap by age 3. American Educator, 27(1), 4–9.

Harwell, M., Maeda, Y., Bishop, K., & Xie, A. (2017). The surprisingly modest relationship between SES and educational achievement. The Journal of Experimental Education, 85(2), 197–214.

Hattie, J. A. C. (1977). Conditions for administering creativity tests. Psychological Bulletin, 84(6), 1249.

Hattie, J. A. C. (1992). Self-concept. Erlbaum.

Hattie, J. A. C. (2005). The paradox of reducing class size and improving learning outcomes. International Journal of Educational Research, 43(6), 387–425.

Hattie, J. A. C. (2008a). Visible learning: A synthesis of over 800 meta-analyses relating to achievement. Routledge.

Hattie, J. A. C. (2008b). The rope model of self. In H. W. Marsh, R. G. Craven, & D. M. McInerney (Eds.), Self-processes, learning, and enabling human potential: Dynamic new approaches (pp. 56–86). Information Age Publishing, Inc.

Hattie, J. A. C. (2009). Narrow the gap, fix the tail, or close the curves: The power of words. In C. Rubie & C. Rawlinson (Eds.), Challenging thinking about teaching and learning. Nova Science.

Hattie, J. A. C. (2015). What doesn’t work in education: The Politics of distraction. Open Ideas at Pearsons. www.pearson.com/hattie/distractions.html Politics of Distraction.

Hattie, J. A. C. (2017, June 28). Misinterpreting the growth mindset: Why we’re doings students a disservice. Finding Common Ground, Education Week. https://blogs.edweek.org/edweek/finding_common_ground/2017/06/misinterpreting_the_growth_mindset_why_were_doing_students_a_disservice.html

Hattie, J. A. C. (2018). Implementing, scaling up, and valuing expertise to develop worthwhile outcomes in schools. William Walker Oration, Presented at the Annual Conference of the Australian Council for Educational Leaders, Sydney. ACEL Monograph #58. www.acel.org.au/acel/ACELWEB/Publications/ACEL_Monograph.aspx

Hattie, J. A. C. (2021). An Ode to expertise. Corwin. https://protect-au.mimecast.com/s/43HsCXLKNwFX9YWqYT62nlB?domain=f.hubspotusercontent10.net

Hattie, J. A. C., Anderson, M., Clinton, J., & Rickards, F. (2015). Developing an evidence base model for the effects of teacher education programs on teacher candidates. In T. O. Seng & W. C. Liu (Eds.), Teacher effectiveness: Capacity building in a complex learning era. Cengage Learning Asia.

Hattie, J. A. C., & Brown, G. T. L. (2004). Cognitive processes in assessment items: SOLO taxonomy (Tech. Rep. No. 43). Auckland, NZ: University of Auckland, Project asTTle.

Hattie, J. A. C., & Clarke, S. (2019). Visible learning: Feedback. Routledge.

Hattie, J. A. C., Clarke, S., Fisher, D., & Frey, N. (2021a). Collective student efficacy. Corwin.

Hattie, J. A. C., & Clinton, J. C. (2008). Identifying accomplished teachers: A validation study. In L. Ingvarson & J. A. C. Hattie (Eds.), Assessing teachers for professional certification: The first decade of the National Board for Professional Teaching Standards (pp. 313–344). Advances in Program Evaluation Series #11. Oxford, Elsevier.

Hattie, J. A. C., Crivelli, J., Van Gompel, K., West-Smith, P., & Wike, K. (2021b). Feedback that leads to improvement in student essays: Testing the hypothesis that “where to next” feedback is most powerful. Frontiers in Education, 6:645758. doi: 10.3389/feduc.2021.645758

Hattie, J. A. C., & Donoghue, G. M. (2016). Learning strategies: A synthesis and conceptual model. Nature: Science of Learning, 1. http://doi.org/10.1038/npjscilearn.2016.13

Hattie, J. A. C., & Donoghue, G. M. (2018). A model of learning: Optimizing the effectiveness of learning strategies. In K. Illeris (Ed.), Contemporary theories of learning (2nd ed., pp. 97–113). Routledge (Also 15 Aktuelle Laringsteorier Forfattern og Samfundslitteratyr, 2019).

Hattie, J. A. C., Fisher, D., Frey, N., & Clarke, S. (2021c). Collective student efficacy: Developing independent and inter-dependent learners. Corwin Press.

Hattie, J. A. C., Gan, M., & Brooks, C. (2016). Instruction based on feedback. In R. Mayer & P. Alexander (Eds.), Handbook of research on learning and instruction (2nd ed., pp. 249–271). Routledge.

Hattie, J. A. C., & Hamilton, A. (2019). Real GOLD vs. Fool’s GOLD: The Visible Learning methodology for finding what works best in education. Corwin. www.visiblelearning.com/sites/default/files/Real%20Gold%20vs.%20Fools%20Gold_FINAL_app.pdf

Hattie, J. A. C., & Hattie, K. (2022). 10 steps to develop great learners: Visible learning for parents. Routledge.

Hattie, J. A. C., Hodis, F. A., & Kang, S. H. (2020). Theories of motivation: Integration and ways forward. Contemporary Educational Psychology, 61, 101865.

Hattie, J. A. C., & Jaeger, R. M. (1998). Assessment and learning. A deductive approach. Assessment in Higher Education, 5, 111–122.

Hattie, J. A. C., & Larsen, S. (2020). The purposes of education: In conversation. Routledge.

Hattie, J. A. C., & Peddie, R. (2003). School reports: “Praising with faint damns”. Set: Research Information for Teachers, 3, 4–9. www.nzcer.org.nz/nzcerpress/set/articles/school-reports-praising-faint-damns

Hattie, J. A. C., & Purdie, N. (1998). The SOLO model: Addressing fundamental measurement issues. In B. Dart & G. M. Boulton-Lewis (Eds.), Teaching and learning in higher education. Australian Council of Educational Research.

Hattie, J. A. C., & Rogers, H. J. (1986). Factor models for assessing the relation between creativity and intelligence. Journal of Educational Psychology, 78(6), 482.

Hattie, J. A. C., & Timperley, H. (2007). The power of feedback. Review of Educational Research, 77(1), 81–112.

Hattie, J. A. C., & Zierer, K. (2018). 10 mindframes for visible learning. Routledge.

Hattie, J.A.C., & Anderman, E. (2013). Handbook on Student Achievement. Schools edition. Routledge.

Hattie, J.A.C., & Anderman, E. (2022). Visible Learning: Guide to Student Achievement – Schools edition. Routledge.

Hattie, K. (2021). Building a language of learning: The harvest home learner. Connect, 248, 7–10.

Hearold, S. L. (1980). Television and social behaviour. Evaluation in Education, 4, 94–95.

Hedges, L. V., & Vevea, J. L. (1998). Fixed- and random-effects models in meta-analysis. Psychological Methods, 3(4), 486.

Henry, G. T., Bastian, K. C., & Fortner, C. K. (2011). Stayers and leavers: Early-career teacher effectiveness and attrition. Educational Researcher, 40(6), 271–280.

Hetland, L. (2000). Listening to music enhances spatial-temporal reasoning: Evidence for the “Mozart effect”. Journal of Aesthetic Education, 34(3/4), 105–148.

Hettleman, K. R. (2019). Mislabeled as disabled: The educational abuse of struggling learners and how we can fight it. Radius Book Group.

Hew, K. F., Bai, S., Huang, W., Dawson, P., Du, J., Huang, G., Jia, C., & Thankrit, K. (2021). On the use of flipped classroom across various disciplines: Insights from a second-order meta-analysis. Australasian Journal of Educational Technology, 37(2), 132–151.

Hiebert, J., Gallimore, R., Garnier, H., Givvin, K. B., Hollingsworth, H., & Jacobs, J. (2003). Teaching mathematics in seven countries: Results from the TIMSS 1999 video study (NCES 2003–013). US Department of Education. Washington, DC: National Center for Education Statistics.

Higgins, S., Xiao, Z., & Katsipataki, M. (2012). The impact of digital technology on learning: A summary for the education endowment foundation. Full Report. Education Endowment Foundation.

Hill, A. J., & Jones, D. B. (2021). Paying for Whose Performance? Teacher Incentive Pay and the Black–White Test Score Gap. Educational Evaluation and Policy Analysis, 43(3), 445–471.

Ho, P., & Kao, G. (2018). Educational achievement and attainment differences among minorities and immigrants. In B. Schneider (Ed.), Handbook of the sociology of education in the 21st century (pp. 109–129). Springer.

Hong, S., & Ho, H. Z. (2005). Direct and indirect longitudinal effects of parental involvement on student achievement: Second-order latent growth modeling across ethnic groups. Journal of Educational Psychology, 97(1), 32.

Hoover, W. A., & Tunmer, W. E. (2020). The cognitive foundations of reading and its acquisition. Springer International Publishing.

Hornby, G., & Greaves, D. (2022). Essential evidence-based teaching strategies: Ensuring optimal academic achievement for students. Springer.

Hosoda, M., Stone-Romero, E. F., & Coats, G. (2003). The effects of physical attractiveness on job-related outcomes: A meta-analysis of experimental studies. Personnel Psychology, 56(2), 431–462.

Houghton, S., Kyron, M., Hunter, S. C., Lawrence, D., Hattie, J., Carroll, A., & Zadow, C. (2022). Adolescents’ longitudinal trajectories of mental health and loneliness: The impact of COVID-19 school closures. Journal of Adolescence, 94(2), 191–205.

Horsley, M., & Walker, R. (2012). Reforming homework: Practices, learning and policy. CQUniversity.

Hoxby, C. (2004). Comment on ‘Why some schools have more underqualified teachers than others’ by R. Ingersol. In D. Ravitch (Ed.), Brookings papers on education policy (pp. 66–77). Brookings Institution Press. www.jstor.org/stable/20067266

Huang, Y., Espelage, D. L., Polanin, J. R., & Hong, J. S. (2019). A meta-analytic review of school-based anti-bullying programs with a parent component. International journal of bullying prevention, 1(1), 32–44.

Hunter, R. (2004). Madeline Hunter’s mastery teaching: Increasing instructional effectiveness in elementary and secondary schools. Corwin.

Ingersoll, R., Merrill, L., & Stuckey, D. (2014). Seven trends: The transformation of the teaching force (CPRE Report# RR-80). Consortium for Policy Research in Education.

Jackson, C. K., & Bruegmann, E. (2009). Teaching students and teaching each other: The importance of peer learning for teachers. American Economic Journal: Applied Economics, 1(4), 85–108.

Jansen, T., Meyer, J., Wigfield, A., & Möller, J. (2022). Which student and instructional variables are most strongly related to academic motivation in K–12 education? A systematic review of meta-analyses. Psychological Bulletin, 148(1–2), 1.

Jennings, J. L., Deming, D., Jencks, C., Lopuch, M., & Schueler, B. E. (2015). Do differences in school quality matter more than we thought? New evidence on educational opportunity in the twenty-first century. Sociology of Education, 88(1), 56–82.

Jenkins, L. (2015). Optimize your school: It’s all about the strategy. Corwin Press.

Jensen, B., Roberts-Hull, K., Magee, J., & Ginnivan, L. (2016). Not so elementary: Primary school teacher quality in top-performing systems. National Center on Education and the Economy (ED588643). ERIC.

Kahneman, D. (2011). Thinking, fast and slow. Penguin Books.

Kalogrides, D., & Loeb, S. (2013). Different teachers, different peers: The magnitude of student sorting within schools. Educational Researcher, 42(6), 304–316.

Kane, T. J., McCaffrey, D. F., Miller, T., & Staiger, D. O. (2013). Have we identified effective teachers? Validating measures of effective teaching using random assignment. Research paper. MET Project. Bill & Melinda Gates Foundation.

Kaplan, L. S., & Owings, W. A. (2002). Teacher quality, teaching quality, and school improvement. Phi Delta Kappa International. Phi Delta Kappa Fastbacks, 499, 3–44.

Kapur, M. (2016). Examining productive failure, productive success, unproductive failure, and unproductive success in learning. Educational Psychologist, 51(2), 289–299.

Kapur, M., Hattie, J., Grossman, I., & Sinha, T. (2022, September). Fail, flip, fix, and feed–Rethinking flipped learning: A review of meta-analyses and a subsequent meta-analysis. Frontiers in Education, 7, p. 956416.

Kapur, M., Hattie, J., Grossman, I., & Sinha, T. (2022, September). Fail, flip, fix, and feed–Rethinking flipped learning: A review of meta-analyses and a subsequent meta-analysis. In Frontiers in Education (Vol. 7, p. 956416). Frontiers Media.

Karbach, J., & Unger, K. (2014). Executive control training from middle childhood to adolescence. Frontiers in Psychology, 5, 390.

Karpicke, J. D., & Blunt, J. R. (2011). Retrieval practice produces more learning than elaborative studying with concept mapping. Science, 331(6018), 772–775.

Kazdin, A. E., Bass, D., Ayers, W. A., & Rodgers, A. (1990). Empirical and clinical focus of child and adolescent psychotherapy research. Journal of Consulting and Clinical Psychology, 58(6), 729.

Kelly, S., Olney, A. M., Donnelly, P., Nystrand, M., & D’Mello, S. K. (2018). Automatically measuring question authenticity in real-world classrooms. Educational Researcher, 47(7), 451–464.

Kennedy, M. (2016). Parsing the practice of teaching. Journal of Teacher Education, 67(1), 6–17.

Kent, S. D. (1992). The effects of acceleration on the social and emotional development of gifted elementary students: A meta-analysis [Unpublished doctoral dissertation]. University of Georgia.

Kim, S. (2022). Fifty years of parental involvement and achievement research: A second-order meta-analysis. Educational Research Review. 1–15. http://doi.org/10.1016/j.edurev.2022.100463

King, R. B., & Areepattamannil, S. (2014). What students feel in school influences the strategies they use for learning: Academic emotions and cognitive/meta-cognitive strategies. Journal of Pacific Rim Psychology, 8(1), 18–27.

Kirschner, P. A., Hendrick, C., & Heal, J. (2022). How teaching happens: Seminal works in teaching and teacher effectiveness and what they mean in practice. Routledge.

Knight, J. (2019). Instructional coaching for implementing visible learning: A model for translating research into practice. Education Sciences, 9(2), 101.

Knogler, M., Hetmanek, A., & Seidel, T. (2022). Bestimmung und Bereitstellung der„ best available “Evidenz für bestimmte Praxisfelder im Bildungsbereich. Optimierung schulischer Bildungsprozesse-What works?, 135.

Koestler, A. (1964). The act of creation. MacMillan.

König, C., & Frey, A. (2022). The impact of COVID-19-related school closures on student achievement – A meta-analysis. Educational Measurement: Issues and Practice, 41(1), 16–22.

Konstantopoulos, S. (2006). Trends of school effects on student achievement: Evidence from NLS: 72, HSB: 82, and NELS: 92. Teachers College Record, 108(12), 2550–2581.

Koo, M., Jung, S., Palmeira, M., & Kim, K. (2022). The timing of help: Receiving help toward the end (vs. beginning) undermines psychological ownership and subjective well-being. Journal of Personality and Social Psychology, 1–24.

Kornhall, P., & Bender, G. (2019). School segregation in Sweden: Evidence from the local level. NESET Ad hoc report. 1/2019.

Korous, K. M., Causadias, J. M., Bradley, R. H., Luthar, S. S., & Levy, R. (2020). A systematic overview of meta-analyses on socioeconomic status, cognitive ability, and achievement: The need to focus on specific pathways. Psychological Reports, 125(1), 55–97.

Kotter, J. P. (2012). Leading change. Harvard Business Press.

Kounin, J. S. (1970). Discipline and group management in classrooms. New Rinehart & Winston.

Kozol, J. (2005). The shame of the nation: The restoration of apartheid schooling in America (1st ed.). Crown Publishers.

Kraft, M. A. (2020). Interpreting effect sizes of education interventions. Educational Researcher, 49(4), 241–253.

Kuchirko, Y. (2019). On differences and deficits: A critique of the theoretical and methodological underpinnings of the word gap. Journal of Early Childhood Literacy, 19(4), 533–562.

Kulik, J. A., & Kulik, C. L. C. (1988). Timing of feedback and verbal learning. Review of Educational Research, 58(1), 79–97.

Kunz, J. (2018). Parental divorce and children’s interpersonal relationships: A meta-analysis. In Divorce and the next generation: Perspectives for young adults in the new millennium (pp. 19–47). Routledge.

Kvarven, A., Strømland, E., & Johannesson, M. (2020). Comparing meta-analyses and preregistered multiple-laboratory replication projects. Nature Human Behaviour, 4(4), 423–434.

Ladd, H. F. (2008). Teacher effects: What do we know. Teacher quality. In G. Duncan & J. Spillane (Eds.), Broadening and deepening the debate (pp. 3–26), Northwestern University.

Ladd, H. F., & Sorensen, L. C. (2017). Returns to teacher experience: Student achievement and motivation in middle school. Education Finance and Policy, 12(2), 241–279.

Lai, P., & Biggs, J. (1994). Who benefits from mastery learning? Contemporary Educational Psychology, 19(1), 13–23.

Lankshear, C., McLaren, P. L., & McLaren, P. (Eds.). (1993). Critical literacy: Politics, praxis, and the postmodern. SUNY Press.

Lazarus, R. S., & Folkman, S. (1984). Stress, appraisal, and coping. Springer.

Lee, H., Chung, H. Q., Abedi, J., & Rashedi, R. (2018). Systematic review of formative assessment: Efficacy and characteristics [Paper presentation]. American Educational Research Association.

Lee, J. (2006, April). Is test-driven external accountability effective? A meta-analysis of the evidence from cross-state causal-comparative and correlational studies. [Paper presentation] Annual meeting of American Educational Research Association. San Francisco, CA.

Lee, S. W. (2018). Pulling back the curtain: Revealing the cumulative importance of high-performing, highly qualified teachers on students’ educational outcome. Educational Evaluation and Policy Analysis, 40(3), 359–381.

Lee, S. W., Kim, C. J., & Kim, D. H. (2015). A meta-analysis of the effect of school-based anti-bullying programs. Journal of Child Health Care, 19(2), 136–153.

Lee, V. E., & Smith, J. B. (1997). High school size: Which works best and for whom? Educational Evaluation and Policy Analysis, 19(3), 205–227.

Lee, W., & Reeve, J. (2012). Teachers’ estimates of their students’ motivation and engagement: Being in synch with students. Educational Psychology, 32(6), 727–747.

Leeson, H. (2016). Catholic education Melbourne collaborative impact program evaluation report, 2015. Cognition.

Leeson, H. (2017). Northern territory collaborative impact program evaluation report, 2015. Cognition.

Lefstein, A., Louie, N., Segal, A., & Becher, A. (2020). Taking stock of research on teacher collaborative discourse: Theory and method in a nascent field. Teaching and Teacher Education, 88, 102954.

Leithwood, K., & Jantzi, D. (2009). A review of empirical evidence about school size effects: A policy perspective. Review of Educational Research, 79(1), 464–490.

Lemov, D. (2012). Teach like a champion field guide: A practical resource to make the 49 techniques your own. John Wiley & Sons.

Lester, S., Lawrence, C., & Ward, C. L. (2017). What do we know about preventing school violence? A systematic review of systematic reviews. Psychology, Health & Medicine, 22(sup1), 187–223.

Levin, H. M. (1988). Cost-effectiveness and educational policy. Educational Evaluation and Policy Analysis, 10(1), 51–69.

Levin, H. M., & Belfield, C. (2015). Guiding the development and use of cost-effectiveness analysis in education. Journal of Research on Educational Effectiveness, 8(3), 400–418.

Levine, A. (2006). Educating school teachers [Electronic Version]. Retrieved April 22, 2008, from www.edschools.org/pdf/Educating_Teachers_Report.pdf

Levy, S. R., Shin, J. E., Lytle, A., & Rosenthal, L. (2017). Systemic approaches to reduce prejudice in schools. In E. Lopez, S. Nahari, & S. L. Proctor (Eds.), Handbook of multicultural school psychology (pp. 237–257). Routledge.

Lietz, P. (2009). Variance in performance between students within schools and between schools. Australian Council for Educational Research.

Lillard, A. S., Lerner, M. D., Hopkins, E. J., Dore, R. A., Smith, E. D., & Palmquist, C. M. (2013). The impact of pretend play on children’s development: A review of the evidence. Psychological Bulletin, 139(1), 1.

Lipnevich, A. A., Berg, D. A., & Smith, J. K. (2016). Toward a model of student response to feedback. In G. Brown & L. R. Harris (Eds.), Handbook of human and social conditions in assessment (pp. 169–185). Routledge.

Little, J. W. (1990). The persistence of privacy: Autonomy and initiative in teachers’ professional relations. Teachers College Record, 91(4), 509–536.

Liu, J., & Cohen, J. (2021). Measuring teaching practices at scale: A novel application of text-as-data methods. Educational Evaluation and Policy Analysis, 43(4), 587–614.

Lomas, J. D., Koedinger, K., Patel, N., Shodhan, S., Poonwala, N., & Forlizzi, J. L. (2017, May). Is difficulty overrated? The effects of choice, novelty and suspense on intrinsic motivation in educational games. In Proceedings of the 2017 CHI conference on human factors in computing systems (pp. 1028–1039). https://dl.acm.org/doi/abs/10.1145/3025453.3025638

Lopez-Agudo, L. A., & Marcenaro-Gutierrez, O. D. (2022). Instruction time and students’ academic achievement: A cross-country comparison. Compare: A Journal of Comparative and International Education, 52(1), 75–91.

Lortie-Forgues, H., Sio, U. N., & Inglis, M. (2021). How should educational effects be communicated to teachers? Educational Researcher, 50(6), 345–354.

Loveless, T. (1999). The tracking wars: State reform meets school policy. Brookings Institution Press.

Lovell, O. (2020). Sweller’s cognitive load theory in action. John Catt Educational.

Lu, L., & Rickard, K. (2014). Value added models for NSW government schools. www.cese.nsw.gov.au//images/stories/PDF/VAPaper_v3-1Final.pdf

Luchini, C., Veronese, N., Nottegar, A., Shin, J. I., Gentile, G., Granziol, U., Soysal, P., Alexinshi, O., Smith, L., & Solmi, M. (2021). Assessing the quality of studies in meta-research: Review/guidelines on the most important quality assessment tools. Pharmaceutical Statistics, 20(1), 185–195.

Lugini, L., Olshefski, C., Singh, R., Litman, D., & Godley, A. (2020, June). Discussion tracker: Supporting teacher learning about students’ collaborative argumentation in high school classrooms. In Conference proceedings of the 28th international conference on computational linguistics. https://par.nsf.gov/servlets/purl/10297590

Luhmann, N. (2013). Theory of society (Vol. 2). Stanford University Press.

Luik, P. (2007). Characteristics of drills related to development of skills. Journal of Computer Assisted Learning, 23(1), 56–68.

Luyten, H., Hendriks, M., & Scheerens, J. (2014). School size effects revisited: A qualitative and quantitative review of the research evidence in primary and secondary education. Springer.

Macklin, P., & Zbar, V. (2020). Driving School Improvement: Practical Strategies and Tools. ACER Press.

Magnusson, S., Krajcik, J., & Borko, H. (1999). Nature, sources, and development of pedagogical content knowledge for science teaching. In J. Gess-Newsome & N. Lederman (Eds.), Examining pedagogical content knowledge (pp. 95–132). Springer.

Mahler, A. G. (2018). From the Tricontinental to the global South: Race, radicalism, and transnational solidarity. Duke University Press.

Manacorda, M. (2012). The cost of grade retention. Review of Economics and Statistics, 94(2), 596–606.

Mandouit, L. (2020). Investigating how students receive, interpret, and respond to teacher feedback [Unpublished doctoral dissertation]. University of Melbourne. https://minerva-access.unimelb.edu.au/items/a044d981-879d-50f6-8427-ba1266b81af7

Marks, G. N. (2015). The size, stability, and consistency of school effects: Evidence from Victoria. School Effectiveness and School Improvement, 26(3), 397–414.

Marks, G. N. (2021). Should value-added school effects models include student-and school-level covariates? Evidence from Australian population assessment data. British Educational Research Journal, 47(1), 181–204.

Marks, G. N., & O’Connell, M. (2021). No evidence for cumulating socioeconomic advantage. Ability explains increasing SES effects with age on children’s domain test scores. Intelligence, 88, 101582.

Marsden, J. (2021). Take risks. MacMillan.

Marsh, H. W., Dicke, T., & Pfeiffer, M. (2019). A tale of two quests: The (almost) non-overlapping research literatures on students’ evaluations of secondary-school and university teachers. Contemporary Educational Psychology, 58, 1–18.

Marsh, H. W., & Kleitman, S. (2005). Consequences of employment during high school: Character building, subversion of academic goals, or a threshold? American Educational Research Journal, 42(2), 331–369.

Marsh, H. W., Pekrun, R., Parker, P. D., Murayama, K., Guo, J., Dicke, T., & Lichtenfeld, S. (2017). Long-term positive effects of repeating a year in school: Six-year longitudinal study of self-beliefs, anxiety, social relations, school grades, and test scores. Journal of Educational Psychology, 109(3), 425.

Martin, A. J. (2006). Personal bests (PBs): A proposed multidimensional model and empirical analysis. British Journal of Educational Psychology, 76(4), 803–825.

Martin, F., Sun, T., Westine, C., & Ritzhaupt, A. (2022). Examining research on the impact of distance and online learning: A second-order meta-analysis study. Educational Research Review, 36, 1–37.

Marton, F. (2014). Necessary conditions of learning. Routledge.

Marton, F., & Pang, M. F. (2006). On some necessary conditions of learning. The Journal of the Learning Sciences, 15(2), 193–220.

Marzano, R. J. (2007). The art and science of teaching: A comprehensive framework for effective instruction. ASCD.

Marzano, R. J. (2016). The Marzano compendium of instructional strategies. Online: Marzano Research.

Marzano, R. J., Gaddy, B. B., & Dean, C. (2000). What Works in Classroom Instruction. ERIC documents. ED 468 434.

Mason, D. A., & Burns, R. B. (1996). “Simply no worse and simply no better” may simply be wrong: A critique of Veenman’s conclusion about multigrade classes. Review of Educational Research, 66(3), 307–322.

Mathur, M. B., & VanderWeele, T. J. (2021). Estimating publication bias in meta-analyses of peer-reviewed studies: A meta-meta-analysis across disciplines and journal tiers. Research Synthesis Methods, 12(2), 176–191.

Mavilidi, M. F., Marsh, H. W., Xu, K. M., Parker, P. D., Jansen, P. W., & Paas, F. (2021). Relative age effects on academic achievement in the first ten years of formal schooling: A nationally representative longitudinal prospective study. Journal of Educational Psychology, 114(2), 308–325

Mayer, R. E. (2019). Illustrations that instruct. In R. Glaser (Ed.), Advances in instructional psychology (pp. 253–284). Routledge.

McCrae, R. R., & Costa Jr, P. T. (1997). Personality trait structure as a human universal. American Psychologist, 52(5), 509.

McDonald, R. P. (2014). Factor analysis and related methods. Psychology Press.

McDonald, T. (2021). Classroom management: Engaging students in learning (3rd ed.). Oxford University Press.

McKnight, L., & Whitburn, B. (2020). Seven reasons to question the hegemony of Visible Learning. Discourse: Studies in the Cultural Politics of Education, 41(1), 32–44.

Meier, D. W. (1996, September). The big benefits of smallness. Educational Leadership, 54, 12–15.

Meinhardt, J., & Pekrun, R. (2003). Attentional resource allocation to emotional events: An ERP study. Cognition and Emotion, 17(3), 477–500.

Meisels, S. J., & Liaw, F. R. (1993). Failure in grade: Do retained students catch up? The Journal of Educational Research, 87(2), 69–77.

Mercer, N., Hennessy, S., & Warwick, P. (2019). Dialogue, thinking together and digital technology in the classroom: Some educational implications of a continuing line of inquiry. International Journal of Educational Research, 97, 187–199.

Merrill, M. D. (2002). First principles of instruction. Educational Technology Research and Development, 50(3), 43–59.

Meyer, B. J., Young, C. J., & Bartlett, B. J. (2014). Memory improved: Reading and memory enhancement across the life span through strategic text structures. Psychology Press.

Mihalca, L., & Mengelkamp, C. (2020). Effects of induced levels of prior knowledge on monitoring accuracy and performance when learning from self-regulated problem solving. Journal of Educational Psychology, 112(4).

Miller-Cotto, D., & Byrnes, J. P. (2016). Ethnic/racial identity and academic achievement: A meta-analytic review. Developmental Review, 41, 51–70.

Mitchell, D., & Sutherland, D. (2020). What really works in special and inclusive education. 3rd ed. Taylor and Francis.

Mitchell, D., & Sutherland, D. (2022). What really works in special and inclusive education using evidence-based teaching strategies. Routledge.

Miyake, A., & Friedman, N. P. (2012). The nature and organization of individual differences in executive functions: Four general conclusions. Current Directions in Psychological Science, 21(1), 8–14.

Modecki, K. L., Minchin, J., Harbaugh, A. G., Guerra, N. G., & Runions, K. C. (2014). Bullying prevalence across contexts: A meta-analysis measuring cyber and traditional bullying. Journal of Adolescent Health, 55(5), 602–611.

Moir, S., Hattie, J. A. C., & Jansen, C. (2014). Teacher perspectives of “effective” leadership in schools. Australian Education Leader, 36(4), 36–41.

Morell, M., Yang, J. S., Gladstone, J. R., Turci Faust, L., Ponnock, A. R., Lim, H. J., & Wigfield, A. (2020). Grit: The long and short of it. Journal of Educational Psychology, 113(5), 1038–1058.

Moreno, S. (2009). Can music influence language and cognition? Contemporary Music Review, 28(3), 329–345.

Muhlenbruck, L., Cooper, H., Nye, B., & Lindsay, J. J. (1999). Homework and achievement: Explaining the different strengths of relation at the elementary and secondary school levels. Social Psychology of Education, 3(4), 295–317.

Murano, D., Sawyer, J. E., & Lipnevich, A. A. (2020). A meta-analytic review of preschool social and emotional learning interventions. Review of Educational Research, 90(2), 227–263.

Murphy, D. (2020). Relationships between innovative learning environments, teacher mind frames and deep learning. Unpublished doctoral dissertation, University of Melbourne.

Narciss, S., & Huth, K. (2004). How to design informative tutoring feedback for multimedia learning. Instructional Design for Multimedia Learning, 181195.

Nasir, N. I. S., Lee, C. D., Pea, R., & McKinney de Royston, M. (2021). Rethinking learning: What the interdisciplinary science tells us. Educational Researcher, 50(8), 557–565.

National Equity Project. (2018). www.nationalequityproject.org/board

Neeleman, A. (2019). The scope of school autonomy in practice: An empirically based classification of school interventions. Journal of Educational Change, 20(1), 31–55.

Newton, D. P., & Newton, L. D. (2019, November). Humanoid robots as teachers and a proposed code of practice. In Frontiers in education (Vol. 4, p. 125). Frontiers Media SA.

Ngoc, P. N., Cheng, C. L., Lin, Y. K., Wu, M. S., Chu, J. S., & Tang, K. P. (2020). A meta-analysis of students’ readiness assurance test performance with team-based learning. BMC Medical Education, 20(1), 1–9.

Noetel, M., Griffith, S., Delaney, O., Harris, N. R., Sanders, T., Parker, P., Cruz, B., & Lonsdale, C. (2022). Multimedia design for learning: An overview of reviews with meta-meta-analysis. Review of Educational Research, 92(3), 413–454.

Nolan, J. M., Hanley, B. G., DiVietri, T. P., & Harvey, N. A. (2018). She who teaches learns: Performance benefits of a jigsaw activity in a college classroom. Scholarship of Teaching and Learning in Psychology, 4(2), 93.

Nuthall, G. (2007). The hidden lives of learners. New Zealand Council for Education Research Press.

Nye, B., Konstantopoulos, S., & Hedges, L. V. (2004). How large are teacher effects? Educational Evaluation and Policy Analysis, 26(3), 237–257.

Nystrand, M., Gamoran, A., & Carbonaro, W. (1998). Towards an ecology of learning: The case of classroom discourse and its effects on writing in High School English and social studies. Report Series, 2, 34.

Nystrand, M., Gamoran, A., & Carbonaro, W. (2001). On the ecology of classroom instruction. In Writing as a learning tool (pp. 57–81). Springer.

Oakes, J. (1992). Can tracking research inform practice? Technical, normative, and political considerations. Educational Researcher, 21(4), 12–21.

Oakes, J. (2005). Keeping track: How schools structure inequality (2nd ed.). Yale University Press.

Oakes, J., Gamoran, A., & Page, R. N. (1992). Curriculum differentiation: Opportunities, outcomes, and meanings. In P. W. Jackson (Ed.), Handbook of research on curriculum: A project of the American Educational Research Association (pp. 570–608). Macmillan.

Oakes, J., Ormseth, T., Bell, R., & Camp, P. (1990). Multiplying inequalities: The effects of race, social class, and tracking on opportunities to learn mathematics and science. RAND.

Oakes, J., & Wells, A. S. (1996). Beyond the technicalities of school reform: Policy lessons from detracking schools. UCLA Graduate School of Education & Information Studies.

O’Connell, M., & Marks, G. N. (2022). Cognitive ability and conscientiousness are more important than SES for educational attainment: An analysis of the UK Millennium Cohort Study. Personality and Individual Differences, 188, 111471.

O’Connor, P. (2020). Visible Learning and whole language: Revisiting the “garbage in, garbage out” problem. Australian Journal of Language and Literacy, 43(2), 141–151.

OECD (2016, January). How is learning time organised in primary and secondary education? Education Indicators in Focus. 1–4.

OECD. (2021). Back to the future of education: Four OECD scenarios for schooling. www.oecd.org/education/back-to-the-future-s-of-education-178ef527-en.htm

Ogbu, J. U. (2004). Collective identity and the burden of “acting White” in Black history, community, and education. The Urban Review, 36(1), 1–35.

O’Leary, T. M., Hattie, J. A., & Griffin, P. (2017). Actual interpretations and use of scores as aspects of validity. Educational Measurement: Issues and Practice, 36(2), 16–23.

Oliver, M., McConney, A., & Woods-McConney, A. (2021). The efficacy of inquiry-based instruction in science: A comparative analysis of six countries using PISA 2015. Research in Science Education, 51(2), 595–616.

Olivola, C. Y., & Todorov, A. (2010). Fooled by first impressions? Reexamining the diagnostic value of appearance-based inferences. Journal of Experimental Social Psychology, 46(2), 315–324.

Organisation for Economic Co-operation and Development, & PISA. (2010). What makes a school successful? Resources, policies and practices. OECD.

Orwin, R. G. (1983). A fail-safe N for effect size in meta-analysis. Journal of Educational Statistics, 8(2), 157–159.

Osborn, A. F. (1963). Applied imagination: Principles and procedures of creative problem-solving. Charles Scribner’s & Sons.

Ota, D. W. (2014). Safe passage: How mobility affects people and what international schools should do about it. Summertime.

Owen, A. M., Hampshire, A., Grahn, J. A., Stenton, R., Dajani, S., Burns, A. S., et al. (2010). Putting brain training to the test. Nature, 465, 775–778.

Page, R. N. (1991). Lower-track classrooms: A curricular and cultural perspective. Teachers College Press.

Papatheodorou, S. (2019). Umbrella reviews: What they are and why we need them. European Journal of Epidemiology, 34(6), 543–546.

Papay, J. P., & Kraft, M. A. (2015). Productivity returns to experience in the teacher labor market: Methodological challenges and new evidence on long-term career improvement. Journal of Public Economics, 130, 105–119.

Parker, P., Dicke, T., Guo, J., Basarkod, G., & Marsh, H. (2021). Ability stratification predicts the size of the big-fish-little-pond effect. Educational Researcher, 50(6), 334–344.

Pascarella, E. T., & Terenzini, P. T. (1991). How college affects students.

Patzer, G. L. (2012). The physical attractiveness phenomena. Springer Science & Business Media.

Pegg, J., & Tall, D. (2010). The fundamental cycle of concept construction underlying various theoretical frameworks. In Theories of mathematics education (pp. 173–192). Springer.

Pekrun, R., Muis, K. R., Frenzel, A. C., & Götz, T. (2017). Emotions at school. Routledge.

Perkins, D. (2014). Future wise: Educating our children for a changing world. John Wiley & Sons.

Peters, R. S. (1960). The concept of motivation. Routledge.

Petit, M. M., Laird, R. E., Marsden, E. L., & Ebby, C. B. (2015). A focus on fractions: Bringing research to the classroom. Routledge.

Petty, G. (2016). Teaching today: A practical guide. Oxford University Press-Children.

Phillips, D. C. (1995). The good, the bad, and the ugly: The many faces of constructivism. Educational Researcher, 24(7), 5–12.

Phillipson, S. (2013). Parental expectations: The influence of the significant other on school achievement. In S. Phillipson, K. Y. L. Ku, S. N. Phillipson (Eds.), Constructing educational achievement: A sociocultural perspective (pp. 87–104). Routledge.

Piaget, J. (1952). The origins of intelligence. International University Press.

Piaget, J., & Inhelder, B. (1969). Intellectual operations and their development. In P. Oleron, J. Piaget, B. Inhelder, & P. Greco (Eds.), Experimental psychology: Its source and method, VII intelligence (pp. 144–205). Routledge.

Picker, K. (2022). Examining the relationship between teacher reading content knowledge, pedagogy and children’s learning experiences [Unpublished doctoral dissertation]. University of Melbourne. https://minerva-access.unimelb.edu.au/items/27a88b87-b782-44f5-b900-6d057956ede1

Piketty, T. (2022). Brief history of equality. Belknap Press.

Pinquart, M., & Kauser, R. (2018). Do the associations of parenting styles with behavior problems and academic achievement vary by culture? Results from a meta-analysis. Cultural Diversity and Ethnic Minority Psychology, 24(1), 75.

Pintrich, P. R. (2000). Multiple goals, multiple pathways: The role of goal orientation in learning and achievement. Journal of Educational Psychology, 92, 544–555.

Podolsky, A., Kini, T., & Darling-Hammond, L. (2019). Does teaching experience increase teacher effectiveness? A review of US research. Journal of Professional Capital and Community, 4(4), 286–308.

Pollock, J. E., & Tolone, L. J. (2020). Improving student learning one teacher at a time. ASCD.

Polya, G. (1971). How to solve it. Princeton University Press.

Popper, K. (1992). The logic of scientific discovery. Routledge.

Porter, A., McMaken, J., Hwang, J., & Yang, R. (2011). Common core standards: The new US intended curriculum. Educational Researcher, 40(3), 103–116.

Postman, N., & Weingartner, C. (1971). Teaching as a subversive activity. Delta.

Powell, D. R., Diamond, K. E., Burchinal, M. R., & Koehler, M. J. (2010). Effects of an early literacy professional development intervention on head start teachers and children. Journal of Educational Psychology, 102(2), 299.

Pratt, S., & George, R. (2005). Transferring friendship: Girls’ and boys’ friendships in the transition from primary to secondary school. Children and Society, 19(1), 16–26.

Pressley, M. (1998). Reading instruction that works: The case for balanced teaching. Guilford Press.

Pressley, M., Borkwski, J. G., & Schneider, W. (1989). Good information processing: What it is and how education can promote it. International Journal of Educational Research, 13(8), 857–867.

Pressley, M., Roehrig, A. D., Raphael, L., Dolezal, S., Bohn, C., Mohan, L., & & Hogan, K. (2003). Teaching processes in elementary and secondary education. In W. M. Reynolds & G. E. Miller (Eds.), Handbook of psychology: Educational psychology (Vol. 7, pp. 153–175). John Wiley & Sons Inc.

Purkey, W. W., Novak, J. M., & Fretz, J. R. (2020). Developing inviting schools: A beneficial framework for teaching and leading. Teachers College Press.

Pychyl, T. A., Lee, J. M., Thibodeau, R., & Blunt, A. (2000). Five days of emotion: An experience sampling study of undergraduate student procrastination. Journal of Social Behaviour and Personality, 15, 239–254.

Quinn, J., McEachen, J., Fullan, M., Gardner, M., & Drummy, M. (2019). Dive into deep learning: Tools for engagement. Corwin Press.

Qvortrup, L. (2006). The new knowledge regions: From simple to complex innovation theory. In P. Cooke & A. Piccalu (Eds.), Regional development in the knowledge economy (pp. 246–271). Routledge.

Raphael, T. E., & McMahon, S. I. (1994). Book club: An alternative framework for reading instruction. The Reading Teacher, 48(2), 102–116. https://doi.org/10.1598/RT.48.2.1

Raymond, M. E., & Hanushek, E. A. (2003). High-stakes research: The campaign against accountability has brought forth a tide of negative anecdotes and deeply flawed research. Solid analysis reveals a brighter picture. Education Next, 3(3), 48–55.

Redding, C., & Grissom, J. A. (2021). Do students in gifted programs perform better? Linking gifted program participation to achievement and nonachievement outcomes. Educational Evaluation and Policy Analysis, 43(3), 520–544.

Reeve, J., Ryan, R. M., Cheon, S. H., Matos, L., & Kaplan, H. (2023). Supporting student motivation. Taylor & Francis.

Reeves, D. B. (2015). Finding your leadership focus: What matters most for student results. Teachers College Press.

Richardson, P. W., & Watt, H. M. (2016). Factors influencing teaching choice: Why do future teachers choose the career? In International handbook of teacher education (pp. 275–304). Springer.

Rickards, F., Hattie, J. A. C., & Reid, C. (2021). The turning point: Growing expertise, evaluative thinking, and the future of the teaching profession. Routledge.

Rindermann, H., & Laura Ackermann, A. (2021). Piagetian tasks and psychometric intelligence: Different or similar constructs? Psychological Reports, 124(6), 2795–2821.

Rios, J. A., Ling, G., Pugh, R., Becker, D., & Bacall, A. (2020). Identifying critical 21st-century skills for workplace success: A content analysis of job advertisements. Educational Researcher, 49(2), 80–89.

Ris, E. W. (2015). Grit: A short history of a useful concept. Journal of Educational Controversy, 10(1), 3.

Ritchhart, R., & Church, M. (2020). The power of making thinking visible: Practices to engage and empower all learners. John Wiley & Sons.

Robinson, B., & Schaible, R. M. (1995). Collaborative teaching: Reaping the benefits. College Teaching, 43(2), 57–59.

Robinson, V. (2011). Student-centered leadership. John Wiley & Sons.

Rogers, C., Lyon, H., & Tausch, R. (2013). On becoming an effective teacher: Person-centered teaching, psychology, philosophy, and dialogues with Carl R. Rogers and Harold Lyon. Routledge.

Rollett, W., Bijlsma, H., & Röhl, S. (2021). Student feedback on teaching in schools: Using student perceptions for the development of teaching and teachers.

Roque, R., Rusk, N., & Resnick, M. (2016). Supporting diverse and creative collaboration in the Scratch online community. In U. Cress, J. Moskaliuk, & H. Jeong (Eds.), Mass collaboration and education (pp. 241–256). Springer.

Rosenblatt, L. M. (1978). The reader, the text, the poem: The transactional theory of the literary work. Southern Illinois University Press.

Rosenshine, B. (2003). High-stakes testing: Another analysis [Electronic Version]. Education Policy Analysis Archives, 11. Retrieved April 29, 2008, from http://epaa.asu.edu/epaa/v11n24/

Rosenthal, R. (1979). The “file drawer problem” and tolerance for null results. Psychological Bulletin, 86, 638–641.

Roseth, C. J., Lee, Y. K., & Saltarelli, W. A. (2019). Reconsidering jigsaw social psychology: Longitudinal effects on social interdependence, sociocognitive conflict regulation, motivation, and achievement. Journal of Educational Psychology, 111(1), 149.

Rowan, B., Correnti, R., & Miller, R. J. (2002). What large-scale, survey research tells us about teacher effects on student achievement: Insights from the Prospects study of elementary schools. Teachers College Record, 104(8), 1525–1567.

Rubie-Davies, C. M. (2014). Becoming a high expectation teacher: Raising the bar. Routledge.

Rubie-Davies, C. M. (2017). Teacher expectations in education. Routledge.

Rubie-Davies, C. M., Hattie, J., & Hamilton, R. (2006). Expecting the best for students: Teacher expectations and academic outcomes. British Journal of Educational Psychology, 76(3), 429–444.

Rumberger, R. W., & Palardy, G. J. (2005). Does segregation still matter? The impact of student composition on academic achievement in high school. Teachers College Record, 107(9), 1999–2045.

Russell, D. (2018). Intention as action under development: Why intention is not a mental state. Canadian Journal of Philosophy, 48(5), 742–761.

Rutter, M. (1979). Fifteen thousand hours: Secondary schools and their effects on children. Harvard University Press.

Ryan, A. M., & Shin, H. (2011). Help-seeking tendencies during early adolescence: An examination of motivational correlates and consequences for achievement. Learning and Instruction, 21, 247–256.

Ryan, R. M., & Deci, E. L. (2017). Self-determination theory: Basic psychological needs in motivation, development, and wellness. Guilford Publications.

Ryan, R. M., & Deci, E. L. (2020). Intrinsic and extrinsic motivation from a self-determination theory perspective: Definitions, theory, practices, and future directions. Contemporary Educational Psychology, 61, 101860.

Ryle, G. (1945, January). Knowing how and knowing that: The presidential address. In Proceedings of the Aristotelian society (Vol. 46, pp. 1–16). Aristotelian Society, Wiley.

Ryle, G. (1949). The concept of mind. Hutchinson’s University Library.

Sackett, D. L., Rosenberg, W. M., Gray, J. M., Haynes, R. B., & Richardson, W. S. (1996). Evidence based medicine: What it is and what it isn’t. British Medical Journal, 312(7023), 71–72.

Sahlberg, P. (2021). Finnish lessons 3.0: What can the world learn from educational change in Finland? Teachers College Press.

Sanders, M., Chonaire, A. N., Tamma, F., Grott, B., Carr, D., Heal, J., & & Lawrence, J. (2021). Visible classroom: Re-grant. Educational Endowment Foundation. https://educationendowmentfoundation.org.uk/projects-and-evaluation/projects/the-visible-classroom-2015

Sanrey, C., Bressoux, P., Lima, L., & Pansu, P. (2021). A new method for studying the halo effect in teachers’ judgement and its antecedents: Bringing out the role of certainty. British Journal of Educational Psychology, 91(2), 658–675.

Santelises, S. B., & Dabrowski, J. (2015). Checking in: Do classroom assignments reflect today’s higher standards? K–12 Practice. Equity in Motion Series. Education Trust. https://eric.ed.gov/?id=ED566668

Scardamalia, M., & Bereiter, C. (2006). Knowledge building: Theory, pedagogy, and technology. In K. Sawyer (Ed.), Cambridge handbook of the learning sciences (pp. 97–118). Cambridge University Press.

Schneider, M., & Preckel, F. (2017). Variables associated with achievement in higher education: A systematic review of meta-analyses. Psychological Bulletin, 143(6), 565.

Schoenfeld, A. H. (2010). How we think: A theory of goal-oriented decision making and its educational applications. Routledge.

Schön, D. A. (2017). The reflective practitioner: How professionals think in action. Routledge.

Schunk, D. H., & DiBenedetto, M. K. (2020). Motivation and social cognitive theory. Contemporary Educational Psychology, 60, 101832.

Scriven, M. (1967). The methodology of evaluation. In R. Tyler, R. Gagne, & M. Scriven (Eds.), Perspectives on curriculum evaluation. AERA monograph series – Curriculum evaluation Chicago. Rand McNally.

Scriven, M. (1991). Chapter II: Beyond formative and summative evaluation. Teachers College Record, 92(6), 19–64.

Scriven, M. (1994). Duties of the teacher. Journal of Personnel Evaluation in Education, 8(2), 151–184.

Scriven, M. (2003). Evaluation in the new millennium: The transdisciplinary vision. Evaluating Social Programs and Problems: Visions for the New Millennium, 19–42.

Scriven, M. (2007). Key evaluation checklist. Evaluation Checklists Project. www.wmich.edu/evalctr/checklists

Smith, M. L., & Glass, G. V. (1980). Meta-analysis of research on class size and its relationship to attitudes and instruction. American Educational Research Journal, 17(4), 419–433.

Specjal, S. (2022). Understanding teacher and student talk for optimising teaching, thinking, and learning. [Unpublished doctoral dissertation]. University of Melbourne. https://minerva-access.unimelb.edu.au/items/5b2b9b11-92a2-407f-8102-e2705f9e1a16

Senko, C., Perry, A. H., & Greiser, M. (2021). Does triggering learners’ interest make them overconfident? Journal of Educational Psychology, 114(3), 482–497.

Shah, C., Richardson, P., & Watt, H. (2020). Teaching ‘out of field’in STEM subjects in Australia: Evidence from PISA 2015 (No. 511). GLO Discussion Paper. https://apo.org.au/node/303739

Shanahan, T. (2016). Relationships between reading and writing development. Handbook of Writing Research, 2, 194–207.

Shanahan, T. (2017). Why you need to be careful about visible learning. The Reading Teacher, 70(6), 749–752.

Shanker, A. (1993). Public vs. private schools. National Forum. Phi Kappa Phi Journal, 73(4), 14–17.

Sharratt, L. (2018). Clarity: What matters most in learning, teaching, and leading. Corwin Press.

Shaw, S. (2020). Mindfulness: Where it Comes from and what it Means. Shambhala Publications.

Shayer, M. (2003). Not just Piaget; not just Vygotsky, and certainly not Vygotsky as alternative to Piaget. Learning and Instruction, 13(5), 465–485.

Shayer, M., & Adey, P. (2002). Learning intelligence: Cognitive acceleration across the curriculum from 5 to 15 Years. Open University Press. Shayer & Adey, 1981.

Shepard, L. A., & Smith, M. L. (1990). Synthesis of research on grade retention. Educational Leadership, 47(8), 84–88.

Shomoossi, N. (2004). The effect of teachers questioning behavior on EFL Classroom interaction: A classroom research study. The Reading Matrix, 4(2), 96–102.

Shuell, T. J. (1990). Teaching and learning as problem solving. Theory into Practice, 29, 102–108.

Shulman, L. (1987). Knowledge and teaching: Foundations of the new reform. Harvard Educational Review, 57(1), 1–23.

Shute, V. J. (2008). Focus on formative feedback. Review of Educational Research, 78(1), 153–189.

Shute, V. J., Torreano, L. A., & Willis, R. E. (2020). DNA: Toward an automated knowledge elicitation and organization tool. In S. Laloie (Ed.), Computers as cognitive tools, volume two: No more walls (pp. 309–335). Routledge.

Sinclair, J., Jang, E. E., & Rudzicz, F. (2021). Using machine learning to predict children’s reading comprehension from linguistic features extracted from speech and writing. Journal of Educational Psychology, 113(6), 1088–1106.

Sinek, S. (2009). Start with why: How great leaders inspire everyone to take action. Penguin.

Slavin, R. E. (2018). John Hattie is wrong. https://robertslavinsblog.wordpress.com/2018/06/21/john-hattie-is-wrong/

Slavin, R. E. (2019). Educational psychology: Theory and practice.

Smith, D. (2017). John Amos Comenius: A visionary reformer of schools. Classical Academic Press.

Smith, K. B., & Meier, K. J. (2016). The case against school choice: Politics, markets and fools. Routledge.

Snook, I., O’Neill, J., Clark, J., O’Neill, A. M., & Openshaw, R. (2009). Invisible learnings?: A commentary on John Hattie’s book-‘Visible learning: A synthesis of over 800 meta-analyses relating to achievement’. New Zealand Journal of Educational Studies, 44(1), 93–106.

Spady, W. G. (1998). Paradigm lost: Reclaiming America’s educational future. AASA Distribution Center.

Spaniol, M., & Danielsson, H. (2022). A meta-analysis of the executive function components inhibition, shifting, and attention in intellectual disabilities. Journal of Intellectual Disability Research, 66(1–2), 9–31.

Specjal, S. (2022). Discourse by design: Understanding the way that talk (Teacher and Student) is used as a tool for optimising teaching, thinking, and learning together through collaborative talk [Unpublished doctoral dissertation]. University of Melbourne. https://minerva-access.unimelb.edu.au/items/5b2b9b11-92a2-407f-8102-e2705f9e1a16

Spielberger, C. D. (Ed.). (1972). Anxiety: Current trends in theory and research (Vol. 1). Academic Press.

Stanley, G. (2008). National numeracy review report. Council of Australia Governments. https://apo.org.au/node/4016

Stanovich, K. E. (2000). Progress in understanding reading: Scientific foundations and new frontiers. Guilford Press.

Stasser, G., & Abele, S. (2020). Collective choice, collaboration, and communication. Annual Review of Psychology, 71, 589–612.

Stern, J., Ferraro, K., Duncan, K., & Aleo, T. (2021). Learning that transfers: Designing curriculum for a changing world. Corwin Press.

Stockard, J., & Wood, T. W. (2017). The threshold and inclusive approaches to determining “best available evidence” an empirical analysis. American Journal of Evaluation, 38(4), 471–492.

Stokas, A. G. (2015). A genealogy of grit: Education in the new gilded age. Educational Theory, 65(5), 513–528.

Storey, N., & Zhang, Q. (2021, September 10). A meta-analysis of COVID learning loss. Johns Hopkins University https://doi.org/10.35542/osf.io/qekw2

Styles, B., & Torgerson, C. (2018). Randomised controlled trials (RCTs) in education research – methodological debates, questions, challenges. Educational Research, 60(3), 255–264.

Sugai, G., & Horner, R. H. (2009). Responsiveness-to-intervention and school-wide positive behavior supports: Integration of multi-tiered system approaches. Exceptionality, 17(4), 223–237.

Sullivan, M. C., Hawes, K., Winchester, S. B., & Miller, R. J. (2008). Developmental origins theory from prematurity to adult disease. Journal of Obstetric, Gynecologic & Neonatal Nursing, 37(2), 158–164.

Sun, J., & Leithwood, K. (2012). Transformational school leadership effects on student achievement. Leadership and Policy in Schools, 11(4), 418–451.

Swanson, H. L., & Alloway, T. P. (2012). Working memory, learning, and academic achievement. In APA educational psychology handbook, Vol 1: Theories, constructs, and critical issues (pp. 327–366). American Psychological Association.

Sweller, J. (2008). Human cognitive architecture. In J. M. Spector, M. D. Merrill, J. van Merrienboer, & M. P. Driscoll (Eds.), Handbook of research on educational communications and technology (pp. 369–381). Routledge.

Sweller, J. (2010). Element interactivity and intrinsic, extraneous, and germane cognitive load. Educational Psychology Review, 22(2), 123–138.

Sweller, J., Kirschner, P. A., & Clark, R. E. (2007). Why minimal guidance during instruction does not work: A reply to commentaries. Educational Psychologist, 47(1), 115–121.

Tamim, R. M., Bernard, R. M., Borokhovski, E., Abrami, P. C., & Schmid, R. F. (2011). What forty years of research says about the impact of technology on learning: A second-order meta-analysis and validation study. Review of Educational Research, 81(1), 4–28.

Tayler, C., Ishimine, K., Cloney, D., Cleveland, G., & Thorpe, K. (2013). The quality of early childhood education and care services in Australia. Australasian Journal of Early Childhood, 38(2), 13–21.

Teacher Education Ministerial Advisory Group. (2014). Australian Government.

Teig, N., Scherer, R., & Nilsen, T. (2018). More isn’t always better: The curvilinear relationship between inquiry-based teaching and student achievement in science. Learning and Instruction, 56, 20–29.

Thrupp, M., Lauder, H., & Robinson, T. (2002). School composition and peer effects. International Journal of Educational Research, 37(5), 483–504.

Thunder, K., Almarode, J., & Hattie, J. (2021). Visible learning in early childhood. Corwin Press.

Timmermans, A., & van der Werf, G. (2017). Accounting for previous performance of students by means of growth curves analyses to estimate the size, stability, and consistency of school effects. Educational Research and Evaluation, 23(5–6), 221–246.

Tomlinson, C. A. (2014). The differentiated classroom: Responding to the needs of all learners. ASCD.

Trachtenburg, P., & Ferruggia, A. (1989). Big books from little voices: Reaching high risk beginning readers. The Reading Teacher, 42(4), 284–289.

Tran, L., & Gershenson, S. (2021). Experimental estimates of the student attendance production function. Educational Evaluation and Policy Analysis, 43(2), 183–199.

Trautwein, U., Köller, O., Schmitz, B., & Baumert, J. (2002). Do homework assignments enhance achievement? A multilevel analysis in 7th-grade mathematics. Contemporary Educational Psychology, 27(1), 26–50.

Trumberg, A., & Urban, S. (2021). School choice and its long-term impact on social mobility in Sweden. Scandinavian Journal of Educational Research, 65(4), 569–583.

Tunmer, W. E., & Hoover, W. A. (2019). The cognitive foundations of learning to read: A framework for preventing and remediating reading difficulties. Australian Journal of Learning Difficulties, 24(1), 75–93.

Tunmer, W. E., & Nesdale, A. R. (1985). Phonemic segmentation skill and beginning reading. Journal of Educational Psychology, 77(4), 417–427.

Turnitin. (2014). Instructor feedback writ large: Student perceptions on effective feedback. Turnitin, LLC. http://go.turnitin.com/paper/student-perceptions-on-effective-feedback

Turnitin. (2015). From here to there: Students’ perceptions on feedback, goals, barriers, and effectiveness. Turnitin, LLC. http://go.turnitin.com/paper/student-feedback-goals-barriers

Tyack, D. B., & Cuban, L. (1995). Tinkering toward utopia: A century of public school reform. Harvard University Press.

Uhry, J. K., & Shepherd, M. J. (1993). Segmentation/spelling instruction as part of a first-grade reading program: Effects on several measures of reading. Reading Research Quarterly, 28(3), 219–233.

Urdan, T., & Kaplan, A. (2020). The origins, evolution, and future directions of achievement goal theory. Contemporary Educational Psychology, 61, 101862.

Urhahne, D., & Wijnia, L. (2021). A review on the accuracy of teacher judgments. Educational Research Review, 32, 100374.

Valdebenito, S., Eisner, M., Farrington, D. P., Ttofi, M. M., & Sutherland, A. (2018). School-based interventions for reducing disciplinary school exclusion: A systematic review. Campbell Systematic Reviews, 14(1), i–216.

Valentine, J. C., Hirschy, A. S., Bremer, C. D., Novillo, W., Castellano, M., & Banister, A. (2011). Keeping at-risk students in school a systematic review of college retention programs. Educational Evaluation and Policy Analysis, 33(2), 214–234.

Van Aert, R. C., Wicherts, J. M., & Van Assen, M. A. (2019). Publication bias examined in meta-analyses from psychology and medicine: A meta-meta-analysis. PLoS ONE, 14(4), e0215052.

van Driel, J. (2021). Teacher knowledge and the knowledge base of teaching. In Science teachers’ knowledge development. Brill. https://edarxiv.org/qekw2/

Van Hees, J. (2011). Oral expression of five and six year olds in low socio-economic schools [Unpublished doctoral dissertation]. University of Auckland.

VanLehn, K. (2011). The relative effectiveness of human tutoring, intelligent tutoring systems, and other tutoring systems. Educational Psychologist, 46(4), 197–221.

Vo, A. D. T. (2013). Toward a definition of evaluative thinking. University of California.

Vygotsky, L. S. (2012). Thought and language. MIT Press.

Walsh, K. (2006). Teacher education: Coming up empty. FWD, 3(1), 1–6. ERIC ED493850

Walters, G. D. (2021). School-age bullying victimization and perpetration: A meta-analysis of prospective studies and research. Trauma, Violence, & Abuse, 22(5), 1129–1139.

Wang, Z., Miller, K., & Cortina, K. (2013). Using the LENA in teacher training: Promoting student involvement through automated feedback. Unterrichtswissenschaft, 4, 290–305.

Wardman, J. (2009). Secondary teachers’, student teachers’ and education students’ attitudes to full year academic acceleration as a strategy for gifted students. Australasian Journal of Gifted Education, 18(1), 25–36.

Wardman, J., & Hattie, J. A. C. (2018). What works better than the rest? The impact of various curricula provisions for gifted learners. In B. Wallace, D. A. Sisk, & J. Senior (Eds.), The Sage handbook of gifted and talented education (pp. 321–334). Sage.

Watt, H. M., Carmichael, C., & Callingham, R. (2017). Students’ engagement profiles in mathematics according to learning environment dimensions: Developing an evidence base for best practice in mathematics education. School Psychology International, 38(2), 166–183.

Webb, N. L. (1997). Criteria for alignment of expectations and assessments in mathematics and science education. Research Monograph, 6.

Webber, A. (2021). Student learning during COVID-19: Literacy and maths in Years 4–10. Education Insights. www.educationcounts.govt.nz/publications/series/he-whakaaro

Webber, M., McKinley, E., & Hattie, J. (2013). The importance of race and ethnicity: An exploration of New Zealand Pākehā, Māori, Samoan and Chinese adolescent identity. New Zealand Journal of Psychology, 42(2), 17–28.

Webster, R., Russell, A., & Blatchford, P. (2015). Maximising the impact of teaching assistants: Guidance for school leaders and teachers. Routledge.

Weinstein, R. S. (2009). Reaching higher: The power of expectations in schooling. Harvard University Press.

Weisz, J. R., Weiss, B., Alicke, M. D., & Klotz, M. L. (1987). Effectiveness of psychotherapy with children and adolescents: A meta-analysis for clinicians. Journal of Consulting and Clinical Psychology, 55(4), 542.

West, M. R., Pier, L., Fricke, H., Hough, H., Loeb, S., Meyer, R. H., & Rice, A. B. (2020). Trends in student social-emotional learning: Evidence from the first large-scale panel student survey. Educational Evaluation and Policy Analysis, 42(2), 279–303.

Westfall, R., Millar, M., & Walsh, M. (2016). Effects of instructor attractiveness on learning. The Journal of General Psychology, 143(3), 161–171.

Westgate, E. C., & Steidle, B. (2020). Lost by definition: Why boredom matters for psychology and society. Social and Personality Psychology Compass, 14(11), e12562.

Westwood, P. (2018). Spelling: Approaches to teaching and assessment. Routledge.

White, J. (2019). Addressing school violence and bullying: Evidence review. NHS Health Scotland.

White, S., & Collin, C. (2021). Learning brief: Coordinating and managing information during the COVID-19 pandemic. Coordinating and managing information during the COVID-19 pandemic. Working Paper. London School of Hygiene and Tropical Medicine, London.

Wienk, M. (2017). Discipline profile of the mathematical sciences. https://amsi.org.au/wpcontent/uploads/2017/10/discipline-profile-2017-web.pdf

Wiggins, G. P., & McTighe, J. (2005). Understanding by design (Expanded 2nd ed.). Association for Supervision and Curriculum Development.

Wiliam, D. (2010). An integrative summary of the research literature and implications for a new theory of formative assessment. In H. Andrade & G. J. Cizek (Eds.), Handbook of Formative Assessment. (pp.18–40). Routledge.

Wiliam, D. (2014). Presentation to the Salzburg Global Seminar. https://www.dylanwiliam.org/Dylan_Wiliams_website/Papers_files/Salzburg%20Seminar%20talk.doc

Wiliam, D. (2016). Leadership for teacher learning: Creating a culture where all teachers improve so that all students succeed. Learning Sciences International.

Williams, P. (2008). Independent review of mathematics teaching in early years settings and primary schools. Department for Children, Schools and Families. https://dera.ioe.ac.uk/8365/

Wilson, R. C., Shenhav, A., Straccia, M., & Cohen, J. D. (2019). The eighty five percent rule for optimal learning. Nature Communications, 10(1), 1–9.

Wittgenstein, L. (1958). Philosophical Investigations (G. E. M. Anscombe, Trans., 2nd ed.). Blackwell.

Witter, M. (2021). Examining multiple dimensions of teacher quality: Attributes, beliefs, behaviours, and students’ perceptions of effectiveness [Unpublished doctoral dissertation]. University of Melbourne. https://minerva-access.unimelb.edu.au/items/baba09b2-c7df-5a05-97f3-ecac0ae238c2

Wrigley, T. (2015). Bullying by numbers. Primary First. https://core.ac.uk/display/286712718

Yates, L. (2017). Curriculum: The challenges and the devil in the details. In Educating Australia: Challenges for the decade ahead (pp. 85–99). Melbourne University Press.

Yates, L., Collins, C., & O’Connor, K. (Eds.). (2011). Australia’s curriculum dilemmas: State cultures and the big issues. Melbourne University Publishing.

Yeager, D. S., Fong, C. J., Lee, H. Y., & Espelage, D. L. (2015). Declines in efficacy of anti-bullying programs among older adolescents: Theory and a three-level meta-analysis. Journal of Applied Developmental Psychology, 37, 36–51.

Young, J. (2017). Technology-enhanced mathematics instruction: A second-order meta-analysis of 30 years of research. Educational Research Review, 22, 19–33.

Young, M. (2013). Overcoming the crisis in curriculum theory: A knowledge-based approach. Journal of Curriculum Studies, 45(2), 101–118.

Zhao, Q. (2022). Absolute standing feedback is more influential than relative standing feedback. Journal of Educational Psychology, 114(4), 701.

Zhao, Y. (2014). Who’s afraid of the big bad dragon? Why China has the best (and worst) education system in the world. John Wiley & Sons.

Zhao, Y. (2018). What works may hurt – Side effects in education. Teachers College Press.

Index

5D implementation model 10–11, 53, 59, 366

ability grouping for gifted students 183, 210

acceleration 73, 82, 115, 163, 210, 222, 224, 288–290, 296, 365, 438

accountability 21, 74, 155, 157–158, 191, 231, 248, 293, 383

acquiring deep learning 348

ADHD 115, 183, 198–199, 223

adjunct aids 367

adopted vs. non-adopted children 134

advance organizers 309–311, 373

after-school programs 419–420

aggression 88–89, 105, 120, 142, 199–200, 206, 294–295

alternative testing methods 335

anger 88–89, 99, 105, 122, 166, 201, 416

anxiety 23, 37, 62, 68, 77, 88–89, 94–95, 99, 103–104, 110–111, 122, 142, 200, 206, 294, 316, 342–343, 435

arts programs 249, 276

attendance 23, 92, 94, 154, 169–170, 196, 200, 207, 254, 299, 366, 386, 400, 420

audio/visual methods 394

background music 344, 359, 368

backward design 49, 61–62, 64, 308, 336, 432

balanced literacy approaches 265

belonging 47–48, 71–73, 92, 111, 148, 166–167, 176, 182–183, 205, 208–209, 211, 431, 433

bilingual programs 248, 251, 301

birth weight 36, 114

boredom 23, 60, 77, 88, 99, 106, 122, 316, 342–343, 347, 370, 435

breast feeding 114

building quality 154, 159

bullying 23–24, 106, 182–183, 195–196, 202, 205, 209, 211

calculators 250, 282–283, 285

career interventions 251

changing school calendars 163

charter schools 37, 154, 160

chess instruction 251, 298

class climate effects 183, 201

class Composition effects 183–184

within-class grouping 183, 190–191

classroom breaks 183, 195

classroom cohesion 182–183, 202

classroom discussion 329–330

classroom effects 182–212

classroom management 92, 182–183, 192, 194, 209–210, 212, 325, 434

class size 37, 156, 182–185, 209–210

clickers 375, 394, 400

climate 47–50, 70–73, 154–155, 173–176, 178–183, 193–194, 196, 201–203, 208–210

coaching 235–238, 252, 294–295

coding 20, 25, 27, 37, 83, 248, 250, 266, 269, 284, 375, 388–389, 420

cognitive acceleration 82, 296

cognitive behavioral programs 363

cognitive-behavioral programs 363

cognitive load 82, 282–283, 285, 313, 317, 346, 362

cognitive task analysis 33, 43, 58, 63, 82, 112, 307, 318–319, 347, 431

collaborative learning 11, 287, 349, 380–381, 383

collective student efficacy 65, 191, 389

collective teacher efficacy 71–75, 156, 172–175, 227–229, 412, 422, 432

college halls of residence 166

competitive vs. individualistic learning 384–385

comprehensive reading programs 249, 253

concentration 67, 88, 91, 122–123

concept mapping 312, 348, 380

conjoint reading programs 249, 255

conscientiousness 67, 91, 96–98, 107, 122, 218, 316

consolidating/merging schools 167

constructivist teaching 379–380

cooperative learning 191, 236, 254, 256, 269, 280–281, 284, 299, 350, 381–384

cooperative vs. competitive learning 384–385

coping strategies 67–69, 100, 120, 122, 176, 299, 343

corporal punishment 127, 138, 140

co-teaching 364–365

co-/team teaching 364

counseling effects 183, 200

COVID 1, 66, 87, 125–126, 129, 142, 164, 305, 332, 399, 408–410, 419, 423–425, 428, 433, 436, 438

creativity 20, 24, 27, 78, 82, 86–87, 102, 153, 171, 186, 193, 225, 247–248, 251, 288, 296–297, 300, 364, 369, 417

creativity programs 24, 193, 251, 296–297

critical thinking 55–56, 78, 87, 152–153, 186, 230, 242, 288, 297, 331–332, 349, 354, 359, 369, 377, 401–402, 431

criticism of visible learning 29–41

criticisms of meta-analysis 27–29

cross-laterality 78, 84

curiosity 67–68, 70, 88, 99, 102, 170, 180, 216, 263, 352–353, 372, 428

curriculum 21–22, 38–39, 44, 46–47, 50, 53, 63, 65, 70, 75, 111, 123, 158, 163, 166, 174, 185, 189, 193, 210–211, 216, 222, 224, 228, 232, 234, 238, 242, 247–307, 318–319, 336–337, 357, 366, 376, 414, 421–423, 438

decade of impact 8–9, 412

decreasing disruptive behavior 183, 196, 212

deliberate practice 33, 65, 70, 123, 163, 347–348, 353, 379, 401, 404, 406, 417, 422, 425

depression 23, 88, 101–102, 104–105, 107, 122, 142, 200, 342

desegregation 155, 164–165

detracking 183, 188–189, 192

dialect use 118

differentiation 61, 175, 220, 222, 240–241, 330

direct instruction 170, 257, 266, 279, 281, 284, 286, 296, 330, 357, 359–361, 381, 389, 391, 413–414, 417, 426, 431

discovery-based teaching 379

distance learning 398–399, 409–410, 419, 423, 425, 429

diversity courses 295

diversity of student body 165

diversity training 248

divorce 133–134, 141, 145–146

drama programs 276–277

early intervention 136, 155, 169, 171, 193

early years non-cognitive skills 84

elaboration and organization 348

elaborative interrogation 348–349

emotional intelligence 37, 88, 100, 342

emotions 23, 59, 87–88, 92, 95–96, 98–100, 109, 111, 296, 343, 352–353, 408, 437

engaged vs. disengaged fathers 140

engagement 12–13, 91–94, 99, 102–103, 145–146, 193–194, 203–205, 332–333, 412–413, 423–424, 435

enjoyment 65, 88, 99, 101, 210, 252–253, 314, 436

enrichment programs 250, 288

environmental structuring 345

errors 68–69, 73, 75, 120–121, 151–152, 186, 191, 202, 221, 243, 275, 316, 318, 320, 324, 329–330, 358, 371, 381, 390, 397, 400, 405, 408, 413, 424, 430, 432

ethnicity 113, 118–119, 129–130, 139, 142, 205, 214, 283, 292

evaluation and reflection 350

evaluative thinking 6, 8, 10, 17–18, 43, 55–56, 58, 63–65, 71, 73–74, 230, 241–242, 244, 296, 388–390, 427, 431, 439

executive functioning 67, 78, 81–82, 114, 117, 122, 300

exercise 37, 117, 195, 344–345, 361, 435

expectations, 7, 46, 48, 53, 57, 59–60, 66–74, 76, 78, 80, 85–86, 95, 98–99, 101, 109–110, 119–120, 125–129, 132, 136–137, 139–141, 143–146, 160, 165, 172–176, 178, 180, 188–189, 192, 195, 197, 203, 209–212, 216–217, 219–220, 222–223 225, 227–228, 235, 240–241, 244, 289, 293, 297, 326, 332, 341, 355, 386, 403, 416, 419, 423, 430; leader expectations 172–174; parental expectations 126–129, 136–137, 139–140, 144–145; student expectations 66–67, 73, 99, 120–121, 223–225; teacher expectations 7, 173, 219–222, 241, 244, 423; teacher expectations - physical attractiveness 222

expelling students 183, 198

explicit teaching strategies 362, 431

exposure to reading 249, 264, 275

extracurricular activities 164, 420–421

families 43–48, 53–54, 69–70, 73–74, 129–134, 136–139

family resources 5, 21, 126–127

family structure 126, 130, 132, 145–146

family on welfare/state aid 130

fathers 127, 133, 140–141, 145

feedback 62–63, 110–112, 272, 307–309, 318–338, 347–348, 390, 406, 433

feedback (from student re quality) 324

feedback (from tests) 335

feedback (self) 320

feedback (technology) 320, 326

field independence 78, 85

finances 155

financial programs 250, 291

flipped classrooms 370, 374

foreign languages 250

formative evaluation 333–334, 413

frequency of testing 334

friendship 70, 132, 142, 146, 179, 182–183, 187, 204–205, 209, 211, 290

frustration 88–89, 92, 99, 105, 122, 361, 416

gaming and simulations 401

gender differences 119

goal commitment 310, 316

grant aid 126, 131

grit 88, 91, 94–96, 152

growth vs. fixed mindsets 89

happiness 87–89, 101–103, 152

head start programs 170

help seeking 343, 346, 349–350

home and family 125–147

home environment 30, 126–127, 135

home-school programs 418

home visiting 144

homework 70, 137–138, 145, 164, 175, 326, 361, 374, 386, 405, 421–425

hope 67, 88, 99, 101–102, 292, 329, 357, 431

humor 359, 369

illness 23, 113, 115, 122, 224

immediacy 367–368

immigrant status 131

implementations using technologies 392–410

inclusion 29, 46, 71, 183, 192

individual instruction 280

inductive teaching 370, 378

initial teacher education programs 216, 229

inquiry-based teaching 337, 377

integrated curricula 291

intelligent tutoring systems 405–406, 408–409

intentional alignment 8, 43–44, 46, 48–50, 52–54, 59–76, 112, 167, 223, 227, 307–308, 319, 336, 353, 358, 376, 388–389, 393, 397, 399, 423, 431

interleaved practice 347–348

interventions for students with learning needs 416

jigsaw method 63, 371, 387–388, 390, 426

juvenile delinquent programs 299

knowing how 43, 65–66, 82, 124, 152–153, 172, 174, 341, 353, 358, 387, 430; deep learning 48, 123, 241, 243–244, 340, 345, 348, 355, 357, 369–370, 376–377

knowing that 44, 52, 72, 92, 153, 241, 243, 312, 341, 358, 387; surface learning 62, 65, 240, 345–347, 352–353, 356

knowing with 341, 358, 387; transfer 8, 12, 21, 44–45, 49–53, 57–58, 61–63, 65–66, 74, 83–84, 124, 142, 168–169, 260, 262, 277, 284, 298–299, 303–304, 307–308, 314, 319, 321, 336–338, 340–341, 343, 345, 350–354, 356, 358, 378, 387–388, 390, 400, 405, 428

labeling students 60, 217, 223, 240–241

language of learning 13, 18, 47, 53, 70–72, 125, 129, 144–146, 186, 349, 413, 424

leadership 12, 17, 24, 137, 154, 158, 170, 172–175, 181, 213–215, 234–235, 299, 398, 413, 436

learning hierarchies 309–310, 312

learning intentions 7, 12–14, 50, 57, 59–60, 63–65, 212, 299, 301, 307–313, 318–319, 336–337, 353, 357, 360–361, 391, 413, 427, 434

learning strategies 6–9, 19, 37–38, 62–65, 68, 85–89, 307–309, 318–319, 336–338, 340–356, 362–363

lectures 227, 326, 359–360, 363–364, 374–375, 390, 396, 402

lesson design 62–63, 66, 87, 336–337, 432

listening 54, 59, 64, 68, 70, 73–74, 104, 111, 121, 137–139, 146, 150, 169, 191, 201, 203, 209–210, 253, 262, 264, 276–277, 302, 318, 326, 349, 367–368, 424, 433, 437

mainstreaming 33, 183, 192, 210

maltreated children 116

manipulative materials on math 250, 282

mastery-approach 109

mastery goals 109, 221, 322, 343

mastery learning 28, 310, 316–317, 343, 359

mathematics curricula & programs 277–279

mathematics direct or guided 250

mathematics instructional processes 279, 306

Matthew effects 67, 121, 145, 251–252, 269, 279, 303, 305

mentoring 135, 214, 229, 297, 371, 386

meta-cognitive strategies 92, 296

Metax™ 18, 41, 430

microteaching 216, 237, 241–242, 245

middle school interventions 168

mind frames 4, 7, 11, 16–18, 43–44, 46–48, 54–55, 57, 120, 123–124, 147, 180–181, 186, 234, 328, 412, 427, 430–431

mindfulness 88, 90, 94, 108, 194, 342

mnemonics 255, 286, 346

mobile phones 396

mobility 23, 132, 141–142

models of implementation 4, 75, 423, 432

moral judgment programs 248, 251, 292

morning vs. evening 88, 106

mother employment 126, 130

motivation 92, 107–113, 122, 314, 343–344; see also thrill

motivation programs 248, 250, 292

multi-grade/age classes 189

multi-tiered support systems 366–367

music-based reading programs 249, 255, 368

music programs 249, 276–277

note-taking 254–255

one-on-one laptops 393

one-on-one tutoring 254

online and digital tools 394

open vs. traditional classrooms 183, 185, 209

out of class communication 385–386

outcomes-based education 293

outdoor/adventure programs 298

outdoor education 248, 298

out-of-school programs 418

parental autonomy 127, 138, 147

parental involvement 20, 127–128, 136–137, 147, 160, 175, 419

parent programs 143–144

passion 7–8, 45, 47–48, 54–55, 57, 65, 91, 216, 226, 427

peer assessment 320

peer- and self-grading 332

peer tutoring 193, 256, 269, 281, 286, 349, 358, 386

perfectionism 88, 98, 107

performance goals 108–109, 221, 314, 323, 344

performance pay 159

persistence 91, 96, 106, 108, 120–122, 131, 221, 263, 295, 314, 342

personality 36, 88, 96–98, 192, 216, 218, 242, 292, 295, 299, 324, 386

philosophy in schools 359, 385

phonics instruction 249, 257–258, 270

phonological awareness 170, 249, 257–259, 261, 264, 268, 270, 275, 365–367, 404

physical environment 170–171, 186

physical syndromes 113, 115

Piagetian levels 78, 80

planning and prediction 309, 311

play programs 248, 251, 300

positive ethnicity self-identity 118

PowerPoint 394

practice testing 347–348

pre-school programs 169–172

prior ability 24, 67, 78, 92, 341, 355

prior achievement 58–60, 65, 67, 78–80, 122, 128, 140, 146, 222, 341, 353, 381

problem-based learning 7, 30, 62, 280, 358, 362, 371–372, 375, 380, 391

problem-solving teaching 350, 373–374

procrastination 23, 89, 107, 109, 122

professional development 156, 214, 216, 233–235, 254, 278, 325, 367, 409, 413

progress to achievement 54, 71, 74–75, 93, 173, 209–210, 306, 355

purposes of schooling 44, 153, 180, 430

questioning 41, 49, 59, 66, 68–69, 93–94, 240, 255, 287, 296, 327–329, 331, 349–350, 353, 359, 373, 375, 378, 385, 391, 417

reading, 12–13, 15, 21, 31, 39, 70, 82–84, 92, 112–114, 116, 118, 128–129, 137, 141–144, 147, 153, 156–158 160–166, 170–171, 180, 187, 192–193, 198–199, 206–207, 212, 216, 224, 231–232, 236, 239–240, 245, 248–249, 252–272, 274–277, 280, 286, 288, 298, 301–302, 304–306, 320, 331, 335, 354, 361, 365, 368, 378, 385, 391, 393, 395, 404, 409, 413, 415–421, 425

reading comprehension programs 249, 262

reading programs for specific groups 254

reading recovery 267–268, 305

reading strategies 249, 254, 256

reciprocal teaching 19, 41, 254, 256, 269, 350, 359, 378–379, 389, 391

record keeping 346

reducing the achievement gap 415, 423

rehearsal and memorization 33, 347

relative age in the class 206

relaxation 88–89, 99, 103, 200, 236

religious schools 155, 160

remarriage 127, 134, 142

remove children into care homes 135

repeated reading 249, 261–262, 270

repeated reading programs 249, 261

rereading 261

response to intervention 88, 201, 252, 365

retention 23, 182, 207–208, 210, 226–227, 291, 302, 352, 400–403, 405, 438

robotics 435

robustness index 23–24, 41

scaffolding and situated learning 380

school calendars/timetables 163

school choice 154, 167–168

school climate effects 175

school compositional effects 164

school effects 23, 160, 176–177, 424

school resourcing 154

school size 155, 164, 166–167, 210

science curricula 285

science programs 278, 291

screen time 23, 143, 394, 396

Scriven, Michael 5, 87, 333, 426, 431, 439

seeking help 98, 152, 349–350, 429

self- and peer-grading 332

self-concept 46, 88, 94–95, 101, 104, 111, 133, 187, 189–190, 193, 207–208, 283, 294, 299, 342, 416, 418, 420, 433

self consequences 350

self-control 88, 95–96, 100, 107, 196–197, 293, 342

self-determination 70, 138, 146, 292

self-directed learning 372

self-efficacy 94, 342–343, 417

self-evaluation 254, 313, 353

self-explanation 349–350

self-monitoring 313, 349–350, 405

self-questioning 66, 349–350, 353

self-regulation 64, 84–87, 95, 98, 112, 196, 199, 273, 299, 321, 324, 348, 352–353, 381, 399, 428, 432

self-reported grades 78, 85, 101

self-talk 66, 353

self-verbalization 349–350

sentence-combining programs 262

service learning 411, 418, 423–424

simulations 22, 379, 394, 401–404

single sex-schools 161

skills 7, 9–10, 19, 42, 51, 60, 70–71, 74, 77–78, 122, 125, 138, 145–146, 152, 166, 170, 173, 203, 208–209, 247, 252–254, 273, 280, 298, 303–305, 308–309, 312, 341, 343, 346, 352–353, 359, 378, 381, 385, 399, 404, 432–433, 437

sleep 107, 111, 113, 116–117, 344–345, 411

small group learning 191

social media 21, 66, 194, 395–398, 408–410, 435–436

social skills programs 198, 200, 251, 293–295

socioeconomic status 37, 66, 127–128, 135, 137, 142, 146, 167, 177, 188, 214, 266, 283, 292

sound or inference 249, 257, 259

spaced versus mass practice 163, 347

spatial skills/training 251

special college programs 248, 302

spelling programs 274

stereotype threat 88, 119, 132

strategy monitoring 348–349

Structure of Observed Learning Outcomes (SOLO) 51–53, 345, 425

student assessment capabilities 432

student control over learning 95, 344

student personality 88, 96

student rating of the quality of teaching 216

students becoming their own teachers 7–8, 43–45, 66–67, 73, 152, 178, 380, 430

students feeling disliked 23, 183, 206

success criteria, 7, 12–13 33, 49–50, 57, 59–64, 66, 68, 76, 94, 96, 104, 106, 112, 122, 174, 191, 202, 210, 221–222, 227, 234, 241, 272, 280, 299, 301, 305, 307–316, 318–319, 322–323, 332, 336–338, 343–344, 352–353, 358, 360–361, 363, 381, 388, 390, 397, 406, 427, 430, 432, 434

sugar 113, 117

summer school 162–163

summer vacation 162, 264

suspension 94, 182–183, 198, 210

task complexity 33, 310

teacher attributes 22, 216–217, 249

teacher clarity 217, 307

teacher collaboration 9, 228

teacher credibility 203, 216, 226

teacher education 156, 192, 213, 216–217, 229–233, 237, 239, 242–243, 249, 358

teacher effects 216–217

teacher estimates of achievement 224–225

teacher performance pay 159

teachers seeing learning through the eyes of students 8, 45, 429

teacher-student dependency 203

teacher-student relationships 183, 194, 202–203, 205, 212, 215, 244

teacher subject matter knowledge 7, 30, 238–239

teacher verbal ability 218

teaching aides 157

teaching aids 282

teaching methods 7–8, 37, 49–50, 57, 62–66, 73, 75–76, 93, 112, 143, 185–186, 203, 209, 242–244, 252, 260, 277, 285–286, 303–305, 307–308, 317, 336–338, 353, 357–359, 364, 374, 376, 383–384, 387–390, 394, 396, 408, 431–432

teaching strategies 7, 22, 49, 63, 65, 193, 243–244, 273, 275, 287, 301, 308, 358–391, 395, 431–432

team teaching 359, 364

technology in science 393

television 121, 137, 143, 344, 393, 396, 436

temperature gauge 23–24

thrill 7–8, 57, 64, 67, 108, 122, 124, 151, 309, 341, 343–344, 352–353

time of day to study 344–345

time on task 265, 305, 347–348, 395, 406

tracking 129, 167, 180, 182–183, 186–189, 211, 220, 225, 365, 408–409

tutoring 137, 193, 236, 252, 254, 256, 258, 269, 281, 286, 302, 319, 349–350, 358, 386, 394, 405–406, 408–409, 420–421

types of schools 21, 154, 161, 179

underlining and highlighting 346

virtual reality 403–404, 435

visual perception programs 249, 261

VL+ 1, 9–13, 15–16, 18, 337, 342, 412–414, 423, 426, 429–430, 432, 436–437

vocabulary programs 249, 259

welfare 126, 130, 132, 135, 141

well-being 37, 46, 53, 70–71, 88, 90, 100–101, 120, 167, 204, 247, 290, 398, 418, 433

whole language 6, 252–253, 265–267, 270

whole-school improvement programs 412

whole-school reduction of achievement gap 414

will 67, 77, 87–88, 341–342, 352–353

worked examples 227, 317–318, 358, 374, 379, 381

working memory 49, 60, 67, 81, 83–84, 114, 122, 195, 256, 274, 341–342, 344, 346–347, 351, 362

writing programs 271, 273–274

zone of proximal development 96, 151, 381