Reading research papers has become somewhat of a pastime for me since I wrote my first book in 2019. Throughout the previous decade, I had become increasingly interested in educational research, progressing from the gateway drug of John Hattie, through to engaging with bloggers who were into ‘evidence-based practice’, and ending up getting my kicks from the unprocessed research papers themselves.
I am not an academic, but I believe that there is a place for practitioners to build bridges between research and practice. There is a need for research (which can be inaccessible to those outside academia) to be translated for practitioners if it is to have an impact in the real world. There is also a need for practitioners to push back on research claims which don’t accurately describe how things appear to be on the ground or do not deliver on promises. There is a divide between academics and practitioners, but the relationship need not be adversarial. As Dr Neil Gilbride points out, there are many valid ways of building knowledge and we should be working together on the task.
Once in a while, I read a paper which particularly resonates – I get a little rush of academic adrenaline. I am aware that this may be because it confirms what I already believe to be true, but this is not always the case. Such papers tend to add something to my understanding, whether it be to tell me something I didn’t know, to help me see something in a different way, or provide me with a conceptual framework which connects together knowledge which is currently floating free in my mind. There is always a ‘click’.
In this post, I’d like to introduce you to one such paper. It is titled ‘Educational leaders’ problem-solving for educational improvement: Belief validity testing in conversations’ and was published in 2021 by Claire Sinemma, Frauke Meyer, Deidre Le Fevre, Hamish Chalmers and Vivianne Robinson. It is a paper that provided many clicks, but it also provides something many research papers do not, that is practical advice which is immediately actionable. I hope to provide a brief summary of some of the ideas in the paper because I think it says some useful and valuable things.
Testing our beliefs
The paper makes four strong claims:
- The ability of educational leaders to solve complex problems is important if we are to improve the school system.
- Leaders hold three types of belief about problems. They hold beliefs about the nature of the problem, what causes it, and how to solve it.
- Testing the validity of these beliefs through conversations is a critical leadership capability.
- In practice, leaders tend to ‘avoid discussion of problem causes, advocate more than inquire, bypass disagreements, and rarely explore logic between solutions and problem causes’.
Immediately, the paper sets up a concrete problem, that is how school leaders engage in dialogue about educational problems. They describe a set of behaviours which I recognise both in my own practice and in that of school leaders I have worked with over the years. At the heart of the problem is a tendency to fixate on a theory which appears to fit the observable features of a problem and to proceed from that theory on the assumption that it is valid. This would not be an issue if those assumptions were correct. The implication is that often they are not. Therefore, the ‘validity testing behaviours’ of leaders are critical in enabling questionable assumptions to be surfaced and for a better theory of action to be formed.
How should leaders test the validity of their beliefs? The authors suggest that ‘conversational moments’ are an opportunity to do so. In these moments, leaders should test their assumptions about:
- The nature of the problem: What is problematic about current practice? What dissonance is there between how things are and how you would like them to be?
- The cause/s of the problem: What circumstances have contrived to take us to this point? To what do we attribute ‘the problem’?
- The likely solution: What actions will lead to resolution and more desirable practices?
The authors find in their study that, instead of the above, leaders frequently assume that they understand the problem, know what caused it and know how to solve it. We may think of this as an illusion of certainty, one which we should aim to dispel.
Why ‘problems’? Why not ‘opportunities’?
Colloquially, the term ‘problem’ is often used negatively. It describes dissatisfaction with the status quo. Robinson uses the term more neutrally to mean a gap between the current and desired state, and the demand that this gap be reduced. An opportunity may also be described as a gap between the current and desired state, but there are an almost infinite number of opportunities, not all of which we feel minded to grasp. When we ‘problematise’ an opportunity to act it is because we feel driven to address it. It is dissatisfaction that motivates us to action. We feel the ‘wrongness’ of circumstance and this, combined with a belief that this can be put right, creates momentum. A problem is therefore born both of dissatisfaction and ambition; despair and desire.
Sinemma et al point to the ill-structured nature of problems we typically find in education, ‘complex rather than straight-forward, and adaptive rather than routine’. Such problems are difficult to solve because ‘they can be construed in various ways and lack clear criteria for what counts as a good solution’. Defining exactly what the problem is becomes difficult, as does identifying a single, or even limited number, of causes.
Furthermore, problem solving in education contexts is typically a social activity as there are many people invested in framing, explaining and solving the problem. Each of these parties will hold their own beliefs about the problem, indeed whether or not they even believe a problem exists. Engaging these vested interests is not merely a matter of truth seeking but a pragmatic response as the information required may be held by multiple people and the enactment of an effective solution will likely depend on the cooperation of many individuals.
To understand the authors’ claim that conversational moments are so important in facilitating wider and substantial change, we must also appreciate the concept of the nesting of problems. Bronfonbrenner’s ecological systems theory provides the basis for a nested model of educational problems. If you are interested in this model I would recommend reading the paper as a detailed description is beyond the scope of this blog post. However, the main assertion is that there is a causal and cumulative relationship between everyday problem solving conversations and the more substantial efforts to reform schools and the wider school system. If we challenge our beliefs about specific problems in our ‘immediate environment’ through everyday encounters, the consequence will be that we think differently about systemic problems.
The paper cites previous research which highlights the difference between novice and expert school leaders in how they perceive and approach ill-structured problems. For example, various studies of the problem solving capabilities of school principals show that expert leaders perceive unstructured problems as more manageable, find them less stressful, are more likely to collect information which challenges their assumptions, and are more reflective on their actions than more novice or aspiring principals. A correlation between how robust and helpful educators perceive their professional discussions to be, and the subsequent improvement in their practice, is also highlighted. The importance of leaders’ problem solving capability, particularly in relation to complex, social problems, is well supported in the literature.
The role of conversation
Conversation, as stated previously, may enable leaders to test the validity of their beliefs and thereby improve the effectiveness of social problem solving. Effectiveness is taken to mean both progressing the task of solving the problem and maintaining or improving the leader’s relationship with those involved. The latter outcome is important as relational trust is foundational to educational problem solving and should not be sacrificed for the benefit of achieving a better theory of action.
One of the difficulties for school leaders is that they may form judgmental beliefs about another’s intentions, attitudes, or motivations. For example, they may attribute unsatisfactory performance or outcomes to an individual or group without considering other explanations fully. Sharing such views candidly may lead to conflict and negative emotion, therefore it is not surprising that leaders may soften their portrayal of the problem or avoid revealing their beliefs fully. Leaders therefore not only require the ability to express contentious views respectfully but the habit of mind to resist making unfounded attributions as to fault or responsibility. This ability requires the leader to develop more sophisticated mental models which allow insight into the complex causal mechanisms at play and help avoid naive and simplistic assumptions about error.
Why is validity testing so important? Sinemma et al argue that leaders’ beliefs have ‘powerful consequences for the lives and learning of teachers and students and can limit or support educational change efforts’. If they act on unsubstantiated beliefs, leaders not only miss opportunities to improve educational outcomes but may make decisions which are counterproductive, even harmful. One of the most important tendencies, the authors claim, is for leaders to be ‘attentive to the information that disconfirms rather than confirms their beliefs’, thereby becoming truth seekers, not truth claimers. This in turn increases their credibility as leaders and reduces the need for coercive influence as teachers are more likely to be internally committed to courses of action which have been opened to scrutiny.
Although the authors do not describe it as such, we may see validity testing conversations as a means of institutional knowledge-building. If beliefs about the school’s problems (their nature, cause and likely solutions) are validated through dialogue, a collective understanding of school improvement will be fostered. Such democratic behaviours normalise organisational learning and support a culture of internal challenge and shared endeavour.
The five validity testing behaviours
Having established a theoretical basis for the importance of validity testing behaviours in leaders, the study set out to explore these behaviours by analysing conversations between leaders and staff. The paper proposes a normative model of effective problem solving conversations which includes five validity testing behaviours:
Beliefs must be disclosed if they are to be tested. This disclosure should be honest and respectful and should include ‘all the information that is believed to be relevant to the problem, including that which might trigger an emotional reaction’. The authors point out the link between respectful disclosure and relational trust.
The reasoning that led to the beliefs should be set out clearly (advocacy) and others should be invited to do the same. Leaders should invite the other party to critique their reasoning and the consequent dialogues can ‘lead to a strengthening, revision, or abandonment of the beliefs for either or both parties’.
It is the differences in beliefs that warrant attention, not convergence. An exploration of differing beliefs is more likely to reveal faulty reasoning, lead to the sharing of asymmetric information, and lead to solutions which integrate divergent perspectives on the problem.
An effective solution will have an internal logic ‘that links problems to their assumed causes and solutions’, what we may call a chain of reasoning. By exploring beliefs about the nature of the problem, it’s possible causes, and possible solutions, this chain will be strengthened.
Agreement in itself is insufficient, else one party persuading the other would suffice. Rather, the goal is what is termed warranted agreement, that is sufficient scrutiny of each others’ beliefs has occurred to provide a strong basis for agreement. Of course, it is not always possible to agree on everything, therefore where agreement does not occur it is important to set out clearly what specifically remains contested and why.
Easier said than done
This paper sets out some credible and well supported claims about how leadership effectiveness can be enhanced through belief validity testing in conversations. The simplicity of the ‘five behaviours’ model is attractive and it is tempting to adapt this as a ‘framework’ to guide leaders in holding such conversations. However, we should be cautious in adopting this as a heuristic or in assuming that these five behaviours constitute a set of generic skills which we can master and apply to the various conversations we have. It may well be that expert leaders are more likely to behave in these ways, but they are unlikely to have developed this expertise by following a formula or explicitly practicing these discrete behaviours.
Indeed, it may be highly counterproductive to encourage a novice leader to suddenly open their beliefs to others for criticism and scrutiny. Not only may their theories be under-developed and embarrassingly naive, a radical change in their behaviour may be unnerving and (initially at least) unwelcome by others. Furthermore, the prevailing culture of the school may mean such behaviour is counter-cultural and lead to stigma or a negative reaction by others leaders. Validity testing behaviours may be a desirable goal but not one that can be achieved overnight if current norms are somewhat different.
It seems likely that experts will be more open and able to validity test their beliefs through conversations because they have well developed mental models which do not fall apart under scrutiny, have moved beyond making simplistic assumptions about causes or where the blame lies for suboptimal performance, and have developed trusting relationships with colleagues over a period of time. When they set out their beliefs for others to scrutinise there is a regularity to this behaviour which means conversations can be less guarded and sufficiently candid. Each party understands that this is about knowledge-building and not an exercise of power or persuasion.
Sinemma et al make a persuasive argument for the importance of belief validity testing as a means of improving leadership effectiveness. But there is some way to go in building relational trust in many schools before we can expect leaders to show the vulnerability required. School leaders may be wrong about many things, but are they ready to confront this?