Month: November 2014
When Martyn Hammersley describes himself as a trouble maker, we should believe him. So troubling is his approach that he seems to want an almost positivist approach to qualitative research. This is of course an exaggeration so extreme that it almost misrepresents him. None-the-less, he certainly leaves me doubtful about the value of qualitative work, while not precisely encouraging me to change direction and embrace big magic numbers.
What he does offer and what I think it worth considering is some discussion of how to evaluate the quality of qualitative research. Putting aside whether it is either necessary or desirable to do this, there is value in the fairly clear set of criteria he suggests. There is little point in basing the credibility of interpretive research on – for example – the numbers involved, or the replicability of the work undertaken, so what’s left?
I don’t agree with everything Hammersly says, but he is in my view a fantastic writer who I enjoy reading, not least of all because of his clarity. He holds his readers’ hand kindly throughout everything he says. He is mindful of wanting to ensure you understand him and that you appreciate the implications of his argument. But the stance he adopts in relation to research is not one I agree with. He would seem to prefer the application of a rigid, tightly defined, almost positivistic template upon qualitative research. I’ve reproduced a brief overview of some aspects of what he says below … troublemaker his is, but these are not unreasonable.
Box 1: Considerations in assessing the adequacy of research reports
The following considerations cover both clarity and sufficiency as standards:
1) The clarity of writing:
- Consistency in use of terms
- Are definitions provided where necessary?
- Are sentences sufficiently well constructed to be intelligible and unambiguous?
- Is there use of inappropriate rhetoric?
2) The problem or question being addressed:
- Is this clearly outlined?
- Is sufficient rationale provided for its significance?
3) The formulation of the main claims:
- Are these made sufficiently clear and unambiguous?
- Are the relations with subordinate claims (including evidence) made sufficiently explicit?
- Is the character of each claim (as description, explanation, theory, evaluation, or prescription) indicated?
4) The formulation of the conclusions:
- Is there a distinction between main claims about the cases studied and general conclusions?
- Is the basis for the conclusions signaled?
5) The account of the research process and of the researcher:
- Is there sufficient, and not too much, information about the research process?
- Is there sufficient, and not too much, information about the researcher? (In other words, is what is necessary and no more provided for assessing the validity of the findings, the value of the methods, the competence of the researcher, according to what is appropriate?)
Hammersley (2008) p162
Hammersley, M. (2008). Questioning qualitative inquiry: Critical essays. Sage.
The quote below is from something a read a while ago. It is the opening sentence of a paragraph, that has been floating around n my head for months. I have never quite found a moment to refer to it.
‘The idea of objects having a social life is a conceit I (Appadurai) coined in 1986 in a collection of essays titled The Social Life of Things. Since then, I (Appadurai) has have continued to be engaged with the idea that persons and things are not radically distinct categories, and that the transactions that surround things are invested with the properties of social relations. Thus, today’s gift is tomorrow’s commodity. Yesterday’s commodity is tomorrow’s found art object. Today’s art object is tomorrow’s junk. And yesterday’s junk is tomorrow’s heirloom.’
There are some things you read that for some reason capture your imagination. Perhaps they are well phrased or present an unexpected line of argument. I have not read very much of Appandurai’s work but I liked this idea of ‘things’ having a social life – dependent on the social transactions they are caught up in. I like the idea of the social life of which they are part having the capacity to transform them.
I connect this to the idea of written notes scribbled in the margins of what you are reading. These tiny little thought pieces – or perhaps conversations between you and the writer of a text – can be transformed into a paragraph and illustrated. They are of enormous value. Not only do they plot the development of your ideas from doctoral candidate to post-doctoral researcher – they represent your personal engagement with a text. They may well be junk (if treated as such), but junk if carefully collated might well of more value then you realise. More importantly, they have a name.
Day one of the second (not a ) weekend – an image based representation of evidence based practice from a medic
Very often the argument about evidence-based practice is a deceptively simple one: tell teachers what works best and they will do it. Why wouldn’t they? Ok – teachers may be involved to some extent in the research design, setting questions and organizing methods, or more often – involved in providing data as sites of experimentation or intervention. But once results are collated and analysed – they are presumably fed back to teachers as a series of pedagogic instructions: teacher proof pedagogy.
Surely this move is a continuation of the long standing ‘discourse of derision’ to which teachers have been subject for some years? They can not be trusted to teach, they are poorly trained, they lack knowledge, they are all left wing progressives – that sort of thing. So what we need is science. Science will instruct them.
But pedagogic practice is more complicated than that. In this table adapted by Frank Coffield from the work of Robin Alexandra, changed slightly by me – teaching is argued as the outcome of a multi-layered filtration process which shapes practice across a number of different dimensions.
Evidence is part of the process but even if the ancient art of positivism is accepted as having some value – it is not the only thing that can (or indeed should) define practice.
Values which define how and what we accept, gather and analyse as
Evidence which is selectively drawn upon to shape what becomes and is experienced as
Policy Pressures which are mediated, translated, interpreted by situational
Pragmatics which is influenced by and influences our
Conceptions of Teaching and Learning which evolve with time, experience and the communities we work alongside to determine our
Practice which is not just what happens in the class room, it is multi-dimensional construct, including
Context, Knowledge, Curriculum, Pedagogy, management, Students, CPD, Society
Coffield, F., & Edward, S. (2009). Rolling out ‘good’,‘best’and ‘excellent’practice. What next? Perfect practice?. British Educational Research Journal, 35(3), 371-390.
While reading: Denzin, N. K. (2009). The elephant in the living room: or extending the conversation about the politics of evidence. Qualitative Research, 9(2), 139-160.
It is unlikely that anyone would argue that policy-making or education practice should be based on anything other than evidence. The issue is in referring to ‘evidence’ advocates of this approach tend towards a narrow belief in research evidence derived from narrowly defined notions of what does and what does not count. Typically research based on positivist methodology is preferred. I get a distinct feeling of déjà vu. Positivism and quantitative methodology are not equivalents as such, they just overlap with remarkable consistency.
The debate about quantitative or qualitative method – in sociology at least – is a long standing one. In 1926 Lundenberg p61 wrote:
The case method is not in itself a scientific method at all, but merely the first step in the scientific method … the statistical method is the best, if not the only scien- tific method …the only possible question … is whether classification of, and gener- alizations from the data should be carried out by random, qualitative, and subjective method … or through the systematic, quantitative and objective proce- dures of the statistical method.
Some years and several conversation later, Becker argued,
life history, when properly conceived and employed can become one of the sociologist’s most powerful observational and analytic tools. (Becker, 1966: xviii)
Denzin’s narrative weaves these together more meaningfully. In 2014, the conversation continues. In response to the current popular debate around Randomised Controlled Trials in education, Dylan Wiliam points out
[…] it is worth noting that RCTs were not required to establish that smoking causes cancer. If we truly wanted “gold standard” evidence that smoking causes cancer, we would have to solicit volunteers for an experiment, divide them into two groups at random, prevent one group from smoking, and ensure that all the members of the other group smoked a certain number of cigarettes per day for a significant length of time (say around 20 years) and then compare the prevalence of cancer in the two groups. Needless to say, this was not the approach adopted. Instead, researchers looked for a way of establishing a causal relationship without an RCT (Hill, 1965).
Denzin’s narrative – interrupted by Dylan William, weaves these quotes together more meaningfully. And then, along comes the elephant:
Consider the parable of the blind men and the elephant. Lillian Quigley (1996):
In my children’s book, The Blind Men and the Elephant (1959) I retell the ancient fable of six blind men who visit the palace of the Rajah and encounter an elephant for the first time. Each touches the elephant, and announces his discovery. The first blind person touches the side of the elephant and reports that it feels like a wall. The sec- ond touches the trunk and says an elephant is like a snake. The third man touches the tusk and says an elephant is like a spear. The fourth person touches a leg and says it feels like a tree. The fifth man touches an ear and says it must be a fan, while the sixth man touches the tail and says how thin, an elephant is like a rope.
There are multiple versions of the elephant in this parable. Multiple lessons. We can never know the true nature of things. We are each blinded by our own perspective. Truth is always partial.
Truth One: The elephant is not one thing. If we call SBR* the elephant, then according to the parable, we can each know only our version of SBR. For SBR advocates, the ele- phant is two things, an all-knowing being who speaks to us, and a way of knowing that produces truths about life. How can a thing be two things at the same time?
Truth Two: For skeptics, we are like the blind persons in the parable. We only see par- tial truths. There is no God’s view of the totality, no uniform way of knowing.
Truth Three: Our methodological and moral biases have so seriously blinded us that we can never understand another blind person’s position. Even if the elephant called SBR speaks, our biases may prohibit us for hearing what she says. In turn, her biases prevent her from hearing what we say.
Truth Four: If we are all blind, if there is no God, and if there are multiple versions of the elephant then we are all fumbling around in the world just doing the best we can.
*Scientifically based research (SBR), or evidence-based movement (EBM)
My take on this is entirely personal. Given that I frequently refer to my preferred style of data analysis as akin to an ‘analytical daydream’, I can see little to gain from qualitative researchers even trying on the methodological straight jacket implied by SBR. More importantly, it misses the question. Teaching is not a technical process of pedagogic intervention leading to predefined learning outcome. Likewise, at what point has any policy maker anywhere ever been interested in what actually works. Policy is politics. Evidence is also politics.
Denzin’s paper aptly illustrates the extent to which all the reservations levelled at qualitative research are evident in qualitative approaches. It is only when truths are settled and accepted by all that flux, change, contestation and controversy subsides.
In the meantime, the question that needs to be answered is not what works in education, but what matters?
An EdD researcher examines her position and wonders: ‘how far back should I go?’
What has influenced me and given shape to my ideas, views and opinions? How far into my past am I going to delve? Does my ancestry influence the me I am today? This feels very much like a chicken and egg question.
My family and upbringing are a major factor but not necessarily directly, one major influence was probably a need to rebel against a Methodist upbringing.How much influence on my position do I have? For instance my position is inevitably as a woman but is it influenced by the history I have lived through.
How far has my education influenced my perspective? My reading has been determined not only by the subjects I have studied but also the preferences of my teachers and the syllabus I have encountered.
How will my new role as a researcher influence my future position? My role as teacher bring the issue of power relationships into focus, if I am to use my students as the subjects of my research I will need to recognize that my relationship with them will be in sharp focus and that I will need to be very clear about this in the planning of my study.
Examining positionality is going to involve a great deal of self examination and personal honesty, It’s going to be an interesting ride!
Reblogged from: http://marycryan2013.wordpress.com/2014/11/02/examining-my-position-2/
Academics argue about everything. One of my favourite essays (written by Richard Edwards) is devoted to a critique of the word ‘and’ when used in teaching ‘and’ learning. Edwards demonstrates how the hardworking ‘and’, a word that usually lurks in the shadows, is capable of misleading. After Edwards, ‘and’ is no longer able to hide behind its sticky anonymity. It’s hardly surprising then that a word like ‘theory’ (not what theory does but what theory is) is hotly contested.
Here are at least five different meanings. A theory is:
A generalizing or explanatory model. Theory tries to distil a range of specific findings or observations into general propositions that explain these findings.
The ‘thinking side’ of practice. Especially in the applied side of the social sciences (such as teaching and social work), theory or ‘theorising’ means thinking and reflecting on practice. It is sometimes called ‘reflective practice’. This has also been called ‘personal theory’ or ‘practical theory’. When people talk about this ‘personal theory’ they mean conjectures, personal thoughts and insights that help people to make sense of the practical world. Many people suggest that in practical fields (such as teaching, nursing or social work) the enhancement of this practical theorisation, coming from our own experience, is what professional development should ultimately be about. The idea is not new: the Greeks had words for this kind of practical theory: phronesis and techne.
A developing body of explanation. Here theory means the broadening bodies of knowledge developing in particular fields. A body of knowledge may be wide, as for example in ‘management theory’ or ‘learning theory’, or tight, based around a particular set of ideas, as for example in ‘Piagetian theory’.
Scientific theory. Theory here is modelled on the idea of theory in the natural sciences. It may ultimately exist in the form of ideas formally expressed in a series of statements. With these statements you are able to predict and explain. As social science progresses it seems less hopeful of discovering this kind of theory.
Grand Theory. ‘Grand Theory’ is a term used mockingly by the great sociologist Wright Mills (1959) to describe social scientists’ expectation that their disciplines should attempt to build systematic theory of ‘the nature of man and society’ (p 23). The theories of Marx and Freud are examples. Wright Mills saw this effort as an obstacle to progress in the human sciences. I take it as given that Grand Theory is not what is generally wanted in social and educational research nowadays. You certainly will not be aiming to develop Grand Theory in your own research, though you may use Grand Theory as a framework or stimulus for it.
Thomas is anti-theory seeing it as a means of limiting what is permitted to count as knowledge by fixing, controlling, setting up boundaries, prohibitions and privileges. It was some time ago I first read Thomas and I must admit that I was not particularly sympathetic to his ideas. (bell hooks writes a essay that I have entitled: one night a theory saved my life). We need theory to make sense of the world, we can observe correlation – theory is what allows us to make sense of what we observe. But in his readable and well argued text “Education and Theory: Strangers in Paradigms’ Thomas views theory in education as antagonistic to pluralism in ideas. When social researchers are committed to theory, they are committed to allowing the generative dance of ideas proliferating to be sacrificed in favour of orderliness.
Re-reading Thomas again over these last few weeks, I have warmed to him. That is, I understand his position differently. I am inclined to agree. If I belong to a theoretical camp, it is the ‘anything goes’ camp. What is needed in educational research is more ad hocery, more thought experiments, more diversity. Theory closes down discussion, limiting the breadth and depth of our view.
Far better I say to let the Otherness of the Other bloom.