Empirical legal research in negligence, judicial review/administrative discretion and contract
For the second seminar (Understanding Quantitative Empirical Legal Research) students are invited to explain what they understand by key concepts in quantitative research such as reliability, replicability, validity (distinguishing internal validity and external validity), generalizability and causality. They are also introduced to the concept of triangulation. This is followed by consideration of the advantages and limitations of secondary analysis of quantitative data, with specific reference to official statistics in the legal and criminal justice fields. Students are asked: 'Why are official statistics on crime likely to be unreliable, and how does the British Crime Survey attempt to avoid this problem?' Students are invited to compare this example with other contexts (such as probation, prison, judicial statistics) which illustrate other problems in the secondary use of quantitative data gathered by government agencies. The rest of the seminar focuses on critical evaluation of a socio-legal article using secondary quantitative analysis, contributing to the compensation culture debate (Lewis et al., 2006). A checklist is provided to encourage students to work through the article systematically and to identify key issues, as follows:
- 1 Is the approach doctrinal, empirical, or a combination?
- 2 What aspect of law, or what 'problem', does the research address? What are the main research questions? What are the researchers trying to find out? What 'gap' in our knowledge does the research seek to fill?
- 3 Is the research qualitative, quantitative, or a combination of these types (n.b. triangulation)? Is the research 'primary' or 'secondary'? If quantitative secondary research, is there any use of official statistics?
- 4 What empirical methods, or techniques, are the researchers using to address their research questions (n.b. triangulation)?
- 5 What empirical data are produced by the methods used?
- 6 Are the data produced by the research methods reliable? Is the research replicable? Have the researchers spelt out their procedures so that the results are capable of replication?
- 7 How do the researchers analyse the data? What analytical methods do they employ? How are the raw data presented or manipulated, and in what form?
- 8 Assuming the data to be reliable, are the results of the research 'externally valid'? How representative is the sample (that segment or part of the phenomenon that the researchers are trying to study)?
- 9 Where research data are claimed to show a causal relationship between 'x' and 'y', does the claim hold water; does 'x' really explain or cause 'y'? Are inferences 'internally valid'?
- 10 What ethical issues were at stake in this research? Did the researchers conform to any ethical code, e.g. SLSA 'Statement of Principles of Ethical Research Practice' (2009)? What ethical principles should they have observed?
- 11 How convinced are you by the authors' overall argument and conclusions based on the empirical research? What are the implications of the research for legal reform, either explicitly addressed by the authors, or implicit? What does the research contribute to our understanding of law in this field? How successful is it in filling 'gaps' in knowledge?
The third seminar (Understanding Qualitative Empirical Legal Research) follows a similar pattern. Students are invited to explain key qualitative research concepts, and to consider why (according to social research methods experts referred to in the core reading) criteria such as reliability and validity have to be adapted or differently applied in this context. The reasons concern the very different aims of qualitative researchers, who as a rule are not claiming to produce findings that are generalizable to other social settings, or to 'measure' social reality with the same degree of accuracy as their quantitative counterparts. Accordingly, while all other points on the checklist for evaluation of the qualitative case study article (Halliday, 2000a) are the same, prompts 6, 8 and 9 differ from those in the previous exercise. So rather than ask whether the data produced by the research are 'reliable' and 'capable of replication', the question is whether they are 'dependable' or 'auditable' (whether the research has been carried out in a manner likely to convince an independent auditor or peer reviewer of their trustworthiness: Lincoln and Guba, 1985). Again, instead of asking whether the results of the research are 'externally valid' and whether the sample is 'representative', the focus here is on the extent of 'thick description' and 'transferability' to other contexts. And, finally, in place of the concern with 'internal validity' and 'causality', the parallel question for qualitative research is whether the results are 'credible' in the sense of being feasible, or likely to be believed by others. Here the empirical focus is on the impact of administrative law, in particular judicial review, on the exercise of discretion in decision-making by local authorities implementing homelessness legislation.
For the final seminar (Understanding Empirical Research on Contract), students are asked to evaluate two articles, one quantitative (Vincent-Jones, 1993) and the other qualitative (Beale and Dugdale, 1975). Whereas in the previous two seminars some of the answers to checklist questions are obvious from the seminar's focus on either quantitative or qualitative research, here no clues are provided as to the nature of the empirical research, so students are required independently to demonstrate understanding of key distinctions and to determine and apply the appropriate criteria for evaluation.
By the end of the module, therefore, the same evaluation technique will have been practised on four occasions across the three different subject fields of Tort, Contract, and Administrative Law. The intention is that students will thereby have acquired a range of critical consumer skills necessary to understand how socio-legal arguments about law and law reform are made on the basis of empirical research, and that these skills should be transferable and of practical use in their independent research for assessed essays and their final- year dissertation. The critical skills element is reinforced by the assessment, which adopts a similar format to the seminar exercises. Students are required to write a 2500-word essay:
With reference to research methods concepts studied in this module, evaluate the argument of the author(s) in one of the following articles, paying particular attention to (i) the methods employed and the use made of empirical data, and (ii) the contribution of the research to the understanding of law in the given field.
For 2010-2011 the articles were: Pick and Sunkin (2001) and Williams (2007).
In addition to the written assignment, for the 2011-2012 session an element of oral assessment has been introduced, weighted 15 per cent of the total mark. Students are informed in the module handbook:
We aim to encourage an open and relaxed atmosphere in which everyone has the opportunity to learn by participating in discussion. For each of seminars 2-4 you will be assessed according to the following criteria:
• Preparation: We expect you to bring notes to the seminar so that we can see how you have prepared answers to the set questions.
Space has been provided in the Workbook for you to make notes for seminar preparation (if you do this electronically, you are expected to bring a printed copy to the seminar).
- • Contribution to discussion: We expect you to make relevant points in participating in and contributing to discussion, but also to show sensitivity to others in the class by responding to them in a respectful and constructive fashion. Listening carefully is just as important as speaking clearly. A good seminar participant will not dominate discussion.
- • Quality of content: We expect you to demonstrate orally your familiarity with the module materials, and your knowledge and understanding of key methodological issues and concepts. You will not be penalised for making mistakes or for being wrong, indeed learning from mistakes is one of the benefits of open discussion.
- • Quality of expression: We are looking for clarity, fluency and conciseness in your oral expression. The ability to 'think on your feet' and respond appropriately to unanticipated situations, comments or questions is a core transferable skill.
-  Other journal articles in the field of tort/negligence drawing on empirical research methodsthat have been found suitable include Gray and Sharpe (1973), Dingwall et al. (2000), Lewiset al. (2002), Williams (2003; 2005), Morris (2007) and Hand (2010).
-  Empirical studies of judicial review/administrative discretion include Halliday (2000b),Machin and Richardson (2000) and Sunkin et al. (2007).
-  On the empirical socio-legal study of contract, see also Macaulay (1963) and Lewis (1982).
-  The assessment articles for 2011-2012 were Halliday (2000b) and Hand (2010).