Survey and interview design

I collected data by interviews and using an online survey, asking the same questions in each setting.^{[1] [2] [3]} Survey design took into account multiple requirements:

Figure 3.3 Respondents by Industry

• • I sought lived experience. Questions were realistic, based on real-life cases.
• • The conversation was with an elite. The survey reflected this is in complexity and the use of open questions.
• • Demographic questions provided quantitative data, which in turn allow analysis of consistency between answers and facilitate some generalisation.
• • In early discussions, trials and pilots it became clear that I should ensure that initial questions were open, with case studies and demographic questions following.^{[4] [5] 48}
• • The availability of online survey tools, easy to distribute by email or social media, made an online survey an easy option despite the risk that vast amounts of data would be returned for analysis.
• • Some questions bore similarity to questions asked by other empirical researchers. This also allowed comparison and strengthened generalising claims.^4.

Context matters. It is vital to place people into contexts they may understand. After conducting an experiment using military officers asked to make a counter-terrorism decision, Mintz replicated it using student avatars. Whereas over one-third of students recommended doing nothing, over 90% of military officers recommended doing something. ⁹ Mintz concludes:

It is unrealistic to expect students to play the role of elites in political science and international relations experiment ... as the groups are very different in their sociodemographic characteristics, expertise, level of professional responsibility, and other significant factors.'

I avoid the use of “x-phi” games or experiments - Ultimatum Games, Prisoner’s Dilemma games, variant trolleyology (a British invention I call tramification) -which all suffer from the disadvantage that real-world application or generalisation to the real world is not possible. In one Prisoner’s Dilemma experiment the game was called the Cooperation Game for half the participants and the Wall Street Game for the others. Those playing the Wall Street Game were “dramatically” less likely to cooperate. Commenting, Jesse Prinz concludes that strategies adjust in “dramatic ways based on culturally meaningful contexts”.' Other studies show, for example, that placing posters with eyes on them on the wall during x-phi experiments varies participant behaviour (Figure 3.4).'

Figure 3.4 The Watching Eyes Experiment - The Posters

• 49 Alex Mintz and others, ‘Can We Generalize from Student Experiments to the Real World in Political Science, Military Affairs, and International Relations?’ (2006) 50 J Conflict Résolut 757 in abstract.
• 50 Ibid. 771.
• 51 Jesse J Prinz, Beyond Human Nature (1st edn, Allen Lane 2012), 313-314. For a note on possible cultural issues in such studies see Natalie Gold, Andrew M Colman and Briony 1) Pulford, ‘Cultural Differences in Responses to Real-Life and Hypothetical Trolley Problems’ (2014) 9 Judgm Decis Mak 65.
• 52 Daniel Nettle, ‘The Watching Eyes Effect in the Dictator Game’ (2013) 34 Evol Hum Behav - abstract. In this game researchers placed a questionnaire and instructions.

This strengthens the case for asking questions designed from real-life case studies and seeking the reaction of experienced managers to them.'³ Experiments should be transferable and should test real-world hypotheses.⁵⁴ Edmonds observes:

In the real world we are not constrained by having just two options, X and Y: we have a multitude of options, and our choices are entangled in complex duties and obligations and motives. In the real world, crucially, there would be no certainty.

Engel makes the point in an extended way, emphasising that

The [iterated I’D] model makes a lot of assumptions. Interaction is anonymous. There is no communication. It is not possible to solve the conflict by agreement. There is no room for third-party' intervention. Interaction is one-shot. It is an interaction between only two individuals. Each individual has to make a yes or no decision and cannot make intermediate choices. There are no dynamics.

All these assumptions matter for the dissolution of cooperation problems.

Individuals are less selfish if they are identified. Communication increases the willingness to take others’ legitimate interests into account. If people are able to choose their partners ... similar partners, they cooperate more. A credible threat with third-party intervention almost perfectly removes the dilemma.⁵⁶

Analysing responses to a questionnaire designed "to explore the circumstances when financial incentives can overcome the moral scruples of contractual parties about breaching the contract", Tess Wilkinson-Ryan asserts that it

is intended to be closely analogous to a real-world contracts context, but there are limits to that analogy'. The stakes were real but very low; there

a cardboard box and two envelopes, one marked “For you to take away”, the other “For a random student”, in the cubicle. Also on the desk was £5 comprising four £1 coins and two £0.5 coins. Participants were instructed to put whatever money they wanted to keep into the "take away” envelope, which they could then pocket, and whatever money they wished to donate to a random student in the “random student” envelope.

• 53 As I observe above, it is difficult to persuade commercial actors to take part in such surveys. Nevertheless, we should be cautious about drawing conclusions from experiments in which the participants are avatars.
• 54 See Russell Korobkin, ‘Empirical Scholarship in Contract Law: Possibilities and Pitfalls’ (2002) U IllLRev 1033 especially at 1052 and Dennis Patterson, ‘The Limits of Empiricism’ (2000)98 MichLRev 2738.
• 55 David Edmonds, Would You Kill the Fat Man? (Princeton UP 2013) at 100.
• 56 Christoph Engel, ‘The Proper Scope of Behavioral Law and Economics’, Discussion Papers of the Max Planck Institute for Research on Collective Goods (2018) at 8 with references. See also Samir Okasha, reviewing The Origins of Human Cooperation: Samuel Bowles and Herbert Gintis: A Cooperative Species (Princeton UP, 2011) at 28 Biology and Philosophy 873: “The [folk] theorem only demonstrates the existence of cooperative Nash equilibria; but this says nothing about how or why agents will arrive at them. ... if players have private rather than public information about each other’s past moves, as seems quite likely, then the folk theorem does not go through.” was no legal framework in which parties were negotiating and operating; there were no reputation costs, transactions costs, or even social costs to breach.^{[6] [7]}

It might have been easier to list the respects in which a real-world analogy existed; and it is unlikely that the subjects had a meaningful understanding of the context.

It is also important to ensure that there are no credible alternative explanations for phenomena experienced; (the error made by Eisenberg and Miller). In another example, Jonathan Morgan, relying on an experiment carried out on 94 Midwestern University students presented with iterated “Prisoner’s Dilemmas”, apparently showing trust increasing with repeated interaction, says: “attempting to enforce vague obligations of trust and cooperation will not only be difficult and expensive, but may be counterproductive”.

In laboratory conditions, the 94 were offered “non-binding” (whatever that means) contracts, then, in a later transaction, offered a “contract”. On acceptance, they were informed, electronically, that “the computer would automatically enforce it”.'^[8] At that point trust apparently decreased. The chief problem with this and similar experiments is the assumption that one can transcribe experimental results from trials involving students to the real world. As Robson notes, this is dangerous.^[9] My survey shows that experience may change perspective, so the use of inexperienced avatars arguably reveals little except the reaction of inexperienced avatars.

The second problem is that the researchers appear not to have appreciated rite nature of the enforcement advice. I double-checked by asking some of my contacts to consider rite short case study in Table 3.2. I received 105 responses and 170 comments.

My respondents are hard-bitten contract professionals; not ingenues. My short survey shows that the “enforce” language may provoke a reduction in trust or other reaction that casts serious doubt on the claim that entering into a formal contract reduces trust. The blatant reference to enforcement arouses suspicion and creates a negative reaction. I cannot pretend that the sample is random. For my purpose, that does not matter. I demonstrate an alternative, plausible, explanation for the so-called reduction in trust.

Interviews work best as conversations, in which the interviewer listens carefully, rather than as question and answer sessions.^[10] Allowing online respondents

Table 3.2 Do Contracts Reduce Trust?

to offer up their opinions in open questions, and to provide written opinion, was intended, as far as possible, to replicate the interviews.

The strength of interviewing is flexibility and the ability to steer conversations.

Good interviews allow the interviewee to do most of the talking. Extracting rich, refined, considered data from very senior participants would be difficult using an online survey. As Dr Jennifer Fleetwood observed, in the qualitative training I undertook, an interview is “easy to do badly and hard to do well”. Managing an elite elite, a super group, required patience and expertise on my part together with a degree of flexibility. Control was not possible with this group but my background, as a peer, enabled me to identify with them, understand their responses and steer them in the desired direction.⁶'

In interviews and online I asked two open questions at the start, asking respondents what they enjoyed about managing contracts and what success meant. I expected this to provide a guide to expert commercial opinion on commercial coherence or business efficacy in generating advice on how these complex contracts work, and are managed, in practice. This definition would be at a relatively high level of abstraction, necessarily so because the question is very' wide.

It was important to ask at least some questions that were not too general. For example, in 2005 Vogenauer and Weatherill surveyed 175 enterprises in eight countries. They found that respondents wanted law that enabled trade (87%), is predictable (79%), fair (78%), flexible (61%) or prescriptive 39%. This is useful, but is clearly an example of survey data which worried Adams and Brownsword; showing expectations shared only at a “very high level of generality”.^{[11] [12] [13] [14]} I followed open questions with four vignettes, case studies, developed from difficult and controversial cases covered in my thesis.^[15] The thread that runs through the vignettes is uncooperative behaviour.

In vignettes, I asked respondents to assess “standard” current options and for advice on how the law and the contract could support them. I expected that a majority would identify communication and governance to create cooperation as important but that legal remedies such as fast-track dispute resolution would be considered extremely useful. I expected enforcement and threat-based

Empirical research: survey and results 91 remedies to be considered helpful but insufficient. In my vignettes fast-track binding adjudication would almost certainly deter some of the behaviour if a duty' to cooperate formed part of the contract.

I considered other vignettes; for example, the Coombes case where a manager had referred to his Secretary as a “bitch”. In informal trials, I detected very strong emotional recoil from this and decided against including it.^[16] I also considered Horkulak v Cantor Fitzgerald International but decided that the use of very strong language would deter respondents.^[17] Case studies were part quantitative, with scaled responses, usually using discrete variables, and part qualitative, allowing respondents to offer alternative solutions or other comment.

After that I asked respondents to rate the importance of cooperation and to tell me what cooperation means and entails and how one achieves it. The purpose of these questions was to determine whether cooperation is considered necessary by my respondents, to allow me to assert that cooperation is necessary to business efficacy or commercial coherence in these complex contracts and, in defining cooperation, whether I can find analogies or authority which would allow me to put forward a transcendent concept of cooperation.

Survey – general results

In analysing results variables were reduced to graphs and tabulated to uncover outliers and major variance. I have included a visual map of the results in Table 3.3, which was produced by visual inspection of the graphical data. From this snapshot one can see that the data are robust. Around 70% of cells show little or no variance, 8% show variance in one answer. Approximately 16% of cells show some variance.

• [1] Miles and Huberman, Qualitative Data Analysis, 267, Robson, Real World Research, 158, Denzin and Lincoln, Collecting, 199-200. Denzin coined the term "triangulation” for carrying out studies in different locations, using multiple theories, multiple researchers, multipledata technologies, different sources, collection methods, quota samples, age and gender, and datatypes to ascertain how far one might generalise from a non-random sample. See also Berg, Qualitative Research Methods, 7, who says that “research literature continues to support Denzin’s recommendation to triangulate”.
• [2] Robson, Real World Research, 277, suggests considering comparing early and late respondents. I took his steer - which is that this is a “counsel of perfection”.
• [3] Brinkmann, Understanding Qualitative Research, 83, saying “the most general rule across paradigmatic differences is; describe what you have done and why”.
• [4] This advice was of a severely practical nature; respondents would complete demographic questions to avoid wasting the work already performed on open and closed questions; which might not be true vice versa. For solid advice on constructing questions and sense-checking them see e.g. Valerie M Sue and Lois A Ritter, Conducting Online Surveys (Sage 2007) (especially to consider whether interviews will add anything), Robson, Real World Research, 255-256, G Nigel Gilbert, Researching Social Life (Sage 2008), 188, and Brinkmann, Understanding Qualitative Research, 49.
• [5] The Bristol Online Survey tool (www.onlinesurveys.ac.uk/) made realising my design easy.
• [6] Tess Wilkinson-Ryan, ‘Incentives to Breach’ (2015) ALER 290.
• [7] Morgan, Contract Law Minimalism, 69, the claim deriving from Deepak Malhotra and J. Keith Murnighan, ‘The Effects of Contracts on Interpersonal Trust’ (2002) 47 Administrative Sci Q 534. Dcakin and others, ‘Contract Law, Trust Relations, and Incentives tor Co-operation: A Comparative Study’, in Jonathan Michie and SF Deakin (eds) Contracts, Co-operation, and Competition: Studies in Economics, Management, and Law (OUP 1997), chapter 5, show using an empirical survey that these results may be hard to replicate in the real world.
• [8] Malhotra and Murnighan. ‘Effects’.
• [9] Robson, Real World Research, 4. And see Mintz and others, ‘Can We Generalize’.
• [10] See Robson, Real World Research, 281.
• [11] I like her advice on methodology (in an email): "I’d say that most academics ... learn on the job ... trialling with students, and then with ex-colleagues will be sufficient. Have confidence -there is no such thing as a perfect interview. ... interviews are all different, so even if your guide is fantastic, it will not do the magic in all circumstances and with all respondents. All you can do is your best, and see what happens!”
• [12] Rebecca E Klatch, ‘The Methodological Problems of Studying a Politically Resistant Community’ (1988) 1 Studies in Qualitative Sociology 73 suggests that young female interviewers may have more success with elites. As a 62-year-old male; I disagree. A peer, an expert, is more likely to be successful. Alvesson and Deetz, Doing Critical Management Research, 195, suggest that I am correct.
• [13] Chapter 7 ‘The European Community’s Competence to Pursue the Harmonisation of Contract Law - An Empirical Contribution to the Debate’ in Stefan Vogenauer and Stephen Weatherill (eds) The Harmonisation of European Contract Law: Implications for European Private Laws, Business and Legal Practice, vol. 1 (Hart 2006) at 137.
• [14] John Adams and Roger Brownsword, Key Issues in Contract (Butterworths 1995) at 326.
• [15] On vignettes see Christiane Atzmuller and Peter M Steiner, ‘Experimental Vignette Studies in Survey' Research’ (2010) 6 Methodology 128.
• [16] Isle of White Tourist Board r Coombes [1976] IRLR413, EAT.
• [17] Horkulak v Cantor Fitzgerald International [2004] EWCA Civ 1287. The Judgment records one incident in which a manager, Mr Amaitis, after a presentation, shouted: “get this shit out of here”, “it will never tucking work”. Stronger language is also recorded.