Starting Out in Research: Resisting the Dominance of 'Nothing Works'
The early 1970s were a time of political and economic instability in Britain. Inflation was rampant, a government was brought down by industrial action and bombs were exploded in crowded pubs and railway stations. An energy crisis led briefly to a three-day working week and power cuts during which people lit their homes with candles. At one time the threat of petrol rationing led to the distribution of extra petrol coupons to ‘essential’ workers; I am pleased to report that this included probation officers. In the meantime, I began to think that my curiosity about the effectiveness of probation demanded deeper study than was easily available within the Probation Service itself, and with some encouragement (from Bill Jordan among others) I began to think of applying for university jobs. I eventually found a post in Swansea (in an area which, among other advantages, offered more affordable housing than Gloucestershire) and was almost immediately challenged to find evidence of social work’s effectiveness.
I found that key figures in my department tended to regard social workers as well-meaning but a bit deluded and not necessarily very bright. Criminologists had read Martinson (1974) and tended to believe that nothing worked, and we also had an anti-collectivist social policy specialist who was a friend of the Conservative intellectual (and Margaret Thatcher’s mentor) Keith Joseph. She co-authored a briefly notorious book about why social work was ineffective and made people worse (Brewer and Lait 1980). These challenges helped to launch me into a 40-year (so far) career of research and teaching which mainly concerned the effectiveness and impact of work with convicted individuals in the community, and I also tried to extend my limited practical experience by working part-time for what was then the West Glamorgan Probation and After-Care Service, and by helping to organise one of the first Victims Support Schemes in Wales. In those early days of victim support many of the people involved thought that by showing some effective support for victims, the criminal justice system might eventually be able to become less punitive and more constructive in the way it dealt with those who offended. As some of us put it a few years later: ‘... penal reform groups were often criticized for not being concerned about “the victim” (although there was no evidence that anyone else was doing anything for victims either)’ (Holtom and Raynor 1988, p. 19); ‘We wanted to avoid reinforcing illusions that victims benefit from harsher sentencing’ (Holtom and Raynor 1988, p. 24).
I also tried to respond to received opinions that probation was ineffective and/or a form of soft correctional coercion in an article in a social work journal which argued that probation should be understood as essentially an agreement entered into by the probationer, the supervisor and the sentencer, and that the defendant’s choice, expressed as consent to the Order, was important. I also pointed to some more positive research findings (mostly not from Britain) as ‘straws in the wind’ which suggested how elements of effectiveness might be developed: ‘we should treat people as people, not as objects to be coercively manipulated for our convenience ... ’ (Raynor 1978, p. 423).
One major methodological problem in social work research seemed to be that we were trying to measure outcomes (‘what works’) when the inputs (social worker activities) were not properly described or defined, and the overall goals and purposes (‘what matters’) were unclear or taken for granted: ‘studies ... based not on rhetoric but on the careful description and evaluation of real situations, do increase our knowledge about the effectiveness of different kinds of social work activity. Global pronouncements, on the other hand, are misleading and pointless’ (Raynor 1979, p. 21); ‘without an awareness of... normative and interpretative dimensions of the social worker’s task and the conceptual equipment to consider them rationally... the empiricist approach to technical effectiveness is an insufficient guide’ (Raynor 1984a, p. 9). Some people misinterpreted these arguments as a rejection of empirical work; for their benefit perhaps, I have pointed out in a more recent issue of the same journal that both qualitative and quantitative research are vital in social work and social science: ‘Without qualitative research, there is not much social science; without measurement and comparison there is not much social science (Raynor and Vanstone 2015, p. 14). We cannot sensibly ask ‘does it work?’ until we know what it is for.
By the mid-1980s my practical work in probation and victim support had largely ceased as I found myself responsible for running and developing what was initially a struggling Applied Social Studies department in the University. The search for resources, for staff, for student numbers and for continuing accreditation from the Central Council for Education and Training in Social Work (CCETSW, now deservedly abolished) took up a large amount of time, including the challenge of engaging with foggy CCETSW concepts like ‘transfer of learning’ and ‘partnership’. However, I was able to continue research through my contacts with practice, including the Afan Alternative to Custody in West Glamorgan, the Day Training Centre in Pontypridd (directed at that time by Maurice Vanstone) and a number of Youth Justice (or, in those days, Juvenile Justice) projects in which I studied diversion and impacts on local sentencing. My central concern was still whether probation and other forms of supervision in the community could demonstrate enough effectiveness to displace prison as a routine response to persistent offending by enabling some positive change and at least a comparable degree of public protection. Essentially these remained my research topics through several decades and over 200 publications. This chapter is not the place to try to summarise a large body of work, but instead I pick out a few examples which I think throw some light on the nature and uses of evidence in and around probation.
During the 1980s I was able to do some further work on the purposes of probation service activities (described in more recent writing as ‘What Matters’ [Mair 2004]). A series of publications concerned SERs, first arguing that they helped sentencers to assess seriousness and were a vehicle for making offers to the Court about what someone could do to turn his or her life around (Raynor 1980). A later study (Gelsthorpe and Raynor 1995) found that reports, if well argued and well presented, did actually have an impact on sentencing in the Crown Courts, making non-custodial sentences more likely. We also developed checklists and training aids to try to make this happen more often (Raynor et al. 1995). (By way of postscript, a small recent study found that since the 1990s reports have become more negative in tone and more concerned with risk and punishment [Gelsthorpe et al. 2010].)
On the question of probation’s purposes, the government in 1984 published a ‘Statement of National Objectives and Priorities’ for probation services (Home; Office 1984) which represented a new level of central prescription. I had an opportunity to comment on the draft version (grandly entitled ‘National Purpose and Objectives’). Although this document contained a number of policies it did not really articulate a core purpose for probation or its place in the criminal justice system. My suggestions about purpose, to fill this gap, were ‘(a) to reduce reliance on coercive solutions to criminal justice problems; (b) to increase and facilitate active participation by offenders, victims and the community both in criminal justice decision-making and in the implementation of decisions’ (Raynor 1984b, p. 46). These proposals, modest as they were, might have provided a yardstick against which to evaluate increasingly diverse probation service activities, and were consistent both with diversion from custody and with engaging people who had been in trouble to take part in constructive activities to improve their lives and make a positive contribution to the community. Later some of these ideas were brought together in a book (Raynor 1985) which had some impact at the time, though now it seems a distinctly uneven piece of work. However, later developments suggest that it was right to commend Joel Fischer’s suggestions of possible routes to more effective social work practice. Fischer, who by this time was a professor of social work in Hawaii, had written one of the best summaries of research which questioned the efficacy of social work (Fischer 1976) and then offered suggestions for improvement (Fischer 1978), highlighting evidence which pointed to the usefulness of core counselling skills (such as empathy, warmth and genuineness, later to be included among ‘core correctional practices’ - see below), time-limited problem-solving approaches and behavioural approaches. This anticipated much of what later became known as ‘What Works’.
Later in the decade I was able to complete a full evaluation of an ‘enhanced probation’ project in West Glamorgan based on group attendance requirements in a probation order, which succeeded in delivering both reductions in custodial sentencing and reduced re-offending, as well as leading to a reduction in the problems and difficulties reported by project members. (This was also the first research in which I used a desktop computer, an IBM ‘portable’ which I remember was about the size of a suitcase, with a five-inch screen.) The report of this study (Raynor 1988) was noticed in Scotland, where it was included in Gill McIvor’s ground-breaking research review (McIvor 1990) but did not seem to attract much attention in England and Wales except from Mollie Samuels, one of the most well-informed Probation Inspectors. I suggested that the effectiveness of the project was largely due to:
... clear gatekeeping to ensure concentration on the intended target group; reasonably effective referral systems to ensure consideration of the project as an option in appropriate cases; clear contracts with project members, including informed consent and a recognition of clients’ obligations and responsibilities in the context of the order; a disciplined framework; a high level of involvement with Courts; high levels of client contact; methods which were demanding and evoked the personal involvement of clients in work on real problems; the social work skills of staff; the support of the Home Office and senior probation management, and the role of the local management committee... (Raynor 1988, p. 172).
By then I felt I was beginning to see how some of the questions with which I began this chapter could be addressed. However, after enjoying reasonable government funding and support during Margaret Thatcher’s supremacy, probably because it was seen as part of the Law and Order budget and as contributing to economies in the criminal justice system through diversion from custody, probation came under attack in 1993 from a new home secretary, Michael Howard. He declared that prison worked, that diversion from custody was no longer an objective and that probation officers should not be trained as or with social workers. If this was intended to transform the culture of the Service it was largely unsuccessful, but it did seem to change the culture of some of the Service’s managers, who no longer talked about ‘alternatives to custody’. The Service’s successes in diversion were no longer measured and, unsurprisingly, actual diversion among adults largely ceased: from 1993 onwards the prison population and the probation caseload grew while the use of fines declined (Raynor 2012). In a review of the annual volume of Probation Statistics I drew attention to the way ‘alternatives to custody’ had become Oldspeak, in spite of significant achievements, without any obvious Newspeak replacement: ‘Orwell fans will remember that the whole purpose of Newspeak was to make history disappear... has there been a decisive shift from promoting non-custodial sentencing as a preferred option at the lower end of the custodial range, to marketing community sentences wherever a market can be found?’ (Raynor 1998, p. 183).
Although the move away from diversion was arguably a mistake, it did have some positive consequences. One was that it focused minds on demonstrating that the probation service could reduce offending.
Attention was drawn by, among others, the editors of this volume to research which was being carried out in Canada on the characteristics of effective work to reduce re-offending (see, for example, Andrews et al. 1990; Ross et al. 1988; and a key work in the British dissemination of these ideas, McGuire 1995). A major focus was on helping people to learn the skills and the ways of thinking which would help them to avoid trouble and lead more satisfying lives, if they wished to make this change. There was strong empirical endorsement of cognitive-behavioural approaches, and the Probation Inspectorate, led by the late Graham Smith, made it clear that probation services were expected to adopt demonstrably effective practices with a focus on reducing re-offending. The Inspectorate’s survey of the extent of effective and evaluated projects in probation services, in which I provided some assistance, produced disappointing results (Underdown 1998): out of 267 projects claimed to be effective by probation managers only four were soundly evaluated and shown to be effective. Wisely delaying the publication of this survey until after the election of a new Labour government in 1997, Graham Smith and his colleagues launched the Effective Practice Initiative, later known as What Works, in an attempt to transform the effectiveness of probation practice.
In the meantime I had had the opportunity, together with Maurice Vanstone, to evaluate an early example of a cognitive-behavioural group programme in probation, with quite encouraging results (Raynor and Vanstone 1996) which were also noted in the Inspectorate’s report. This was the pioneering STOP programme, Straight Thinking On Probation, started by David Sutton, then chief probation officer of Mid Glamorgan. There was a tendency to seize on the positive results without looking at the detail of what went into the project: for example, we emphasised the need for detailed and patient preparation and consultation to embed the underlying ideas in practice culture (this took many months); we pointed out that effectiveness was not simply about the group programme but about the whole experience of probation including case management, individual supervision and support; and we took a strong interest in informed consent, and in participants’ accounts of what they learned (Raynor and Vanstone 1997, 2002). Unfortunately the central directors of the ‘What Works’ project felt they had to go much faster: the main development funding, provided through the Crime Reduction Programme, was only available for 3 years and there was a great deal of pressure to show results quickly. One result was that many practitioners felt that a new way of working was being imposed on them by diktat, some of the Pathfinder programmes established as part of the ‘What Works’ initiative were not well run and many of the wrong people were allocated to programmes in an attempt to hit target numbers:
... researchers noted a large number of implementation difficulties: projects were often not running in a fully developed form when the evidence... was collected ... projects tended to make a slow start and not to achieve their target numbers... the top-down management style... alienated parts of the workforce... little attention was paid to the need for effective case management.’ (Raynor 2004, pp. 317-319)
Evidence alone was not enough: attention was also needed to the human processes involved in its practical application.
Later I became involved in research on risk and need assessment, piloting it in England and Wales with Colin Roberts (Raynor et al. 2000; Raynor 2007) and in a series of studies of post-prison resettlement with Mike Maguire, Sam Lewis and others (see, for example, Maguire and Raynor 1997, 2006; Maguire et al. 2000; Lewis et al. 2003, 2007). Again the results tended to confirm emerging models of effective practice: the most effective resettlement projects combined attention to ex-prisoners’ pressing practical needs with work to influence attitudes, self-management and thinking. Regular contact with a supportive supervisor or mentor also helped. Further indications of how and why probation might be effective came from a study of Black and Asian probationers’ experiences (Calverley et al. 2004; Lewis et al. 2006). A long-term research partnership with the Jersey Probation and After-Care Service, where standards are in some respects higher than on the mainland, led to a series of studies (e.g. Raynor and Miles 2007; Miles and Raynor 2014). Recently these have included a study, with Maurice Vanstone and Pamela Ugwudike, of the practice skills (‘core correctional practices’) used by probation staff. This quasi-experimental study has confirmed the effectiveness of appropriate practice skills: officers who were typically more skilful and resourceful in their individual supervision achieved better results and lower reconvictions: ‘... skills matter in probation work: when practice is skilful, reconvictions are reduced... One cost-effective route [towards reduced reoffending] might be to focus on developing staff skills... The lessons of the evidence-based approach might have been learned better without attempts by headline-hunting politicians to impose a more punitive culture’ (Raynor et al. 2014, pp. 245—247). Improving staff skills might in fact be a much more cost- effective approach to improvement than is offered by the current vogue for privatisation.
By this time I felt I had found at least the beginning of answers to several of the questions I had set out with 40 years earlier. However, it was also clear that many developments and changes in probation were not evidence driven, and that the application of evidence to policy and practice was far from straightforward. Finding evidence is not in itself the answer: persuading people that it is in their interests to pay attention to it is another challenge, and the nature of this challenge, and the uses and meanings of evidence, change over time. Before the 1990s the limited research available had little impact on policy and practice in England and Wales (though an early attempt by McGuire and Priestley 1985 to promote evidence-based practice was widely read). The Home Office had largely given up on the search for effectiveness and showed limited interest in research from overseas, or even from Scotland. This changed when probation’s search for ‘what works’ coincided with the dissemination (often by psychologists) of new research on effective practice. The New Labour government elected in 1997 promised evidence-based policy-making and was prepared to invest in the ‘Pathfinder’ experiments although, as described above, the time scale was too short and the implementation too uneven to deliver the kind of results that probation’s leaders hoped for. The peak years for evidence-based development in probation lasted from 1997 to about 2003; by then, politicians who had learned to see evidence as a useful resource began to look for evidence to support preferred policies rather than choosing policies to fit the evidence. Outside criminal justice the most glaring example of this was the manufacturing of evidence to manipulate opinion in favour of the invasion of Iraq in 2003. Within criminal justice evidence has been used to support practice developments such as the accreditation of programmes, but major policy changes are based on little or no evidence, or on reports specially commissioned to support them. An early example was the abolition of consent to probation in 1997 (see Raynor 2014); more recent examples are the abolition of probation orders in 2000; the creation of the national Probation Service and marginalisation of the judiciary in 2001; the creation of the National Offender Management Service in 2004 following a report by a businessman (Carter 2003); the inclusion of a ‘punitive requirement’ in every community sentence in 2013, in line with an earlier report by a civil servant (Casey 2008); and finally, in the clearest example of evidence-free policy-making, the decision to break up and sell off the majority of the Probation Service to private providers in 2014—2015.
Pilot studies intended to inform these latest changes were discontinued in order to accelerate the process and complete it before the General Election of 2015 (just as the break-up and privatisation of rail services was accelerated before the election of 1997). The Justice Secretary who led the privatisation process, Christopher Grayling, is rumoured to have said ‘you don’t pilot a revolution’. The twenty-first century so far shows a progression from being guided by evidence, to using evidence as a resource to support policy decisions already made, to creating evidence to support policies and eventually to dispensing with evidence altogether. The erosion of the traditional basis of probation has been accelerated by a series of Home Secretaries and Justice Secretaries who seemed overconcerned with maintaining a tough image, from Michael Howard through to Jack Straw, David Blunkett, John Reid and finally Grayling. (Exceptions were Kenneth Clarke and, for a short while, the recent incumbent Michael Gove.) Some of these gentlemen may have felt that too much support for probation would compromise their tough image. As a result, the majority of probation staff trying to contribute to rehabilitation find themselves in a legislative and organisational environment which is not particularly helpful. In a recent article I compared the Ministry of Justice to the ancient allegory of the Ship of Fools, in which the quarrelsome and disorganised crew can never decide together how and where to sail: ‘Part of the tragedy of life aboard the ship is that not everyone on it is a fool, but those who have a good idea of the appropriate destination and how to get there cannot find enough others to agree with them, and their warnings about the rocks ahead are ignored’ (Raynor 2014, p. 304).