Step 3: Conduct reflective observation

Following on from the ‘feeling’-based observational work of Step 2, Step 3 involves a more structured observation process that can be broken down into three stages.

Carry out a formal consultation

The care process analysis, shadowing, and discovery interviews should have yielded a substantial amount of information from various sources. To make sense of this information, grouping findings into common themes can be valuable, identifying which patterns emerge from multiple sources and looking for how the views of the different groups (e.g. staff, patients, relatives, or members of the public) both converge and differ. Although Step 2 was inherently exploratory and not intended to be deeply analysed, there is still a general rule that if a problem seems to occur multiple times from multiple sources, it is less likely that it is a rogue view and more likely to be a common opinion affecting at least a proportion of the target group.

The ideas that begin to surface from Step 2 can be more thoroughly tested through a formal consultation. The aim of the formal consultation is threefold:

• 1. To gather more comprehensive data about the section of the care process and the target group that have been selected to have a ‘benchmark’ of what the current situation is like
• 2. To test whether problems identified in Step 2 are commonly felt by a wider sample of the target group and are indeed commonly perceived problems (rather than individual views)
• 3. To gather additional open feedback that could have a bearing on how an intervention is designed

Unlike discovery interviews, the formal consultation should aim to involve larger numbers of participants. The exact number will depend on the care process and the target group involved. For a busy hospital waiting area that sees 200 people a day, for example, it may be advisable to collect at least 100 consultation forms across a range of days (e.g. weekdays and weekends) and times of day (e.g. morning, evening or night). However, for a nursing home that has only 20 residents and 10 staff, it might be that collecting responses from two-thirds of these provides enough information to get a good overview of opinion. If a low volume of forms is needed, it may be possible for somebody to hand these to respondents individually. For a high volume, or for forms that need distributing on both day and night shifts, it may be possible to add them to a routine already in place. For example, if all visitors to a waiting area are given a registration form on arrival, the receptionist may be able to hand over a consultation form at the same time. When visitors return their registration form, they can return the consultation form too, meaning that forms are distributed without additional staff time being involved. Alternatively, if there is a bank of email addresses of people who attend a particular community centre, say, it could be that the form can be emailed to them as an online survey for people to complete in their own time.

To gather information on the three themes, a formal consultation questionnaire could contain the following types of questions:

• 1. Demographic questions—It is important to ascertain when and by whom consultation forms are completed. Asking the date and time of day is advised, as well as asking for people’s age and gender along with other demographic factors that are likely to stratify responses. However, to preserve anonymity, the form should allow questions to be optional and not ask too many personal questions that could lead to a participant being identified. It may also be helpful to know how many times a person has experienced the section of the care process under focus, such as how many times they have visited the waiting area or how long they have had a particular condition and been visiting a clinic for a particular treatment as this may affect their responses. As a guide, demographic questions often take up around 10% of a consultation form
• 2. Baseline data—Another important element of a consultation is gathering information on the current situation. This is helpful for a number of reasons, including (i) illuminating more about the initial problem to help understand it, (ii) providing additional evidence that the current situation is inadequate to support buy-in or funding for the planned intervention, (iii) providing a baseline against which the impact of an arts intervention can be measured. Depending on the area of interest, this might be identifying people’s overall view on a hospital waiting room environment, or assessing how supported people feel during a treatment programme. It might also involve constructing a set of ‘standards’: targets that the part of the patient’s care process should aim for, such as ‘create a waiting environment that 90% of people feel is calming’ or ‘have every patient undergoing a particular treatment feeling moderately or very well supported in their care’. Some organizations such as hospitals may already have their own set of standards that can be used here. Alternatively, standards could be established in consultation with patients and staff during discovery interviews. Testing whether standards are met in this consultation can demonstrate if there is a shortfall in the current situation and show to people involved what the arts intervention is aiming to achieve. Consultation questions can be numeric or text-based. For baseline data, numeric answers will provide the easiest benchmarking information but text-based answers will illuminate more about a problem, so both are advisable. As a guide, baseline data could take up around 30% of a consultation form

Brace’s Questionnaire Design: How to Plan, Structure and Write Survey Material for Effective Market Research (2008) provides a good introduction to designing questionnaires for market research such as this.(7)

3. Test the validity of identified themes—The core part of the consultation form should be testing whether the themes that emerged from the small sample involved in the discovery interviews are representative of the larger population and also to tease out more about these themes or clarify issues surrounding them. This should involve around 50% of the consultation form. The themes tested should include the initial problems, to see whether other people agree that they are problems, and some of the proposed solutions that may have been suggested in the discovery interviews, or that may already be being considered for an arts intervention. Results can then be analysed to give the average value or most common response. A common problem in testing themes is inadvertently biasing responses, such as pushing agreement with a particular ‘solution’. There are a number of ways to avoid this, including phrasing questions in a neutral way, providing a range of answers so people highlight the ones they think are best, or providing sliding scales of agreement (such as ‘Likert’ scales).

Matell and Jacoby discuss Likert scales in more detail in an article from 1971 entitled ‘Is there an optimal number of alternatives for Likert scale items?’.(8).

4. Invite open responses—The consultation form can end with an invitation for more opinions on an issue. This can extend the work of the discovery interviews. However, unlike discovery interviews, it is not possible to ask people more questions based on their answers, and people are also less likely to spend time explaining their thoughts properly. So to provide useful data, as well as providing truly open-ended responses such as ‘do you have any comments’, open-ended questions can be more targeted, such as asking people if there are any other problems with the part of the care process being explored, or if they have specific solutions to help with one of the identified problems

Consultations may need to be slightly different for staff, patients, and relatives, but as much as possible the questions should be kept similar so that results can be compared between groups. And as with discovery interviews, it is important to canvas the views of a wide range of people, including patients of different ages and staff in different roles and levels of seniority.

In analysing consultation forms, descriptive statistics should be calculated, such as the current levels of ‘standards’ and the percentage of people agreeing with the identified themes. Graphics such as bar charts are helpful for quickly understanding the data. It is also useful to examine a diagram of the demographics of participants who were involved, to check whether they were representative of the people being targeted with the intervention (such as the right age range). If responses have been gathered from lots of people, it may also be worth exploring whether there were differences in response between certain groups, such as men and women or older and younger people. For open-ended responses, it can help to group feedback into themes. Although this is just a planning stage, care should be taken over the interpretation of the consultation data as this could make the difference between a pleasant arts activity and a really well-targeted and high-impact intervention.