Summary

This chapter describes how to get a final, reliable scale from the pool of items that your participants have responded to. This involved coding your questionnaire, making a database and using a reliability analysis to see how strongly each item, in turn, relates to all the other items in the scale (or subscale). Those that do not relate well are removed, and the result should be a consistent, reliable scale.

Factor and principal components analysis are briefly outlined, as is item weighting.

Appendix 5: Reliability

Notes on statistics packages

The specific instructions for computing given in these appendices are for IBM SPSS for Windows users. Full instructions are given for carrying out the necessary operations to enter and check data, and to compute and improve reliability.

IBM SPSS for Windows is frequently updated. As of this writing, SPSS 26 is the most modern, but this will almost certainly not be the case by the time you read this. Fortunately the changes made with each new edition of SPSS generally consist of the addition of new statistical tests. The general menus and output formats remain constant, except for a change informatting in SPSS 7.5 and later versions. This means that these instructions should be appropriate for any later editions of SPSS.

Loewenthal (1996) gives instructions for calculating reliability under circumstances without a suitable statistics package.

Other statistics packages for reliability analysis include

• • SYSTAT
• • SAS/STAT
• • Analyse-it

Appendix 5.1: Types of data, coding, and statistics

You need to be aware what kind of number scale you are using when you code your data. The type of number scale will affect the choice of statistics that are appropriate. Consult the guide that follows (and, if necessary, Chapter 5), or consult a suitable book (such as Coolican, 2019, de Vaus, 2013, or Gravetter & Wallnau, 2016) or an appropriate person if you feel unsure on any of the points.

• • Interval data: Age and other numerical data (such as number of children, years of education and the like) can be entered straight onto the computer. It is called interval data because you know the size of the interval between two numbers - the difference between three and two is the same as the difference between two and one. If this sounds like a strange thing to be concerned about, be assured that it is important. It is quite likely that some of the data you deal with will not be interval data: other types of data are described later, and in Table A5.1. Interval data are suitable for parametric statistical analysis including analysis of variance (ANOVA), Pearson correlation, t-test, and multiple regression analysis.
• • Ordered (or ordinal) data: Here the data may look like numerical data, but there is a difference. With ordinal data you can only be sure that one number is greater than another, but you cannot be sure by how much. Supposing three brands of chocolate are ranked in order of preference, with brand X first (rank = 1), brand Y second (rank = 2) and brand Z third (rank = 3). We know that 1, 2 and 3 are in order of magnitude, but we do not know anything about the size of the intervals between them. Brand X may be liked very much, much better than brand Y. However, brand Y is only a bit nicer than brand Z, but this is not reflected in rank-order data. Ordinal data may be used in non-parametric statistics, suitable for ordinal data, such

Table A5.1 Different types of number scales: Interval, ordinal and nominal

Abbreviation: ANOVA, analysis of variance.

as Mann-Whitney U test, Kendall’s tau, Kruskal-Wallis one-way ANOVA and Jonkheere’s trend test. In some cases it may be useful to treat the data as categorical, particiularly as dichotomous (see later), but never treat it as interval data. Taking the most likely scenarios, you can use ordinal data in the non-parametric tests of association listed previously. Ordinal data may also be used in cross-tabs and chi-squares, though you may have to collapse some categories to get a result that can be interpreted. Ordinal data should not be used in tests suitable only for parametric data, such as ANOVA, or most other forms of multivariate analysis.

Nominal (or categorical) data: Here, you do not even know that one number is greater or less than another number. Numbers are just used to indicate category differences. In Table A5.1 there are three categories for food taste: mainly sweet, mainly spicy and bland. If you are going to code these, you should represent them by numbers (because computer statistical packages do not like and cannot normally use alphabetical data). You can see that the numbers 1, 2 and 3 are not ordered. It can be very advantageous to have just two categories, rather than more than two. This is called dichotomous data. Statistics and interpretation become much clearer and simpler, and important statistics are possible with dichotomous data. Notably, statistics packages can run correlations and reliability statistics on dichotomous data. The techniques are called, respectively, point biserial correlation, and the Kuder-Richardson KR-20 reliability coefficient. It is also allowable to include variables involving dichotomous data as independent variables in a multiple regression analysis. Unlike categorical data with more than two categories, dichotomous data are ordered. The moral of all this is to try and reduce categorical data to dichotomies, where it is sensible to do so. You do not have to do this when you code the data; you can make lots of categories and then collapse them into two when you are computing (see the instructions for recoding in Appendix 5.3).

• Rating scales: What about ratings? Are they interval or ordinal scales? If I rate my liking for chocolate on a 5-point scale, and brand X gets a 5, brand Y gets a 4, and brand Z gets a 3, is the difference between the 5 and the 4 the same as the difference between the 4 and the 3? Generally, rating scales may be treated as interval scales, unless you suspect the contrary. Likert (1932) suggested this when he developed his best-selling method of scale construction involving rating. Likert’s (cunning) suggestion was that the intervals in such rating scales should be equal-appearing.