Descriptive statistics (norms) for the final scale

You define your final scale by listing only those items with satisfactory item-total correlation and obtain Cronbach’s alpha (or other reliability coefficient), the number of cases, mean, range and variance of the final scale, and of any subscales.

Some would prefer to divide the scale mean by the number of items in the scale, to give an item mean. The advantage of this is that if the number of items in the scale is ever changed at a later date, then comparisons involving the item mean would still make some sense.

If you have decided to use item weightings, then answers to each item must be multiplied by the relevant weighting before the mean score on the scale is determined.

The production of standardised scores is beyond the scope of this book, and Kline (1999) or Urbina (2014) may be followed if these are desired.

Summary of steps for data entry and reliability

• 1. Code data where needed.
• 2. Enter data.
• 3. Check data.

• 4. Recode where necessary (for ‘reverse-meaning’ items).
• 5. Save data file, making at least one backup copy.
• 6. Compute the reliability coefficient (usually Cronbach’s alpha) for the scale, and for any subscales. If satisfactory, you can go straight to stage 9.
• 7. If this is unsatisfactory (less than 0.70-0.80) examine item-total correlations (scale cohesiveness).
• 8. Remove items with unsatisfactory item-total correlations.
• 9. Produce descriptive statistics (norms) for the final scale, containing only those items with satisfactory item-total correlations: number of cases, mean, range and variance (or standard deviation), and reliability coefficients.

If this fails to produce a scale with a satisfactory reliability coefficient, you could attempt a factor (principal components) analysis, as described later and in Appendix 5.6.

If ever describing any scale with dubious reliability, you should state clearly that the scale does not meet criteria for reliability.

Factor and principal-components analyses

If you are developing a scale for professional use or major research purposes, some psychometricians would regard factor analysis as an important or even essential procedure in the construction of psychological scales and tests. If, however, you have obtained satisfactory reliability or you are not applying the scale in a major way, or both, you could skip this bit. As discussed earlier in this chapter, you might consider this analysis as a way of identifying possible subscales, if your measure has poor internal consistency.

The underlying theory is a specialist aspect of statistics, and there is a good deal of controversy surrounding the applications of factor analysis. Chapter 1 refers to relevant discussions.

For those taking first steps in scale construction, you could consider using factor analysis if you thought there were several factors in your scale, but you were not sure which items were contributing to them. Exploratory factor analysis will tell you the answers to these questions, and so, too will principal components analysis. Chapter 1 gives a brief description of factor and principal components analyses and explains the difference between them, advising you to be clear about which you are doing - they are sometimes confused with each other, and they will give similar results. It is suggested that principal components analysis may give clearer answers than factor analysis. Appendix 5.6 offers brief instructions for carrying out factor or principal components analysis.

If you do a factor (or principal components) analysis, you need to examine which items load heavily on each factor. Common criteria are as follows:

• • Normally, loadings of 0.4 and above, or
• • Loadings of 0.3 and higher where there are few or no high loadings, for example, and where it would make sense in naming the factors to include items with slightly lower loadings

Name the factor according to the items that load heavily on it. To do this, make a list of high-loading items, and look for a common feature. Note that negative loadings indicate that the item is negatively associated with the factor.

You do not have to accept and use all the factors. A rough guide to deciding which factors to accept is as follows:

• • The first one to four factors will probably account for quite a lot of variance each, and after that there may be a sudden drop. A likely type of scenario would be for factor 1 to account for about 13%, factor 2 about 8%, and then factors 3 and 4 to account only for a mere 3% or 4% each. Look for such a drop, and use it as a cut-off point in accepting which factors to use. SPSS will normally drop factors that account for insufficient variance.
• • Only accept factors accounting for, say, about 8% or more of the variance. Thus, in this example, only consider using factors 1 and 2.
• • Only accept factors that make sense in terms of the constructs you are assessing. Of course, a reasonable amount of variance should be accounted for. This means that if a factor accounts for a lot of variance, and it does not make sense or is not of interest to you - then you do not have to use it.

If you do a factor analysis, you may want to consider whether to use item weightings. Weighting the items means that an item with high weighting contributes more to the score than an item with low weighting. The main advantage of weighting is that it could give a more sensitive scale, possibly with improved validity. The main disadvantage is that calculation of scale totals and norming data is more complicated and time consuming. Responses to each item have to be multiplied by the item weighting (of course this is normally done by the computer). You could certainly consider using weighted scores if you find the results of your validity analyses are disappointing. There are several ways of developing item weightings: factor loadings can be used as item weightings, or the factor

Coding, database, and analysis 79 analysis facility on a statistics package will calculate factor scores (consult your statistics package manual if necessary). If you are thinking of using item weightings, a more advanced manual on test construction should be consulted.

A simpler application of factor analysis, which has some of the advantages of item weighting without the disadvantages, is to use the factor loadings as criteria for retaining items in the scale. Decide whether to use a loading of 0.3 or 0.4 as a cut-off, and accept only items with this loading or higher in your scale. This may improve the sensitivity and reliability of your scale, without the extra computational labour of item weightings. However, unless you had an undiscovered subscale structure embedded in your scale, which only factor analysis could reveal, this method is unlikely to give better results than a reliability (item) analysis.

If the results of factor analysis look useful, and you need more information than is provided in this book, you could consult Field (2017) for a more detailed guide on doing factor analysis. Tabachnick and Fidell (2013) present a much more detailed account of factor analysis, including a comparison of relevant statistics packages.