Testing the Integrity of Formal Data Collected by Supervisors
When a supervisor is conducting the majority of observations of participant activity involvement, he/she may gradually drift from the original guidelines when scoring meaningful activities as discussed in Chapter 5, Assessing Meaningful Versus Nonmeaningful Task
Participation. In particular, over the course of conducting numerous monitoring sessions, the supervisor may begin to score certain activities as meaningful when the activities do not conform to the guidelines for meaningful activities. Subtle changes in how activities are scored can happen without the supervisor being aware that he/she is drifting from the guidelines but nevertheless compromise the integrity of the formal data. As a guard against this potential drift, the clinician should conduct interobserver agreement checks by occasionally observing with the supervisor as also discussed in Chapter 5, Assessing Meaningful Versus Nonmeaningful Task Participation. By comparing the scoring of activities between the two observers, any disagreements in how activities are scored can be resolved by referring to the guidelines for meaningful activities.
The act of the clinician periodically observing with the supervisor can also serve other useful functions. For example, the clinician can monitor and help maintain the supervisor’s competence in providing feedback to staff following the monitoring sessions. The clinician can also obtain first-hand information about how well respective staff are providing meaningful activities. This can provide the clinician with opportunities to offer ideas for improvement where necessary that the supervisor can then relay to the staff.