Results Validity Stage
For the validity stage, data was gathered from a new sample deemed to be representative of the scale’s target population, in order to evaluate the consistency of the results between studies and to re-examine scale properties. During reliability stage, a split sample was used for scale development. While one half of the sample served as the primary development sample, results were cross-checked with the holdout sample which was used to replicate the findings. As discussed, one can infer with a certain degree of comfort that the scale properties are not distorted by chance, as the results of this stage were quite promising. However, additional new data had to be collected to avoid shortcomings due to scale development out of a single sample and to comply with scientific standards (DeVellis, 2012; MacKenzie et al., 2011).
Insights from samples 3a and 3b could be distorted due to a number of reasons. First, the two sub-samples which were divided randomly from the entire reliability sample very likely represent a homogeneous but very specific population (i.e., Austrian university students). Whereas this population has some very appealing characteristics (e.g., high familiarity with the trust object), it is desirable to compare the scale properties to a different sample which is more representative for the online consumer population in the two German speaking countries. Second, the data collection period for the two sub-samples was not separated by time and it can be assumed that any special condition under which data was collected in the reliability stage may equally apply for both samples. In order to rule out such circumstances restricting scale generalizability and internal validity (e.g., history effects), the collection of new data sets was deemed reasonable.
Particularly, two different samples were used for scale finalization. The first one (sample 4; n = 526) was taken from an online panel and provided data for conduct EFA as well as additional item analyses as precursors to confirmatory factor analysis, and to re-estimate the model by using CFA to finalize and confirm the hypothesized structure of the higher-order construct over multiple data sets. A second sample (samples 5(a-e); n = 824) consisting of pooled data from a series of interviews among (Austrian) online consumers served cross-validation purposes. Together with additional data, this sample enabled this research to exercise supplemental validity and reliability testings, establish norms, and to apply generalizability theory.