N400 amplitude results

For modeling N400, we have even more combinations at our disposal, since we can combine corpus-based measures with the baseline, the predictability performance and with both. We evaluate on all 343 data points for N400 amplitude fitting. Without using corpus-based predictors, the baseline predicts a mere 0.032 of variance, predictability alone explains 0.192 of variance and their combination explains 0.193 of variance - i.e. the baseline is almost entirely subsumed by CCP-based predictability. As can be observed from Table 10.3, this is a score that is not yet reached by the language models, even when combining all of them.

Table 10.3. r² explained variance of the N400 for various combinations of the corpus-based predictors, in combination with the baseline, and with the empirical predictability

When comparing the performance of the computationally defined predictors, a picture similar to the prediction of the empirical predictability emerges. The n-gram model scores best, particularly for the larger NEWS and WIKI corpora. This confirms a generally accepted hypothesis that larger training data trumps smaller, more focused training data, see e.g. [BAN 01] and others. The n-gram model is, however, immediately followed by the neural model, and again, the topic predictor provides the poorest performance in explaining N400 amplitude variance, which suggests that the N400 does not reflect long-range semantic processes. The best combination without predictability, with a score of r² = 0.167, approaches the performance of the predictability and baseline (see Figure 10.2).

Figure 10.2. Prediction models exemplified for the NEWS corpus in the x-axes and the N = 334 mean N400 amplitudes on the y-axes. A) Prediction by baseline + n-gram (r² = 0.161), and B) a standard approach to N400 data, consisting of the baseline of position and frequency, as well as the empirical predictability (r² = 0.193; e.g. [DAM 06]). Fisher's r-to-z tests revealed no significant differences in explained variance (P = 0.55). For a color version of this figure, see www.iste.co.uk/ sharp/cognitive. zip

The experiments with predictability as an additional predictor confirm the results from the previous section: n-grams + baseline and predictability capture slightly different aspects of human reading performance, thus their combination explains up to 6% more net variance than predictability alone.