Procedure

Our first idea on how to compute the stimulus given the responses was to look at the associations of the responses, and to determine their intersection. However, in preliminary experiments, we found out that this does not work well. The reason appears to be asymmetry in word association. But what do we mean by asymmetry in this context?

The co-occurrence counts that we extract from the corpus are symmetric, because whenever word A co-occurs with word B, word B also co-occurs with word A. Whether an association matrix computed from the cooccurrence matrix is also symmetric depends on the association measure used. However, even in the case of symmetric weights, associations can still be asymmetric. Let us illustrate this using Figure 4.2. This is the graphical equivalent of a symmetric association matrix^[1]. As can be seen, the strongest association with blue is black. However, the opposite is not true: the strongest association with black is not blue as black has an even stronger association with white.

Figure 4.2. Associative lexical network with symmetric weights

To give an idea about the situation in the EAT: not considering multiword units, the EAT comprises 8,210 stimulus words and likewise 8,210 primary responses. However, there is not a complete overlap between these two vocabularies: only 7,387 words occur in both, which means that, only for these words, symmetry considerations are possible. Of these 7,387 cases, 63% of the responses were symmetric, and 37% were asymmetric. Table 4.3 shows some examples from the EAT for both types of associations.

Table 4.3. Examples for symmetric and asymmetric associations (PR = primary response)

Let us now return to our above example, namely circus, nose, funny ^ clown. Here, circus and clown are an example for the symmetric case. Both are each other’s primary associative responses in the EAT, and therefore circus is the strongest association with clown, and likewise clown is the strongest association with circus. If this were always true, things would be straightforward. However, this is not the case. For example, clown is strongly associated with nose, but nose is not strongly associated with clown. In the EAT, among 97 test persons, given the stimulus word nose, none responded with clown. Likewise, given the word funny, among 98 test persons, again nobody answered with clown. Therefore, if we take the intersection of the associations with circus, nose, and funny, clown would be out. This is why an approach based on intersecting associations does not work well.

Instead, like in word sense disambiguation and like in multiword semantics, it appears that we have to take contextual information into account^[2]. For example, in the context of circus, nose is clearly related to clown, but, in the context of doctor, it is not.

Such considerations resulted in the following approach: we utilize the observation that a stimulus word must have strong weights to all of its top associations, and that a strong association with only some of them does not suffice. Such a behavior can usually be put into practice by using a multiplication.

However, we do not multiply the association strengths, as the log- likelihood ratio has an inappropriate (exponential) value characteristic. This value characteristic has the effect that a weak association with one of the stimuli can easily be overcompensated by a strong association with another stimulus, which is not desirable. Instead of multiplying the association strengths, we therefore multiply their ranks. This improves the results considerably.

These considerations lead us to the following procedure (see [RAP 08]): given an association matrix of vocabulary V containing the log-likelihood ratios between all possible pairs of words, to compute the stimulus word causing the responses a, b, c, ..., the following steps are conducted:

• 1) For each word in V (by considering its association vector), look up the ranks of words a, b, c, ... in its association vector, and compute the product of these ranks (“Product-of-Ranks algorithm”).
• 2) Sort the words in V according to these products, with the sort order such that the lowest value obtains the top rank (i.e. conduct a reverse sort).

Note that this procedure is somewhat time consuming as these computations are required for each word in a large vocabulary^[3]. On the plus side, the procedure is in principle applicable to any number of given words, and with an increasing number of given words, there is only a slight increase in computational load.

A minor issue is the assignment of ranks to words that have identical log- likelihood scores, especially in the frequent case of zero co-occurrence counts. In such cases, the assignment of almost arbitrary ranks within such a group of words could adversely affect the results. We therefore suggest assigning corrected ranks, which are to be chosen as the average ranks of all words with identical log-likelihood scores.

In principle, the algorithm can also be used if there is only a single given word. However, this does not make much sense as the algorithm is computationally far more expensive than what we described in section 4.2, and the results are typically worse for the reason that ranks do not allow as fine-grained distinctions as do association strengths. For example, given the word white, the algorithm might find several words in the vocabulary where white is on rank 1 (e.g. black and snow). However, as (without further sophistication) no distinction is made between these, they will end up in arbitrary order, without taking into account that the top rank of black is more salient than that of snow^[4].

On the other hand, if the number of given words becomes larger, depending on the application, it can be helpful to introduce a limit to the maximum rank, thereby reducing the effects of statistical variation, which is especially severe for the lower ranks. Note that for the current work, we used a rank limit of 10,000. However, the exact value is not critical because this usually has little impact if the focus is mainly on the top ranks, as is the case here.

• [1] In the asymmetric case, we would require two directed connections of (usually) differentwidths between each pair of nodes.
• [2] Further reflections on this may lead to the fundamental question whether asymmetry ofword associations is the consequence of word ambiguity, or whether word ambiguity is theconsequence of asymmetry of word associations.
• [3] Considerable time savings are possible by using an index of the non-zero co-occurrences.
• [4] In the EAT, 57 and 40 subjects, respectively, responded with white for these two stimuluswords; another example is lily, where the primary associative response is also white, but isproduced by only 19 subjects. In this case, the next frequent response, namely flower, is veryclose as it is produced by 17 subjects.