The Weight of Argument
The somewhat novel question
In Chapter 6 of A Treatise on Probability (1973 , hereafter TP), John Maynard Keynes raised a question he described as "somewhat novel." This concerned the problem he labeled "The weight of arguments."
Keynes claimed that the magnitude of the probability of an argument depends upon "a balance between what may be termed the favourable and the unfavourable evidence" (TP, p. 77). In contrast to this, Keynes introduced a notion of weighing the amount of available evidence.
Peirce on balancing reasons
Charles Peirce had exploited the metaphors of balancing and weighing evidence in 1878 in his "Probability of Induction." Unlike Keynes, however, he did not contrast "balance" and "weight" but used them more or less interchangeably.
Peirce devoted considerable space in that essay to examining the "conceptualist" view of probability, according to which probability "is the degree of belief that ought to attach to a proposition" (Peirce, 1878, p. 291).
According to conceptualists, degree of belief can be measured by a "thermometer" whose reading is proportional to the "weight of evidence" (Peirce, 1878, p. 294). To explain this, Peirce focused on cases where one is assessing evidence for and against a hypothesis H and where each datum is probabilistically independent of every other relative to information prior to data collection. He derived the probability that two items testify in favor of H conditional on both items agreeing in testifying either for or against H. He then defined the "chance of H" as what is now called the "odds for H given E" or ratio of P(H/E)/P(~H/E) where E describes the data. He took the conceptualist practice of invoking insufficient reason to be claiming that the prior odds P(H)/P(~H) = 1 and with that took the "final odds" (that is, the odds for H given E) to be equal to the "likelihood ratio" P(E/H)/P(E/~H). He then argued that the logarithm of this ratio is a suitable "thermometer" for measuring degrees of belief. This logarithm is equal to log P(E/H) — log P(E/~H). E is a reason for H if this value is positive. E is a reason against H if the value is negative. Peirce also added:
But there is another consideration that must, if admitted, fix us to this choice for our thermometer. It is that our belief ought to be proportional to the weight of evidence, in this sense, that two arguments which are entirely independent, neither weakening nor strengthening each other, ought, when they concur, to produce a belief equal to the sum of the intensities of belief which either would produce separately (Peirce, 1878, p. 294).
In spite of Irving John Good's allegations to the contrary (Good, 1981), Peirce's characterization of the "independence" of two arguments concurring in their support for or against the hypothesis H is perfectly correct. If E = E1&E2 and independence here means that P(E/H) = P(E1/H) P(E2/H) and P(E/~H) = P(E1/~H)P(E2/~H), the logarithms of the products become sums. The likelihood ratio becomes a sum of two differences logP(E1/H) — logP(E1/~H) and logP(E2/H) — logP(E2/~H). If the two differences concur—that is, show the same sign—Peirce's contention is that the Fechnerian intensities of belief ought to be the sum. The "weight" of the support has increased. If the two differences are of different signs so that one bit of evidence supports H and the other undermines it, a "balancing of reasons" assesses the overall support (Peirce, 1878, p. 294). Peirce does not say so in so many words but we may call the result an assessment of the "net weight" of the reasons or evidence.
Peirce explicitly associated the balancing of reasons with the con- ceptualist view of probability that he opposed. He took the balancing reasons procedure to be a way of presenting the best case for conceptualism. Peirce pointed out that the approach presupposes a dubious use of the principle of insufficient reason to identify "prior" probabilities for use with Bayes's theorem to obtain posterior probabilities and final odds.
Peirce insisted that the best case is not good enough. Peirce's writings are strewn with diverse arguments attacking the conceptualist view.
Some of the attacks are against the principle of insufficient reason. Some of them are attacks on conceptualism even when it dispenses with insufficient reason and takes a more radically subjectivist turn. Peirce wrote:
But probability, to have any value at all, must express a fact. It is,
therefore, a thing to be inferred from evidence.
(Peirce, 1878, p. 295)
There is some evidence to think that neither Peirce nor John Venn (whom Peirce clearly admired) always interpreted judgements of probability to be judgements of fact—that is, relative frequency or physical probability. Peirce was plainly concerned with issues where a decision maker is faced with a momentous decision in the here and now without the prospect of referring the outcome to a long-run series of outcomes. Judgements of probability used in the evaluation of prospects facing the decision maker are not beliefs about relative frequencies or physical probabilities.
According to Peirce, the probability judgements the inquirer is prepared to make in a given context of choice should be derived from the information the inquirer has concerning long-run relative frequencies or physical probabilities in situations of the kind the decision maker takes himself or herself to be addressing. Judgements of numerically determinate belief probability are worthless as a warrant for assessing risks (they have no value at all as Peirce puts his point) unless they "express" a fact. As I understand Peirce, the judgements of numerically determinate belief probability are derived in accordance with principles of "probabilistic syllogism," "statistical syllogism," or, as later authors might put it, "direct inference" from information about objective statistical probabilities of outcomes of an experiment of some kind. That an experiment of type S is also of type T is legitimately ignored if the statistical probabilities of outcomes of trials satisfying both S and T are known (or fully believed) to be equal to the statistical probabilities of those outcomes on trials of kind S. (The extra information that the trial is of kind T is known to be "statistically irrelevant.") If the inquirer knows the extra information to be statistically relevant or does not know it to be irrelevant, it may not be ignored even if this means that no determinate belief probability may be assigned to a hypothesis about the outcome of experiment.
Appealing to insufficient reason to assign probabilities is a way of deriving belief probability judgements that fail to "express" a fact.
For Peirce such derivation would be unacceptable. His opposition to conceptualism was opposition not so much to belief probabilities per se but to the appeal to considerations such as insufficient reason to ground the assessment of belief probabilities.
Peirce offered an illustration of his point of view and in that setting mounted an argument against the "proceeding of balancing reasons."
A bean is taken from a large bag of beans and hidden under a thimble. A probability judgement is formed of the color of the bean by observing the colors of beans sampled from the bag at random with replacement. Peirce considered three cases: (i) two beans are sampled with replacement where one is black and one is white; (ii) ten beans are sampled where four, five, or six are white; and (iii) 1000 are sampled and approximately 500 are black.
The conceptualist (known to us as Laplacian or Bayesian) invokes insufficient reason to assign equal prior probability to each of the n + 1 hypotheses as to the number r of white beans in the bag of n beans. Bayes's theorem yields a "posterior distribution" over the n + 1 hypotheses. One can then obtain an estimate of the average number r of white beans in the bag of n that is equal to the probability on the data that the bean under the thimble is white. Peirce found this derivation acceptable in those situations where the inquirer could derive the prior probability distribution from knowledge of statistical probabilities. To achieve this in every case, the inquirer would have to assume absurdly that worlds are as plentiful as blackberries.
The alternative favored by Peirce is to consider a rule of inference that specifies the estimate to make of the relative frequency of whites in the bag for each possible outcome of sampling. Indeed, if the rule specifies that the estimate of the relative frequency of whites in the bag falls in an interval within к standard deviations from the observed relative frequency of whites in the sample, it can be known prior to finding out the result of sampling what the statistical probability of obtaining a correct estimate will be. So, in making a judgement about the color of the ball under the thimble, we can no longer take the excess of favorable over unfavorable data as a thermometer measuring degree of belief. Indeed, no single number will do.
In short, to express the proper state of our belief, not one number but two are requisite, the first depending on the inferred probability, the second on the amount of knowledge on which that probability is based (see Peirce, 1878, p. 295).
In a footnote to this passage Peirce made it clear that "amount of knowledge" is tied to the probable error of the estimate of the "inferred probability." Indeed, Peirce suggested that an infinite series of numbers may be needed. We might need the probable error of the probable error, and so on.
Thus, the "amount of knowledge" or what I am calling the "gross weight of evidence," according to Peirce, cannot be accounted for on the conceptualist view but can be on the view that insists that belief probabilities be derivable via direct inference from statistical probability.