Keynes on frequentism
In spite of their differences, Keynes agreed with Peirce that the belief probability judgements one can reach by balancing reasons cannot tell the entire story of how evidence is assessed:
The magnitude of the probability of an argument . . . depends upon a balance between what may be termed the favourable and the unfavourable evidence; a new piece of evidence which leaves the balance unchanged, also leaves the probability of the argument unchanged. But it seems that there may be another respect in which some kind of quantitative comparison between arguments is possible. This comparison turns upon a balance, not between the favourable and the unfavourable evidence, but between the absolute amounts of relevant knowledge and of relevant ignorance respectively.
(TP, p. 77)
As the relevant evidence at our disposal increases, the magnitude of the probability of the argument may either decrease or increase, according as the new knowledge strengthens the unfavorable or the favorable evidence; but something seems to have increased in either case—we have a more substantial basis upon which to rest our conclusion. I express this by saying that an accession of new evidence increases the weight of an argument. New evidence will sometimes decrease the probability of an argument, but it will always increase its "weight."
(TP, p. 77)
Despite the terminological difference between Peirce who used "weight of evidence" in the sense of net weight or balance of argument, and Keynes who, in the passage cited, was using "weight" in the sense of gross weight, Peirce and Keynes were in agreement that balance of argument or probability alone could not characterize all important aspects of evidential appraisal.
That is where the agreement ended. Peirce insisted that numerically determinate belief probability judgements be grounded or supported by knowledge of statistical or physical probability. Such statistical knowledge itself may be imprecise and subject to some kind of random error. It is that random error that Peirce thought could be used to characterize some of the other aspects of evidential appraisal.
In discussing the frequency interpretation of probability, Keynes complained about the narrowness of Venn's approach, which makes no claims about how to ground belief probabilities in statistical probabilities or long-run frequencies.2
Keynes did consider a view of the frequency theory different from Venn's, which he attributed tentatively to Karl Pearson (TP, p. 109). On this view, as it is according to Peirce's view, knowledge of statistical probability is used to ground judgements of belief probability. Keynes appreciated, as did Peirce, that, according to this version of the frequency view, the problem of choosing a reference class (or kind of trial) becomes critical.
According to both Peirce and the Keynesian reconstruction of the frequency view, the reference class used in direct inference should contain all relevant information about the specific experiment (see Keynes, TP, p. 113).
For Peirce, the judgement of relevance is a judgement of statistical relevance—that is, information about statistical or physical probabilities. This is information X must know. It is, in this sense, grounded in fact.
According to Keynes, the judgement of relevance is not a judgement of fact or grounded in fact but is a direct judgement about belief probabilities. Indeed, Keynes claimed that when belief probability can be "measured" by a known "truth-frequency," the same result can be obtained by a proper use of his Principle of Indifference—that is, insufficient reason. I believe that Keynes had in mind here a thesis suggestive of Bruno de Finetti's use of symmetry conditions on probability judgements to relate belief probabilities of single states to frequencies in reference classes to which they belong. De Finetti thought that he could replace allusion to physical or statistical probability in all contexts where it seemed useful to replace judgements of statistical probability by judgements of subjective or belief probability. It appears that Keynes did also.
In many cases, information available to the inquirer may not be known to be either statistically irrelevant or statistically relevant, so that Peirce's approach offers no clear guidance as to how to draw conclusions about the outcomes of some kind of trial of a certain kind even when the chances of outcomes of that kind on such a trial are known. We may be convinced that a cab from the city is involved in an accident and 85 per cent of the city's cabs are yellow and the remainder blue without being able to judge via direct inference that the belief probability that the cab in the accident is yellow is 0.85. We may not know whether 85 per cent of cabs in the city involved in accidents are yellow. That is to say, we may not know what percentage of cabs in the city involved in accidents are yellow and, hence, we may be ignorant of the stochastic relevance or irrelevance of the information that the cab in question was involved in an accident. In that case, Peirce himself insisted in 1867 that credal probability judgement goes indeterminate. Ignoring the "base rate" information given is no fallacy.
Keynes's criticism of Venn's frequency view carries over to Peirce. The applicability of Peirce's theory is severely limited. However, Keynes should not have made this objection since he himself conceded that credal probability can be indeterminate.
Keynes could have insisted that indeterminacy would be intolerably widespread if one insisted on Peirce's demands. One needs to be in a position to make moderately determinate judgements of belief probability without grounding in objective or statistical chance—at least for the purpose of assessing relevance.
This is not the place to elaborate on the controversy. I shall say only that I am inclined to think that Peirce's position is overly demanding. This point is not sufficient to undermine the importance of frequentist, statistical, or physical probability as Keynes seems to suggest. It does mean that we cannot restrict the use of determinate or relatively determinate belief probabilities to those grounded in knowledge of statistical probabilities.
Return now to the question of the weight of argument. In his 1878 paper and even more emphatically in 1883 Peirce proposed an account of how to make estimates from data about frequencies without appeal to Bayes' theorem that are probabilistically reliable. In this way, Peirce was able to avoid the use of prior belief probabilities while using data as "input" into the procedure without using it as evidence or premises of his "inference" and violating his strictures on direct inference. The method is essentially the method of confidence interval estimation of Neyman-Pearson (see Levi, 1980, ch. 17).
Such a method of estimation can be associated with a measure of its accuracy determined by the standard deviation, as we noted Peirce proposed to do. Keynes recognized the possibility of using measures of dispersion of a probability distribution as measures of weight of argument. In many contexts the variance of posterior distribution of a certain parameter decreases with an increase of information on which the posterior is based. But as Keynes illustrated this need not be so (see TP,
pp. 80-2). Examples of dispersion increasing with more evidence can be constructed.