Archive for the ‘Uncategorized’ Category

DA VINCI DIDN’T PAINT BY NUMBERS: MAYO’S CRITIQUE OF THE BAYESIAN WAY

May 24, 2012

DA VINCI DIDN’T PAINT BY NUMBERS:
MAYO’S CRITIQUE OF THE BAYESIAN WAY
Sam George
6/15/99

PROLOGUE:

On what epistemological grounds do we make scientific truth claims? In Error and the Growth of Experimental Knowledge (all page numbers without an author’s name preceding them refer to this text), Deborah Mayo claims the appropriate focus is on small-scale statistical inferences grounded in actual experimental practice (p. 59). Rather than exclusively seeking experimental results that support or cast doubt on a theory-laden supposition, Mayo asserts along with the New Experimentalists that “experimental knowledge remains despite theory change” (p. 62). This puts her in direct opposition to the proponents of the Bayesian Way. The various branches of Bayesianism strongly assert a theoretical presumption that prior degrees of belief are modified by the evidence to yield posterior degrees of belief. Mayo concedes that statistical testing approach results can be reconstructed after the fact by the Bayesian approach, yet one of her basic theses is that the Bayesian approach is not and cannot be the method used by scientists who seek the growth of experimental knowledge. Even though masterpieces of painting, such as the Mona Lisa, can be reproduced by children following a paint-by-the-numbers kit, this does not mean that the original artist did or even could make his or her original contributions through such a method.

ACT I: MAYO ON THE BAYESIANS

Mayo began her pointed critique of Bayesianism with a contrast between evidential-relationship and testing approaches.

E-R approaches . . . commonly seek quantitative measures of the bearing of evidence on hypotheses. What I call testing approaches, in contrast, focus on finding general methods or procedures of testing with certain good properties. (p. 72)

A major difference between the two is that E-R approaches generally (although not exclusively) assign probabilities to hypotheses, while testing approaches do not. Objective Bayesians in the tradition of Rudolph Carnap attempted to deduce the prior probabilities from the logical structure of some first-order language (p. 73). Unfortunately, the first-order languages used were inadequate to articulate actual science. The majority of Bayesians, on the other hand, subscribe to subjective degrees of belief for prior probabilities. Mayo quotes Howson and Urbach,

we are under no obligation to legislate concerning the method people adopt for assigning prior probabilities. These are supposed merely to characterise their beliefs subject to the sole constraint of consistency with the probability calculus (p. 75).

Strict Bayesians must accept that different persons will have radically dissimilar opinions, both between people and also for the same person from time to time. The Bayesian Way provides for them the only rational method of fitting together prior and posterior degrees of belief via Bayes’s theorem. The argument goes along the lines that “if we are rational, we will be coherent in the Bayesian sense” (p. 76), e.g. avoiding Dutch Book arguments, which would insure that the gambler would lose money. Mayo quoted the Bayesian Leonard Savage as writing “that the theory of personal probability ‘is a code of consistency for the person applying it, not a system of predictions about the world around him’” (p. 76).
E-R approaches ultimately lead to subjectivism, according to Mayo. While some things, such as binomial probabilities, can be calculated in an objective fashion, other probabilities — such as the composition of matter by atoms — can never be rigorously objective. Were we in the context of multiple universes, it would make sense to talk about a given hypothesis, H, having a nontrivial probability in a given universe. A person without complete knowledge, on the other hand, could meaningfully assign a subjective degree of belief between 0 and 1 for H even in a model of a singular universe.
Bayes’s Theorem can be generalized from probability theory.

For an exhaustive set of disjoint hypotheses, H1, H2. . ., Hn, whose probabilities are not zero, and outcome e where P(e) > 0:

P(e|H1)P(H1)
P(H1|e) = _________________________________________

P(e|H1)P(H1) + P(e|H2)P(H2) + . . . + P(e|Hn)P(Hn)
(pp. 81-82).

In certain well-defined cases, such as Mayo’s contrived second-order game of playing rouge et noir OR single-number roulette determined by the flip of a coin, the theorem works well and non-Bayesians agree with Bayesians on the appropriateness of the application. The rub comes, of course, when there are not well-defined and objective prior probabilities, as will be the case the vast majority of the time. The common person (along with the statistically-minded individual) will want to know how well a Bayesian model corresponds to reality; to the Bayesians, however, the issue is coherence, not correspondence.
Mayo pointed out that the Bayesians often defend themselves with the assertion that the differences in prior probabilities make no difference in posterior probabilities with sufficient additional evidence. This is not a credible defense of the Bayesian Way for Mayo for at least the reasons that (1) some agents could assign zero probabilities and different agents could disagree among themselves as to the assignments of other degrees of belief, and (2) the Bayesian Way claims to be a good (according to Howson and Urbach THE ONLY good) way for evaluating non-statistical as well as statistical hypotheses. Howson and Urbach apparently also claim the whole point of (Bayesian) inductive reasoning is inference from premise(s) to conclusion (p. 85). Analogously to deductive reasoning, the point is not the truth of the premises–that much to them is irrelevant–but the form of the inference. According to Mayo, this move seems to be an attempt to subsume induction under deduction, making an ampliative deduction, and as such, this Bayesian assertion seems immediately suspect at the very best.
This is the heart of the controversy, according to Mayo: a basic difference in the aims of the Bayesians from the various non-Bayesians. Agreeing to disagree with the Bayesians over the goals of an ampliative induction removes much of the controversy between the two groups, she claims. Mainstream statistical reasoning fails by Bayesian standards, as does the Bayesian Way fail by normative statistical standards. Is there room here for compromise? “Conceding the limited scope of the Bayesian algorithm might free the Bayesian to concede that additional methods are needed, if only to fill out the Bayesian account” (p. 86). Unfortunately, the strictest Bayesians do not appear likely to concede their major thesis that “All You Need is Bayes” to quote Mayo’s (p. 88) paraphrasing the Beatles. Without an imperialist perspective, the Bayesian Way becomes one tool among a set of many other tools (which in my judgment is how it ought to be–more on this at the end of the paper). Bayesians of the Howson and Urbach variety wish to demote mainstream scientists as being somehow covert Bayesian agents to the extent that they are producing any useful results. Of course, this can only be maintained by the most dogmatic ideologue–mainstream science can and does get along quite nicely without subjective Bayesianism (p. 89). “The Bayesian model is neater [than error statistics methods], but it does not fit the actual procedure of inquiry” (p. 99).

ACT II: GEORGE ON MAYO ON THE BAYESIANS

The first question in evaluating Mayo’s critique of the Bayesians is whether or not she has represented them accurately. Some important points of Mayo’s inadequacy should be addressed, including Bayesian extremism and the direction of movement from prior to posterior degrees of belief.
While the various flavors of Bayesians would take different positions on the issue, the plain vanilla ones–such as Howson and Urbach–are uncompromising absolutists–their way or no way, “Bayes or Bust” (p. 84). Knowing what I now know about statistically-based reasoning, this assertion borders on the absurd. Surely someone can be a rational agent without following the Bayesian Way or even a way that could be approximated by Bayesianism. Yet just because the Bayesians stake an extremist claim, is that warranted grounds to dismiss their reasoning? Certainly not, for some truths are radical truths not admitting of compromise. As unlikely as it seems, it may indeed be the case that it is indeed a matter of Bayes or bust, and an implied ridicule of such a position is not helpful.
Mayo quotes Kyburg as saying,

for any body of evidence there are prior probabilities in a hypothesis H that, while nonextreme, will result in the two scientists having posterior probabilities in H that differ by as much as one wants (p. 84f).

This point is technically true, but misleading. Starting off, any two scientists could have such different prior degrees of belief that the posterior probabilities can be as arbitrarily far apart as one would like to make them, but the important thing to note is the direction of the movement. With additional evidence, the posterior probabilities will converge on the actual value. The difference will indeed wash out, it is not a matter of movement away from the actual value.
Mayo’s basic thesis, that the Bayesian Way does not reflect scientific practice, is well-taken. We do well to recall “considerations of probability have played an important part in the history of science, [but] until very recently, explicit probabilistic arguments for the confirmation of various theories, or probabilistic analyses of data, have been great rarities in the history of science” (Papineau, p. 292). The analogy comes to mind with recent developments in Kosovo: after NATO airstrikes apparently forced Serbian forces out of Kosovo, the Russian military has moved in, claiming critical strategic positions. Likewise, after centuries of painstaking progress, mainstream scientists find a noisy bunch of Bayesians laying claim (in this case) to the entire field of enterprise.
I find Bayesian imperialist claims to be obnoxious and overstated. That statistically-based evaluation can be reproduced after the fact by Bayesian algorithms does not demonstrate that the Bayesian Way is even equal to, much less superior to, statistics. Indeed, the Mona Lisa can be reproduced by a child using a paint-by-the-numbers kit, but no artist worthy of the title paints in such a fashion.
Mayo claimed the Bayesian model was neater (p. 99), but it seems to me that keeping track of successive generation of prior and posterior probabilities would become incredibly cumbersome, even in an age of computers. The theory certainly has an elegant simplicity to it, but in practice, it would be far too complex to execute. Again, the theory does not fit accepted scientific practice.

EPILOGUE: GEORGE ON THE BAYESIANS

Most generally, are people inclined to be Bayesians? If this were the natural inclination of human beings in evaluating logical inferences, such would be a powerful (albeit nonconclusive) argument in favor of Bayesianism. Apparently most Bayesians believe as much, such as “what we ourselves take to be correct inductive reasoning is Bayesian in character that there should be observable and sometimes systematic deviations from Bayesian precepts” (Howson and Urback, p. 423). This strikes mf belief in what I have said, but it seems reasonable to propose that most of the putatively rational agents in my classes will have at least a significant degree of belief in my words, yet I always have some students who attend class erratically at best and still claim that they expect to pass, even when they have failed every assignment all semester long. Hence it seems that Bayesianism is not normative among scientists and is not the exclusive course of action of all putatively rational individuals in everyday life.
Bayesians are dogmatic imperialists. This seems most convenient for the Bayesians; it sounds analogous to asking a person of great faith if there is a God. “Of course there is,” might be the initial reply. “What about the presence of evil?” might be the next question, to which the next answer might be, “Evil only exists to sort the faithful from the non-faithful.” In short, the person’s belief system is non-falsifiable–there is no evidence one could bring to such a person that would change his or her beliefs. The opportunities for a strict Bayesian to admit of error also seem diminishingly small. However, despite the epistemic circle of coherence within which the Bayesians confine themselves, the rest of the world does want to know if something is true, not simply if it is self-consistent. I am always disturbed when ideology takes precedence over correct reasoning. In this regard, the orthodox Bayesians are as obnoxious as any group of religious fundamentalists, which is rather ironic given the subject matter of the Bayesian faith. Yet, as Mayo pointed out, Bayesians are not speaking in the same categories as concern the rest of us–they are making puddings, while statisticians and other error-conscious reasoners are making entrees of various kinds. The problem is that the Bayesians insist that there is no need of anything else, that pudding is by itself sufficient, and that simply cannot be the case. Pudding has its proper place, but it does not have the nutritional value to support a human being by itself, and any person who tried to subsist on pudding alone would rapidly succumb to the diseases of malnutrition. Likewise, a pseudo-science confined to Bayesian orthodoxy would not permit the kind of explosion of knowledge humanity has witnessed over the past few hundred years–it simply is not the way science is or should be done.
As I said earlier, that one makes extreme claims does not rule out the truth of one’s position. However, for an extraordinary claim that purports to account for all science, the Bayesian Way must offer extraordinary proof of its position, and I fail to see as much. I believe Bayesianism could and should be retained as one tool among many. Mayo implies as much through the discussion on pages 86-88. I can imagine situations wherein a Bayesian approach could give useful information, even without completely objective prior degrees of belief. I did not spend much time watching the trial of O.J. Simpson, but it seems to me that the prosecution could have profitably cited Bayes’s theorem. Certainly not ALL abused women who are murdered are murdered by their husbands/boyfriends–perhaps police statistics would indicate that only 75% of that population were killed by their significant others–certainly not a powerful enough argument by itself to satisfy the American legal system’s standards for determining guilt beyond a reasonable doubt. That statistic, though, could credibly serve as a Bayesian prior belief and interact with the evidence of the blood tests (which also have well-documented statistical probabilities) to yield a posterior degree of belief that would be highly unfavorable for the defense. Of course, such would not prove Mr. Simpson’s guilt in any deductively valid fashion, but it would make for a strongly persuasive argument.
Generally, it seems to me, Bayesianism is at its strongest when dealing with the subjective decisions of human beings. Mayo only alluded to this briefly. However, it seems good to me as well that we bring to the surface all of our implicit reasoning and avail ourselves of all the tools at our disposal when making important decisions, and in this more limited sense, the Bayesian Way is a useful tool among others.

CITED WORKS

Howson, Colin and Urbach, Peter. Scientific Reasoning: The Bayesian Approach,
second edition. Open Court: Chicago and La Salle, IL, 1989, 1993.

Mayo, Deborah G. Error and the Growth of Experimental Knowledge.
The University of Chicago Press: Chicago and London, 1996.

Papineau, David, ed. The Philosophy of Science. Oxford University Press: New York,
1996.

PARABLE OF AMERICA (an allegory about gay marriage)

December 21, 2009

            Once upon a time, there was a small village in our land; the village was named “America.”  Because the village was so small, there was only one church, America Baptist Church.

            One fine day at choir practice, Steve asked why all the baptisms at ABC were by full immersion.  “Surely some people are too scared of the water to go all the way under,” he said.  The choir members suggested Steve go to the Board of Deacons.

            Unfortunately, the deacons were not pleased with Steve’s question.  They said only deviant people even considered such things.  Baptisms were always by full immersion and that was the way it had always been.  In truth, one of the deacons had been sprinkled as a child in another village, but it was a secret he was determined to take to his grave.

            “I don’t want to be fully immersed, I want to be sprinkled,” declared Steve.  “No one else wants to be sprinkled,” replied the deacons, “so why must you be so difficult?”  Steve started asking the people at ABC and discovered about 10% agreed with his views on baptism and other things, too.

            Steve and his friends decided if they did not fit in with the way things were done at ABC, they would form their own church, to be named America Methodist Church.  There was an old, abandoned community center downtown that had been an eyesore for many years, but Steve and his friends thought with some loving renovation it would make a splendid home for the AMC.

            Steve asked the Town Council if he could buy the community center from the town so that AMC could renovate it and make it their beautiful new sanctuary.  Sadly, though, all of the members of the Town Council were also members of ABC; they declared that Steve did not have the right to submit a bid on the property.

            Steve and his friends were upset they had been denied their opportunity.  Leslie suggested talking to the American Civil Liberties Union, which they did.  With the help of the ACLU, Steve and Leslie and the other prospective members of the AMC sued the Town Council for the right to purchase the property.  Although Steve was no longer alone, he was being called “unpatriotic,” “ungodly,” and many other unkind things.  Some people would no longer talk to him in public and he started receiving hate mail.

            During the trial, people said many things.  The chair of the deacons claimed that it would weaken the institution of baptism at ABC if Methodists were allowed to sprinkle.  Leslie answered that the folks of ABC were quite welcome to continue to practice baptism as they saw fit, but the folks of AMC should be entitled to do it their own way.  The judge listened carefully to what both sides said and eventually ruled that Methodists were entitled to the same rights as every other person in America and that Steve and Leslie and the others had the right to bid on the abandoned community center.

            Elections for Town Council were held not long after the judge issued her ruling.  Steve decided he would run for election on a pro-AMC platform.  However, the Mayor declared he thought that America should have only one church and proposed that the Town Council pass a “Protection of Baptism” act that would designate America as a sprinkling-free zone and turn the old community center into a Sanctity of Baptism Memorial.  Many people who personally agreed with baptism by immersion thought that the Mayor and Town Council were going too far in trying to contradict the judge and restrict the rights of the Methodists.  However, 56% of the people who went to the polls agreed with the Mayor, and to this day, America has only one church.

Whither Truth?

January 25, 2009

While the intuitive meaning of truth seems obvious, contemporary philosophers and other scholars have no broadly-accepted definition. The epistemological (epistemology: the nature of knowledge) stakes are high and have gone in many interesting directions. In illustrating some of the dilemmas, permit me self-consciously to mix categories to show a couple of more extreme options with fideism, a theory of truth, vs. postmodernism, a cross-disciplinary perspective. The careful observer will see both of these at work in contemporary Western societies.

Fideism (from Wikipedia):
Alvin Plantinga defines “fideism” as “the exclusive or basic reliance upon faith alone, accompanied by a consequent disparagement of reason and utilized especially in the pursuit of philosophical or religious truth.” The fideist therefore “urges reliance on faith rather than reason, in matters philosophical and religious,” and therefore may go on to disparage the claims of reason. The fideist seeks truth, above all: and affirms that reason cannot achieve certain kinds of truth, which must instead be accepted only by faith. Plantinga’s definition might be revised to say that what the fideist objects to is not so much “reason” per se — it seems excessive to call Blaise Pascal anti-rational — but evidentialism: the notion that no belief should be held unless it is supported by evidence.
The fideist would claim we ascend the mountain of human knowledge, but even having ascended to the top, there are ultimate truths and realities that are still unobtainable by rationality, so one must make the leap of faith into the arms of God. A current expression of this heard in church circles would be to “let go and let God.” There is no rational (or as per Plantiga above, evidential) reason to believe that there is a god to catch the aspirant, but because she believes, she moves into places beyond the pale of the merely rational.

Postmodernism (also Wikipedia):
Postmodern philosophy is skeptical or nihilistic toward many of the values and assumptions of philosophy that derive from modernity, such as humanity having an essence which distinguishes humans from animals, or the assumption that one form of government is demonstrably better than another…. Postmodern philosophy is often particularly skeptical about simple binary oppositions characteristic of structuralism, emphasizing the problem of the philosopher cleanly distinguishing knowledge from ignorance, social progress from reversion, dominance from submission, and presence from absence.
Most varieties of post-modernism despair that there may be no absolutes and everything is relative, or if there are absolutes, they are unknowable. Note that when pushed to the extreme, believing that that can be no absolutes, particularly absolute truth, this becomes a non-rational faith perspective that is self-contradictory: how can we know absolutely there is no absolute truth when there IS no absolute truth from which to learn such?

Is all of this just “the talk on a cereal box”? Interestingly, many postmodernists would agree, even as their hermeneutic (interpretive scheme) rests on a deep, philosophical foundation. And, too, the fideists would claim the only meaningful truths are revealed by God, so rational and evidential inquiry is ultimately fruitless. Even if some of the fideists were right, the truths they arrive at contradict the truths of other fideists coming out of different faith perspectives, unless one wants to claim that fideist knowledge is absolutely subjective, in which case it is not truth in a widely-accepted sense. Likewise, it seems to me that the radical postmodernist gives up without trying and ultimately lapses into nihilism and hedonism. I would argue aesthetically (so again, with a non-rational motivation) that the pursuit of truth is a beautiful, noble, and worthy thing; life is a journey and not a destination.