{"id":535,"date":"2012-10-03T11:00:43","date_gmt":"2012-10-03T11:00:43","guid":{"rendered":"https:\/\/skepticink.com\/humesapprentice\/?p=535"},"modified":"2013-05-06T16:58:10","modified_gmt":"2013-05-06T16:58:10","slug":"proving-darwin-math-and-tomatoes","status":"publish","type":"post","link":"https:\/\/skepticink.com\/humesapprentice\/2012\/10\/03\/proving-darwin-math-and-tomatoes\/","title":{"rendered":"Proving Darwin: Math and Tomatoes"},"content":{"rendered":"

Bayes\u2019 Theorem: A Gentle Introduction<\/em><\/p>\n

To illustrate Bayesian reasoning I\u2019ll use a couple of common-sense examples. Suppose that your socks go missing. Why did they go missing? There are many possible explanations, but let\u2019s just consider two. The first explanation is that you dropped your socks as you were leaving your local Laundromat. The second explanation is that evil sock-stealing gnomes broke into your home and took the socks without leaving any signs of entry or taking anything else. Either hypothesis would explain the fact that your socks went missing. However, it is clear that one is more likely than the other. Why is that? It is because you already know from your own observations that you could have easily dropped your socks on the way out of the Laundromat. You have probably lost things because you carelessly dropped them a hundred times in the past. On the other hand, you have no evidence that any pair socks that have ever gone missing were stolen by evil sock-stealing gnomes. So, even though both theories explain the evidence (the fact that your socks went missing) equally well, you reject the evil gnome hypothesis because it is inherently<\/em> less likely than the hypothesis that you were careless. A case of missing socks is not usually the result of gnomes, which is to say that it is inherently unlikely or that it has what Bayesians call a low prior probability<\/em>.<\/p>\n

Let\u2019s take another example: suppose that you reach into a purse that you know contains an even mix of ordinary coins and trick coins that always land on \u201cheads.\u201d You pull out a coin. You flip it five times and find that it always comes up heads. Did you get a trick coin? Probably so. Even though it is possible<\/em> that your coin is an ordinary one (sometimes people flip a normal coin and it lands heads five times in a row) it isn\u2019t likely. You are justified in believing your coin is a trick coin because the evidence you have, which in this case is your observation of the coin-toss outcomes, is far more likely if you got a trick coin than if you got an ordinary coin. In other words, the evidential probability<\/em> of the trick coin hypothesis is greater than the evidential probability of the ordinary coin hypothesis.<\/p>\n

In the first scenario, the evidential probability of both theories were equal, and so to find the right answer we only had to figure out which theory had the greater prior probability. In the second example the tables were turned; The prior probability was the same for getting a trick coin as it was for getting a regular coin, but the evidence (seeing the coin consistently turn up heads after five flips) was more likely if it was a trick coin. Obviously the real world is rarely this ideal. Sometimes a theory explains a few pieces of evidence better than its competitors, but that does not mean that it is necessarily more likely. It could be the case that other evidence is predicted so poorly by the theory (and so much better by one or more competing theories) that it is not, on final consideration, the most likely theory. In reality, judging a theory as the most likely out of many contenders requires looking at the prior<\/em> and evidential <\/em>probabilities of all our theories and carefully comparing them to see which one has the greatest final probability<\/em> (that is, the probability that a theory is correct after all factors are considered). The equation that allows us to do this correctly is Bayes\u2019 Theorem.<\/p>\n

Bayes\u2019 Theorem can be stated as the following:<\/p>\n

P(h|b)\u00a0 x\u00a0\u00a0 P(e|h&b)<\/p>\n

_________________________________________<\/p>\n

[ P(h|b)\u00a0 x P(e|h&b) ]\u00a0 +\u00a0 [ P(~h|b)\u00a0 x P(e|~h&b) ]<\/p>\n

 <\/p>\n

Despite appearances this equation is actually very easy to understand.<\/p>\n

The term \u201cP(h|b)\u201d stands for the prior probability and it means \u201cthe probability of the hypothesis we are examining given our background knowledge.\u201d Our background knowledge is everything we know about the world excluding the evidence we are currently investigating. Remember the missing sock example I gave earlier? When I said that the evil-gnome hypothesis was inherently unlikely, that\u2019s just another way of saying that the the probability of the hypothesis, given only our background knowledge (everything else we already know about the world), is very low. We know that it\u2019s low because we have never encountered evil-gnomes stealing socks, and so the frequency of that type of event is either zero or extraordinarily low.<\/p>\n

The term \u201cP(e|h&b)\u201d means \u201cthe probability that we would have the evidence that we do if our hypothesis is true<\/em>, given our background knowledge.\u201d Think back to the trick coin example I gave earlier: we know that an ordinary coin lands on heads about fifty-percent of the time. This fact is part of our background knowledge, and it helps us find the probability that our outcome will be observed if the \u201cordinary coin\u201d hypothesis is correct.<\/p>\n

Bayes\u2019 Theorem simply says that you should multiply the two numbers I just discussed and then divide the product of that multiplication by itself plus the product of the prior and evidential probability of all alternative hypotheses (the term ~h stands for an alternative hypothesis in our other two terms, which are otherwise the same as the two terms we discussed previously).<\/p>\n

Let\u2019s try one more example. Suppose that you have two neighbors, Mrs. Smith and Mrs. Jones, who grow tomatoes. Mrs. Smith\u2019s tomato garden is fifty percent red and fifty percent green. Mrs. Jones\u2019 garden is one-hundred percent green tomatoes. If you find a green tomato in your backyard, how likely is it that it came from Mrs. Jones\u2019 garden? Since all Mrs. Jones\u2019 tomatoes are green while only half of Mrs. Smith\u2019s tomatoes are, the odds are two out of three. Think about it: if you randomly took four tomatoes from each garden (making eight total) then all four tomatoes you took from Mrs. Jones\u2019 garden would be green (because that\u2019s all she grows) while only two of the four tomatoes you took from Mrs. Smith would be green. To help you decide the probability, you could throw away the two red tomatoes you got from Mrs. Smith, because you already know your tomato is green, and just focus on the green tomatoes. If you had eight tomatoes and you tossed out the two red ones, you\u2019d be left with six green tomatoes. Again, four of those six green tomatoes came from Mrs. Jones. Four out of Six, or 4\/6, is the same as \u201ctwo out of three\u201d (2\/3). The probability that it came from Mrs. Smith is one out of three.<\/em><\/p>\n

Now let\u2019s mix things up: pretend that you now notice that the tomato you found in your yard is what you call \u201ca big \u2018un.\u201d Let\u2019s pretend that one out of every three tomatoes in Mrs. Jones\u2019 garden is a big \u2018un, while all of Mrs. Smith\u2019s tomatoes are big \u2018uns (\u201cMust be that new fertilizer,\u201d you say to yourself). We\u2019ll treat this new observation as \u201cevidence.\u201d If the tomato came from Mrs. Smith\u2019s garden, then the probability of the evidence, that is, the probability that the tomato would be a big \u2018un, is one hundred percent. If it came from Mrs. Jones garden, then the probability is one out of three, or 33 percent. Remember, this isn\u2019t the probability that your<\/em> tomato would come from one garden instead of the other, these are just the probabilities that your tomato would be a big \u2018un if it came from one or the other. In each case, we are talking about the probability of the evidence if our hypothesis is true given our background knowledge<\/em>. In this case, our background knowledge is that the tomatoes from Mrs. Smith\u2019s garden are always big \u2018uns while tomatoes from Mrs. Jones\u2019 garden are big \u2018uns thirty-three percent of the time.<\/p>\n

What is the likelihood that your tomato came from Mrs. Smith\u2019s garden? If we want to know the overall probability of the hypothesis that the tomato came from Mrs. Smith, we need to use Bayes Theorem, and to use Bayes Theorem, we need to know the prior and evidential probabilities of that hypothesis and the hypothesis that it came from Mrs. Jones. How do we find the prior probability? Since the prior probability is the probability of a hypothesis given only our \u201cbackground knowledge\u201d (not our evidence), and since we have said that we are using the size of the tomato (\u201cIt\u2019s a big \u2018un!\u201d) as our evidence, what we need to know is the probability that the tomato came from Mrs. Smith before we took into account the evidence. Take a look at the paragraph before last to find the answer: one out of three, which can be written as the decimal number .33. What is the probability of the evidence if the tomato came from Mrs. Smith? Take a look at the last paragraph: 100 percent (or 1). The prior and evidential probabilities for the Mrs. Jones hypothesis are two out of three, or .67, and one out of three, or .33, respectively, and these numbers both come from the last two paragraphs.<\/p>\n

Bayes\u2019 Theorem dictates that in order to know the overall probability of the Mrs. Smith hypothesis, we must take the prior and evidential probabilities of that hypothesis and multiply them, then divide that number times itself and the multiplication of the prior and evidential probabilities of the Mrs. Jones hypothesis. Let\u2019s take this one step at a time: What is the prior probability of the Mrs. Smith hypothesis? One out of three. What is its evidential probability? One. Multiplying those two numbers together equals one out of three. Let\u2019s put that number over to the side for the moment. What is the prior probability of the Mrs. Jones hypothesis? Two out of three. What is its evidential probability? One out of three. What do we get when we multiply those two numbers? Two out of nine, or the decimal .22. What do we do now? We take the number .33 (the product of multiplying the prior and evidential probabilities of the Mrs. Smith hypothesis) and divide it by itself (.33) plus the product of multiplying the prior and evidential probabilities of the Mrs. Jones hypothesis (.22). So we divide .33 by .55. The answer we end up with is .6, which means that the probability that your tomato came from Mrs. Smith\u2019s garden is sixty percent. We can visualize this entire procedure and know that the number here is correct. Imagine that we picked six random tomatoes from each garden. As I\u2019ve described above, here\u2019s what that collection would look like:<\/p>\n

MRS. SMITH \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 MRS. JONES<\/p>\n

\"\"<\/a>\"\"<\/a>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \"\"<\/a>\"\"<\/a><\/p>\n

\"\"<\/a>\"\"<\/a>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \"\"<\/a>\"\"<\/a><\/p>\n

\"\"<\/a> \"\"<\/a>\u00a0 \u00a0\u00a0\u00a0 \u00a0 \u00a0 \u00a0 \u00a0\u00a0 \u00a0\u00a0 \u00a0 \"\"<\/a>\"\"<\/a><\/p>\n

Since we already know our tomato is green, we can erase all the red ones and just focus on the green:<\/p>\n

MRS. SMITH\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 MRS. JONES<\/p>\n

\"\"<\/a>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \"\"<\/a>\"\"<\/a><\/p>\n

\"\"<\/a>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \"\"<\/a>\"\"<\/a><\/p>\n

\"\"<\/a>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \"\"<\/a>\"\"<\/a><\/p>\n

One out of every three of Mrs. Jones\u2019 tomatoes are \u201cbig \u2018uns\u201d and all of Mrs. Smith\u2019s tomatoes are \u201cbig \u2018uns,\u201d so we let\u2019s update the picture to show that:<\/p>\n

MRS. SMITH\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 MRS. JONES<\/p>\n

\"\"<\/a>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \"\"<\/a>\"\"<\/a><\/p>\n

\"\"<\/a>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0 \u00a0\u00a0\u00a0\u00a0 \"\"<\/a>\"\"<\/a><\/p>\n

\"\"<\/a>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \u00a0 \u00a0 \u00a0 \u00a0\u00a0 \u00a0 \u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \"\"<\/a>\"\"<\/a><\/p>\n

 <\/p>\n

Since we know our tomato is big, we can erase all the ones which are not, since we only care about the frequency of large tomatoes:<\/p>\n

MRS. SMITH\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 MRS. JONES<\/p>\n

\"\"<\/a>\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0\u00a0 \"\"<\/a>\"\"<\/a><\/p>\n

\"\"<\/a><\/p>\n

\"\"<\/a><\/p>\n

Look at what the last illustration shows: we have five large tomatoes, and 3 out of 5 (or sixty percent) of those come from Mrs. Smith\u2019s garden, which is the same answer we got with Bayes\u2019 Theorem. That\u2019s why we know Bayes\u2019 Theorem is right: it is nothing but a way to describe what is self-evidently true. It takes the numbers and percentages that human beings know (the prior and evidential probabilities of a hypothesis) and helps us realize what follows from those numbers.<\/p>\n

End Note: I should add that this example assumes for the sake of argument that there is no correlation between the color and size of a tomato. That is, a red tomato is no more likely to be a “big ‘un” than a green tomato, and vice versa.<\/p>\n","protected":false},"excerpt":{"rendered":"

Bayes\u2019 Theorem: A Gentle Introduction To illustrate Bayesian reasoning I\u2019ll use a couple of common-sense examples. Suppose that your socks…<\/p>\n","protected":false},"author":11,"featured_media":0,"comment_status":"open","ping_status":"open","sticky":false,"template":"","format":"standard","meta":{"footnotes":""},"categories":[17],"tags":[131,31,43,135],"class_list":["post-535","post","type-post","status-publish","format-standard","hentry","category-bayes-theorem","tag-bayes","tag-darwin","tag-evolution","tag-tomato"],"_links":{"self":[{"href":"https:\/\/skepticink.com\/humesapprentice\/wp-json\/wp\/v2\/posts\/535","targetHints":{"allow":["GET"]}}],"collection":[{"href":"https:\/\/skepticink.com\/humesapprentice\/wp-json\/wp\/v2\/posts"}],"about":[{"href":"https:\/\/skepticink.com\/humesapprentice\/wp-json\/wp\/v2\/types\/post"}],"author":[{"embeddable":true,"href":"https:\/\/skepticink.com\/humesapprentice\/wp-json\/wp\/v2\/users\/11"}],"replies":[{"embeddable":true,"href":"https:\/\/skepticink.com\/humesapprentice\/wp-json\/wp\/v2\/comments?post=535"}],"version-history":[{"count":0,"href":"https:\/\/skepticink.com\/humesapprentice\/wp-json\/wp\/v2\/posts\/535\/revisions"}],"wp:attachment":[{"href":"https:\/\/skepticink.com\/humesapprentice\/wp-json\/wp\/v2\/media?parent=535"}],"wp:term":[{"taxonomy":"category","embeddable":true,"href":"https:\/\/skepticink.com\/humesapprentice\/wp-json\/wp\/v2\/categories?post=535"},{"taxonomy":"post_tag","embeddable":true,"href":"https:\/\/skepticink.com\/humesapprentice\/wp-json\/wp\/v2\/tags?post=535"}],"curies":[{"name":"wp","href":"https:\/\/api.w.org\/{rel}","templated":true}]}}