{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# 贝叶斯公式的应用\n", "\n", "## 红球与白球\n", "我们先设想一个问题：\n", "\n", "假设有两只碗:\n", "\n", ">碗 1 装着 30 颗红球和 10 颗白球\\\n", ">碗 2 装着 20 颗红球和 20 颗白球\\\n", ">请问，随机挑一只碗，然后随机抽一个球，假如抽到的是一颗红球，那这颗红球来自碗一的概率是多少？\n", "\n", "最简单也最直接的想法是，我只看最后结果，你抽到的是红球，那这颗红球来自碗一的概率是 $\\frac{30}{30 + 20} = \\frac{3}{5}$.\n", "\n", "当然，我希望大家可以用贝叶斯公式来计算：\n", "\n", "$$\\begin{aligned} P(碗_1 | 红球) &= \\frac{P(碗_1)P(红球 | 碗_1)}{P(红球)} & = \\frac{\\frac{1}{2} \\cdot \\frac{3}{4}}{\\frac{5}{8}} &= \\frac{3}{5} \\end{aligned}$$\n", "\n", "贝叶斯公式普遍的情形是这样：\n", "\n", "$$P(h|d) = \\frac{P(h) P(d|h)}{P(d)}$$\n", "\n", "其中 $h$ 表示 hypothesis，$d$ 表示数据。我们把 $p(h)$ 称为 prior，$p(d|h)$ 称为 likelihood，$p(h|d)$ 称为 posterior. \n", "\n", "## 贝叶斯表" ] }, { "cell_type": "code", "execution_count": 41, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
prior
碗10.5
碗20.5
\n", "
" ], "text/plain": [ " prior\n", "碗1 0.5\n", "碗2 0.5" ] }, "execution_count": 41, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pandas as pd\n", "table = pd.DataFrame(index = [\"碗1\", \"碗2\"])\n", "\n", "table['prior'] = 1/2, 1/2\n", "table" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "在上面那个球与碗的情形中，hypothesis 是该红球来自碗一，data 是抽到一颗红球，prior 是 $p(碗_1)$，likelihood 是 $p(红球|碗_1)$，posterior 是 $p(碗_1 | 红球)$。\n", "\n", "那我们接着：" ] }, { "cell_type": "code", "execution_count": 42, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
priorlikelihood
碗10.50.75
碗20.50.50
\n", "
" ], "text/plain": [ " prior likelihood\n", "碗1 0.5 0.75\n", "碗2 0.5 0.50" ] }, "execution_count": 42, "metadata": {}, "output_type": "execute_result" } ], "source": [ "table['likelihood'] = 3/4, 1/2\n", "table" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "然后 $prior * likelihood$，也就是 $P(碗_1)P(红球 | 碗_1)$:" ] }, { "cell_type": "code", "execution_count": 43, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
priorlikelihoodunnorm
碗10.50.750.375
碗20.50.500.250
\n", "
" ], "text/plain": [ " prior likelihood unnorm\n", "碗1 0.5 0.75 0.375\n", "碗2 0.5 0.50 0.250" ] }, "execution_count": 43, "metadata": {}, "output_type": "execute_result" } ], "source": [ "table['unnorm'] = table['prior'] * table['likelihood']\n", "table\n", "## unnorm 表示 unnormalized posteriors" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "因为我们知道 $p(d)$ 也就是 $p(红球)$ 为 $\\frac{5}{8}$，所以：" ] }, { "cell_type": "code", "execution_count": 44, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
priorlikelihoodunnormposterior
碗10.50.750.3750.6
碗20.50.500.2500.4
\n", "
" ], "text/plain": [ " prior likelihood unnorm posterior\n", "碗1 0.5 0.75 0.375 0.6\n", "碗2 0.5 0.50 0.250 0.4" ] }, "execution_count": 44, "metadata": {}, "output_type": "execute_result" } ], "source": [ "table['posterior'] = table['unnorm'] / (5/8)\n", "table" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "这个结果和我们算出来的 $3/5$ 是一致的。我们甚至还知道了 $p(碗_2|红球) = \\frac{2}{5}$。\n", "\n", "其实，更简单的做法是，既然 `unnorm` 是 `posterior` 的非标准化版，那我们把 `unnorm` 标准化一下不就好了吗？怎么标准化？除以 `unnorm` 的和就可以了。\n", "\n" ] }, { "cell_type": "code", "execution_count": 45, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
priorlikelihoodunnormposteriorposterior_again
碗10.50.750.3750.60.6
碗20.50.500.2500.40.4
\n", "
" ], "text/plain": [ " prior likelihood unnorm posterior posterior_again\n", "碗1 0.5 0.75 0.375 0.6 0.6\n", "碗2 0.5 0.50 0.250 0.4 0.4" ] }, "execution_count": 45, "metadata": {}, "output_type": "execute_result" } ], "source": [ "table['posterior_again'] = table['unnorm'] / table['unnorm'].sum()\n", "table" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 骰子问题\n", "\n", "假设我们有三个骰子：6 面的、8 面的、12 面的。A 君随机抽一个骰子，然后抛向空中，落下来的数字是 1。A 君让 B 君说出这个骰子是 6 面的概率。\n", "\n", "我们来分析一下，我们想知道的是 $p(dice = 6 | number = 1)$，所以：\n", "\n", "- prior: $p(dice = 6)$\n", "- likelihood: $p(number = 1 | dice = 6)$\n", "- data: $p(number = 1)$\n", "- posterior: $p(dice = 6 | number = 1)$" ] }, { "cell_type": "code", "execution_count": 46, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
priorlikelihood
61/31/6
81/31/8
121/31/12
\n", "
" ], "text/plain": [ " prior likelihood\n", "6 1/3 1/6\n", "8 1/3 1/8\n", "12 1/3 1/12" ] }, "execution_count": 46, "metadata": {}, "output_type": "execute_result" } ], "source": [ "table2 = pd.DataFrame(index = [6, 8, 12])\n", "from fractions import Fraction\n", "table2['prior'] = Fraction(1,3)\n", "table2['likelihood'] = Fraction(1,6), Fraction(1,8), Fraction(1, 12)\n", "table2" ] }, { "cell_type": "code", "execution_count": 47, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
priorlikelihoodunnorm
61/31/61/18
81/31/81/24
121/31/121/36
\n", "
" ], "text/plain": [ " prior likelihood unnorm\n", "6 1/3 1/6 1/18\n", "8 1/3 1/8 1/24\n", "12 1/3 1/12 1/36" ] }, "execution_count": 47, "metadata": {}, "output_type": "execute_result" } ], "source": [ "table2['unnorm'] = table2['prior'] * table2['likelihood']\n", "table2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "我们知道 \"unnorm\" 基本上已经是 posterior 了，唯一的区别是其还未标准化。" ] }, { "cell_type": "code", "execution_count": 48, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
priorlikelihoodunnormposterior
61/31/61/184/9
81/31/81/241/3
121/31/121/362/9
\n", "
" ], "text/plain": [ " prior likelihood unnorm posterior\n", "6 1/3 1/6 1/18 4/9\n", "8 1/3 1/8 1/24 1/3\n", "12 1/3 1/12 1/36 2/9" ] }, "execution_count": 48, "metadata": {}, "output_type": "execute_result" } ], "source": [ "table2['posterior'] = table2['unnorm']/table2['unnorm'].sum()\n", "table2" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 三重门\n", "\n", "假设你在参加一个电视节目。你面前是三扇门。其中一扇门后面是一辆法拉利，其余两扇门后面各是一只山羊。你随机挑一扇门，不管打开后是什么，都归你了。你当然希望后面是一辆法拉利。你随机挑了门 A，告诉了主持人，主持人打开了门 C，后面是一只山羊，主持人问你，你是坚持选择 A 还是换成 B。\n", "\n", "主持人通常这么行事：\n", "1. 主持人总会打开一扇门，给你机会重新选择\n", "2. 他从来不会打开有车的门\n", "3. 如果你选择了有车的那扇门，他会从另外两扇门中随机选一扇门打开\n", "\n", "现在问你，汽车在门 A 和门 B 后的概率分别是多少？\n", "\n", "你可能会觉得各是 1/2，但其实不是。我们用贝叶斯来计算一下。因为我们要选择的是门，所以 hypothesis 是门，故 prior 是 $p(门)$：车在 A, B, C 之后。\n", "\n", "Data 是主持人打开了 C 我们看到了一只山羊。\n", "\n", "Likelihood 是 $p(车 | 门)$。\n", "\n", "Posterior 是 $p(门 | 车)$。\n", "\n", "记住 prior 是在我们什么都不知道、没有任何数据的情况下所做的预测。" ] }, { "cell_type": "code", "execution_count": 49, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
prior
A1/3
B1/3
C1/3
\n", "
" ], "text/plain": [ " prior\n", "A 1/3\n", "B 1/3\n", "C 1/3" ] }, "execution_count": 49, "metadata": {}, "output_type": "execute_result" } ], "source": [ "table3 = pd.DataFrame(index = ['A', 'B', 'C'])\n", "table3['prior'] = Fraction(1, 3)\n", "table3" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "我们来分析 Likelihood。Likelihood 指的是在某一 hypothesis 的情况下，这个 data 的概率。我们挨个分析。\n", "\n", "如果 hypothesis 是车在 A, 那么主持人打开 C 的概率是 $1/2$。\n", "\n", "如果 hypothesis 是车在 B，他必须给你选择的机会，但他不能打开 A，只能打开 C，所以这个 data 的概率是 1。\n", "\n", "如果 hypothesis 是车在 C，那么主持人打开 C 的概率是 0，因为他不能打开有车的那扇门。\n", "\n", "所以：" ] }, { "cell_type": "code", "execution_count": 50, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
priorlikelihoodunnormposterior
A1/31/21/61/3
B1/311/32/3
C1/3000
\n", "
" ], "text/plain": [ " prior likelihood unnorm posterior\n", "A 1/3 1/2 1/6 1/3\n", "B 1/3 1 1/3 2/3\n", "C 1/3 0 0 0" ] }, "execution_count": 50, "metadata": {}, "output_type": "execute_result" } ], "source": [ "table3['likelihood'] = Fraction(1, 2), 1, 0\n", "table3['unnorm'] = table3['prior'] * table3['likelihood']\n", "table3['posterior'] = table3['unnorm']/(table3['unnorm'].sum())\n", "table3" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "由此我们看到，车在 B 门之后的概率要更高。" ] } ], "metadata": { "kernelspec": { "display_name": "ica", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.11.6" } }, "nbformat": 4, "nbformat_minor": 2 }