Bayesian Updating

Each observation updates the distribution over θ by Bayes' rule, with the previous posterior as the prior:

$$p(\theta |x_1,\dots ,x_k)\propto p(x_k|\theta )p(\theta |x_1,\dots ,x_{k-1}).$$

The observations are independent given θ, so after k steps the result is the posterior given all k observations at once, in whatever order they arrived:

$$p(\theta |x_1,\dots ,x_k)\propto p(\theta )\prod _{i=1}^{k}p(x_i|\theta ).$$

Each observation is the number of heads in m flips, x_i ~ Binomial(m, θ), and θ ~ Beta(α, β). After k observations with h heads and t tails in total, the posterior is Beta(α + h, β + t): the heads add to α and the tails to β, so the prior acts as α heads and β tails counted before the data.

Each observation is the number of heads in m flips, x_i ~ Binomial(m, θ), and θ ~ ½ Beta(α, β) + ½ Beta(β, α): a coin believed to be biased, in a direction not known in advance. Each component updates as a Beta prior does, and its weight is multiplied by how well it predicted the data. With h heads and t tails, the posterior is w Beta(α + h, β + t) + (1 − w) Beta(β + h, α + t), where w is B(α + h, β + t) divided by B(α + h, β + t) + B(β + h, α + t) and B is the beta function.

Each observation is a count x_i ~ Poisson(θ), and θ ~ Gamma(α, β) with shape α and rate β. After k observations with total count s, the posterior is Gamma(α + s, β + k): the counts add to α and the number of observations to β.

Each observation is x_i ~ N(θ, σ²) with σ known, and θ ~ N(μ₀, τ²). After k observations with sample mean x̄, the posterior is N(m, v) with 1/v = 1/τ² + k/σ² and m = v (μ₀/τ² + kx̄/σ²): the precisions add, and the posterior mean is the precision-weighted average of the prior mean and the sample mean.

Each observation is a waiting time x_i ~ Exponential(θ) with rate θ, and θ ~ Gamma(α, β) with shape α and rate β. After k observations with total time s, the posterior is Gamma(α + k, β + s): the number of events adds to α and the elapsed time to β.

The curves and statistics on this page are computed on a grid of θ values rather than from these closed forms, so the prior need not be conjugate; the closed form is printed above the plot for comparison.