A number of days in the past I wrote a submit entitled Does further knowledge at all times cut back posterior variance?. In a nutshell, the reply isn’t any, not at all times.
That led the earlier submit which checked out posterior means for 3 Bayesian fashions, displaying how the posterior imply is a weighted common of the prior imply and the imply of the brand new knowledge. The weights are precisions, which implies one thing totally different for every mannequin.
For the beta-binomial mannequin, variance might enhance when seeing sudden knowledge (particulars right here), however precision at all times will increase.
For the normal-normal mannequin precision is the reciprocal of variance. Each new knowledge level makes precision go up and posterior variance go down.
The Poisson-gamma mannequin stands out as the most fascinating. As said within the earlier submit, if knowledge has a Poisson distribution with parameter λ, and λ has a gamma(α0, β0) prior distribution, then the posterior distribution on λ after observing okay occasions over time t has a gamma(α0 + okay, β0 + t) posterior distribution. Due to this fact the posterior variance is
(α0 + okay) / (β0 + t)².
Observe the posterior variance is an rising operate of okay and a reducing operate of t. Because of this the posterior variance will increase each time an occasion is noticed, and it decreases quadratically between observations.
Right here’s an illustration. I simulated knowledge from a Poisson course of with λ and used a gamma(1, 1) prior on λ. Right here’s a plot of the posterior variance.
