Arranging a Beta Distribution into Exponential Family Form

A family of PMFs or PDFs is an exponential family if it can be arranged into the form

$f(x|\theta) = h(x)c(\theta)\exp\left(\sum_{i=1}^kw_i(\theta)t_i(x)\right)$

where $\theta$ is the vector of parameters.

The Beta distribution PDF takes the form

$f_X(x) = \frac{x^{\alpha – 1}(1 – x)^{\beta – 1}}{\textrm{B}(\alpha, \beta)}$

Given that the $\alpha$ and $\beta$ parameters are unknown, we now arrange $f_X(x)$ into an exponential family form:

$f(x|\alpha,\beta) = \frac{x^{\alpha – 1}(1 – x)^{\beta – 1}}{\textrm{B}(\alpha, \beta)}$
$= \frac{e^{(\alpha – 1)\ln(x)}e^{(\beta – 1)\ln(1 – x)}}{\textrm{B}(\alpha, \beta)}$
$= \frac{e^{(\alpha – 1)\ln(x) + (\beta – 1)\ln(1 – x)}}{\textrm{B}(\alpha, \beta)}$
$= \frac{1}{\textrm{B}(\alpha, \beta)}e^{(\alpha – 1)\ln(x) + (\beta – 1)\ln(1 – x)}$

The log identity $x^b = e^{b\ln(x)}$ is a very useful logarithmic identity to remember when trying to arrange PDFs into exponential family form.

We observe:
$h(x) = I_{x \in (0,1)}(x)$ (If you see that h(x) = 1, that is a cue to use an indicator function that ranges through the support of $x$.)
$c(k,\beta) = \frac{1}{\textrm{B}(\alpha, \beta)$
$w_1(k,\beta) = \alpha – 1$
$w_2(k,\beta) = \beta – 1$
$t_1(x) = \ln(x)$
$t_2(x) = \ln(1-x)$

Hence, the Beta distribution given unknown parameters $\alpha$ and $\beta$ is an exponential family with a two-dimensional parameter vector $\theta$.

A similar process will apply for showing that a Beta PDF with one unknown parameter, $\beta$ or $\alpha$ is an exponential family.

Leave a Reply

Captcha * Time limit is exhausted. Please reload CAPTCHA.