Arranging a Gamma Distribution Family into Exponential Family Form

A family of PMFs or PDFs is an exponential family if it can be arranged into the form

$f(x|\theta) = h(x)c(\theta)\exp\left(\sum_{i=1}^kw_i(\theta)t_i(x)\right)$

where $\theta$ is the vector of parameters.

Now, the shape-scale parameterization of the Gamma distribution PDF takes the form

$f_X(x) = \frac{1}{\Gamma(k)\beta(k)}x^{k-1}e^{-\frac{x}{\beta}}$

$\beta$ is typically used as the symbol for the rate parameter in the shape-rate parameterization of the Gamma PDF, but the default symbol for the scale parameter, $\theta$, would conflict with the symbol for our parameter vector.

Can we arrange that PDF into an exponential family form? Spoiler: yes.

Here, we demonstrate that a Gamma PDF given two unknown parameters, $\beta$ and $k$, is an exponential family.

$f(x|k,\beta) = \frac{1}{\Gamma(k)\beta(k)}x^{k-1}e^{-\frac{x}{\beta}}$
$ = \frac{1}{\Gamma(k)\beta(k)}e^{(k-1)\ln(x)}e^{-\frac{x}{\beta}}$
$ = \frac{1}{\Gamma(k)\beta(k)}e^{(k-1)\ln(x) – \frac{x}{\beta}}$

The log identity $x^b = e^{b\ln(x)}$ is a very useful logarithmic identity to remember when trying to arrange PDFs into exponential family form.

We observe:
$h(x) = I_{x>0}(x)$ (If you see that h(x) = 1, that is a cue to use an indicator function that ranges through the support of $x$.)
$c(k,\beta) = \frac{1}{\Gamma(k)\beta(k)}$
$w_1(k,\beta) = k – 1$
$w_2(k,\beta) = -\frac{1}{\beta}$
$t_1(x) = \ln(x)$
$t_2(x) = x$

Hence, the Gamma distribution given unknown parameters $\beta$ and $k$ is an exponential family with a two-dimensional parameter vector $\theta$.

A similar process will apply for showing that a Gamma PDF with one unknown parameter, $\beta$ or $k$ is an exponential family.

Leave a Reply

Your email address will not be published. Required fields are marked *

Captcha * Time limit is exhausted. Please reload CAPTCHA.