Archive for November, 2016

Arranging a Beta Distribution into Exponential Family Form

Sunday, November 20th, 2016

A family of PMFs or PDFs is an exponential family if it can be arranged into the form

$f(x|\theta) = h(x)c(\theta)\exp\left(\sum_{i=1}^kw_i(\theta)t_i(x)\right)$

where $\theta$ is the vector of parameters.

The Beta distribution PDF takes the form

$f_X(x) = \frac{x^{\alpha – 1}(1 – x)^{\beta – 1}}{\textrm{B}(\alpha, \beta)}$

Given that the $\alpha$ and $\beta$ parameters are unknown, we now arrange $f_X(x)$ into an exponential family form:

$f(x|\alpha,\beta) = \frac{x^{\alpha – 1}(1 – x)^{\beta – 1}}{\textrm{B}(\alpha, \beta)}$
$= \frac{e^{(\alpha – 1)\ln(x)}e^{(\beta – 1)\ln(1 – x)}}{\textrm{B}(\alpha, \beta)}$
$= \frac{e^{(\alpha – 1)\ln(x) + (\beta – 1)\ln(1 – x)}}{\textrm{B}(\alpha, \beta)}$
$= \frac{1}{\textrm{B}(\alpha, \beta)}e^{(\alpha – 1)\ln(x) + (\beta – 1)\ln(1 – x)}$

The log identity $x^b = e^{b\ln(x)}$ is a very useful logarithmic identity to remember when trying to arrange PDFs into exponential family form.

We observe:
$h(x) = I_{x \in (0,1)}(x)$ (If you see that h(x) = 1, that is a cue to use an indicator function that ranges through the support of $x$.)
$c(k,\beta) = \frac{1}{\textrm{B}(\alpha, \beta)$
$w_1(k,\beta) = \alpha – 1$
$w_2(k,\beta) = \beta – 1$
$t_1(x) = \ln(x)$
$t_2(x) = \ln(1-x)$

Hence, the Beta distribution given unknown parameters $\alpha$ and $\beta$ is an exponential family with a two-dimensional parameter vector $\theta$.

A similar process will apply for showing that a Beta PDF with one unknown parameter, $\beta$ or $\alpha$ is an exponential family.

Regarding Identity Politics And Their Guaranteed Persistence

Sunday, November 20th, 2016

I absolutely hate this talk of “we must end identity politics” that has become pervasive after the election, especially as people are trying to prescribe Democrats solutions for how to recover from this election. First off, how are people defining “identity politics”? It appears that a lot of people are using it as a blanket term for political organization and mobilization by gender, race, or ethnic group. Identity politics exist and will not go away because your geographic origins, gender, phenotypes, genotypes determine your lot in life, the hand you’ve been dealt, et cetera (insert favored idiom here). Your predetermined traits (like those of an RPG character class) come with a set of pros and cons that relate to your probabilities of success in life as defined by societal and cultural values. People are motivated to maintaining or improving their probabilities, so it is easy to motivate people to advocate for the blocs they belong to. Yes, identity politics make it easy to hate on another group of people as The Others, but it has also mobilized mass sectors of the American population to advocate for egalitarian goals, such as women’s suffrage, desegregation, and elimination of sodomy laws, which allowed members of one American bloc to live significantly more comfortably at little or no cost (and perhaps benefit) to opposing blocs.

As an alternative, people are promoting an emphasis on “class politics.” But how are identity politics so different from class politics? You are still slicing a population into sub-groups, and then trying to get them to oppose the interests of another group. Yes, having no hive-mind sub-group jousting would be ideal, but it is a platonic ideal that is impossible given human brain heuristics. Identity politics will survive the 2016 election and persist as humans continue to exist in their current form.

Arranging a Gamma Distribution Family into Exponential Family Form

Sunday, November 20th, 2016

A family of PMFs or PDFs is an exponential family if it can be arranged into the form

$f(x|\theta) = h(x)c(\theta)\exp\left(\sum_{i=1}^kw_i(\theta)t_i(x)\right)$

where $\theta$ is the vector of parameters.

Now, the shape-scale parameterization of the Gamma distribution PDF takes the form

$f_X(x) = \frac{1}{\Gamma(k)\beta(k)}x^{k-1}e^{-\frac{x}{\beta}}$

$\beta$ is typically used as the symbol for the rate parameter in the shape-rate parameterization of the Gamma PDF, but the default symbol for the scale parameter, $\theta$, would conflict with the symbol for our parameter vector.

Can we arrange that PDF into an exponential family form? Spoiler: yes.

Here, we demonstrate that a Gamma PDF given two unknown parameters, $\beta$ and $k$, is an exponential family.

$f(x|k,\beta) = \frac{1}{\Gamma(k)\beta(k)}x^{k-1}e^{-\frac{x}{\beta}}$
$ = \frac{1}{\Gamma(k)\beta(k)}e^{(k-1)\ln(x)}e^{-\frac{x}{\beta}}$
$ = \frac{1}{\Gamma(k)\beta(k)}e^{(k-1)\ln(x) – \frac{x}{\beta}}$

The log identity $x^b = e^{b\ln(x)}$ is a very useful logarithmic identity to remember when trying to arrange PDFs into exponential family form.

We observe:
$h(x) = I_{x>0}(x)$ (If you see that h(x) = 1, that is a cue to use an indicator function that ranges through the support of $x$.)
$c(k,\beta) = \frac{1}{\Gamma(k)\beta(k)}$
$w_1(k,\beta) = k – 1$
$w_2(k,\beta) = -\frac{1}{\beta}$
$t_1(x) = \ln(x)$
$t_2(x) = x$

Hence, the Gamma distribution given unknown parameters $\beta$ and $k$ is an exponential family with a two-dimensional parameter vector $\theta$.

A similar process will apply for showing that a Gamma PDF with one unknown parameter, $\beta$ or $k$ is an exponential family.