The logit and logistic functions

Linear regression works on real numbers \(\mathbb{R}\), that is, the input and output are in \(\mathbb{R}\). For probabilities, this is problematic because the linear regression will happily give a probability of \(-934\), where we know that probabilities should always lie between \(0\) and \(1\). This is only by definition, but it is an useful definition in practice. Informally, the logistic function converts values from real numbers to probabilities and the logit function does the reverse.

Logistic

The logistic function converts values from \((-\infty, \infty)\) to \((0, 1)\):

$$\text{logistic}(x) = \frac{1}{1 + e^{-x}}.$$

We can easily define this function ourselves:

But, also load it from StatsFuns.jl:

Graphically, this looks as follows:

If you care enough, you could decide to remember the plot by heart. Some people advise to remember the following numbers by heart.

$$\begin{aligned} \text{logistic}(-3) &\approx 0.05, \\ \text{logistic}(-1) &\approx \tfrac{1}{4}, \\ \text{logistic}(1) &\approx \tfrac{3}{4}, \: \text{and} \\ \text{logistic}(3) &\approx 0.95. \end{aligned}$$

since

Logit

The inverse of the logistic function is the logit function:

$$\text{logit}(x) = \log\frac{x}{1 - x}$$

This function goes from \((0, 1)\) to \((-\infty, \infty)\):