Linear regression works on real numbers \(\mathbb{R}\), that is, the input and output are in \(\mathbb{R}\). For probabilities, this is problematic because the linear regression will happily give a probability of \(-934\), where we know that probabilities should always lie between \(0\) and \(1\). This is only by definition, but it is an useful definition in practice. Informally, the *logistic* function converts values from real numbers to probabilities and the *logit* function does the reverse.

The logistic function converts values from \((-\infty, \infty)\) to \((0, 1)\):

$$\text{logistic}(x) = \frac{1}{1 + e^{-x}}.$$

We can easily define this function ourselves:

`mylogistic(x) = 1 / (1 + exp(-x));`

But, also load it from `StatsFuns.jl`

:

`using StatsFuns: logistic`

`mylogistic(1) == logistic(1)`

true

Graphically, this looks as follows:

If you care enough, you could decide to remember the plot by heart. Some people advise to remember the following numbers by heart.

$$\begin{aligned} \text{logistic}(-3) &\approx 0.05, \\ \text{logistic}(-1) &\approx \tfrac{1}{4}, \\ \text{logistic}(1) &\approx \tfrac{3}{4}, \: \text{and} \\ \text{logistic}(3) &\approx 0.95. \end{aligned}$$

since

`r2(x) = round(x; digits=2);`

`logistic(-3) |> r2`

0.05

`logistic(-1) |> r2`

0.27

`logistic(1) |> r2`

0.73

`logistic(3) |> r2`

0.95

The inverse of the logistic function is the *logit* function:

$$\text{logit}(x) = \log\frac{x}{1 - x}$$

`mylogit(x) = log(x / (1 - x));`

`using StatsFuns: logit`

`logit(1) == mylogit(1)`

true

This function goes from \((0, 1)\) to \((-\infty, \infty)\):

Built with Julia 1.10.4 and

CairoMakie 0.11.8StatsFuns 1.3.0

Rik Huijzer.
The text is licensed under CC BY-SA 4.0.
The code is licensed under the MIT License.
Website built with Franklin.jl and the Julia programming language.
Last update: 2024-08-03.