The logit and logistic functions

Linear regression works on real numbers \mathbb{R}, that is, the input and output are in \mathbb{R}. For probabilities, this is problematic because the linear regression will happily give a probability of -934, where we know that probabilities should always lie between 0 and 1. This is only by definition, but it is an useful definition in practice. Informally, the logistic function converts values from real numbers to probabilities and the logit function does the reverse.

Logistic

The logistic function converts values from (-\infty, \infty) to (0, 1):

\text{logistic}(x) = \frac{1}{1 + e^{-x}}.

We can easily define this function ourselves:

mylogistic(x) = 1 / (1 + exp(-x))

But, also load it from StatsFuns.jl:

using StatsFuns: logistic

@assert mylogistic(1) == logistic(1)

Graphically, this looks as follows:

I = -8:0.01:8
fig = Figure()
ax = Axis(fig[1, 1]; xlabel="x", ylabel="logistic(x)")
lines!(ax, I, logistic.(I))
xlims!(ax, -6, 6)
fig

If you care enough, you could decide to remember the plot by heart. Some people advise to remember the following numbers by heart.

\begin{aligned}
\text{logistic}(-3) &\approx 0.05, \\
\text{logistic}(-1) &\approx \tfrac{1}{4}, \\
\text{logistic}(1) &\approx \tfrac{3}{4}, \: \text{and} \\
\text{logistic}(3) &\approx 0.95.
\end{aligned}

since

r2(x) = round(x; digits=2);

logistic(-3) |> r2

0.05

logistic(-1) |> r2

0.27

logistic(1) |> r2

0.73

logistic(3) |> r2

0.95

Logit

The inverse of the logistic function is the logit function:

\text{logit}(x) = \log\frac{x}{1 - x}

mylogit(x) = log(x / (1 - x));

using StatsFuns: logit

@assert logit(1) == mylogit(1)

This function goes from (0, 1) to (-\infty, \infty):

I = 0:0.001:1
fig = Figure()
ax = Axis(fig[1, 1]; xlabel="x", ylabel="logit(x)")
lines!(ax, I, logit.(I))
xlims!(ax, 0, 1)
ylims!(ax, -6, 6)
fig

Appendix

Built with Julia 1.11.5 and

CairoMakie 0.12.16
StatsFuns 1.3.2