The logit and logistic functions

Linear regression works on real numbers $\mathbb{R}$, that is, the input and output are in $\mathbb{R}$. For probabilities, this is problematic because the linear regression will happily give a probability of $-934$, where we know that probabilities should always lie between $0$ and $1$. This is only by definition, but it is an useful definition in practice. Informally, the logistic function has been designed to convert values from real numbers to probabilities and the logit function is the inverse.

Logistic

The logistic function converts values from $(-\infty, \infty)$ to $(0, 1)$:

logistic(x) = 1 / (1 + exp(-x))

We can visualise this with

using Gadfly

plot(y = [logistic], xmin = [-6], xmax = [6], 
	Geom.line, Stat.func, Guide.xlabel("x")
)

Some people advise to remember the following numbers by heart.

since

logistic(-3) = 0.04742587317756678
logistic(-1) = 0.2689414213699951
logistic(1) = 0.7310585786300049
logistic(3) = 0.9525741268224334

Logit

The inverse of the logistic function is the logit function,

logit(x) = log(x / (1 - x))

This function goes from $(0, 1)$ to $(- \infty, \infty)$.

plot(y = [logit], xmin = [0], xmax = [1],
	Geom.line, Stat.func, Guide.xlabel("x")
)