Linear regression works on real numbers , that is, the input and output are in . For probabilities, this is problematic because the linear regression will happily give a probability of , where we know that probabilities should always lie between and . This is only by definition, but it is an useful definition in practice. Informally, the logistic function has been designed to convert values from real numbers to probabilities and the logit function is the inverse.
The logistic function converts values from to :
logistic(x) = 1 / (1 + exp(-x))
We can visualise this with
using Gadfly
plot(y = [logistic], xmin = [-6], xmax = [6],
Geom.line, Stat.func, Guide.xlabel("x")
)
Some people advise to remember the following numbers by heart.
since
logistic(-3) = 0.04742587317756678
logistic(-1) = 0.2689414213699951
logistic(1) = 0.7310585786300049
logistic(3) = 0.9525741268224334
The inverse of the logistic function is the logit function,
logit(x) = log(x / (1 - x))
This function goes from to .
plot(y = [logit], xmin = [0], xmax = [1],
Geom.line, Stat.func, Guide.xlabel("x")
)