Logistic Regression is a statistical method used for binary classification, which means it helps predict one of two possible outcomes based on one or more input variables.
Instead of predicting a continuous value (like linear regression), logistic regression predicts the probability that a given input belongs to a certain class ā usually labeled as 0 or 1.
Linear Combination of inputs:
Like linear regression, it starts by computing a weighted sum:
z=w0+w1x1+w2x2+āÆ+wnxnz = w_0 + w_1x_1 + w_2x_2 + \dots + w_nx_n
Apply Sigmoid Function:
The output z is passed through a sigmoid (logistic) function:
Ļ(z)=11+eāz\sigma(z) = \frac{1}{1 + e^{-z}}
This maps the result to a value between 0 and 1, representing the probability of class 1.
Decision Rule:
If Ļ(z)>0.5\sigma(z) > 0.5, predict class 1; otherwise, predict class 0.
Suppose you're predicting whether an email is spam (1) or not spam (0), based on features like:
Logistic regression finds the weights for these features to best separate spam from not spam.