Textbook
In binary linear classification, an object has features $x\in\mathbb R^d$. A weight vector $w\in\mathbb R^d$ and bias $b\in\mathbb R$ define the score:
$$s(x)=w^{\mathsf T}x+b=\sum_{j=1}^{d}w_jx_j+b.$$
The rule is $h(x)=1$ when $s(x)\ge0$, and $h(x)=0$ when $s(x)<0$. For $w\ne0$, the boundary $w^{\mathsf T}x+b=0$ is a hyperplane: a straight line for two features. When $w=0$, the score is constant and this rule does not produce a separating hyperplane.
Each weight multiplies its corresponding coordinate, and the bias is added once. Parameters may be fitted using labelled examples; the learning procedure is a separate choice. This formula applies parameters that have already been selected.
A linear boundary cannot represent arbitrary class arrangements in the original features. It may misclassify training objects; a smooth boundary alone does not establish performance on unseen data.
Related concepts: Decision boundaries.
For fixed parameters, the features and the zero-score convention completely determine the result. Multiplying every weight and the bias by one positive constant preserves score signs and therefore class labels. The scoreβs numerical value is thus not a probability by itself. Negative scaling changes signs and cannot be treated as the same classifier expressed in another unit.
In plain language
A linear rule resembles scoring an application: one feature adds two points, another subtracts one, and a bias sets the starting amount. The final number determines the class through its side of the zero threshold. Doubling every weight and the bias doubles the score without changing decisions, so a larger number alone does not establish greater confidence. Weights also depend on feature scales and do not explain causes. Points awarded for an umbrella may help prediction, but the formula gains no authority over rain.
Example
The toy parameters $w=(2,-1)$, $b=-1$, and features $x=(2,1)$ are chosen by hand. The first prediction is followed by a change in the second coordinate.
Linear classifiers
Feature contributions
The first feature contributes 4, the second minus 1. The bias is also minus 1.
Step 1 of 3: Feature contributions
Read all steps
Feature contributions
The first feature contributes 4, the second minus 1. The bias is also minus 1.
- Contributions
- 2Γ2 = 4currentβ1Γ1 = β1currentb = β1
Score and label
The sum is 2. A positive score gives class 1.
- Score
- 2included
- Class
- 1included
A different point
With x = (2,4), the second feature contributes a different value. The score becomes negative and the prediction is 0.
- Contributions
- 4β4currentβ1
- Score
- -1excluded
- Class
- 0included
The first input produces the positive score 2 and receives class 1. For fixed $x_1=2$, the boundary satisfies $4-x_2-1=0$, giving $x_2=3$. This boundary input also receives class 1 under the declared convention $s\ge0$. At $x_2=4$, the score is negative. The calculation checks both predictions and the exact location where the rule switches between them.
Source of the original notes: lecture notes.