Textbook
A classifier $h$ assigns a class to every feature point. A class decision region is the set of points assigned that label. A decision boundary separates regions with different predictions: points from different regions occur arbitrarily close to its points.
In two feature dimensions, a boundary may be a line or a collection of curves. In higher dimensions, its shape depends on the model. It comes from the prediction rule, rather than simply connecting labelled observations.
For the illustrative continuous score $s(x)=x_1+x_2-3$, class 1 is assigned when $s(x)\ge0$ and class 0 when $s(x)<0$. The boundary is $s(x)=0$. Points exactly on it belong to class 1 by convention; this convention does not change the boundary’s location.
The same labelled sample can admit different boundaries. Separating known points successfully therefore does not determine where prediction regions should lie on new data.
Related concepts: Supervised classification.
In this linear example, fixing $x_2$ places the transition at $x_1=3-x_2$. At any point on that line, arbitrarily small changes of the first coordinate in opposite directions produce opposite score signs. This verifies the boundary property. Away from the line, a sufficiently small neighbourhood preserves the sign, so changing a feature need not always change the assigned class.
In plain language
A delivery map can assign different rates on opposite sides of a line. An address is a point, a region groups identical answers, and crossing the boundary can change the rate after a small movement. The rule also covers a new building with no previous orders; fencing every familiar address separately would turn delivery into a zoning conspiracy. A separate convention assigns the line itself without moving the boundary. The map describes decision regions, but distance to the line alone measures neither prediction probability nor confidence.
Example
In this toy model, only $x_1$ changes while $x_2=1$ remains fixed. The score $s=x_1+x_2-3$ makes the boundary crossing visible without any hidden learning procedure.
Decision boundaries
Before the boundary
At (1,1), the score is negative and the prediction is class 0.
Step 1 of 3: Before the boundary
Read all steps
Before the boundary
At (1,1), the score is negative and the prediction is class 0.
- x
- 1current1current
- Score
- -1excluded
- Class
- 0included
On the boundary
At (2,1), the score is zero. The stated convention already assigns class 1.
- x
- 2current1
- Score
- 0current
- Class
- 1included
Beyond the boundary
At (3,1), the score is positive. The class stays 1 although the point has moved.
- x
- 3current1
- Score
- 1included
- Class
- 1included
With $x_2=1$, the rule reduces to $s=x_1-2$, placing the transition exactly at $x_1=2$. Values 1 and 3 lie on opposite sides. The class at the threshold is a separate convention: equality belongs to class 1 here. Assigning equality to class 0 would change the prediction at that point while leaving the neighbouring regions and their dividing line in the same location.
Source of the original notes: lecture notes.