5.1 Linear prediction, margins and perceptron

Original supplement on binary linear classification.

1. The output is a label, not a fitted response

Regression and classification can use the same feature vectors while solving different mathematical problems. A regression score is judged by how closely its numerical value matches a response. A binary classifier uses a score to choose between two labels. A good fit to the numbers minus one and plus one need not put the decision boundary in a good place.

The necessary foundations are real vectors and inner products, Cauchy–Schwarz, affine functions, finite sums, and bounded-sequence compactness in finite-dimensional Euclidean space. The existing least-squares note supplies the design-matrix and normal-equation results used in Question 1. Fisher discriminants already select scores from labelled scatter; they solve a different objective from the separating inequalities below. Learned metrics use a squared-distance margin, which must also be distinguished from distance to a hyperplane.

Fix a finite nonempty training list

$$(x_i,y_i),\qquad i=1,\ldots,m,\qquad x_i\in\mathbb R^d,\quad y_i\in\{-1,+1\}, \quad m\geq1,\quad d\geq1.$$

Repeated inputs are allowed; they may have inconsistent labels. All statements in this note concern this fixed finite list. No sampling distribution or claim about future prediction error is assumed.

Score and prediction convention. A parameter pair $(w,b)\in\mathbb R^d\times\mathbb R$ gives the affine score and classifier

$$f_{w,b}(x)=w^\top x+b,\qquad h_{w,b}(x)= \begin{cases} +1,&f_{w,b}(x)\geq0,\\ -1,&f_{w,b}(x)<0. \end{cases}$$

Thus a zero score predicts the positive class. The score is linear in its parameters, but is affine in x when the intercept b is nonzero. The empirical classification error is

$$\operatorname{err}(w,b) =\frac1m\sum_{i=1}^m 1_{\{h_{w,b}(x_i)\ne y_i\}}.$$

This is not the least-squares objective $\sum\limits_i(y_i-f_{w,b}(x_i))^2$. The latter penalizes the sizes of numerical discrepancies; the former counts incorrect labels after thresholding. Question 1 gives exact data where their preferred conclusions differ.

The signed score $y_i f_{w,b}(x_i)$ is convenient: a positive value means the example is correctly classified away from the boundary. At zero it no longer decides correctness by itself. A positive-labelled zero is correct under our rule, and a negative-labelled zero is incorrect. Consequently the condition $y_i f_{w,b}(x_i)\geq0$ for every i is not equivalent to zero classification error under this tie convention.

We call the list strictly affinely separable if some $(w,b)$ satisfies $y_i f_{w,b}(x_i)>0$ for every i. Strict separation is stronger than merely accepting positive ties. It is the hypothesis used in the margin and termination theorems.

2. Distance, functional margin and geometric margin

Suppose first that $w\ne0$. The equation $w^\top x+b=0$ defines a genuine hyperplane H. It is nonempty, since $-bw/\|w\|^2$ lies on it. The vector w specifies an orientation as well as a normal direction: the positive scores lie on the side to which w points.

Distance proposition. For every $x\in\mathbb R^d$, its unique closest point on H and its distance to H are

$$p(x)=x-\frac{f_{w,b}(x)}{\|w\|^2}w,\qquad \operatorname{dist}(x,H)=\frac{|f_{w,b}(x)|}{\|w\|}.$$

Indeed, substitution gives $w^\top p(x)+b=0$. For any $q\in H$, $w^\top(x-q)=f_{w,b}(x)$, so Cauchy–Schwarz implies $\|x-q\|\geq |f_{w,b}(x)|/\|w\|$. The displayed p attains this bound. To verify uniqueness explicitly, decompose $x-q=(x-p)+(p-q)$. The first vector is parallel to w and the second is perpendicular to w, hence $\|x-q\|^2=\|x-p\|^2+\|p-q\|^2$. Equality with the smallest distance forces $q=p$.

In particular, the distance from the origin is $|b|/\|w\|$, not generally b. The closest point to the origin is $-bw/\|w\|^2$, whose coordinate along the oriented unit normal is $-b/\|w\|$.

Worked distance calculation. Let $w=(3,4)^\top$, $b=-10$, and $x=(2,2)^\top$. Then $\|w\|=5$, $f(x)=4$ and

$$p=\left(\frac{38}{25},\frac{34}{25}\right)^\top, \qquad 3p_1+4p_2=10,\qquad \|x-p\|=\frac45.$$

If this point has label plus one and the origin has label minus one, their signed distances are $4/5$ and two. The minimum is therefore $4/5$.

For a fixed list define its functional margin $a(w,b)=\min\limits_i y_i f_{w,b}(x_i)$. For $w\ne0$, define its signed geometric margin

$$\delta(w,b)=\frac{a(w,b)}{\|w\|} =\min_i\frac{y_i(w^\top x_i+b)}{\|w\|}.$$

If $a>0$, the geometric margin is the smallest distance of a training point to the hyperplane, with all points on their prescribed sides. If $a\leq0$, it is a signed quantity and is not a positive robustness radius.

For $\lambda>0$, replacing $(w,b)$ by $(\lambda w,\lambda b)$ preserves every prediction and the hyperplane. It multiplies a by $\lambda$ and preserves $\delta$. Thus a strictly positive functional margin can be made arbitrarily large by parameter scaling without moving the decision boundary. Both w and b must be scaled. For example, $2x+2=0$ and $x+1=0$ give the same boundary, whereas changing only w produces $x+2=0$. Negative scaling reverses signs away from the boundary; our positive tie convention still applies on the boundary.

Finite-data robustness. If $\delta(w,b)>0$ and $\|r_i\|<\delta(w,b)$, then the perturbed training point $x_i+r_i$ remains strictly correctly classified. In fact,

$$y_i f_{w,b}(x_i+r_i) =y_i f_{w,b}(x_i)+y_i w^\top r_i \geq\|w\|\big(\delta(w,b)-\|r_i\|\big)>0.$$

The radius is strict. A point attaining the finite minimum can be moved to the hyperplane by $r_i=-y_i\delta(w,b)w/\|w\|$. At that displacement strict separation is lost; the tie rule determines its predicted label. A slightly larger displacement in the same direction crosses the boundary. This proves a deterministic perturbation statement on the observed list, not a probability bound or a generalization theorem.

When $w=0$, the score is constant. Its zero set is either all of $\mathbb R^d$ or empty, not a nondegenerate hyperplane; division by $\|w\|$ is undefined. A list containing both labels cannot be strictly separated by a constant score. A list with one label can be, which is why the two cases must remain separate.

3. The augmented perceptron: specify the update before proving it

Encode the intercept in a final coordinate:

$$z_i=(x_i,1)\in\mathbb R^{d+1},\qquad \theta=(w,b),\qquad \theta^\top z_i=f_{w,b}(x_i),\qquad R=\max_i\|z_i\|=\max_i\sqrt{\|x_i\|^2+1}.$$

The finite number R is at least one. Using the largest norm of the unaugmented x alone would omit the intercept coordinate.

Assume that there is a unit vector $v\in\mathbb R^{d+1}$ and a number $\gamma>0$ such that

$$\|v\|=1,\qquad y_i v^\top z_i\geq\gamma \quad\text{for every }i.$$

Every finite strictly separable list admits such a pair: take any strict parameter $\theta_*$, which is nonzero, put $v=\theta_*/\|\theta_*\|$ and take $\gamma=\min\limits_i y_i\theta_*^\top z_i/\|\theta_*\|$. The minimum is positive because there are finitely many strictly positive values. Conversely this assumption itself supplies a strict separator. Cauchy–Schwarz also gives $\gamma\leq R$. There is no need to know an optimal margin or solve a margin optimization problem first.

Here is one exact algorithm.

1. Set $\theta=0$ and the update count $t=0$.
2. Make a complete pass through indices $1,\ldots,m$, starting with an update flag set to false.
3. At a visited index i, if $y_i\theta^\top z_i\leq0$, replace $\theta$ by $\theta+y_i z_i$, increase t by one and set the flag to true. Otherwise leave it unchanged.
4. Stop if the entire pass had no update. Otherwise begin another full pass.

Ties are included deliberately. Starting at zero with a strict-negative update test would make no first update at all. Our condition also updates a positive-labelled zero, even though the chosen prediction rule calls that tie positive. We therefore count mistake-or-tie updates. Ordinary prediction mistakes are a subset of them; the two counts need not be equal.

No normalization is performed between updates. The unit step size and zero initialization are part of the algorithm being proved.

The update bound

Let $\theta_t$ mean the parameter after exactly t updates, regardless of how many data visits occurred. If the next updated index is i, then

$$\theta_{t+1}^\top v =\theta_t^\top v+y_i z_i^\top v \geq\theta_t^\top v+\gamma.$$

Induction from $\theta_0=0$ gives $\theta_T^\top v\geq T\gamma$ for every realized update count T. At the same update,

$$\begin{aligned} \|\theta_{t+1}\|^2 &=\|\theta_t\|^2+2y_i\theta_t^\top z_i+y_i^2\|z_i\|^2\\ &\leq\|\theta_t\|^2+R^2, \end{aligned}$$

because the update condition makes the middle term nonpositive and $y_i^2=1$. A second induction gives $\|\theta_T\|^2\leq TR^2$.

For $T=0$, the bound below is immediate. For $T>0$, Cauchy–Schwarz with the unit vector v yields

$$T\gamma\leq\theta_T^\top v\leq\|\theta_T\| \leq R\sqrt T, \qquad T\leq\left(\frac R\gamma\right)^2.$$

This is a bound on the total number of updates, and hence also on the number of actual prediction mistakes encountered by this algorithm. The proof includes the initial zero vector and never divides by its norm. It does not require v to be a maximum-margin vector.

Why the stated scan really stops

The bound applies to every finite realized prefix of updates. An infinite update sequence would have a prefix with integer length greater than $(R/\gamma)^2$, a contradiction. Our full-pass algorithm cannot run forever without stopping: every nonfinal pass contains an update, so infinitely many nonfinal passes would give infinitely many updates.

If its total update count is T, it has at most T nonfinal passes and one final pass. It therefore uses at most $m(T+1)$ data visits. During the final pass the parameter never changes; every visited index satisfies $y_i\theta^\top z_i>0$. Because all m indices were checked, the final parameter strictly separates the entire training list.

The initial nonempty list forces at least one update, since all signed scores initially vanish. Thus the final parameter is nonzero, also seen from $\theta_T^\top v\geq T\gamma>0$. One may rescale the final parameter by its positive norm without changing predictions. Doing so after every update would change the recurrence and would require a different proof.

A different algorithm may search the entire finite list for any violating index at each stage and stop only when none exists; the same update bound works. An unfair scan that keeps ignoring a violating point has no such certificate. A fixed iteration cap is likewise not a certificate unless all final inequalities are checked. The bound does not turn an arbitrary scan or stopping rule into this algorithm.

A useful constant-predictor case

For one example $x_1=0$ with label plus one, $z_1=(0,1)$, $R=1$, and $v=(0,1)$ has $\gamma=1$. The first update gives $\theta=(0,1)$, followed by a no-update pass. The bound $T\leq1$ is attained. The result is the constant score one: its augmented parameter is nonzero, but its feature normal w is zero. It is a valid perceptron result, while the feature-space hyperplane-distance formula is inapplicable. Augmentation does not erase this distinction.

4. Standard hard margin: existence must include the intercept

For this section assume in addition that both labels occur and that the finite list is strictly affinely separable. The standard hard-margin problem is

$$\min_{w\in\mathbb R^d,\ b\in\mathbb R}\frac12\|w\|^2 \quad\text{subject to}\quad y_i(w^\top x_i+b)\geq1,\quad i=1,\ldots,m.$$

The intercept b is unpenalized. This is not minimization of $\|\theta\|^2/2=(\|w\|^2+b^2)/2$ in the augmented space.

The feasible set is nonempty. Indeed, if $(u,c)$ strictly separates and $a=\min\limits_i y_i(u^\top x_i+c)>0$, then $(u/a,c/a)$ is feasible. Conversely feasibility implies strict separation. With both labels, a feasible pair cannot have w zero: a positive label would require $b\geq1$ and a negative one $b\leq-1$.

Attainment by a bounded feasible sublevel

Choose one feasible pair $(w_0,b_0)$, and put $C=\|w_0\|>0$. Fix a positive-labelled index p and a negative-labelled index q. For every feasible pair with $\|w\|\leq C$, its two corresponding constraints imply

$$b\geq1-w^\top x_p\geq1-C\|x_p\|, \qquad b\leq-1-w^\top x_q\leq-1+C\|x_q\|.$$

Thus b is bounded on the relevant sublevel, even though the objective does not penalize it. Define K to be the feasible pairs with $\|w\|\leq C$ and b in this fixed closed interval. The benchmark belongs to K. Every constraint is a continuous affine inequality, so K is closed; the displayed bounds make it bounded in the finite-dimensional space $\mathbb R^{d+1}$.

For completeness, take a sequence in K whose objective values tend to its infimum. Finite-dimensional bounded-sequence compactness gives a convergent subsequence; closedness puts its limit in K. Continuity of $\|w\|^2/2$ gives a minimum there. Any feasible pair outside K must have $\|w\|>C$, since feasible pairs inside that norm ball already obey the b bounds. Its objective is then greater than the benchmark's. The minimum on K is therefore a global minimum over all feasible pairs.

This argument earns attainment. Merely saying that the constraints are closed would not suffice on an unbounded set, and the w penalty alone does not automatically bound b without the two labels.

The normal is unique

The feasible set is convex: any convex combination of two feasible pairs satisfies every affine constraint. Suppose $(w_1,b_1)$ and $(w_2,b_2)$ are minimizing pairs. Their feasible midpoint has objective

$$\frac12\left\|\frac{w_1+w_2}{2}\right\|^2 =\frac12\left(\frac{\|w_1\|^2}{2}+\frac{\|w_2\|^2}{2}\right) -\frac18\|w_1-w_2\|^2.$$

If $w_1\ne w_2$, this is strictly below the common minimum, impossible. Hence the minimizing w is unique. This argument does not penalize or establish strict convexity in b; no general assertion about b is inferred from it. In Question 4 the particular constraints themselves will determine b uniquely.

Why this minimizes the right norm for geometric margin

At any optimizer $(w_*,b_*)$, let $a_*=\min\limits_i y_i(w_*^\top x_i+b_*)$. Feasibility gives $a_*\geq1$. If $a_*>1$, divide both parameters by $a_*$. This preserves feasibility and strictly reduces the objective because $w_*\ne0$, a contradiction. Thus $a_*=1$ and

$$\delta(w_*,b_*)=\frac1{\|w_*\|}.$$

For any strict separator $(u,c)$, both labels imply $u\ne0$. With $a=\min\limits_i y_i(u^\top x_i+c)>0$, the feasible pair $(u/a,c/a)$ has normal norm $\|u\|/a=1/\delta(u,c)$. Optimality of $w_*$ implies $\|w_*\|\leq1/\delta(u,c)$, or $\delta(u,c)\leq1/\|w_*\|$. The optimizer itself attains equality, so a maximum geometric margin really exists and is attained.

Conversely, if a strict separator attains this largest geometric margin, rescaling it to functional minimum one gives objective $1/(2\delta^2)$, equal to the optimal value. Hence the hard-margin formulation and feature-space geometric-margin maximization agree under the stated hypotheses. The transformation rescales parameters, not the observed feature coordinates.

5. The augmented geometry is a different geometry

The perceptron estimates progress against a unit vector in $\mathbb R^{d+1}$, using the norm $\sqrt{\|w\|^2+b^2}$. Its separator assumption is an augmented-space assumption. Standard hard margin instead minimizes the feature-normal norm $\|w\|$ and leaves b unpenalized.

Even when the same constraints $y_i(w^\top x_i+b)\geq1$ are imposed, minimizing $(\|w\|^2+b^2)/2$ specifies a different objective. Its distance is measured in the augmented space, where the final data coordinate was fixed to one; this is not automatically the distance obtainable by perturbing only the original features. Question 4 derives different optimizers and different feature-space margins on the same two observations.

The role of both labels is also substantive. For a single positive example, choosing w zero and $b\geq1$ minimizes the standard objective at zero, and b need not be bounded. Such minimizers exist, but their feature hyperplane is degenerate. The geometric-margin equivalence just proved is not a statement about that case. If opposite labels share the same input, the two hard constraints contradict one another and there is no feasible pair. The next examples make the failure of a separation assumption explicit.

6. What the theorems certify

The perceptron theorem certifies a finite number of mistake-or-tie updates and termination of the specified fair scan on strictly separable finite data. Its proof does not minimize the standard hard-margin objective and does not claim that the final parameter is a maximum-margin solution.

The hard-margin theorem supplies an attained optimum for its stated feasible model, with a unique feature normal. It says neither that every classification dataset is separable nor that a perfect training separator predicts new data accurately. No soft margin, kernel, duality, KKT theorem, probability bound or floating-point stability result is being used.

Common checks are therefore mathematical checks of the model itself:

• Write the tie convention before interpreting a zero signed score.
• Include the constant coordinate when computing the perceptron radius R.
• Count updates separately from visits and verify an entire no-update pass.
• Normalize w and b together when comparing the same hyperplane.
• Check whether the objective penalizes the intercept.
• Use both-label and strict-separation hypotheses when applying the margin optimizer theorem.

7. Questions and complete Solutions

Question 1: a least-squares score can choose the wrong boundary

Take one-dimensional inputs $x=(-1,0,1,100)$ and labels $y=(-1,-1,+1,+1)$. Fit the affine least-squares score $b+wx$ using the existing normal-equation theorem. Write its two-by-two system, calculate all four predictions with the stated positive tie rule, and compare it with the score $x-1/2$. Explain what a zero score predicts for each possible label, and why a zero feature normal has no hyperplane-distance formula.

Solution. The design rows are $(1,x_i)$, with coefficient vector $(b,w)^\top$. Direct finite sums give

$$A^\top A= \begin{bmatrix}4&100\\100&10002\end{bmatrix}, \qquad A^\top y=\begin{bmatrix}0\\102\end{bmatrix}.$$

The first normal equation gives $b=-25w$. Substituting into the second gives $(10002-2500)w=102$, so

$$w=\frac{51}{3751},\qquad b=-\frac{1275}{3751},\qquad (f(x_1),f(x_2),f(x_3),f(x_4)) =\frac1{3751}(-1326,-1275,-1224,3825).$$

There are no ties in these four scores. Thresholding gives $(-1,-1,-1,+1)$, with the single error at input one and empirical error $1/4$. The normal-equation result guarantees the optimum of the numerical squared-loss problem, not the optimum of label-error count.

The score $g(x)=x-1/2$ has values $(-3/2,-1/2,1/2,199/2)$ at the same inputs. Every signed score is positive, so it has zero classification error. Its large numerical output at100 can be costly for squared loss while remaining a correct positive prediction. There is no contradiction between the two objectives.

At a zero score our classifier outputs plus one: a positive label is correct and a negative label is incorrect. Yet the signed product is zero for both, so it cannot distinguish them. Finally w zero makes the score constant. Its zero level is either empty or all of the line, and dividing by $|w|$ is invalid. A constant prediction rule can be defined, but the nonzero-normal distance proposition cannot be applied to it.

Question 2: follow every visit before using an update bound

Use the ordered list $((1,0),+1),((0,1),-1),((1,1),+1)$. Start the stated augmented perceptron at $\theta=(0,0,0)$ with $z=(x_1,x_2,1)$. Record each visit, its signed score, and the resulting parameter. Stop only after a full no-update pass. Give a unit separator, R and $\gamma$, and compare the actual updates and visits with their bounds.

Solution. The signed score is evaluated before any update at that visit. The full calculation is:

For example, the second visit uses $z_2=(0,1,1)$ with label minus one, so it subtracts that vector from $(1,0,1)$. The third visit has positive label and score zero; it is an update even though the positive tie rule predicts that label correctly.

There are $T=4$ updates, three passes and nine visits. Take

$$v=\frac1{\sqrt5}(2,-1,0),\qquad (y_1v^\top z_1,y_2v^\top z_2,y_3v^\top z_3) =\frac1{\sqrt5}(2,1,1).$$

Thus $\gamma=1/\sqrt5$. The first two augmented vectors have norm $\sqrt2$ and the third has norm $\sqrt3$, so $R=\sqrt3$. The theorem gives $T\leq R^2/\gamma^2=15$, a valid but loose bound. The visit bound using the actual T is $m(T+1)=3(5)=15$, while the actual number is nine.

The final pass matters: its three positive signed scores certify the whole list for one fixed parameter. Stopping at the earlier positive score of a single observation would not provide that certificate. The final parameter is a strict separator, but this computation has not established a maximum-margin optimization claim.

Question 3: distinct-input failure and inconsistent-label cycling

First give positive labels to $(1,1),(-1,-1)$ and negative labels to $(1,-1),(-1,1)$. Prove that no affine score strictly separates these XOR data. Next use the single one-dimensional input zero twice, with labels plus one and minus one in that order. Show exactly how the zero-initialized perceptron cycles under this repeated scan, and identify the theorem hypothesis that fails.

Solution. Write the affine parameters as $(w_1,w_2,b)$. The positive constraints would be $w_1+w_2+b>0$ and $-w_1-w_2+b>0$. Adding them gives $2b>0$. The negative constraints would be $w_1-w_2+b<0$ and $-w_1+w_2+b<0$. Adding them gives $2b<0$. This is impossible. Distinct inputs alone do not imply affine strict separability.

For the repeated zero input, both augmented vectors are $(0,1)$, with opposite labels. Starting at $(w,b)=(0,0)$, the first positive signed score is zero, so the update gives $(0,1)$. The negative example then has signed score minus one, so subtracting $(0,1)$ gives $(0,0)$ again. Each complete pass repeats

$$(0,0)\longrightarrow(0,1)\longrightarrow(0,0).$$

Every pass has updates, so the specified stopping test never succeeds. There cannot be a unit v and positive $\gamma$ with both $v^\top(0,1)\geq\gamma$ and $-v^\top(0,1)\geq\gamma$; adding would give $0\geq2\gamma$. The update theorem is therefore inapplicable. Normalizing the recurring zero parameter would also be undefined.

The second data list is more than a failure of affine geometry: one deterministic label rule at the same input cannot output two different labels simultaneously. Neither example is repaired by citing a theorem whose separation hypothesis is false. No nonlinear or kernel construction is claimed here.

Question 4: the bias penalty changes the separating line

In one dimension take a positive example at1 and a negative example at minus five. Derive the optimizer of standard hard margin with unpenalized b, including its feature-space geometric margin. Then minimize $(w^2+b^2)/2$ under the same constraints and compare boundaries and margins. Finally delete the negative example and explain the resulting degeneracy without falsely claiming nonexistence.

Solution. The two constraints are

$$w+b\geq1,\qquad 5w-b\geq1.$$

Adding gives $6w\geq2$, so $w\geq1/3$. The standard objective $w^2/2$ is therefore minimized at $w=1/3$, provided it is feasible. At that value the separate constraints require $b\geq2/3$ and $b\leq2/3$, hence $b=2/3$. Both are equalities, proving feasibility and the unique b in this particular example. The boundary is $x=-b/w=-2$; each example is distance three from it. Equivalently the functional minimum is one and $\delta=1/|w|=3$. The standard objective value is $1/18$.

For the augmented objective, the positive constraint and Cauchy–Schwarz imply

$$w^2+b^2\geq\frac{(w+b)^2}{2}\geq\frac12.$$

Equality requires $w=b$ and $w+b=1$, so the only possible equality pair is $w=b=1/2$. It also satisfies the other constraint, since $5/2-1/2=2\geq1$. Hence it is the unique augmented-norm optimizer, with objective value $1/4$.

Its boundary is $x=-1$. The positive point is distance two and the negative point distance four from it, so the minimum feature-space margin is two. Both classifiers have zero training error, but standard feature-space margin three is larger. Their objective values belong to different objectives and should not be treated as competing measurements of the same loss.

The augmented margin at $(1/2,1/2)$ is $1/\sqrt{1/2}=\sqrt2$, whereas its feature margin is two. At the standard pair the augmented norm is $\sqrt5/3$, giving augmented margin $3/\sqrt5$. These different values display the two geometries explicitly.

If the negative example is deleted, the only constraint is $w+b\geq1$. All pairs $w=0,b\geq1$ attain objective zero in the standard problem. Existence has not failed, but b is not unique and the feature normal is zero. The nondegenerate hyperplane-margin theorem was stated for both labels and cannot be used here. In fact fixed unit normal $w=1$ with b increasing gives arbitrarily large feature margins for this one point; there is no finite largest such margin among genuine hyperplanes. This is precisely why the constant-predictor optimum and the both-label geometric equivalence are different cases.

8. Connections and the boundary of this supplement

The same real feature table can support numerical regression, labelled projection, learned distance geometry and binary classification. The objective and the output determine which calculation is appropriate. Fisher's direction need not solve the hard-margin constraints, and a nearest-neighbor squared-distance margin is not the geometric quantity derived here.

For computation with smooth quadratic losses, continue with gradient descent on quadratics. For distinguishing fitting from evaluation, use validation and ridge regression. Those connections do not turn the finite-data theorems in this note into statistical prediction guarantees.

The proved endpoints are binary affine geometry, the specified perceptron update and stopping bounds, and attained standard hard margin under finite strict separation with both labels. Soft margins, kernels, dual formulations, multiclass models and probabilistic generalization are separate subjects. The four Questions and their complete solutions above are original.