Bivariate Case (OLS)

Given
\[ x = x_1,x_2,\cdots, x_n   \,\,\ \text{and}  \,\,\,  y = y_1,y_2,\cdots, y_n\]

We define the optimal minimum or the optimized mininum as:
\[ \text{Opt Min F} = \sum{[y_\text{actual}-f(x)]^2}\]
where
\[ \text{line  } y = f(x)= a +bx\]
Substituting the line $f(x)$ to the optimized minimum $F$, we have
\[ F = \sum{[y-a -bx)]^2}\]
Taking the first derivative with respect to $a$ and $b$,
\[ \begin{split} \frac{\partial F}{\partial a} &= 2 \sum{(y-a-bx)(-1)} = 0\\
\frac{\partial F}{\partial b} &= 2 \sum{(y-a-bx)(-x)} = 0 \end{split}  \]
We can now derive the normal equation for the OLS as:

\[ \begin{split} \sum{y} &= an + b \sum{x}\\
\sum{xy} &= a \sum{x} + b \sum{x^2} \end{split} \]

Solve for $a$ and $b$ to get critical points and find out if they really produced the optimum.