Xrfgtjtk

I am confused by the language of my math text book where it says $hat y = alpha times$basis function summation from $i=1$ to $n$? What are basis functions? And why does it says there are more data points than basis functions? I know how to derive $alpha$ using linear algebra and calculus by using $mx + c$ as the equation of the regression line.

Also I am a bit confused on the fact that how does it know that the smaller distance of the actual data point and $hat y$ is perpendicular to the line of $hat y$? I will be very very grateful if anyone can help me with this.

You are approximating the data points with a linear combination of independent functions. In the case of the straight line of equation

$$y=mx+p$$

there are two functions which are the identity,

$$y=x$$ and the unit,

$$y=1.$$

You can generalize with more functions, such as the powers $x^2,x^3,cdots$ or any other. Basis function comes from the terminology of linear algebra.

Every data point lets you write an equation, such as

$$(x_k,y_k)to y_k=mx_k+p.$$

When there are more points than basis functions, i.e. than unknown coefficients, the system is overdeterminate and has no solution. In this case, you try to find a good compromise, i.e. one that approximately satisfies all equations.

Notice that the minimization via the model $y=mx+p$ does not achieve the minimum distance to the lines, but the vertical distance to the line. See total least squares for the former. https://en.wikipedia.org/wiki/Total_least_squares

A model which is linear with respect to the parameters write
$$y=sum_{k=1}^n a_k,f_k(x)$$ and the $f_k(x)$ are the $k$ basis functions.

Suppose that you want to fit data to the model
$$y=a_0+a_1sin(x)+a_2log(x)+a_3 e^{-pi x}$$ Define $t_i=sin(x_i)$, $u=log(x_i)$, $v=e^{-pi x_i}$. So the model is just
$$y=a_0+a_1t+a_2u+a_3v$$ which corresponds to a multilinear regression.