Where (X'X) -1X'X = I, the identity matrix. To solve for regression coefficients, simply pre-multiply by the inverse of X'X: Just as the regression equation can be expressed compactly in matrix form, so can the normal equations. It is sort of cool that this simple expression describes the regression equation for 1, 2, 3, or any number of independent variables.Īdvertisement Normal Equations in Matrix Form Given these matrices, the multiple regression equation can be expressed concisely as: Values for each independent variable in the regression equation.
Matrix X has a column of 1's plus k columns of K + 1 x 1 vector that holds estimated regression coefficients. Y is an n x 1 vector that holds predicted values of the dependent variable and b is a Each record includes scores for 1 dependent variable and k independent variables. Here, the data set consists of n records. To express the regression equation in matrix form, we need to define three matrices: Y, b, and X. Where ŷ is the predicted value of the dependent variable, b k are regression coefficients,Īnd x k is the value of independent variable k. With multiple regression, there is one dependent variable and k dependent variables. If you are unfamiliar with these topics, check out the free matrix algebra tutorial Specifically, you should beįamiliar with matrix addition, matrix subtraction, and matrix multiplication. To follow the discussion on this page, you need to understand a little matrix algebra.