- Linear regression in Python is most commonly implemented with the LinearRegression class in scikit-learn 1.9.0, which fits an Ordinary Least Squares model by minimising the residual sum of squares
- For statistical inference with p-values and confidence intervals, statsmodels 0.14.6 provides the OLS class
- Both libraries are free, open-source, and widely used across data science.
Linear regression is one of the first algorithms every data scientist learns, and Python makes it easy to apply with just a few lines of code. This guide explains what linear regression is, the assumptions behind it, how to build and evaluate a model with scikit-learn and statsmodels, and where the technique is used in practice.
What Is Linear Regression in Python?
Linear regression in Python is a statistical method that models the relationship between a dependent variable and one or more independent variables by fitting a straight line. In Python, it is implemented mainly through scikit-learn's LinearRegression class and the statsmodels OLS class, both of which estimate coefficients using ordinary least squares.
At its core, linear regression fits a straight line of the form y equals a plus b times x, where a is the intercept and b is the slope. Simple linear regression uses a single predictor, while multiple linear regression uses several. Python estimates these coefficients automatically using ordinary least squares, which minimises the sum of the squared differences between the observed and predicted values.
What Are the Assumptions of Linear Regression?
Linear regression relies on four core assumptions, often summarised as LINE: a Linear relationship between predictors and response, Independent errors with little autocorrelation, Normally distributed errors, and Equal variance of errors, known as homoscedasticity. A further requirement is little multicollinearity, meaning the independent variables should not be strongly correlated with one another.
- Linearity: the relationship between each predictor and the response is linear, which you can check with scatter plots.
- Independence: the errors are independent, with little or no autocorrelation between residuals.
- Normality: the errors at each predictor value are normally distributed with a mean of zero.
- Equal variance (homoscedasticity): the errors have constant variance across all values of the predictors.
- Little multicollinearity: the independent variables are not strongly correlated with one another.
How Do You Build a Simple Linear Regression Model in Python?
To build a simple linear regression model, import LinearRegression from sklearn.linear_model, prepare your feature matrix X and target vector y, then call the fit method to train the model. Use the predict method for new values, and read the estimated slope from coef_ and the intercept from intercept_ after fitting completes.
A minimal example looks like this. First import the library with the line from sklearn.linear_model import LinearRegression. Prepare X as a two-dimensional array of features and y as the target. Then run model equals LinearRegression().fit(X, y). Call model.predict(X_new) to score new data, and inspect model.coef_ for the slope and model.intercept_ for the constant term.
- Import numpy, pandas, and LinearRegression from sklearn.linear_model.
- Load your data and split it into a feature matrix X and target vector y.
- Split into training and test sets with train_test_split from sklearn.model_selection.
- Create the model and train it by calling model.fit on the training data.
- Generate predictions with model.predict and evaluate them with model.score.
How Do scikit-learn and statsmodels Differ for Regression?
scikit-learn focuses on prediction and machine learning workflows, offering a simple fit and predict interface plus easy integration with pipelines and cross-validation. statsmodels focuses on statistical inference, so its OLS summary reports coefficients, standard errors, p-values, R-squared, and diagnostic tests. Many analysts use scikit-learn for modelling and statsmodels for interpretation.
| Aspect (2026) | scikit-learn 1.9.0 | statsmodels 0.14.6 |
|---|---|---|
| Primary use | Prediction and ML pipelines | Statistical inference |
| Import | sklearn.linear_model.LinearRegression | statsmodels.api.OLS |
| Fit step | model.fit(X, y) | OLS(y, X).fit() |
| Key output | coef_, intercept_, score | summary with p-values and R-squared |
| Best for | Building predictive models | Interpreting relationships |
Where Is Linear Regression Applied?
Linear regression is applied across many fields. In finance it helps forecast consumption, investment, and asset returns, and it underpins the beta term in the capital asset pricing model. Economists use it for trend analysis and demand forecasting, while researchers in the natural and social sciences use it to quantify causal relationships between variables.
- Trend lines: modelling how a measurable quantity such as GDP or prices changes over time.
- Finance: forecasting consumption, fixed investment, and inventory or import demand.
- Economics: quantifying systematic risk through the beta term in the capital asset pricing model.
- Natural and social sciences: measuring causal relationships between variables in experiments.
How Do You Evaluate a Linear Regression Model?
Evaluate a linear regression model using the coefficient of determination, R-squared, which the score method returns and where 1.0 is a perfect fit. Complement it with mean squared error, root mean squared error, and mean absolute error from sklearn.metrics. Finally, plot the residuals to confirm the linearity, independence, and equal variance assumptions hold.
- R-squared: the coefficient of determination returned by the score method, where 1.0 is a perfect fit.
- Mean squared error and root mean squared error from sklearn.metrics, which penalise larger errors.
- Mean absolute error: the average size of the prediction errors in the original units.
- Residual plots: visual checks that the linearity, independence, and equal variance assumptions hold.