Python Linear Regression

Last modified on: October 20, 2024

<h2>Tutorial: Linear Regression in Python</h2>
Linear regression is a fundamental technique in statistics and machine learning that models the relationship between a dependent variable yyy and one or more independent variables xxx. In its simplest form, linear regression finds the best-fitting straight line through the data points, which can be represented as:
y=&beta;0+&beta;1xy = \beta_0 + \beta_1 xy=&beta;0+&beta;1x
Where:
<ul>
<li>&beta;0\beta_0&beta;0 is the intercept (the value of yyy when x=0x = 0x=0).</li>
<li>&beta;1\beta_1&beta;1 is the slope of the line (how much yyy changes for a one-unit change in xxx).</li>
</ul>
In Python, linear regression can be performed using libraries such as <code>scikit-learn</code>, <code>numpy</code>, and <code>matplotlib</code> for visualization.
<hr />
<h2>Tutorial: Linear Regression in Python</h2>
<h3>1. Importing Required Libraries</h3>
<div class="dark bg-gray-950 contain-inline-size rounded-md border-[0.5px] border-token-border-medium relative">
<div class="flex items-center text-token-text-secondary bg-token-main-surface-secondary px-4 py-2 text-xs font-sans justify-between rounded-t-md h-9">
<pre class="language-python"><code>import numpy as np
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression
from sklearn.metrics import mean_squared_error, r2_score</code></pre>
</div>
</div>
<ul>
<li><code>numpy</code>: For handling numerical data and arrays.</li>
<li><code>matplotlib</code>: For plotting the data and the regression line.</li>
<li><code>LinearRegression</code>: From <code>sklearn</code> to perform linear regression.</li>
<li><code>mean_squared_error</code>, <code>r2_score</code>: To evaluate the regression model.</li>
</ul>
<hr />
<h3>2. Generating Sample Data</h3>
Let's generate some random data for the independent variable xxx and dependent variable yyy that has a linear relationship.
<div class="dark bg-gray-950 contain-inline-size rounded-md border-[0.5px] border-token-border-medium relative">
<div class="flex items-center text-token-text-secondary bg-token-main-surface-secondary px-4 py-2 text-xs font-sans justify-between rounded-t-md h-9">
<pre class="language-python"><code>np.random.seed(0) # For reproducibility
x = 2 * np.random.rand(100, 1) # 100 random data points for x
y = 4 + 3 * x + np.random.randn(100, 1) # y = 4 + 3x + noise

# Scatter plot of the data
plt.scatter(x, y, color='blue')
plt.title("Scatter Plot of the Data")
plt.xlabel("X")
plt.ylabel("Y")
plt.show()</code></pre>
</div>
</div>
Here, we generate a simple linear relationship y=4+3xy = 4 + 3xy=4+3x with some random noise added to simulate real-world data.
<hr />
<h3>3. Fitting a Linear Regression Model</h3>
To fit a linear regression model, we'll use the <code>LinearRegression()</code> class from <code>sklearn</code>.
<div class="dark bg-gray-950 contain-inline-size rounded-md border-[0.5px] border-token-border-medium relative">
<div class="flex items-center text-token-text-secondary bg-token-main-surface-secondary px-4 py-2 text-xs font-sans justify-between rounded-t-md h-9">
<pre class="language-python"><code>lin_reg = LinearRegression()
lin_reg.fit(x, y) # Fit the model

# Predict y values based on the model
y_pred = lin_reg.predict(x)</code></pre>
</div>
</div>
In this step:
<ul>
<li><code>fit(x, y)</code> computes the best-fit line by finding the optimal slope and intercept.</li>
<li><code>predict(x)</code> generates the predicted values of yyy based on the fitted model.</li>
</ul>
<hr />
<h3>4. Visualizing the Linear Regression Line</h3>
We can plot the original data points and the fitted regression line to visualize the result.
<div class="dark bg-gray-950 contain-inline-size rounded-md border-[0.5px] border-token-border-medium relative">
<div class="flex items-center text-token-text-secondary bg-token-main-surface-secondary px-4 py-2 text-xs font-sans justify-between rounded-t-md h-9">
<pre class="language-python"><code># Plot original data and the regression line
plt.scatter(x, y, color='blue', label='Original data')
plt.plot(x, y_pred, color='red', label='Fitted line')
plt.title('Linear Regression Fit')
plt.xlabel('X')
plt.ylabel('Y')
plt.legend()
plt.show()</code></pre>
</div>
</div>
The red line represents the linear regression fit, showing how well the model predicts the relationship between xxx and yyy.
<hr />
<h3>5. Interpreting the Model Parameters</h3>
Once the model is fitted, you can extract the coefficients (slope) and intercept of the best-fit line.
<div class="dark bg-gray-950 contain-inline-size rounded-md border-[0.5px] border-token-border-medium relative">
<div class="flex items-center text-token-text-secondary bg-token-main-surface-secondary px-4 py-2 text-xs font-sans justify-between rounded-t-md h-9">
<pre class="language-python"><code>slope = lin_reg.coef_[0]
intercept = lin_reg.intercept_
print(f"Slope (Coefficient): {slope}")
print(f"Intercept: {intercept}")</code></pre>
</div>
</div>
The output gives us:
<ul>
<li>The slope, which indicates how much yyy changes for a one-unit increase in xxx.</li>
<li>The intercept, which indicates the value of yyy when x=0x = 0x=0.</li>
</ul>
<hr />
<h3>6. Evaluating the Model</h3>
You can evaluate the performance of the regression model using metrics such as Mean Squared Error (MSE) and R-squared (R&sup2;).
<div class="dark bg-gray-950 contain-inline-size rounded-md border-[0.5px] border-token-border-medium relative">
<div class="flex items-center text-token-text-secondary bg-token-main-surface-secondary px-4 py-2 text-xs font-sans justify-between rounded-t-md h-9">
<pre class="language-python"><code>mse = mean_squared_error(y, y_pred)
r2 = r2_score(y, y_pred)
print(f"Mean Squared Error (MSE): {mse}")
print(f"R-squared (R&sup2;): {r2}")</code></pre>
</div>
</div>
<ul>
<li>MSE measures the average squared difference between the actual and predicted values. Lower values are better.</li>
<li>R-squared indicates how well the model explains the variance in the data. Values close to 1 indicate a good fit.</li>
</ul>
<hr />
<h3>7. Predicting New Values</h3>
You can use the fitted model to make predictions for new, unseen values of xxx.
<div class="dark bg-gray-950 contain-inline-size rounded-md border-[0.5px] border-token-border-medium relative">
<div class="flex items-center text-token-text-secondary bg-token-main-surface-secondary px-4 py-2 text-xs font-sans justify-between rounded-t-md h-9">
<pre class="language-python"><code>x_new = np.array([[1.5], [3.0]]) # New values of x
y_new_pred = lin_reg.predict(x_new)
print(f"Predicted values for new x: {y_new_pred}")</code></pre>
</div>
</div>
This predicts the yyy-values for the new data points x=1.5x = 1.5x=1.5 and x=3.0x = 3.0x=3.0 based on the fitted regression model.
<hr />
<h3>Conclusion</h3>
In this tutorial, we covered:
<ol>
<li>How to generate and visualize data for linear regression.</li>
<li>How to fit a linear regression model using <code>scikit-learn</code>.</li>
<li>How to visualize the best-fit line.</li>
<li>How to extract and interpret the model parameters (slope and intercept).</li>
<li>How to evaluate the model using MSE and R-squared.</li>
<li>How to make predictions using the trained model.</li>
</ol>
Linear regression is a simple but powerful tool for modeling relationships between variables, and it's a foundational concept for more complex machine learning models.

Web Technology View More