Python Polynomial Regression

Last modified on: October 19, 2024

<h2>Polynomial Regression - Tutorial: Polynomial Regression in Python</h2>
<p><strong>Polynomial regression</strong> is a type of regression analysis in which the relationship between the independent variable <span class="katex"><span class="katex-mathml">xx</span><span class="katex-html" aria-hidden="true"><span class="base"><span class="mord mathnormal">x</span></span></span></span> and the dependent variable <span class="katex"><span class="katex-mathml">yy</span><span class="katex-html" aria-hidden="true"><span class="base"><span class="mord mathnormal">y</span></span></span></span> is modeled as an <span class="katex"><span class="katex-mathml">nn</span><span class="katex-html" aria-hidden="true"><span class="base"><span class="mord mathnormal">n</span></span></span></span>-degree polynomial. It extends linear regression by fitting a polynomial equation to the data, which is particularly useful when the data shows a non-linear relationship.</p>
<p>In Python, we can perform polynomial regression using the following steps:</p>
<h2><strong>Tutorial: Polynomial Regression in Python</strong></h2>
<h3><strong>1. Import Required Libraries</strong></h3>
<div class="dark bg-gray-950 contain-inline-size rounded-md border-[0.5px] border-token-border-medium relative">
<div class="flex items-center text-token-text-secondary bg-token-main-surface-secondary px-4 py-2 text-xs font-sans justify-between rounded-t-md h-9">
<pre class="language-python"><code>import numpy as np
import matplotlib.pyplot as plt
from sklearn.linear_model import LinearRegression
from sklearn.preprocessing import PolynomialFeatures</code></pre>
</div>
</div>
<ul>
<li><code>numpy</code> is used for generating and manipulating data.</li>
<li><code>matplotlib</code> is used for visualizing the results.</li>
<li><code>sklearn.linear_model.LinearRegression</code> is used to perform regression.</li>
<li><code>sklearn.preprocessing.PolynomialFeatures</code> is used to generate polynomial features.</li>
</ul>
<hr />
<h3><strong>2. Generate Sample Data</strong></h3>
<p>Let's create some synthetic data that follows a quadratic relationship between <span class="katex"><span class="katex-mathml">xx</span><span class="katex-html" aria-hidden="true"><span class="base"><span class="mord mathnormal">x</span></span></span></span> and <span class="katex"><span class="katex-mathml">yy</span><span class="katex-html" aria-hidden="true"><span class="base"><span class="mord mathnormal">y</span></span></span></span>, with some added noise.</p>
<div class="dark bg-gray-950 contain-inline-size rounded-md border-[0.5px] border-token-border-medium relative">
<div class="flex items-center text-token-text-secondary bg-token-main-surface-secondary px-4 py-2 text-xs font-sans justify-between rounded-t-md h-9">
<pre class="language-python"><code>np.random.seed(0)  # For reproducibility
x = 2 - 3 * np.random.normal(0, 1, 100)
y = x - 2 * (x ** 2) + np.random.normal(-3, 3, 100)

# Reshape x to make it compatible with sklearn (2D array)
x = x[:, np.newaxis]

# Scatter plot the data
plt.scatter(x, y, color='blue')
plt.title("Scatter Plot of Original Data")
plt.xlabel("X")
plt.ylabel("Y")
plt.show()</code></pre>
</div>
</div>
<p>Here, we generate a quadratic relationship: <span class="katex"><span class="katex-mathml">y=x&minus;2x2y = x - 2x^2</span><span class="katex-html" aria-hidden="true"><span class="base"><span class="mord mathnormal">y</span><span class="mrel">=</span></span><span class="base"><span class="mord mathnormal">x</span><span class="mbin">&minus;</span></span><span class="base"><span class="mord">2</span><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist"><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span></span></span></span></span></span></span>, with some random noise added to the <span class="katex"><span class="katex-mathml">yy</span><span class="katex-html" aria-hidden="true"><span class="base"><span class="mord mathnormal">y</span></span></span></span>-values.</p>
<hr />
<h3><strong>3. Perform Linear Regression</strong></h3>
<p>We'll first try to fit a <strong>linear regression</strong> model to the data to see how well it fits.</p>
<div class="dark bg-gray-950 contain-inline-size rounded-md border-[0.5px] border-token-border-medium relative">
<div class="flex items-center text-token-text-secondary bg-token-main-surface-secondary px-4 py-2 text-xs font-sans justify-between rounded-t-md h-9">
<pre class="language-python"><code>lin_reg = LinearRegression()
lin_reg.fit(x, y)
y_pred_linear = lin_reg.predict(x)

# Plot the linear regression fit
plt.scatter(x, y, color='blue')
plt.plot(x, y_pred_linear, color='red')
plt.title('Linear Regression Fit')
plt.xlabel('X')
plt.ylabel('Y')
plt.show()</code></pre>
</div>
</div>
<p>You will notice that the <strong>linear regression</strong> line does not fit the data well because the data follows a non-linear pattern.</p>
<hr />
<h3><strong>4. Perform Polynomial Regression</strong></h3>
<p>Now, let's fit a polynomial regression model, starting with a quadratic (degree 2) polynomial.</p>
<div class="dark bg-gray-950 contain-inline-size rounded-md border-[0.5px] border-token-border-medium relative">
<div class="flex items-center text-token-text-secondary bg-token-main-surface-secondary px-4 py-2 text-xs font-sans justify-between rounded-t-md h-9">
<pre class="language-python"><code># Create polynomial features (degree 2)
poly = PolynomialFeatures(degree=2)
x_poly = poly.fit_transform(x)

# Fit the polynomial regression model
poly_reg = LinearRegression()
poly_reg.fit(x_poly, y)
y_pred_poly = poly_reg.predict(x_poly)

# Plot the polynomial regression fit
plt.scatter(x, y, color='blue')
plt.plot(x, y_pred_poly, color='red')
plt.title('Polynomial Regression Fit (Degree 2)')
plt.xlabel('X')
plt.ylabel('Y')
plt.show()</code></pre>
</div>
</div>
<p>Here&rsquo;s what happens:</p>
<ul>
<li><code>PolynomialFeatures(degree=2)</code> generates the polynomial features (i.e., <span class="katex"><span class="katex-mathml">xx</span><span class="katex-html" aria-hidden="true"><span class="base"><span class="mord mathnormal">x</span></span></span></span>, <span class="katex"><span class="katex-mathml">x2x^2</span><span class="katex-html" aria-hidden="true"><span class="base"><span class="mord"><span class="mord mathnormal">x</span><span class="msupsub"><span class="vlist-t"><span class="vlist-r"><span class="vlist"><span class="sizing reset-size6 size3 mtight"><span class="mord mtight">2</span></span></span></span></span></span></span></span></span></span>, etc.).</li>
<li>We use <code>LinearRegression()</code> to fit a linear model to these polynomial features.</li>
<li>The red line represents the quadratic polynomial that fits the data much better than the linear model.</li>
</ul>
<hr />
<h3><strong>5. Higher-Degree Polynomial Regression</strong></h3>
<p>You can experiment with higher-degree polynomials to see if they fit the data better, though be cautious of <strong>overfitting</strong>.</p>
<h4><strong>Fitting a 4th Degree Polynomial:</strong></h4>
<div class="dark bg-gray-950 contain-inline-size rounded-md border-[0.5px] border-token-border-medium relative">
<div class="flex items-center text-token-text-secondary bg-token-main-surface-secondary px-4 py-2 text-xs font-sans justify-between rounded-t-md h-9">
<pre class="language-python"><code># Create polynomial features (degree 4)
poly = PolynomialFeatures(degree=4)
x_poly = poly.fit_transform(x)

# Fit the polynomial regression model
poly_reg = LinearRegression()
poly_reg.fit(x_poly, y)
y_pred_poly = poly_reg.predict(x_poly)

# Plot the polynomial regression fit (degree 4)
plt.scatter(x, y, color='blue')
plt.plot(np.sort(x, axis=0), y_pred_poly[np.argsort(x, axis=0)], color='red')
plt.title('Polynomial Regression Fit (Degree 4)')
plt.xlabel('X')
plt.ylabel('Y')
plt.show()</code></pre>
</div>
</div>
<p>A degree 4 polynomial may capture more details of the data but can lead to overfitting if the degree is too high.</p>
<hr />
<h3><strong>6. Evaluating the Model</strong></h3>
<p>To measure how well the polynomial model fits the data, you can evaluate it using metrics like <strong>R-squared</strong> and <strong>Mean Squared Error (MSE)</strong>.</p>
<h4><strong>R-squared Score:</strong></h4>
<div class="dark bg-gray-950 contain-inline-size rounded-md border-[0.5px] border-token-border-medium relative">
<div class="flex items-center text-token-text-secondary bg-token-main-surface-secondary px-4 py-2 text-xs font-sans justify-between rounded-t-md h-9">
<pre class="language-python"><code>from sklearn.metrics import r2_score

# Calculate R-squared
r2_poly = r2_score(y, y_pred_poly)
print(f'R-squared (Polynomial Regression, degree 4): {r2_poly}')</code></pre>
</div>
</div>
<p>A higher R-squared value indicates a better fit.</p>
<hr />
<h3><strong>7. Conclusion</strong></h3>
<p>In this tutorial, we:</p>
<ul>
<li>Generated synthetic data following a polynomial relationship.</li>
<li>Tried to fit a <strong>linear regression</strong> model, which didn&rsquo;t fit well.</li>
<li>Applied <strong>polynomial regression</strong> (degree 2 and 4), which provided a better fit to the data.</li>
<li>Showed how to evaluate the performance of the model using R-squared.</li>
</ul>
<p>Polynomial regression is a useful tool when dealing with non-linear relationships, but be cautious of <strong>overfitting</strong> as you increase the polynomial degree.</p>

Web Technology View More