Python Confusion Matrix

Last modified on: October 19, 2024

<div class="flex-shrink-0 flex flex-col relative items-end">
<div>
<div class="pt-0">
<div class="gizmo-bot-avatar flex h-8 w-8 items-center justify-center overflow-hidden rounded-full">
<h2 class="relative p-1 rounded-sm flex items-center justify-center bg-token-main-surface-primary text-token-text-primary h-8 w-8">Confusion Matrix in Python</h2>
</div>
</div>
</div>
</div>
<div class="group/conversation-turn relative flex w-full min-w-0 flex-col agent-turn">
<div class="flex-col gap-1 md:gap-3">
<div class="flex max-w-full flex-col flex-grow">
<div class="min-h-[20px] text-message flex w-full flex-col items-end gap-2 whitespace-normal break-words [.text-message+&amp;]:mt-5" dir="auto" data-message-author-role="assistant" data-message-id="9c9972aa-451f-4ad4-a068-4d9c3fc3976d">
<div class="flex w-full flex-col gap-1 empty:hidden first:pt-[3px]">
<div class="markdown prose w-full break-words dark:prose-invert light">
<p>A <strong>confusion matrix</strong> is a performance measurement tool for classification problems, helping us understand how well a classification model performs by comparing predicted and actual labels. It is particularly useful for binary and multiclass classification.</p>
<hr />
<h3><strong>Confusion Matrix Structure</strong></h3>
<p>For a binary classification problem, the confusion matrix looks like this:</p>
<span class="katex-display"><span class="katex"><span class="katex-mathml">Predicted:&nbsp;NoPredicted:&nbsp;YesActual:&nbsp;NoTrue&nbsp;Negative&nbsp;(TN)False&nbsp;Positive&nbsp;(FP)Actual:&nbsp;YesFalse&nbsp;Negative&nbsp;(FN)True&nbsp;Positive&nbsp;(TP)\begin{array}{|c|c|c|} \hline &amp; \textbf{Predicted: No} &amp; \textbf{Predicted: Yes} \\ \hline \textbf{Actual: No} &amp; \text{True Negative (TN)} &amp; \text{False Positive (FP)} \\ \textbf{Actual: Yes} &amp; \text{False Negative (FN)} &amp; \text{True Positive (TP)} \\ \hline \end{array}</span><span class="katex-html" aria-hidden="true"><span class="base"><span class="mord"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist"><span class="mtable"><span class="col-align-c"><span class="mord text"><span class="mord textbf">Actual:&nbsp;No</span></span><span class="mord text"><span class="mord textbf">Actual:&nbsp;Yes</span></span><span class="vlist-s"></span></span><span class="col-align-c"><span class="mord text"><span class="mord textbf">Predicted:&nbsp;No</span></span><span class="mord text">True&nbsp;Negative&nbsp;(TN)</span><span class="mord text">False&nbsp;Negative&nbsp;(FN)</span><span class="vlist-s"></span></span><span class="col-align-c"><span class="mord text"><span class="mord textbf">Predicted:&nbsp;Yes</span></span><span class="mord text">False&nbsp;Positive&nbsp;(FP)</span><span class="mord text">True&nbsp;Positive&nbsp;(TP)</span><span class="vlist-s"></span></span></span></span><span class="vlist-s"></span></span></span></span></span></span></span></span>
<p>Where:</p>
<ul>
<li><strong>True Positive (TP)</strong>: The model correctly predicted the positive class.</li>
<li><strong>True Negative (TN)</strong>: The model correctly predicted the negative class.</li>
<li><strong>False Positive (FP)</strong>: The model incorrectly predicted the positive class (Type I error).</li>
<li><strong>False Negative (FN)</strong>: The model incorrectly predicted the negative class (Type II error).</li>
</ul>
<hr />
<h3><strong>Metrics Derived from the Confusion Matrix</strong></h3>
<ol>
<li><strong>Accuracy</strong>: <span class="katex"><span class="katex-mathml">TP+TNTP+TN+FP+FN\frac{TP + TN}{TP + TN + FP + FN}</span><span class="katex-html" aria-hidden="true"><span class="base"><span class="mord"><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist"><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">TP</span><span class="mbin mtight">+</span><span class="mord mathnormal mtight">TN</span><span class="mbin mtight">+</span><span class="mord mathnormal mtight">FP</span><span class="mbin mtight">+</span><span class="mord mathnormal mtight">FN</span></span></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">TP</span><span class="mbin mtight">+</span><span class="mord mathnormal mtight">TN</span></span></span></span><span class="vlist-s"></span></span></span></span></span></span></span></span></li>
<li><strong>Precision</strong>: <span class="katex"><span class="katex-mathml">TPTP+FP\frac{TP}{TP + FP}</span><span class="katex-html" aria-hidden="true"><span class="base"><span class="mord"><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist"><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">TP</span><span class="mbin mtight">+</span><span class="mord mathnormal mtight">FP</span></span></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">TP</span></span></span></span><span class="vlist-s"></span></span></span></span></span></span></span></span></li>
<li><strong>Recall (Sensitivity)</strong>: <span class="katex"><span class="katex-mathml">TPTP+FN\frac{TP}{TP + FN}</span><span class="katex-html" aria-hidden="true"><span class="base"><span class="mord"><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist"><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">TP</span><span class="mbin mtight">+</span><span class="mord mathnormal mtight">FN</span></span></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord mathnormal mtight">TP</span></span></span></span><span class="vlist-s"></span></span></span></span></span></span></span></span></li>
<li><strong>F1 Score</strong>: <span class="katex"><span class="katex-mathml">2&sdot;Precision&sdot;RecallPrecision+Recall2 \cdot \frac{\text{Precision} \cdot \text{Recall}}{\text{Precision} + \text{Recall}}</span><span class="katex-html" aria-hidden="true"><span class="base"><span class="mord">2</span><span class="mbin">&sdot;</span></span><span class="base"><span class="mord"><span class="mfrac"><span class="vlist-t vlist-t2"><span class="vlist-r"><span class="vlist"><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord text mtight">Precision</span><span class="mbin mtight">+</span><span class="mord text mtight">Recall</span></span></span><span class="sizing reset-size6 size3 mtight"><span class="mord mtight"><span class="mord text mtight">Precision</span><span class="mbin mtight">&sdot;</span><span class="mord text mtight">Recall</span></span></span></span><span class="vlist-s"></span></span></span></span></span></span></span></span></li>
</ol>
<hr />
<h3><strong>Tutorial: Confusion Matrix in Python</strong></h3>
<p>Let&rsquo;s walk through an example of creating a confusion matrix using Python&rsquo;s <code>scikit-learn</code> library.</p>
<h3><strong>1. Import Required Libraries</strong></h3>
<div class="dark bg-gray-950 contain-inline-size rounded-md border-[0.5px] border-token-border-medium relative">
<div class="flex items-center text-token-text-secondary bg-token-main-surface-secondary px-4 py-2 text-xs font-sans justify-between rounded-t-md h-9">
<pre class="language-python"><code>import numpy as np
from sklearn.model_selection import train_test_split
from sklearn.linear_model import LogisticRegression
from sklearn.metrics import confusion_matrix, classification_report, accuracy_score</code></pre>
</div>
</div>
<h3><strong>2. Preparing the Data</strong></h3>
<p>For this example, let's generate a simple binary classification dataset using <code>scikit-learn</code>'s <code>make_classification()</code>.</p>
<div class="dark bg-gray-950 contain-inline-size rounded-md border-[0.5px] border-token-border-medium relative">
<div class="flex items-center text-token-text-secondary bg-token-main-surface-secondary px-4 py-2 text-xs font-sans justify-between rounded-t-md h-9">
<pre class="language-python"><code>from sklearn.datasets import make_classification

# Generate a binary classification dataset
X, y = make_classification(n_samples=1000, n_features=5, n_classes=2, random_state=42)

# Split into training and test sets (80% train, 20% test)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)</code></pre>
</div>
</div>
<hr />
<h3><strong>3. Training a Logistic Regression Model</strong></h3>
<p>We'll train a <strong>logistic regression</strong> model for binary classification.</p>
<div class="dark bg-gray-950 contain-inline-size rounded-md border-[0.5px] border-token-border-medium relative">
<div class="flex items-center text-token-text-secondary bg-token-main-surface-secondary px-4 py-2 text-xs font-sans justify-between rounded-t-md h-9">
<pre class="language-python"><code># Initialize the model
model = LogisticRegression()

# Train the model on the training data
model.fit(X_train, y_train)

# Predict on the test set
y_pred = model.predict(X_test)</code></pre>
</div>
</div>
<hr />
<h3><strong>4. Creating the Confusion Matrix</strong></h3>
<p>Now, let&rsquo;s create the confusion matrix to evaluate the model&rsquo;s performance.</p>
<div class="dark bg-gray-950 contain-inline-size rounded-md border-[0.5px] border-token-border-medium relative">
<div class="flex items-center text-token-text-secondary bg-token-main-surface-secondary px-4 py-2 text-xs font-sans justify-between rounded-t-md h-9">
<pre class="language-python"><code># Create the confusion matrix
cm = confusion_matrix(y_test, y_pred)

print("Confusion Matrix:\n", cm)</code></pre>
</div>
</div>
<ul>
<li>The confusion matrix compares the actual labels (<code>y_test</code>) with the predicted labels (<code>y_pred</code>).</li>
</ul>
<hr />
<h3><strong>5. Visualizing the Confusion Matrix</strong></h3>
<p>You can use <code>seaborn</code> to create a heatmap for better visualization of the confusion matrix.</p>
<div class="dark bg-gray-950 contain-inline-size rounded-md border-[0.5px] border-token-border-medium relative">
<div class="flex items-center text-token-text-secondary bg-token-main-surface-secondary px-4 py-2 text-xs font-sans justify-between rounded-t-md h-9">
<pre class="language-python"><code>import seaborn as sns
import matplotlib.pyplot as plt

# Visualize the confusion matrix using seaborn's heatmap
sns.heatmap(cm, annot=True, fmt="d", cmap="Blues", xticklabels=['Pred: No', 'Pred: Yes'], yticklabels=['Actual: No', 'Actual: Yes'])
plt.xlabel('Predicted')
plt.ylabel('Actual')
plt.title('Confusion Matrix')
plt.show()</code></pre>
</div>
</div>
<p>This heatmap helps in quickly interpreting the confusion matrix.</p>
<hr />
<h3><strong>6. Classification Report and Other Metrics</strong></h3>
<p>In addition to the confusion matrix, it&rsquo;s helpful to look at metrics like <strong>Precision</strong>, <strong>Recall</strong>, <strong>F1-Score</strong>, and <strong>Accuracy</strong>.</p>
<div class="dark bg-gray-950 contain-inline-size rounded-md border-[0.5px] border-token-border-medium relative">
<div class="flex items-center text-token-text-secondary bg-token-main-surface-secondary px-4 py-2 text-xs font-sans justify-between rounded-t-md h-9">
<pre class="language-python"><code># Generate the classification report
report = classification_report(y_test, y_pred)

# Print the classification report
print("Classification Report:\n", report)

# Calculate and print accuracy
accuracy = accuracy_score(y_test, y_pred)
print(f"Accuracy: {accuracy:.2f}")</code></pre>
</div>
</div>
<ul>
<li><strong>Classification Report</strong> provides Precision, Recall, F1-Score, and Support (number of true instances for each class).</li>
<li><strong>Accuracy</strong> measures the overall performance of the model.</li>
</ul>
<hr />
<h3><strong>Example Output</strong></h3>
<p>For a binary classification problem, the output might look like this:</p>
<h4><strong>Confusion Matrix:</strong></h4>
<div class="dark bg-gray-950 contain-inline-size rounded-md border-[0.5px] border-token-border-medium relative">
<div class="flex items-center text-token-text-secondary bg-token-main-surface-secondary px-4 py-2 text-xs font-sans justify-between rounded-t-md h-9">lua</div>
<div class="sticky top-9 md:top-[5.75rem]">
<div class="absolute bottom-0 right-2 flex h-9 items-center">
<div class="flex items-center rounded bg-token-main-surface-secondary px-2 font-sans text-xs text-token-text-secondary">
<pre class="language-python"><code>[[84 11]
 [ 7 98]]</code></pre>
</div>
</div>
</div>
</div>
<h4><strong>Classification Report:</strong></h4>
<div class="dark bg-gray-950 contain-inline-size rounded-md border-[0.5px] border-token-border-medium relative">
<div class="flex items-center text-token-text-secondary bg-token-main-surface-secondary px-4 py-2 text-xs font-sans justify-between rounded-t-md h-9">markdown</div>
<div class="sticky top-9 md:top-[5.75rem]">
<div class="absolute bottom-0 right-2 flex h-9 items-center">
<div class="flex items-center rounded bg-token-main-surface-secondary px-2 font-sans text-xs text-token-text-secondary">
<pre class="language-python"><code>              precision    recall  f1-score   support

0       0.92      0.88      0.90        95
           1       0.90      0.93      0.91       105

accuracy                           0.91       200
   macro avg       0.91      0.91      0.91       200
weighted avg       0.91      0.91      0.91       200</code></pre>
</div>
</div>
</div>
</div>
<ul>
<li><strong>Precision</strong> for class 0 (negative class) is 0.92, meaning that when the model predicts "No", it's correct 92% of the time.</li>
<li><strong>Recall</strong> for class 1 (positive class) is 0.93, meaning that the model correctly identifies 93% of all actual "Yes" cases.</li>
</ul>
<hr />
<h3><strong>Conclusion</strong></h3>
<p>In this tutorial, we:</p>
<ol>
<li>Built a <strong>confusion matrix</strong> for a binary classification problem.</li>
<li>Visualized the confusion matrix using <strong>seaborn</strong>.</li>
<li>Generated a <strong>classification report</strong> to view precision, recall, F1-score, and support for each class.</li>
<li>Calculated the <strong>accuracy</strong> of the model.</li>
</ol>
<p>The confusion matrix provides a clear, detailed summary of model performance, especially in classification problems where the balance between false positives and false negatives matters.</p>
</div>
</div>
</div>
</div>
</div>
</div>

Web Technology View More