Python Grid Search

Last modified on: October 19, 2024

<div class="flex-shrink-0 flex flex-col relative items-end">
<div class="pt-0">
<div class="gizmo-bot-avatar flex h-8 w-8 items-center justify-center overflow-hidden rounded-full">
<div class="flex-shrink-0 flex flex-col relative items-end">
<div>
<div class="pt-0">
<div class="gizmo-bot-avatar flex h-8 w-8 items-center justify-center overflow-hidden rounded-full">
<h2 class="relative p-1 rounded-sm flex items-center justify-center bg-token-main-surface-primary text-token-text-primary h-8 w-8">Grid Search in Python</h2>
</div>
</div>
</div>
</div>
<div class="group/conversation-turn relative flex w-full min-w-0 flex-col agent-turn">
<div class="flex-col gap-1 md:gap-3">
<div class="flex max-w-full flex-col flex-grow">
<div class="min-h-[20px] text-message flex w-full flex-col items-end gap-2 whitespace-normal break-words [.text-message+&amp;]:mt-5" dir="auto" data-message-author-role="assistant" data-message-id="d9c2ad47-a497-4810-a401-ecf0f2e0b82f">
<div class="flex w-full flex-col gap-1 empty:hidden first:pt-[3px]">
<div class="markdown prose w-full break-words dark:prose-invert light">
<p><strong>Grid Search</strong> is a hyperparameter tuning technique used to find the best combination of hyperparameters for a given model. It systematically works through multiple combinations of hyperparameters, cross-validating as it goes to determine which combination produces the best performance.</p>
<hr />
<h3><strong>Key Concepts of Grid Search</strong></h3>
<ul>
<li><strong>Hyperparameters</strong>: Settings that govern the training process of machine learning models (e.g., learning rate, number of trees in a forest).</li>
<li><strong>Cross-Validation</strong>: A technique used to assess the performance of a model by partitioning the data into training and validation sets.</li>
</ul>
<h3><strong>Using Grid Search with <code>scikit-learn</code></strong></h3>
<p>We will use Python&rsquo;s <code>scikit-learn</code> library to perform grid search with a sample dataset and a machine learning model.</p>
<h3><strong>1. Import Required Libraries</strong></h3>
<div class="dark bg-gray-950 contain-inline-size rounded-md border-[0.5px] border-token-border-medium relative">
<div class="flex items-center text-token-text-secondary bg-token-main-surface-secondary px-4 py-2 text-xs font-sans justify-between rounded-t-md h-9">
<pre class="language-python"><code>import numpy as np
import pandas as pd
from sklearn.datasets import load_iris
from sklearn.model_selection import train_test_split, GridSearchCV
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import classification_report</code></pre>
</div>
</div>
<h3><strong>2. Preparing the Data</strong></h3>
<p>For this example, we&rsquo;ll use the <strong>Iris dataset</strong>.</p>
<div class="dark bg-gray-950 contain-inline-size rounded-md border-[0.5px] border-token-border-medium relative">
<div class="flex items-center text-token-text-secondary bg-token-main-surface-secondary px-4 py-2 text-xs font-sans justify-between rounded-t-md h-9">
<pre class="language-python"><code># Load the Iris dataset
iris = load_iris()
X = iris.data  # Features
y = iris.target  # Target variable

# Split the dataset into training and test sets (80% train, 20% test)
X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.2, random_state=42)</code></pre>
</div>
</div>
<h3><strong>3. Define the Model and Hyperparameters</strong></h3>
<p>We will define a <strong>Random Forest Classifier</strong> and specify the hyperparameters we want to tune.</p>
<div class="dark bg-gray-950 contain-inline-size rounded-md border-[0.5px] border-token-border-medium relative">
<div class="flex items-center text-token-text-secondary bg-token-main-surface-secondary px-4 py-2 text-xs font-sans justify-between rounded-t-md h-9">
<pre class="language-python"><code># Initialize the Random Forest Classifier
model = RandomForestClassifier()

# Define the parameter grid
param_grid = {
    'n_estimators': [10, 50, 100],  # Number of trees in the forest
    'max_depth': [None, 10, 20, 30],  # Maximum depth of the tree
    'min_samples_split': [2, 5, 10]  # Minimum number of samples required to split an internal node
}</code></pre>
</div>
</div>
<h3><strong>4. Perform Grid Search</strong></h3>
<p>We will use <code>GridSearchCV</code> to perform the grid search.</p>
<div class="dark bg-gray-950 contain-inline-size rounded-md border-[0.5px] border-token-border-medium relative">
<div class="flex items-center text-token-text-secondary bg-token-main-surface-secondary px-4 py-2 text-xs font-sans justify-between rounded-t-md h-9">
<pre class="language-python"><code># Initialize GridSearchCV
grid_search = GridSearchCV(estimator=model, param_grid=param_grid, cv=5, scoring='accuracy')

# Fit the model on the training data
grid_search.fit(X_train, y_train)</code></pre>
</div>
</div>
<h3><strong>5. Evaluate the Best Model</strong></h3>
<p>After performing the grid search, we can evaluate the best model and its performance.</p>
<div class="dark bg-gray-950 contain-inline-size rounded-md border-[0.5px] border-token-border-medium relative">
<div class="flex items-center text-token-text-secondary bg-token-main-surface-secondary px-4 py-2 text-xs font-sans justify-between rounded-t-md h-9">
<pre class="language-python"><code># Get the best parameters and best score
best_params = grid_search.best_params_
best_score = grid_search.best_score_

print(f"Best Parameters: {best_params}")
print(f"Best Cross-Validation Score: {best_score:.2f}")

# Predict on the test set using the best estimator
best_model = grid_search.best_estimator_
y_pred = best_model.predict(X_test)

# Print the classification report
print("Classification Report:\n", classification_report(y_test, y_pred))</code></pre>
</div>
</div>
<h3><strong>Conclusion</strong></h3>
<p>In this tutorial, we:</p>
<ol>
<li>Introduced <strong>Grid Search</strong> for hyperparameter tuning using the <strong>Iris dataset</strong>.</li>
<li>Defined a <strong>Random Forest Classifier</strong> and specified a grid of hyperparameters to explore.</li>
<li>Used <code>GridSearchCV</code> to perform the grid search and identify the best hyperparameters.</li>
<li>Evaluated the performance of the best model on the test set.</li>
</ol>
<p>Grid search is a powerful tool for improving model performance by systematically exploring the hyperparameter space, ensuring that the best model configuration is identified.</p>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>
</div>

Web Technology View More