Ridge Regression Preliminaries /* Load libraries */ from sklearn.linear_model import Ridge from sklearn.datasets import load_boston from sklearn.preprocessing import StandardScaler Load Boston Housing Dataset /* Load data */ boston = load_boston() X = boston.data y = boston.target Standardize Features /* Standarize features */ scaler = StandardScaler() X_std = scaler.fit_transform(X) Fit Ridge Regression The hyperparameter, αα, lets us …
Day: May 24, 2021
Linear Regression Using Scikit-Learn Preliminaries /* Load libraries */ from sklearn.linear_model import LinearRegression from sklearn.datasets import load_boston import warnings /* Suppress Warning */ warnings.filterwarnings(action=”ignore”, module=”scipy”, message=”^internal gelsd”) Load Boston Housing Dataset /* Load data */ boston = load_boston() X = boston.data y = boston.target Fit A Linear Regression /* Create linear regression */ regr = …
Lasso Regression Preliminaries /* Load library */ from sklearn.linear_model import Lasso from sklearn.datasets import load_boston from sklearn.preprocessing import StandardScaler Load Boston Housing Dataset /* Load data */ boston = load_boston() X = boston.data y = boston.target Standardize Features /* Standarize features */ scaler = StandardScaler() X_std = scaler.fit_transform(X) Fit Lasso Regression The hyperparameter, αα, lets us …
Effect Of Alpha On Lasso Regression Often we want conduct a process called regularization, wherein we penalize the number of features in a model in order to only keep the most important features. This can be particularly important when you have a dataset with 100,000+ features. Lasso regression is a common modeling technique to do regularization. The …
Adding Interaction Terms Preliminaries /* Load libraries */ from sklearn.linear_model import LinearRegression from sklearn.datasets import load_boston from sklearn.preprocessing import PolynomialFeatures import warnings /* Suppress Warning */ warnings.filterwarnings(action=”ignore”, module=”scipy”, message=”^internal gelsd”) Load Boston Housing Dataset /* Load the data with only two features */ boston = load_boston() X = boston.data[:,0:2] y = boston.target Add Interaction Term …
Hyperparameter Tuning Using Random Search Preliminaries /* Load libraries */ from scipy.stats import uniform from sklearn import linear_model, datasets from sklearn.model_selection import RandomizedSearchCV Load Iris Dataset /* Load data */ iris = datasets.load_iris() X = iris.data y = iris.target Create Logistic Regression /* Create logistic regression */ logistic = linear_model.LogisticRegression() Create Hyperparameter Search Space /* …
Find Best Preprocessing Steps During Model Selection We have to be careful to properly handle preprocessing when conducting model selection. First, GridSearchCV uses cross-validation to determine which model has the highest performance. However, in cross-validation we are in effect pretending that the fold held out as the test set is not seen, and thus not part of …
Recall Preliminaries /* Load libraries */ from sklearn.model_selection import cross_val_score from sklearn.linear_model import LogisticRegression from sklearn.datasets import make_classification Generate Features And Target Data /* Generate features matrix and target vector */ X, y = make_classification(n_samples = 10000, n_features = 3, n_informative = 3, n_redundant = 0, n_classes = 2, random_state = 1) Create Logistic Regression …
Precision Preliminaries /* Load libraries */ from sklearn.model_selection import cross_val_score from sklearn.linear_model import LogisticRegression from sklearn.datasets import make_classification Generate Features And Target Data /* Generate features matrix and target vector */ X, y = make_classification(n_samples = 10000, n_features = 3, n_informative = 3, n_redundant = 0, n_classes = 2, random_state = 1) Create Logistic Regression …
Plot The Validation Curve Preliminaries /* Load libraries */ import matplotlib.pyplot as plt import numpy as np from sklearn.datasets import load_digits from sklearn.ensemble import RandomForestClassifier from sklearn.model_selection import validation_curve Load Digits Dataset /* Load data */ digits = load_digits() /* Create feature matrix and target vector */ X, y = digits.data, digits.target Plot Validation …