上QQ阅读APP看书，第一时间看更新

How to do it...

Here, we have a dataset based on the properties of cancerous tumors. Using this dataset, we'll build multiple classification models with diagnosis as our response variable. The diagnosis variable has the values, B and M, which indicate whether the tumor is benign or malignant. With multiple learners, we extract multiple predictions. The weighted averaging technique takes the average of all of the predicted values for each training sample.

In this example, we consider the predicted probabilities as the output and use the predict_proba() function of the scikit-learn algorithms to predict the class probabilities:

Import the required libraries:

# Import required libraries
from sklearn.tree import DecisionTreeClassifier
from sklearn.svm import SVC
from sklearn.linear_model import LogisticRegression

Create the response and feature sets:

# Create feature and response variable set
# We create train & test sample from our dataset
from sklearn.cross_validation import train_test_split

# create feature & response variables
X = df_cancerdata.iloc[:,2:32]
Y = df_cancerdata['diagnosis']

We retrieved the feature columns using the iloc() function of the pandas DataFrame, which is purely integer-location based indexing for selection by position. The iloc() function takes row and column selection as its parameter, in the form: data.iloc(<row selection>, <column selection>). The row and column selection can either be an integer list or a slice of rows and columns. For example, it might look as follows: df_cancerdata.iloc(2:100, 2:30).

We'll then split our data into training and testing sets:

# Create train & test sets
X_train, X_test, Y_train, Y_test = \
train_test_split(X, Y, test_size=0.20, random_state=1)

Build the base classifier models:

# create the sub models
estimators = []

dt_model = DecisionTreeClassifier()
estimators.append(('DecisionTree', dt_model))

svm_model = SVC(probability=True)
estimators.append(('SupportVector', svm_model))

logit_model = LogisticRegression()
estimators.append(('Logistic Regression', logit_model))

Fit the models on the test data:

dt_model.fit(X_train, Y_train)
svm_model.fit(X_train, Y_train)
logit_model.fit(X_train, Y_train)

Use the predict_proba() function to predict the class probabilities:

dt_predictions = dt_model.predict_proba(X_test)
svm_predictions = svm_model.predict_proba(X_test)
logit_predictions = logit_model.predict_proba(X_test)

Assign different weights to each of the models to get our final predictions:

weighted_average_predictions=(dt_predictions * 0.3 + svm_predictions * 0.4 + logit_predictions * 0.3)