Ensemble learning is a powerful technique that combines the predictions of multiple models to produce a more accurate and robust model. VotingClassifier and VotingRegressor are two ensemble methods provided by scikit-learn library that can be used to combine the predictions of multiple classifiers or regressors respectively.
There are three types of voting strategies:
- “hard” - the predicted class or regression value is the one with the highest number of votes from the base models.
- “soft” - the predicted class or regression value is the weighted average of the predicted probabilities or values of the base models.
- “voting” - allows you to define your own voting strategy by specifying a list of weights for each base model.
1. VotingRegressor Method
In a VotingRegressor model, multiple regression models are trained on the same training data. The predictions of each of these models are then combined by taking the average of the predicted values.
It can be used for both linear and non-linear regression problems, and it is often used in situations where no single regression model performs well on its own.
Here is an example of how to use VotingRegressor in Scikit-Learn:
In this example, we load the California housing dataset using fetch_california_housing() and split it into training and test sets. We then create two individual regression models - a linear regression model and a decision tree regression model. We combine these models using the VotingRegressor class, and then train the voting regressor on the training data.
Finally, we make predictions on the test data using the voting regressor, and evaluate the performance of the model using mean squared error.
2. VotingClassifier Method
In a VotingClassifier model, multiple classification models are trained on the same training data. The predictions of each of these models are then combined by taking the majority vote of the predicted classes.
It can be used for both binary and multi-class classification problems, and it is often used in situations where no single classification model performs well on its own.
Here is an example of how to use VotingClassifier in Scikit-Learn:
This code builds a VotingClassifier to combine three individual models, DecisionTreeClassifier , LogisticRegression , and KNeighborsClassifier , to predict the target variable of the iris dataset. The VotingClassifier is trained on the training data and used to make predictions on the test data. The performance of the VotingClassifier is evaluated using the accuracy_score metric. To avoid a convergence warning produced by the LogisticRegression model, the code uses the warnings module with the "ignore" mode and ConvergenceWarning category.