what is Ensemble learning and why it is so powerful? -2021

Ensemble learning is a powerful technique for improving the performance of any model. Its performance is so fascinating that it has and is used in many competitive programming for example kaggle competition. So, let’s see what and how of it.

What is ensemble learning?

The idea of ensemble learning is simple, instead of relying on and using one model for a problem, it uses multiple models for a single problem and each output or prediction is made on the basis of votes from each model. The value with majority votes is elected as the final output

For example, if we are doing cat and dog classification with 10 models and 7 of them classified a data point as Cat then the final output will be a cat as the value has the majority of votes.
Also read –> What is Transfer learning, and why it is the next driver of ML success (According to Andrew Ng)

How ensemble learning works?

In ensemble learning, we take the help of multiple models. These models in ensemble learning could be different (like a combination of KNN, linear regression, decision tree) or the same (like all can be linear regression models) and can have any number of them. Also, we can create ensemble models for any problem like regression and classification.

Just like normal model training we train each individual model on a different sample of data. Which are extracted randomly from the training data. These individual models are called weak models as they hold high chances of making mistakes. After the training process, we use them on test data, and the output of each data is collected from each model. Those are passed to the final model which is called strong models as the chances of making mistakes are pretty low for it, as compared to individuals.

And in the final step, it concludes the final output based on each individual model’s output. If the model is for a regression problem then it takes the average of those output values. Or in the case of the classification problem, it checks the class with a maximum vote (number of predictions of each class from each model) and makes it the final output.
Also read –> Complete understanding of deep learning for beginners without math

Why do we use ensemble learning?

To understand this let’s take an example, suppose after exploring some posts on Facebook, you got confused about the shape of Earth and you want to know if the Earth is flat or not.

Similarly, to reduce the chances of getting wrong answers or predictions we just don’t rely on the answer of one model and we use ensemble learning

Category of ensemble learning

Ensemble learning is of two categories: parallel ensemble learning and sequential ensemble learning. And the type of ensemble methods mainly used is bagging and boosting.

Difference between parallel and sequential ensemble learning

Parallel Ensemble learning

In parallel ensemble learning base learners are generated in parallel format, to encourage independence between the base layer. The independence of base learners reduces the error slightly, due to the application of averages.

These include homogeneous ensembles (in which the same base learning algorithm is used to train base estimators) such as bagging and random forests and heterogeneous ensemble methods (in which different base learning algorithms are used to train base estimators) such as stacking.

Sequential Ensemble learning

While in sequential ensemble learning base learners are generated in sequential format, to encourage the dependence between the base learners. The model’s performance is improved by assigning higher weights to previously misrepresented learners. Examples include boosting.
If you want to know more about it then you can read it here

Challenges of ensemble learning

Apart from providing better accuracy, ensemble learning also have some downside, which are

1. Take much more time
2. Require extra memory
3. Lack of decision explainability

Summary

Ensemble learning is one of the cool techniques which improve the performance of a model significantly. Instead of using a single model for prediction, ensemble learning uses multiple models, which decreases the chances of error or error rate. Also, it can be used for both regression and classification. But despite performance improvement, it also holds some downside like time, memory, and explainability issue

Aman Kumar

Data Scientist with 3+ years of experience in building data-intensive applications in diverse industries. Proficient in predictive modeling, computer vision, natural language processing, data visualization etc. Aside from being a data scientist, I am also a blogger and photographer.