Ensemble Learning Methods
For a better version of this article please stop now and read here. This post will just be an aggregate of information, links, and blurbs from various articles to try and encapsulate as much as possible about ensemble learning methods without getting too deep into the weeds of how to do it.
Origins and Active Development
The term Ensemble Learning is not directly attributed to being coined by a single person. Its concept seems to have been the result of groups of people from all over working on it, not together, but with an overlap in the relevance of their work.
That said here are some people who were critical members of its creation, as well as it’s active development:
Leo Breiman
What is it Ensemble Learning?
Ensemble learning has a few different classifications. The commonality between the types of ensemble learning is they all use the output of different models evaluating the same dataset to create a sort of set of predictions. This prediction “set” is then combined together in ways, or weighted in ways so that the output of that is more accurate than the output of any single model on its own (from within the group of models).
When & Where to use it?
Instead of saying when to use it, I’ll say when not to use it:
When computational loads need to be smaller (multiple models processing data can be expensive)
When storage capacity is limited
When the data sets themselves are smaller
If you don’t have a very clear question with value that’s being answered
I wouldn’t implement ensemble methods (expensive on developer time, compute time, and storage size) unless I knew a very useful question was being answered.
Why use it?
Over anything else - I’m not sure. I can’t guarantee the accuracy of the following statement.
From a few articles I’ve found (paywalled unfortunately), I’ve been able to glean that people in Data Science and ML fields right now are opting for these ensemble methods when they want higher degrees of accuracy, as well as when the data is noisy, or if the individual models from within the group of models are noisy.
Conclusion
Ensemble learning produces an output that is based on the output of 2 or more ML Models working together on the same data set. The idea is to create a sort of ‘collective intelligence’ that gets closer to a desirable result by reducing variances, biases, and other errors that can occur from within a single model. The error magnitude across the output across multiple models can be reduced more easily than turning a single model perfectly. If the funding, time, and need is present for a system, ensemble learning systems are a great choice to produce more reliable predictions.
