One type of Machine Learning that we have not spoken about yet is “Ensemble Learning.”
A recent post on Google’s AI blog does an excellent job of shedding some light on what this is. Essentially, in general, Ensemble Learning is when an AI team combines different prediction models to try to achieve their ideal results more efficiently.
Imagine that several neural networks are connected together, working to tackle the same issue. In connection with this, they are also using the same training data.
The overall aim here, according to current research on the subject, is to generate multiple hypotheses that let them attack the problem at hand in different ways, all at once. At the start of this process, the AI system involved only has neural networks called base learners, that are typically weaker than a group of neural networks in terms of their overall efficiency. Over time, these base learners are connected until they become better learners in terms of their increased efficiency.
If we jump back to Google’s post, they actually do a good job of honing in on the importance of all of this, through the example of their new development framework called AdaNet. Through their description of the launch of AdaNet, they mention that the original idea for it came from a combination of research into Reinforcement Learning and what they call “evolutionary-based Auto ML.” After a bit of research on the subject, it quickly becomes clear that this could refer to a mix of evolutionary algorithms and unsupervised learning principles.
If you are not clear on what the idea of an evolutionary algorithm is referring to, take a look at our suggested resource on the subject below.
For now, just imagine a Machine Learning algorithm that is based on some small element of natural selection. Overall, what this amounts to is a form of an algorithm that lets go of what it terms to be inefficient solutions, in favor of those that it terms to be more efficient. In a sense, as is mentioned in the source below, this is like what natural selection does with a species.
In the end, only the strong survive.
In future pieces, we’ll give further examples of how this plays out in practice. For now, keep in mind that some of these algorithms were combined with Reinforcement Learning algorithms in order to develop a framework based on Ensemble Learning.
As Google claims that ADANet will save Machine Learning professionals time related to constructing neural networks, it will be interesting to see if this is confirmed to be true through external research.
References:
Original Blog Post: https://ai.googleblog.com/2018/10/introducing-adanet-fast-and-flexible.html
Secondary Source: https://cs.nju.edu.cn/zhouzh/zhouzh.files/publication/springerEBR09.pdf
Evolutionary Algorithms: https://towardsdatascience.com/introduction-to-evolutionary-algorithms-a8594b484ac
Evolutionary Based Auto ML: https://ai.googleblog.com/2018/03/using-evolutionary-automl-to-discover.html
