Since Thoughts on Reinforcement learning and after reading that paper on DQN and being a bit more sure about how reinforcement learning is implemented algorithmically (Bellman update), I started wondering about other unrelated things, like what a Bayesian networks is.
I've seen references to Bayesian networks in literature I've read without having an intuitive understanding of what it is and and how they work and, more importantly, what applicability they might have to me in general - because why not know? I've also felt this way about probability and Markov Chains, delving into aspects about probability distributions, Hidden Markov chains (HMM), Markov decision processes (MDP) and this ultimately lead me to Bayesian networks probably (no pun intended!) because it also has to do with probability. Also, I had recently conducted a research task for Brunel where I needed to review papers on types and applications of Deep Neural Networks (DNN) which are very much grounded in probability which is probably where this whole foray in learning about probability probably started from. However, I digress...
Why I've been interested Bayesian networks is because they are said to be usable to make intuitive decisions in machines/computers.
Specifically, they allow for decisions to be made in a way similar to how humans might make decisions by indirectly inferring certain situations despite not directly witnessing the situation, i.e they use other indications or conditions that the situation depends on (in certain degrees) as a means to suggest that the situation is occurring. They do this systematically, where humans do it more intuitively or perhaps even superstitiously.
This is interesting if you'd like to simulate decision making in a more human way within an artificial entity such as a game character or a robot, for example. The key is that it can be achieved through a systematic, well-definable process (which is what machines like, and what can be implemented as an algorithm) and it produces human-like behaviour (as a result of seemingly plausible human-like decision making process) which is what we'd like to achieve in a simulated artificial intelligent entity.
While I've suggested that bayesian networks can be used to help make decisions (I'm not going to explaining exactly how yet), they can also be used to learn and indicate/detect the probability that a past situation is currently occurring despite only knowing some aspects about the situation right now.
It learns by gaining more experience about the make up of historical situational data, i.e what the conditions were when situations occurred, and uses the frequency of certain situational aspects as a means to predict the situation when only some of the situational aspects are known at this very moment. This means that at this very moment, you can predict if currently the situations is occurring with only fragments of knowledge about the situation.
The more experience you gain of the conditions of the situations, the more accurate the prediction will be when only presented with some of the conditions. It might be challenging to realise the impact of this idea.
For example, these ideas are used by spam detection algorithms. They collect aspects/conditions about emails and ask you to add another condition which indicates if the situation is a spam email or not. As more instances of when those aspects/conditions are marked as spam accumulate in the historical data, this will increase the probability that those aspects/conditions lead to the probability (detection) as spam, specifically when you don't know its spam (but you know the other conditions), and you have historical data where some of these aspects have contributed to the situation of spam before (in the historical data).
Additionally, if you know more spam-related conditions/aspects about the spam email, the probability of the email being detected as spam increases, i.e as the more you know about the spam conditions, the more likely it will be detected as spam.
This is extremely useful/interesting and incidentally this is also how weather is predicted.
For example, they look at historical data and from it they work out the conditions that contribute to the probability of rain, they then take what they know about today's conditions and this determines how the conditions likely contribute to the probability of rain today. Again, if more knowledge about the conditions that cause rain is known, the more probable the prediction of rain will be.
There is more to be said about how Bayesian networks work, specifically how they are implemented algorithmically and mathematically but this will be reserved for a future article.