Since Thoughts on Reinforcement learning and after reading that paper on DQN and being a bit more sure about how reinforcement learning is implemented algorithmically (Bellman update), I started wondering about other unrelated things, like what a Bayesian networks is.

I've seen references to Bayesian networks in literature I've read without having an intuitive understanding of what it is and and how they work and, more importantly, what applicability they might have to me in general - because why not know? I've also felt this way about probability and Markov Chains, delving into aspects about probability distributions, Hidden Markov chains (HMM), Markov decision processes (MDP) and this ultimately lead me to Bayesian networks probably (no pun intended!) because it also has to do with probability. Also, I had recently conducted a research task for Brunel where I needed to review papers on types and applications of Deep Neural Networks (DNN) which are very much grounded in probability which is probably where this whole foray in learning about probability probably started from. However, I digress...

Why I've been interested Bayesian networks is because they are said to be usable to make intuitive decisions in machines/computers.

Specifically, they allow for decisions to be made in a way similar to how humans might make decisions by indirectly inferring certain situations despite not directly witnessing the situation, i.e they use other indications or conditions that the situation depends on (in certain degrees) as a means to suggest that the situation is occurring. They do this systematically, where humans do it more intuitively or perhaps even superstitiously.

This is interesting if you'd like to simulate decision making in a more human way within an artificial entity such as a game character or a robot, for example. The key is that it can be achieved through a systematic, well-definable process (which is what machines like, and what can be implemented as an algorithm) and it produces human-like behaviour (as a result of seemingly plausible human-like decision making process) which is what we'd like to achieve in a simulated artificial intelligent entity. 

While I've suggested that bayesian networks can be used to help make decisions (I'm not going to explaining exactly how yet), they can also be used to learn and indicate/detect the probability that a past situation is currently occurring despite only knowing some aspects about the situation right now.

It learns by gaining more experience about the make up of historical situational data, i.e what the conditions were when situations occurred, and uses the frequency of certain situational aspects as a means to predict the situation when only some of the situational aspects are known at this very moment. This means that at this very moment, you can predict if currently the situations is occurring with only fragments of knowledge about the situation. 

The more experience you gain of the conditions of the situations, the more accurate the prediction will be when only presented with some of the conditions. It might be challenging to realise the impact of this idea.

For example, these ideas are used by spam detection algorithms. They collect aspects/conditions about emails and ask you to add another condition which indicates if the situation is a spam email or not. As more instances of when those aspects/conditions are marked as spam accumulate in the historical data, this will increase the probability that those aspects/conditions lead to the probability (detection) as spam, specifically when you don't know its spam (but you know the other conditions), and you have historical data where some of these aspects  have contributed to the situation of spam before (in the historical data).

Additionally, if you know more spam-related conditions/aspects about the spam email, the probability of the email being detected as spam increases, i.e as the more you know about the spam conditions, the more likely it will be detected as spam.

This is extremely useful/interesting and incidentally this is also how weather is predicted.

For example, they look at historical data and from it they work out the conditions that contribute to the probability of rain, they then take what they know about today's conditions and this determines how the conditions likely contribute to the probability of rain today. Again, if more knowledge about the conditions that cause rain is known, the more probable the prediction of rain will be.

There is more to be said about how Bayesian networks work, specifically how they are implemented algorithmically and mathematically but this will be reserved for a future article.

Since writing about my 3D OpenGL Project, I've been recently wondering why I have a growing appreciation of Mathematics. What is it that I appreciate specifically? 

It may be obvious but I feel that answering these types of personal, inward and exploratory questions is the best at helping you discover your innate drive and motivations. 

Over the last couple of years, I've started to realise that while Mathematics can be difficult to interpret at times, this is due largely I feel, primarily due to the lack of contextual and background understanding which is then required to interpret its notation and syntax. This might seem obvious but what this means is that the majority of Mathematics must happen in your head, not on paper, and that the primary activity is conceptual manipulation of ideas and thinking. My education never highlighted this, and equally, I didn't see it.

I was of the impression growing up that mathematics was firstly and primarily learnt through syntax and notation, and that through notation and syntax, we derive mathematical understanding. For example, that it is through the manipulation of fundamental rules that are represented as particular algorithms and syntactical notation. 

A part of my growing interest in Mathematics is having realised that its syntax is not of primary importance, and it has been detrimental for me up to this point to learn and gain an understanding of mathematics by seeing it this way. I've also realized that understanding is a very time-consuming process but learning notation is lightening fast.

For example, I took at least 6-8 months to understand the ideas behind Calculus but the mathematic syntax, algebra, algorithms and rules only took maybe half a page. And for all the manipulation of those algorithms, they never provided actual understanding or insight. This might seem obvious to some but I only really realised this recently.

The opposite is true: insight is achieved without syntax, and the most effective way to understand it is through conceptual thinking and manipulation of ideas without requiring pages of ink.

Having said that, the reason why I've tried to understand my deficiencies is that I see Mathematics as fundamentally important to me because of its utility in representing ideas and my growing interest in understanding them.

I like the idea of representing ideas. For example, I usually represent conceptual ideas as code every day using syntax and notation for the particular programming language I'm using. Those ideas live an entirely separate life from their final representation in code. I spend more time thinking and trying to understand the implications of what I'm planning to represent/do, and coding that thinking and understanding into the syntax and notation is merely a brief formality. The same is true of Mathematics.

I've come to realise the link between Mathematical notation and Programming too. They both represent ideas.

Mathematics is a programming language used to define ideas using rules to confine/define them.

Thus to answer my question, why do I appreciate Mathematics? I appreciate Mathematics for its ability to define and represent ideas.

What a wonderful realisation.

The acquisition of knowledge to inform an understanding of anything has, throughout the ages, produced multiple theories that try to explain how it is achieved.

For example, Empiricism suggests that knowledge is derived primarily from sensory experience alone. Thomas Hobbes believed that everything in the universe is purely physical in nature and that cognitive processes are determined by predictable physical laws, and that ideas originate from sensation. Indeed, Hobbes believed that, “…sense perception is epistemically fundamental” and that, “…all knowledge is ultimately traceable back to, and validated by, sense perceptions”(Machamer, 2014). In this way, he suggested that behaviour derives purely from physical experience in response to physical stimuli or sensation.

Similarly, empiricist John Locke suggested that all knowledge is derived from experience, both through our physical sensory mechanisms, but also through cognitive reflection on that physical experience, and that this together is what determines all our sensations, feelings and ideas. Incidentally, he also suggested that at birth the mind begins as a blank slate or tabula rasa, and that, “…the mind is initially dependent upon experience for its operation”, and then evolves as experience is gathered (Duschinsky, 2012).

Rationalism, on the other hand, claims that, “…we have substantive a priori knowledge of the world and, typically, that we have non-empirical concepts (Vanzo, 2016). Following from this, Rene Descartes, often seen as the father of modern philosophy, suggested that our human senses are, in fact, an unreliable source of knowledge because they can easily be deceived, and that we acquire knowledge primarily through active mental processes, i.e cognitive mechanisms like thinking, intuition and reasoning about circumstance and he makes a,”… clear distinction between the idea of mind and the idea of body” (Ventriglio and Bhugra, 2015).

Yet another view, Behaviourism, suggests that it is only through learning alone that knowledge is created. Russian physiologist Ivan Pavlov showed that pairing a new physical/environmental stimulus with an pre-existing and well-known stimulus could, though repetition, produce the same response to the well-known stimulus, which as a consequence, invoked what appeared to be a fundamental form of learning, i.e the learnt association between a perception of a stimuli and its response, which “…ultimately led to the theory of respondent conditioning” (Guercio, 2018).

Pavlov’s results also suggest that there exists an underlying cognitive mechanism which appears to control this associative learning behaviour. One potential model for this is the simulation of an artificial experiential environment that is based on this stimulus-response (S-R) theory for the purpose of eliciting and gathering environmental stimuli and responses to create contextual and situational awareness and knowledge. (see Figure 5)

J.B. Watson, who researched associative learning, suggested that stimulus-response (S-R) learning is the basis of all human experience, and that learning could only be studied by observing behaviour in circumstances, and that “…observations of behaviour could be used to infer the nature…” of behaviour itself (Hall, 2009).

References

Machamer, P. (2014) ‘Thomas Hobbes’, Hobbes studies, 27(1), pp. 1–12. doi: 10.1163/18750257-02701003.

Duschinsky, R. (2012) ‘Tabula Rasa and Human Nature’, Philosophy, 87(4), pp. 509–529. doi: 10.1017/S0031819112000393.

Vanzo, A. (2016) ‘Empiricism and Rationalism in Nineteenth-Century Histories of Philosophy’, Journal of the history of ideas, 77(2), pp. 253–282. doi: 10.1353/jhi.2016.0017.

Ventriglio, A. and Bhugra, D. (2015) ‘Descartes’ dogma and damage to Western psychiatry’, Epidemiology and psychiatric sciences, 24(5), pp. 368–370. doi: 10.1017/S2045796015000608.

Hall, G. (2009) ‘Watson: The thinking man’s behaviourist’, The British journal of psychology, 100(S1), pp. 185–187. doi: 10.1348/000712609X413656.