Since Policy Gradient Methods, I've been curious about how LLMs are taught/trained.  It turns out that folks use reinforcement learning to train LLMs (Large Language Models) too, and it amounts, very similarly, to the Policy Gradient Methods I recently discussed. For example, an LLM predicts the next token given the prior tokens and therefore, there must be a way to evaluate the generated token against what the next token should be, i.e there must be a loss function or a reward function. Let's...

Read More ...

Since reading a survey on Agentic AI, I've formulated a basic blueprint of what I feel it is. Agentic AI basically concerns creating AI workers that are: Autonomous, Adaptable and goal-driven Use a combination of reinforcement learning (RL) and goal-oriented architectures/approaches Implements adaptive control strategies and techniques Designed to be resilient to change (and the unknown) Implements smart, opportunistic learning such as RL, imitation, pattern knowledge, priorities, social...

Read More ...

Introduction The paper reviewed in this article is "Playing Atari with Deep Reinforcement Learning" by Mnih et al. , which describes the first time Deep Neural Networks (DNNs) were integrated with Reinforcement Learning.  Q-Learning has been used as a solution to Markov Decision Processes (see Markov Decision Processes), which uses reinforcement learning (RL) to determine the best policy that an agent can use to reap the maximum rewards through its moves. See Rationalizing Q-Learning It has...

Read More ...

I've been recently trying to piece together my learning around how reinforcement learning is actually implemented algorithmically. The fundamental idea is that you'd like to simulate decision-making in an agent, but specifically that the decision-making process involves having the agent learn (and then make) moves that are the most beneficial to it. This, therefore, is an admirable and useful decision-making process to try and simulate. This also means that the agent must learn what 'good'...

Read More ...

Since A Model of Belief and the Capacity to Know, I'm increasingly convinced that to understand how to artifically model human-like behaviour requires a methodical and fundamental understanding of human behaviour (in general), and, more importantly, what causes it. This, of course, is what Psychology pursues. This might seem implicitly reasonable, but here I want to make it robustly evident. To begin with, humans make decisions with remarkable flair; they are pretty good at considering the...

Read More ...

Since Thoughts on Bayesian NetworksI've been thinking about using Bayesian networks as a means to identify situations. In How Bayesian Networks learn, various conditions, aspects, circumstances or situational occurrences are captured within a single observation. For example, if you were observing/recording weather conditions, each observation could be composed of co-occurring aspects, such as the current humidity reading, the sunshine level, or whether it is raining or cloudy. These...

Read More ...

I've recently started thinking about how to simulate/model reinforcement learning and how it is implemented.  I knew what reinforcement learning was because I knew it described the learning that took place when Ivan Pavlov conducted his famous experiments on conditioning behaviour/learning in dogs. This however, of course, is just the theory and is different to actually implementing it as an algorithm to model learning in a computer. I read a paper by Mnih et al.on how reinforcement learning was...

Read More ...

Introduction As part of my academic research endeavours, I'm undertaking to train myself to analyse research papers with a more methodical and critical eye.  The particular paper reviewed in this article is "A Fast Learning Algorithm for Deep Belief Nets" by Hinton et al. and is part of a larger survey entitled "A survey of deep neural network architectures and their applications" by Liu, W. et al.  The approach I've used to structure my review process is outlined in Research Review Process. Table...

Read More ...

Introduction As part of my academic research endeavours, I'm undertaking to train myself to analyse research papers with a more methodical and critical eye.  The particular paper reviewed in this article is, "Multimodal Deep Autoencoder for Human Pose Recovery" by Hong et al., and is part of a larger survey entitled "A survey of deep neural network architectures and their applications" by Liu, W. et al.  The approach I've used to structure my review process is outlined in Research Review...

Read More ...

 I've created a systematic approach that aims at assessing certain aspects about papers that I'm reviewing. The first phase (Context and Understanding) is meant to cut to the core of the underlying research that is being presented. It is also hoped that by analysing the papers in this fashion, justification and reflection can be undertaken when considering individual aspects. The second phase (Methodological Issues) aims to consider the threats to the research's validity with a view to...

Read More ...

As early Greek philosophers tried to make sense and explain the world they perceived, so too might an artificial agent attempt to reason about the virtual world and circumstances it finds itself in. An understanding of the physical world as interpreted by physical senses and the non-physical world as interpreted by abstract concepts such as ideas, thoughts, emotion and our capacity to think and feel, has grounded much of how humans model their perceived reality. This research aims to...

Read More ...

Cognitive behaviourism researcher E.C. Tolman showed that behavioural conditioning alone, i.e learning purely through stimulus-response, is not enough to inform learning, and that cognitive mechanisms play a role. Through experiments, it was shown that when a learning agent, such as a rat, was put into a maze, it solved the maze faster next time if it knew there was a reward at the end of it, specifically that, “…rats do learn to expect goals in specific locations” (Tolman, Ritchie and Kalish,...

Read More ...

A stimulus may be defined as a class of relationships which exists between two parties (a sender and a receiver), specifically where the sender initiates the relationship and the receiver is primarily subject to it. Circumstance may then be defined as a particular response to a stimulus, and a situation as a set of coordinated circumstances that can characterise behaviour. This leads to defining observation as the witnessing or collection of circumstances and situations. In this model, a...

Read More ...

In addition to the research’s primary objectives, which are derived from autonomous observational exploration, this model could also allow for more directed, intervening and controlled exploration, one which is encouraged through reinforcement learning. Based on Pavlov’s work, B.F Skinner showed that S-R links can be re-enforced by pleasant or unpleasant environmental consequences, and that, “…behavior is principally controlled by schedules of reinforcement”, where a positive re-enforcement...

Read More ...

The experience sampling method (ESM) is a means of describing experiences. This method is a field-driven approach that aims to allows participants to more easily fill in questionnaires or surveys (relevant to their experiences) while they are actually situated within the context or field of the experiences that the questions are targeted towards, and this usually, “…involves sampling participants’ experiences in natural environments, in real time (or close to it), and on multiple measurement...

Read More ...

A Possible Approach The premise of this research, is that through an autonomous entity or entities that have been provided the facilities to experience and perceive situational information, and an environment that provides sensory stimulation indicative of those situations, that arbitrary situational experiences that are encountered can, in the first instance be identified and studied, detected, codified and measured, and therefore provide situational data for analysis such as learning,...

Read More ...