My research focuses on using computer game technologies to simulate real-time experiences of autonomous and self-aware artificial entities (NPCs or non-player characters).
This aims to develop new and novel real-time models for characterising the 'experienced' unknown, thereby improving contextual adaptation, behaviour, and situation detection in NPCs through exploration and learning within dynamic and sensory-based virtual environments. See Research Proposal, and A Philosophical Introduction.
Also see Research Terminology, Avenues of research and Simulating real-time experiences of autonomous and self-aware NPCs: An Approach
Project details can be found here: Research Project
Research Areas:
Activity:
-
Defining and Conceptualizing Agentic AI Friday 3rd of October 2025 10:53:48 PM
Since reading a survey on Agentic AI, I've formulated a basic blueprint of what I feel it is. Agentic AI basically concerns creating AI workers that are: Autonomous, Adaptable and goal-driven Use a combination of reinforcement learning (RL) and goal-oriented architectures/approaches Implements adaptive control strategies and techniques Designed to be resilient to change (and the unknown) Implements smart, opportunistic learning such as RL, imitation, pattern knowledge, priorities, social...
Understanding Q-Networks Thursday 25th of September 2025 08:53:32 PMQ-Learning is used as a solution to solve Markov Decision Processes (MPDs) (see Markov Decision Processes), i.e., its goal is to determine (learn) the best policy, i.e., moves, that an agent can take to reap the maximum rewards, which in most cases results in reaching the goal in the most optimal way possible. In this way, optimization is a key aspect and gives rise to the equation that is used to achieve this in Q-Learning, the Bellman Optimality equation. In Q-Learning, the Bellman...
Reviewing Playing Atari with Deep Reinforcement Learning Wednesday 24th of September 2025 06:06:39 PMIntroduction The paper reviewed in this article is "Playing Atari with Deep Reinforcement Learning" by Mnih et al. , which describes the first time Deep Neural Networks (DNNs) were integrated with Reinforcement Learning. Q-Learning has been used as a solution to Markov Decision Processes (see Markov Decision Processes), which uses reinforcement learning (RL) to determine the best policy that an agent can use to reap the maximum rewards through its moves. See Rationalizing Q-Learning It has...
Understanding Markov Decision Processes Wednesday 17th of September 2025 07:49:39 PMI've been reading papers presenting various approaches to Deep Learning using CNNs (Convolutional Neural Networks) and DBNs (Deep Belief Networks) to yield important solutions to existing problems. These are collectively called Deep Neural Networks (DNNs). Other approaches are using AE (Autoencoders) to simplify input for better learning. These are collectively referred to as Deep Learning approaches. Generally, these approaches are good for learning and approximating functions, i.e the...
Understanding Q-Learning Sunday 14th of September 2025 10:22:04 AMI've been recently trying to piece together my learning around how reinforcement learning is actually implemented algorithmically. The fundamental idea is that you'd like to simulate decision-making in an agent, but specifically that the decision-making process involves having the agent learn (and then make) moves that are the most beneficial to it. This, therefore, is an admirable and useful decision-making process to try and simulate. This also means that the agent must learn what 'good'...
Architecture for a single simulation Friday 12th of September 2025 11:00:01 PMSince Contextualizing Artificial Intelligence and Psychology, I've been designing a drawing that aims to represent the current architecture I have for creating real-time 2D simulations. This can be considered a blueprint for each game simulation. The design goal is to provide a means that allows the most common or otherwise reusable components, which make up almost all simulations (considered as separate games), to be reused in each simulation/game. An example would be providing a means for...
Contextualizing Artificial Intelligence and Psychology Monday 8th of September 2025 05:02:28 PMSince A Model of Belief and the Capacity to Know, I'm increasingly convinced that to understand how to artifically model human-like behaviour requires a methodical and fundamental understanding of human behaviour (in general), and, more importantly, what causes it. This, of course, is what Psychology pursues. This might seem implicitly reasonable, but here I want to make it robustly evident. To begin with, humans make decisions with remarkable flair; they are pretty good at considering the...
A Model of Belief and the Capacity to Know Sunday 31st of August 2025 06:27:51 PMIntroduction I read an article in issue 4 of Philosophy Now, entitled "Knowledge & Reasons" (read it here), in which the author (Joe Cruz) gives an interesting evolutionary narrative perspective of epistemology (theory of knowledge). It made me reflect on how beliefs in general might form and how knowledge is used to inform them. Table of Contents Model for 'belief'The capacity to 'know'A model for behaviourPerception and UnderstandingCause and effect Model for 'belief' I think,...
Situation detection using Bayesian networks Thursday 31st of July 2025 04:58:52 PMSince Thoughts on Bayesian NetworksI've been thinking about using Bayesian networks as a means to identify situations. In How Bayesian Networks learn, various conditions, aspects, circumstances or situational occurrences are captured within a single observation. For example, if you were observing/recording weather conditions, each observation could be composed of co-occurring aspects, such as the current humidity reading, the sunshine level, or whether it is raining or cloudy. These...
How Bayseian Networks learn Friday 25th of July 2025 11:32:49 AMSince Thoughts on Bayesian Networks,I've been thinking about how they actually work and why they work. I'm going to walk through the process behind the theory I presented previously. From a learning perspective, i.e., how they learn, my research suggests that they rely on statistics about the increasing number of observations over time. As they increase, this affects the average occurrence of any particular situation as they occur (or do not reoccur). For example, if you're designing a spam...
Thoughts on Reinforcement learning Wednesday 23rd of July 2025 01:39:44 PMI've recently started thinking about how to simulate/model reinforcement learning and how it is implemented. I knew what reinforcement learning was because I knew it described the learning that took place when Ivan Pavlov conducted his famous experiments on conditioning behaviour/learning in dogs. This however, of course, is just the theory and is different to actually implementing it as an algorithm to model learning in a computer. I read a paper by Mnih et al.on how reinforcement learning was...
Thoughts on Bayesian networks Sunday 20th of July 2025 12:30:12 PMSince Thoughts on Reinforcement learningand after reading that paper on DQN and being a bit more sure about how reinforcement learning is implemented algorithmically (Bellman update), I started wondering about other unrelated things, like what a Bayesian networks is. I've seen references to Bayesian networks in literature I've read without having an intuitive understanding of what it is and and how they work and, more importantly, what applicability they might have to me in general - because...
Reviewing A Fast Learning Algorithm for Deep Belief Nets Monday 16th of June 2025 02:42:14 PMIntroduction As part of my academic research endeavours, I'm undertaking to train myself to analyse research papers with a more methodical and critical eye. The particular paper reviewed in this article is "A Fast Learning Algorithm for Deep Belief Nets" by Hinton et al. and is part of a larger survey entitled "A survey of deep neural network architectures and their applications" by Liu, W. et al. The approach I've used to structure my review process is outlined in Research Review Process. Table...
Review of A survey of deep neural network architectures and their applications Monday 16th of June 2025 01:20:06 PMIntroduction As part of my academic research endeavours, I'm undertaking to train myself to analyse research papers with a more methodical and critical eye. The particular paper reviewed in this article is "A survey of deep neural network architectures and their applications" by Liu, W. et al. The approach I've used to structure my review process is outlined in Research Review Process. Table of Contents Research questionResearch aimType of researchMode of enquiryMethodologyResearch...
Review of Multimodal Deep Autoencoder for Human Pose Recovery Monday 16th of June 2025 01:02:58 PMIntroduction As part of my academic research endeavours, I'm undertaking to train myself to analyse research papers with a more methodical and critical eye. The particular paper reviewed in this article is, "Multimodal Deep Autoencoder for Human Pose Recovery" by Hong et al., and is part of a larger survey entitled "A survey of deep neural network architectures and their applications" by Liu, W. et al. The approach I've used to structure my review process is outlined in Research Review...
Reviewing A Real-Time Hand Posture Recognition System Using Deep Neural Networks Monday 16th of June 2025 11:42:50 AMIntroduction As part of my academic research endeavours, I'm undertaking to train myself to analyse research papers with a more methodical and critical eye. The particular paper reviewed in this article is "A Real-Time Hand Posture Recognition System Using Deep Neural Networks" by Tang et al.and is part of a larger survey entitled "A survey of deep neural network architectures and their applications" by Liu, W. et al. The approach I've used to structure my review process is outlined in...
A Systematic Research Review Process Wednesday 2nd of April 2025 09:33:13 AMI've created a systematic approach that aims at assessing certain aspects about papers that I'm reviewing. The first phase (Context and Understanding) is meant to cut to the core of the underlying research that is being presented. It is also hoped that by analysing the papers in this fashion, justification and reflection can be undertaken when considering individual aspects. The second phase (Methodological Issues) aims to consider the threats to the research's validity with a view to...
Research Proposal Tuesday 1st of April 2025 08:56:06 PMQuestions How can we create more realistic and adaptive non-player characters (NPCs) in video games? How can the pursuit for autonomous, self-aware NPCs help to model, develop and test new and novel real-time learning methods and techniques? Can modelling intelligent virtual entities in real-time computer games help to inform models for self-awareness in broader fields such as robotics or more general software applications? Introduction NPCs in games often follow predefined scripts for...
A Philosophical Introduction Friday 6th of September 2024 09:04:35 PMAs early Greek philosophers tried to make sense and explain the world they perceived, so too might an artificial agent attempt to reason about the virtual world and circumstances it finds itself in. An understanding of the physical world as interpreted by physical senses and the non-physical world as interpreted by abstract concepts such as ideas, thoughts, emotion and our capacity to think and feel, has grounded much of how humans model their perceived reality. This research aims to...
Models of social learning Friday 6th of September 2024 07:34:37 PMWhen considering a single learning agent in a virtualised world, research has shown that in social contexts, humans seeing or experiencing other people’s reactions or emotions, i.e observing stimuli and resultant responses in a social environment, can cause the same reactions to recur, for example, shared disgust (Sowden, Khemka and Catmur, 2022). This phenomenon is called Mirroring, and appears to be a mechanism that humans and animals, and possibly agents, could use to begin learning in...
Reflective Thinking and Cognition Friday 6th of September 2024 07:29:32 PMCognitive behaviourism researcher E.C. Tolman showed that behavioural conditioning alone, i.e learning purely through stimulus-response, is not enough to inform learning, and that cognitive mechanisms play a role. Through experiments, it was shown that when a learning agent, such as a rat, was put into a maze, it solved the maze faster next time if it knew there was a reward at the end of it, specifically that, “…rats do learn to expect goals in specific locations” (Tolman, Ritchie and Kalish,...
Detecting situations from experiences Friday 6th of September 2024 07:27:22 PMAn approach to situation detection is to represent the world objects in a hierarchical scene graph, and model sensory events as influences on those objects. This can allow for the identification of effects of causality and the influence it has on multiple related objects, such as the propagation of effects to their dependencies, e.g pushing a box with a pen in it, will also move the pen relative to the box. In this way, when a stimulus affects an object, the resulting change/response can be...
Modeling Observations Friday 6th of September 2024 07:26:01 PMA stimulus may be defined as a class of relationships which exists between two parties (a sender and a receiver), specifically where the sender initiates the relationship and the receiver is primarily subject to it. Circumstance may then be defined as a particular response to a stimulus, and a situation as a set of coordinated circumstances that can characterise behaviour. This leads to defining observation as the witnessing or collection of circumstances and situations. In this model, a...
Exploratory and behavioural Inclinations based on Fear Friday 6th of September 2024 07:23:59 PMFear and the intolerance of the unknown appear to significantly influence human inclination to explore (Carleton, 2016). Similarly, and perhaps inherently related, is motivation and emotion, which appear to be cognitive constructs that appear to be inherent in human experiences and which influence their behaviour. This suggests that evaluating stimuli and determining how they affect the agent's sensitivity to fear is likely an important aspect in processing observations. In this way, fear also...
Exploration : A philosophical approach to curiosity and learning Friday 6th of September 2024 07:20:51 PMIn addition to the research’s primary objectives, which are derived from autonomous observational exploration, this model could also allow for more directed, intervening and controlled exploration, one which is encouraged through reinforcement learning. Based on Pavlov’s work, B.F Skinner showed that S-R links can be re-enforced by pleasant or unpleasant environmental consequences, and that, “…behavior is principally controlled by schedules of reinforcement”, where a positive re-enforcement...
Expectations : A Psychological perspective Friday 6th of September 2024 07:16:46 PMForming Expectations As the previous research by Pavlov showed, meeting any expectation, from a learning perspective, appears to be an important mechanism which can reinforce prior learning of cause and effect (stimulus and response). Through exploration, we proposed to utilise this mechanism to reinforce or redefine prior situational learning. In this way, expectations are a high-level abstraction about causality (events) and their results/outcomes, while causality itself is an abstraction of...
Psychological Perspective: Describing, Defining and Interpreting Experiences Friday 6th of September 2024 07:11:54 PMThe experience sampling method (ESM) is a means of describing experiences. This method is a field-driven approach that aims to allows participants to more easily fill in questionnaires or surveys (relevant to their experiences) while they are actually situated within the context or field of the experiences that the questions are targeted towards, and this usually, “…involves sampling participants’ experiences in natural environments, in real time (or close to it), and on multiple measurement...
The pursuit of knowledge Friday 6th of September 2024 07:08:39 PMThe acquisition of knowledge to inform an understanding of anything has, throughout the ages, produced multiple theories that try to explain how it is achieved. For example, Empiricism suggests that knowledge is derived primarily from sensory experience alone. Thomas Hobbes believed that everything in the universe is purely physical in nature and that cognitive processes are determined by predictable physical laws, and that ideas originate from sensation. Indeed, Hobbes believed that, “…sense...
Research Approach Friday 6th of September 2024 10:42:53 AMA Possible Approach The premise of this research, is that through an autonomous entity or entities that have been provided the facilities to experience and perceive situational information, and an environment that provides sensory stimulation indicative of those situations, that arbitrary situational experiences that are encountered can, in the first instance be identified and studied, detected, codified and measured, and therefore provide situational data for analysis such as learning,...