As early Greek philosophers tried to make sense and explain the world they perceived, so too might an artificial agent attempt to reason about the virtual world and circumstances it finds itself in.
An understanding of the physical world as interpreted by physical senses and the non-physical world as interpreted by abstract concepts such as ideas, thoughts, emotion and our capacity to think and feel, has grounded much of how humans model their perceived reality.
This research aims to investigate how artificial learning agents can acquire arbitrary and initially unknown situational awareness and meaning through similar environmental perceptions, focusing specifically on defining and coping with the ‘unknown’ encountered within their corresponding virtual realities.
In understanding the nature of the unknown, the research will explore simulating models for defining it.
A possible model, for example, is through observing causality, for example, using Stimuli-Response theory to inform an agent’s perception of reality from its experiences of causality within it. By sensing stimuli and response sequences that are encountered and observing their coordination and organisation patterns (reasoning) could provide a model to describe what is initially unknown, but which is otherwise empirically ‘experienced’, thereby constructing a conceptual model of what happened, i.e, explaining the unknown by defining what is experienced of it.
This research is primarily focused on simulating virtual realities in real-time, using computer game technologies to produce variations of ‘experience’ for game agents to model an understanding from. This approach allows for the application and development of models within a dynamic and real-time experimental environment, creating an observation-based, situational Epistemic learning environment.
By defining models of knowledge formation and understanding from arbitrary experiences based on real-time interactions within virtual worlds, this research offers the ability to not only define and systematically characterise the encountered unknown, but also to explore the additional affordances that modelling it might bring.
For example, within a simulated sensory environment that uses the aforementioned example, this research can also explore the potential for defining and detecting contextual variations, i.e, change, in those signatures as a basis for defining or recognising ‘situations’ or other abstractions from experience.
Figure 1 shows a theoretical approach that might use elementary stimuli and responses to characterise and formalise a model of ‘experience’ of the unknown in order to help understand its nature.
With potentially unlimited environmental configurations and sensory stimuli that can be placed within virtual realities, this research has the opportunity to draw deep correlations between what the unknown is, what causes it to occurs, how to define it, and what it might mean (as far as the agent is concerned) and possibly automatically determining what an appropriate response could be, but crucially putting in place an empirical methodology for rationalizing it.
The benefits of characterising the unknown could be dealing with the unknown, or identifying types of unknown behaviours, i.e, revealing hidden truths, or abnormalities and the situations that cause them. This might help inform a capability to define how to reproduce them, which is often key in understanding them. Equally, identifying when situations fall outside of known parameters could be useful in determining how well a system is adapting to new situations or correctly identifying exceptional situations from otherwise known ones.
Table of Contents
The Unknown
An interesting and enigmatic concept is how to represent the encountered unknown, and perhaps as important, is composing knowledge from that representation to derive an intuitive sense that is definable, repeatable and explainable.
Figure 2 illustrates that the formation of various abstractions can be considered as the process of reasoning.
Stimuli-Response modelling or other primitives could be used to form abstractions and generalisations for the complexity of interpreting the ‘experienced’ unknown. An important aspect is to systematically define knowledge and use it for future behaviour or reasoning.
Real-time computer simulations of events, situations, and encounters by an artificial learning entity provide ample opportunity to respond to this task in an experimental way that extends beyond purely theoretical models, allowing for empirical application and measurement of models in the research.
Virtual Reality
The benefit of using simulation and virtual realities is that abstractions can be easily manipulated and iterated through, while the extent to which models of understanding can be simulated allows for a relatively short extension of simulations to the real world. This means that the ability to successfully abstract and define game agent experience could more easily translate to effective real-world applications than more theoretical approaches. For example, the simulation of sensory perception in games, or understanding the ways in which sensory perception can form knowledge, can have real-world implications in education.
Simulation also brings the opportunity to detect consistency and patterns in behaviour, or indeed abnormalities, while monitoring, which could be a serendipitous by-product of this research, particularly by reasoning about the experienced unknown by composing identifiable and discernible components to define it.
This might identify or recognise the effects of the unknown (or known situations) in a more formalised way, e.g as the composition of simpler primitives such as stimuli-response sequences and temporal networks. This might also have an impact on the effect that sensory perception in computer games could be implemented to enhance game-play, realism and the communication of knowledge.
Lastly, simulation can circumvent some ethical constraints, for example, modelling and simulating threats, and exploring the effect that safety has on interpreting the unknown, which opens the avenue to explore the influence of threat and protection might have on the behaviour of NPCs when confronted with the unknown.
Autonomy
The use of autonomy can be well-simulated in real-time computer games using Artificial Intelligence. This allows the use of available contexts and the models within the simulation environments to inform the creation of autonomous behaviour, not only in achieving simulated exploration but in applying and interpreting models in more unknown situations.
Combining simulation and autonomy also provides a means to codify and define rules and research outcomes actively and test their effectiveness.
Through encounters with the unknown, exploration is often an inevitable precursor. In this way, autonomy and influencing exploratory behaviour in agents brings with it more random and realistic encounters with the unknown, which is useful when studying the reaction of agents to the unknown, including the effects of curiosity and fear. In this way, the ability to simulate autonomous encounters using Artificial Intelligence can be used to good effect to produce organic experiential data for interpreting the unknown by observant agents.
Exploring, interpreting and reacting to the unknown means that observation or sensation is likely to play a key aspect of forming knowledge during simulation. In this respect, there is an opportunity to explore how agents can observe and interpret the data they need to input into knowledge models that define what they experience.
Through reacting to the unknown, there is potential for determining or understanding how the agent should or might behave, suggesting that the research could dig deeper into the origins of creating ‘appropriate’ behaviour in the face of interpreting the unknown.
Lastly, we find that teaching and sharing knowledge makes learning easier and looking into how models of pre-existing understanding might be incorporated into other game agents is also an interesting avenue of investigation relevant to this research, particularly in light of advances in distributed computing, networking protocols for coordination and agent-based, collective intelligence. This can be considered as the externalisation of experience to teach others.
Questions
- To what extent can a game character function by being able to observe, sense and experience its own environment?
- Is it possible to define the characteristics of an unknown object or learn about its nature through exploratory observation?
- Is it possible to uniquely define and distinguish between situations based on observation, particularly while they are unfolding in real time?
- Can circumstances and situations be represented through the collection of sensory observations?
- Can the nature of situations be defined through observation?
- Can situation-specific behaviour be learnt and reused to respond to observations?
- How can observations be simulated in virtual reality and implemented in a game character?
- To what extent can observations be transferred and used by collaborating game characters?
- Can developing sensory virtual realities provide a suitable means to define and make observations within them? How would this be done, and how useful can it be?
- Can experience be defined using observations, and how can these experiences be used in a game character?