The experience sampling method (ESM) is a means of describing experiences. This method is a field-driven approach that aims to allows participants to more easily fill in questionnaires or surveys (relevant to their experiences) while they are actually situated within the context or field of the experiences that the questions are targeted towards, and this usually, “…involves sampling participants’ experiences in natural environments, in real time (or close to it), and on multiple measurement occasions” (Meers et al., 2020).

ESM usually defines experience as the collection of contextual experience information through surveys and questionnaires; however, our research aims to base experiences on observational data (S-R links and relevant CSIs) collected by agents. However, like ESM, the underlying goal is the same, which is to obtain descriptive information about raw sensory experiences from situational contexts (the environment), as proposed by Lewin’s Field theory, but to be more generic and applicable to all gathered sensory data.

While there needs to be research into determining how best to compose this sensory data into contextualised experience, the initial approach is that it would be a composition of S-R links, other environmental context and CSIs, that is translated into a usable and cohesive representation of experience.

To implement such an ESM, a possible approach is to process S-R links and CSIs through a processing pipeline which uses methods to attempt to filter, validate, aggregate, and contextualise the raw observational (sensory) data into experience. Figure 9 shows how the identification of observations precedes the increasingly higher level of abstraction that results from processing observations.

Figure 9 illustrates how the reasoning process might be defined by producing abstractions.

Our research proposes to analyse agent-based observations in a similar manner that clinical psychology uses to describe and rationalise experiences, namely through:

1. Eidetic Reduction
2. Eidetic Variation
3. Phenomenological Reduction

For example, Eidetic Reduction is a consultation and deliberation process, usually performed by humans to determine the underlying essence of something, such as experiences or observations. This is done to try to remove any unneeded context that may occur in one or multiple variances of that experience, but keep the relevant context. What remains after the irrelevant variances have been removed is said to be the essence, which, once obtained, would allow one to define the unique identification traits of the experience. The essence of anything is typically an abstraction formed by simplifying and choosing to disregard information. This might be useful in trying to identify and discern different situations from similar observational experiences.

Defining situations

One example of a further abstraction from experience (which is itself an abstraction based on observation) is the identification and formation of situational boundaries (situations). Situations can provide context such that different situations can be responded to differently, such that learning and a specific rule creation might be specifically targeted to a unique situation, and not apply in others. In forming situations, time and duration of observations are likely to be a factor when determining when one situation ends and another begins, and this research would explore this.

Generally, Eidetic Reduction includes various approaches and considerations, including:

• Profiling, which is the need to recognise the same thing that is experienced or manifested differently each time
• Detecting essence, which is the detection of essence by looking for the underlying essence, devoid of unneeded and generally irrelevant information or context
• Seeking Universals, which is the process of utilising a collection of several instances of the same situation to identify to remove/reduce things that are not general or always inherent to identify so-called Universals in situations. This could be achieved through variations of situations in the simulation environment.
• Identifying Objects of awareness/consciousness, which is the process of attempting to find the objects that have been included in the subject’s experience (or in our case, observations), and can extend to looking for meaning in those objects.

The last technique might be a useful way to define observations as about various objects that make up the context of the agent’s situation, particularly in light of Husserl's suggestion that, “..any experience we have includes three elements: a subject, a predicate and an object” (Deurzen, 2015). This supports the perhaps obvious idea that before reasoning can occur, identification of conceptual objects must occur.

A conceptual illustration that aims to implement a sensory-based system is illustrated in Figure 10, which outlines a possible approach.

The opposite of Eidetic reduction, is Eidetic Variation which specifically varies instances of the same thing, to establish universals (things that do not change), and this is usually applied, like Eidetic reduction, to non-material objects such as as ideas, acts, conceptual simulations such as a theoretical situation - or as this research proposes, variances provided by similar experiences observed in a simulated virtual reality. This could be used to learn about a situation by varying it until the fundamental characteristics or rules emerge, much like what happens in back-propagation (machine learning).

Phenomenological Reductions, which form part of the discipline of Phenomenology, are also a study of conscious phenomena, and are a “… methodical study of the process of human awareness and the experiences” (Deurzen, 2015). Here, a conscious phenomenon might be an arbitrary description of a human experience or, as proposed, one that is observed by an agent, for example.

Likewise, Phenomenological reductions follow similar but different prescriptive methods to Eidetic Reduction:

• Epoche, which is removing biases, assumptions and any prejudices when interpreting an experience (noise). This is similar to seeking essence.
• Horizontalization which is identifying the limits or unknowns that may need to be explored. This could help determine what to explore or look for next, in terms of acquiring knowledge. This effectively identifies gaps

These methods and approaches could be combined with Eidetic reduction to refine and clarify, define and refine experiential agent observations. Naturally, an agent is not human; however, there are aspects of this approach that might apply to an artificial intelligence entity describing and clarifying its observations to identify what to do about them.

Both Eidetic and Phenomenological reductions are often used in therapeutic consultations between a Therapist who is listening (applying reductions) to observations that the patient (subject) is describing about their experience from memory (recollection). Figure 11 illustrates Phenomenological observation used in such a setting, where the goal is for the patient (Client) to gain an underlying understanding of their experiences, through dialogue with the Therapist to help obtain the essence of the meanings behind the client’s experiences.

One could conceivably, in this respect, replace the Client with the agent (to make observations about its experiences), and the Therapist with the thinking or reasoning cognitive construct in the agent that is trying to make sense of its observed experiences (forming abstractions). This would be done by reducing irrelevant situational complexity to find an experience’s unique, identifiable core. An interesting aspect of this approach is the continual looping feedback mechanism (like a game loop) where the experiential observations made by the client and assessments of these observations, made primarily by the therapist, are validated with the client, effectively providing training data for defining experiences.

Using a framework centred around Eidetic and Phenomenological reductions to analyse awareness and experiences that an agent observes might provide the basis for a structured and methodical means to process the situational complexities experienced in the agent's reality.

In practice, Phenomenology, like Eidetic reduction, operates on cognitive, conceptual constructs such as ideas and experiences, and one might suggest that anything that is to be reasoned about (or processed), whether its derived from a physical perceptions or not, can be thought of as a conceptual simulation at some point merely by thinking about it, i.e cognitively simulating it. In this way, collecting virtual sensory observations which are then reduced to software ideas/models and which ultimately need to be processed in order to establish reason and quantify any experience, might also be seen as being equivalent to conceptual simulation.

In this way, agent observations could ultimately manifest as cognitive simulations modelled as software models, which then arguably apply to being modelled using Phenomenology as a structural and rigorous formal basis on which to quantify experiences. This would enable a systematic approach to agent observation in order to make sense of experienced situations and the associated situational contexts that occur within the virtual environment.

References

Meers, K. et al. (2020) ‘mobileQ: A free user-friendly application for collecting experience sampling data’, Behavior Research Methods, 52(4), pp. 1510–1515. doi: 10.3758/s13428-019-01330-1.

Deurzen, E. van (2015) ‘Structural Existential Analysis (SEA): A Phenomenological Method for Therapeutic Work’, Journal of contemporary psychotherapy, 45(1), pp. 59–68. doi: 10.1007/s10879-014-9282-z.

A Possible Approach

The premise of this research, is that through an autonomous entity or entities that have been provided the facilities to experience and perceive situational information, and an environment that provides sensory stimulation indicative of those situations, that arbitrary situational experiences that are encountered can, in the first instance be identified and studied, detected, codified and measured, and therefore provide situational data for analysis such as learning, behavior creation and situation detection.

This pursuit is specifically in the context of forming knowledge in order for an agent to formulate an understanding or rationalisation of new and unknown situations it experiences while exploring.

Observations might include the perception of unknown entities, such as game objects, such as characters, walls, etc., within the virtual reality, or equally classification of multiple aspects of an observation to form a sense of defining the makeup of a situation that is currently occurring.

In this way, objects and the environment that the agent interacts with will need to possess characteristics and properties that can be sensed.

Modelling circumstantial perception in agents based on the assimilation of experience, specifically through empirical agent observation, is a key aspect of the simulations proposed in this research.

Figure 4 provides a basic illustrative example of the scope of the research. 

The overall research approach considers a linear progression around phases of modelling and understanding experience within virtual worlds:

1. Simulating experiences (phase 1)
2. Observing experiences (phase 2)
3. Defining experiences (phase 3)
4. Using experiences (phase 4)
5. Distributing experience (phase 5)

The methods and research design for each of these phases of research are discussed in more detail next.

Phases 1 and 2

Simulating and Sensing Virtual Environments

The first research goal is to be able to sense and collect observational data systematically about arbitrary in-situ experiences when the agent experiences them.

This will likely involve the ability to create a primary autonomous learning agent and the ability to perceive its observations through the acquisition of environmental stimuli in order to acquire knowledge about situations that are encountered and experienced by the agent.

Our research proposes to position an artificial learning agent within a simulated virtual reality where that environment exerts sensory forces that it can observe through simulated sensation based on stimulus-response. By establishing relationships between stimuli that occur and resultant responses, and an ability to define estimations of ongoing situations through observing environmental contexts that which these occur, provides a model for our research to pursue the establishment of an experience-based autonomous learning environment.

Perceiving Virtual Reality

An important aspect of such a learning simulation is engineering of an environment that can be designed to emit circumstantial information as part of the virtual reality, including housing variables, constantly evolving artificial entities or game objects like characters, behaviours and other environmental variations that the learning agents can perceive and learn from. Examples of possible 2D and 3D virtual environments that might be created are shown in Figure 6.

Our research proposes to use sense detection architectures to simulate the production of situational stimuli within the simulation environment as a means to acquire sensory experience, while the simulation of S-R links is to be derived from an approach like Epic’s Unreal AI Perception System (‘AI Perception’, no date) which is based on the generation and distribution of stimuli events throughout the environment.

Objectives (phases 1 and 2)

Key research goals in phases 1 and 2 include:

1. Creating a reality (world) to serve as the simulation environment that can be controlled to produce varying perceivable situations to agents within it, likely using stimulus-response theory.
2. Incorporating a means for the environment to be sensed such that it has an impact on the experience of its inhabitants, i.e the agents. This is likely to include producing sensation.
3. Creating controllable and autonomous entities that can function within this sensory environment, that can perceive it, and objects within it.
4. Establishing an observational mechanism for collecting and recording experiential/situational data perceived by the agent. (modelling the unknown)

The combination of these two phases would focus on the technical aspects of constructing the experimental platform on which this research will be used to sense and simulate sensory knowledge acquisition.

The simulation system design itself and implementation of supporting subsystems, and the architecture necessary to unify them, such as event and resource management, networking, scripting, artificial intelligence and computer graphics, will also require implementation. In this way, a technical aim is to produce a virtual environment that can visualise and demonstrate the experiments of models simulated in this research.

It is in this way that this research is not only concerned with how experiences influence an agent’s behaviour and understanding, but also explores how perception and experience can be implemented as sensing software models and data structures, particularly within a context of real-time perceptual learning.

In many respects, this phase forms the foundation on which further exploration and research will be based. It can largely be seen as a building phase, focusing on the software design and engineering required to model and simulate agent observations. This phase constitutes an application of various computer games technologies, including graphics, simulation and engineering.

Phase 3

Formation of Abstractions

Modelling the unknown

The primary goal of the phase is the creation of higher-level situational experiences from the previously collected sensory data in order to act on them. This phase provides a means to formulate a model to define experiences and distinguish between situations (modelling the unknown). This data can then be used in subsequent phases as the basis of a variety of possible outcomes, including to learn about the nature of situations, behaviours, and decision-making in unexpected or new and unknown situations, etc.

While raw stimuli-response sequences might be used to underlie raw sensory experiences, there could be a process that uses those basic sensory experiences to compose a more holistic, higher-level description of that experience - or to interpret it in order to find meaning in that experience.

Allowing a simulation of reality and a means to observe it opens the possibilities to study the effects of situations and, therefore, the resultant observational data that represents them. This, when synthesised into experience, can then provide learning material for subsequent reasoning and understanding of the causes and effects of situations.

A targeted understanding of how and why situations occur and what happens when they do can then allow for the formulation of unique patterns or rules that can define them, ultimately making it possible to detect when rules are (and still are) applicable to a situation.

Objectives

Key research goals in this phase include gaining a discernible definition of abstractions (circumstance, situation and experience) from raw experiential (experienced) data such as S-R links and CSIs:

1. Defining and detecting, generalising unique and discernible situations and circumstances from experiential data, likely utilising sensory data, scene graphs and temporal networks.
2. Describing observations, eg, processing through Eidetic and Phenomenological reduction processes.
3. Producing aggregate, higher-order structures from observations, i.e. defining basic situation-specific knowledge such as experience.
4. Construction or use of cognitive models for processing sensory experiences.

These represent the formation of abstractions from experiences.

Phase 4

Using and Sharing Experiences

Once a notion of circumstance, situation and experiences can be identified, an agent may be used to become more contextually aware to inform behaviour and gain an understanding in response to those situations, i.e using abstractions to influence their behaviours.

Defining Behavior

Typically, behaviours in games are not generated from experience and instead are defined by pre-defined behavioural logic or scripts that, when interpreted, define how the agent should behave in various circumstances. This would be pre-defined actions to predefined circumstances, meaning behaviour is not created, but pre-existing behaviour is triggered.

Our research proposes that it may be possible though an experiential learning model/approach to dynamically generate these behavioral scripts, as part of continual “online” experience gathering exercise, and then refine them through continual variations in experiences in the virtual reality through a real-time feedback-response loop, and utimately being able to use them to produce behavior in the future in response to similar contexts that were used to generate them.

An approach developed by (Robertson and Watson, 2015) was to generate behaviour trees (BTs) that were based on the supply of action sequences, where the focus was on predefined actions as textual representations such as “train”, “evolve”, etc, in order to remove redundant common behaviours. Our research could instead represent actions as more detailed compositions of S-R sequences derived from actual agent observations. BTs are appropriate because they produce reactive behaviour which aligns well with the stimulus-response model. This would allow agents to define and create behaviour as they experience situations, as new experience-based rules, which the agent can use as part of its behavioural knowledge to either carry out those behaviours in the future or recognise specific behaviours based on their S-R sequences encountered in real-time or “online” while experiencing its virtual reality. Equally, these learnt behaviours could be transferred to other agents for them to use when situations (context) call for it.

The results of externalising “online” experience and other dynamic abstractions could also be used to inform more traditional, static and “offline” training models, which might help make static artificial neural networks (ANNs) more contextually aware by incorporating the variable experiences that only “live” and online training data can provide. This combination of experience and offline deliberation might mitigate failures in traditional ANNs that often occur because, “…internal rules become fixed and do not adapt to new situations of use”, and this could help augment traditional ANNs that traditionally could not, “…adapt fast enough because the training algorithms are too slow to meet real-time requirements”, that occur in the real-world. (Denning and Arquilla, 2022). It's not clear if ANNs would need to be designed initially to be able to train from sensory input or if existing networks that have never witnessed causality data would need to be adapted or re-engineered.

This externalisation of learning and experience, such as saving S-R link sequences or higher-level synthesised situational abstractions, might also be useful for a static analysis of learning performance, providing empirical evidence about what learning actually took place and what has been learnt from any particular experience of a situation. This might include the provenance and formulation details of the underlying expectations/rules/knowledge that were synthesised as part of that learning. This can be used as a basis for evaluating the performance of the underlying model to measure progress, intelligence or learning capability of our model, but more importantly, in showing what specific data/situation/experience contributed to a learnt behaviour.

Social Context and Shared Experience

An extension of the experiential system proposed by this research is the possibility of gaining experience by interacting with other similar learning agents. This might be realised through the networking of autonomous agents. This is the field of multi-agent research.

If primitive experience might be simulated through the evaluation and establishment of S-R links as the basis of experiential knowledge formation, this may be externalised (saved), and then reused and transferred between agents. Such a mechanism may form the basis for distributed advice or learning among collaborating, social agents.

The ramifications of this are that learning need not require initial practical experience once that situational experience is already formalised or when CSIs match (akin perhaps to transfer learning in ML). An agent might be able to pick up foundational sensory-experiential derived advice from other agents as it progresses through levels, situational tests or other virtual realities, for example.

It would be interesting to find out if, as research suggests (Nonaka and Krogh, 2009), tacit knowledge, which is not initially captured as explicit knowledge, can be acquired through additional transferred experiences and therefore enhance and improve prior learnt knowledge. This might, however, be subject to the same data loss that occurs when any simplification/abstraction is formed.

The variation of experiences and perspectives as described would be achievable in the controlled simulation environment through loading of different situational trials, levels and alternative environmental or world circumstances.

In addition, through distributed observational experience, research suggests that even more distributed and communicative agents, which produce swarm-like behaviour, can allow multiple individuals to contribute to the actual monitoring of environmental change and provide feedback about observed variances. This is particularly relevant for the creation of swarms of virtual perception sensors, as previously discussed, to contribute to environmental or situational monitoring within virtual worlds. Virtual worlds in this respect are themselves an abstraction of the execution environment in which reasoning can be established from data that is present within them. This would likely alleviate a single point of sensing failure (or bias), and form part of a decentralised, distributed observational capability that senses change in general. (Greengard, 2022),

Objectives

Analysing abstractions

Key research goals in this phase are based on using established situational knowledge to create and inform a situational perspective or basis from which an agent can:

1. Learn about the emergent nature of captured situations through analysis (possibly ML) of newly constructed situational knowledge.
2. Identify relationships and expectations between/in experiences and underlying goals.
3. Produce respectable reaction in response to situations (behaviour selection), e.g, through the generation of situational behaviour trees
4. Externalise experience about situations and distribute learning and sharing of situational knowledge between collaborating agents.

Research into simulating agent autonomy based on effective reasoning about circumstances is likely to require research into contemporary AI approaches and theory. In order to implement a strategy for agent autonomy, looking into the types of intelligence models that may help discover emergent rules from complex circumstantial data, such as machine learning (ML), and how the formation of cognitive maps of the circumstance might be developed, is likely to require more research.

This phase might produce some interesting opportunities to explore new synthesised strategies (based on analysis of sensory experiences) and then feed this back into the research simulation based on analysis of patterns and results.

This phase is also concerned with the creation of distributed and shared experiences between agents.

Ultimately, the goal in this area of research is to determine the requirements and theory needed to model novel uses of the abstractions formed by the agent.

Phase 5

Reporting and Outcomes

The final phase of the research is to write up the research write-up of the findings and observations from the research objectives performed in the prior phases.

The outcomes that might be reported, but are not limited to, might include:

• The learnings discovered while creating perception in agents and environments, including the construction of perceptual software models
• The analysis of experiences and situations
• The empirical outcomes of the speculated conjecture as posed throughout this proposal, such as those on expectations and modelling the unknown
• The usages of the externalised or formalised representation of experience and situations, including relevance to AI and learning
• Approaches to solving the problem of context and fixed rules for responses.

Our research has the opportunity to explore and contribute to multiple disciplines, for example, Epistemology, Psychology, Artificial Intelligence, Simulation and Computer games technologies such as Graphics and Networking.

An example of topics that are relevant to our research is listed below.

Appendix B - Illustrations

1. Crytek’s Target Tracks Perception System (TTPS) (Rabin, 2014a)
2. Subscribing to stimuli in the environment (Epic’s Unreal’s AI Perception System)
3. How an agent might actively observe its environment.

References

 

‘AI Perception’ (no date). Available at: https://docs.unrealengine.com/5.1/en-US/ai-perception-in-unreal-engine/ (Accessed: 5 January 2023).

Burnes, B. and Cooke, B. (2013) ‘Kurt Lewin’s Field Theory: A Review and Re-evaluation’, International journal of management reviews : IJMR, 15(4), pp. 408–425. doi: 10.1111/j.1468-2370.2012.00348.x.

Ceria, A. and Wang, H. (2023) ‘Temporal-topological properties of higher-order evolving networks’, Scientific Reports, 13(1), p. 5885. doi: 10.1038/s41598-023-32253-9.

Denning, P. J. and Arquilla, J. (2022) ‘The context problem in artificial intelligence’, Communications of the ACM, 65(12), pp. 18–21. doi: 10.1145/3567605.

Greengard, S. (2022) ‘Swarm robotics moves forward’, Communications of the ACM, 65(12), pp. 12–14. doi: 10.1145/3565979.

Kim, S. D. (2012) ‘Characterizing Unknown Unknowns’. Available at: https://www.pmi.org/learning/library/characterizing-unknown-unknowns-6077 (Accessed: 6 December 2023).

Masuda, N. and Holme, P. (2019) ‘Detecting sequences of system states in temporal networks’, Scientific Reports, 9(1), p. 795. doi: 10.1038/s41598-018-37534-2.

Mathews, S. (2022) ‘Cppgamelib’. Available at: https://github.com/stumathews/cppgamelib (Accessed: 21 January 2023). 

Nonaka, I. and Krogh, G. von (2009) ‘Tacit Knowledge and Knowledge Conversion: Controversy and Advancement in Organizational Knowledge Creation Theory’, Organization science (Providence, R.I.), 20(3), pp. 635–652. doi: 10.1287/orsc.1080.0412.

Oettershagen, L. and Mutzel, P. (2022) ‘TGLib: An Open-Source Library for Temporal Graph Analysis’. arXiv. doi: 10.48550/arXiv.2209.12587.

Rabin, S. (2014a) ‘Crytek’s Target Tracks Perception System’, in Game AI Pro. A K Peters/CRC Press, pp. 432–441. doi: 10.1201/b16725-37.

Rabin, S. (2014b) ‘How to Catch a Ninja: NPC Awareness in a 2D Stealth Platformer’, in Game AI Pro. A K Peters/CRC Press, pp. 442–451. doi: 10.1201/b16725-38.

Robertson, G. and Watson, I. (2015) ‘Building behavior trees from observations in real-time strategy games’, in. IEEE, pp. 1–7. doi: 10.1109/INISTA.2015.7276774.

Wu, H. et al. (2014) ‘Path problems in temporal graphs’, Proceedings of the VLDB Endowment, 7(9), pp. 721–732. doi: 10.14778/2732939.2732945.

Zhan, X.-X. et al. (2021) ‘Measuring and utilizing temporal network dissimilarity’, arXiv.org. Available at: https://arxiv.org/abs/2111.01334v1 (Accessed: 1 January 2024).

Introduction

I read an article in issue 4 of Philosophy Now, entitled "Knowledge & Reasons" (read it here), in which the author (Joe Cruz) gives an interesting evolutionary narrative perspective of epistemology (theory of knowledge). 

It made me reflect on how beliefs in general might form and how knowledge is used to inform them.

Table of Contents

Model for 'belief'

I think, fundamentally, and simply, that the progenitor of all knowledge must be certainty. For example, any known or learnt cause and effect, predictable pattern or pure function would qualify in my mind as knowledge.

Furthermore, to derive a belief in anything likely requires acting on knowledge and requires an initial set of beliefs to be based upon, which the author exemplifies using axioms or rules in Mathematics. These, I consider, are the origins of belief and upon which any new rational belief may begin. These are apriori beliefs.

A belief may be true/false, good or bad, and so there is an element of subjectivity that influences what one may consider a belief's corresponding quality or correctness, or even appropriateness given the situation.

Indeed, a set of apriori beliefs may be incorrect, misinterpreted or subject to personal and subjective rendering. Either way, any measure of quality derived from a belief system/process that necessitates subjectivity is subjective in and of itself. In this way, a measure of quality must be agreed upon by a shared agreement or consensus that the belief is good/bad/wrong/right, etc. This is (necessarily) how the judicial system has determined that a collective balance of probabilities from a jury is required to inform a good and appropriate measure of agreement/consensus when individual subjectivity is too dangerous to consider solely. 

Following a systematic derivation from a priori beliefs that are used in conjunction with known rules of causality (knowledge) and coupled with the subjectivity of the reasoner, an opinion or belief based on the prior components can be modelled:

See A Model for Belief.

The capacity to 'know'

Another interesting thought presented in that article is, "What makes knowledge possible at all?".

For having a capacity to know anything (a model for the capability of knowing), I consider the requirements that provide the capacity to 'know' being Memory and a place to store models of cause and effect (such as the Brain) and the ability to interpret (or process) cause and effect, which I consider to be logic.

The models of cause and effect are representations of what we consider knowledge - they provide us with a clear idea of what causes what. Lastly, there must be a need for knowledge. For example, knowledge might help you survive or provide a reward.

Then you need personality to provide subjectivity to qualify the reasoner as an individual, with freedom and all the unpredictability this brings and which often appears chaotic when externally observed by others. This helps provide subjectivity, a potential for error, misinterpretation, freedom and autonomy, etc.

My feeling is that the mind is either an extension of the brain or an abstraction that the brain creates/uses to make it possible to provide or draw upon the subjectivity above. This, I think, is probably what makes us human and likely drives us to be conscious. Being conscious of things that influence you, I believe, leads to the inherent subjectivity that makes us unique and individual beings. In my model, I represent the engine of subjectivity to be the mind.

With a capacity to store notions of cause and effect (memory) and an ability to interpret them (logic), one can then draw on one's subjective experiences accessed via the mind.

One might consider the brain to be the physical device that the mind uses to produce abstract concepts and ideas to form knowledge that is stored/retrieved in the brain. 

Whether it is the mind or the brain that is the playground for deriving beliefs, models of causality are likely developed using abstractions, exploration and theoretical use and planning of these models of causality. They are likely simplified representations of how our brain/mind simulates or represents the world in terms of cause and effect.

Either way, these 'representations' are models of cause and effect that we've learnt to represent in our minds/brains. In this way, I consider learning to be the process of knowing cause -> effect, just like a neural network provides a model for a mapping from cause -> effect by 'learning' based on observation of cause and effect. Similarly, other models like Bayesian networks use statistical averages of such observations to derive this mapping of cause and effect.

It is not certain what actual models exist in the brain/mind; however, our artificial models (Neural networks and Bayesian models, etc.)  aim to simulate them. 

A model for behaviour

An intriguing idea is how all of this informs behaviour, and an idea presented by Cruz is that different representations inform different behaviours, such that one can "decouple behaviour from stimuli through representations". 

It's not explicitly stated that 'representations' in Cruz's mind are what I consider them to be, i.e actual instances of models of cause and effect, but the inherent usefulness in this thinking that that flexible outcomes or behaviours can be seen to be derived from whichever model/representation is triggered by an external stimulus. This provides a useful/interesting model for behaviour, particularly in simulating an artificial agent.

For example, an agent might possess different models of causality, such as neural networks, pattern recognition, and predefined rules, and each provides a different form of behavioural outcome based on the nature and limitations of the model.

The quality of the behavioural response is also then subject to a shared agreement that forms a consensus about its appropriateness given the situation.

The diversity of responses possible is based on which model or representation was triggered/selected, and this provides for flexible or adaptive outcomes and might therefore simulate more closely human behaviour when these stimuli are coupled with subjectivity to predicate the selection of a model of causality to base its behaviour/response on.

Perception and Understanding

To understand or make sense of anything is to be prompted to do so. That is, a stimulation causes us to attend to that stimulation and in attending to it, we might choose to make sense of its nature, or understand why it occured or what it will mean for us etc... This process of attending to stimulation is what is referred to as understanding/perception, and thus, I consider all understanding to start with attention.

My view is that Understanding (is perception), and is the process of finding or matching appropriate existing models of cause -> effect that already exist in the mind (knowledge). Which models match or are found relevant are based on what is being observed/perceived/sensed, etc. The process of finding might in itself update and contribute to the existing models of cause -> effect, in which case, I think this would be a reasonable definition of learning (defining cause -> effect) while observing, for example. Understanding is, therefore, a process.

As an artificial example, a neural network is by definition knowledge by virtue of the fact that it maps cause to effect. If by observing additional evidence for cause and effect externally (maybe through sight), for example, one might supply that that model/representation with information that helps make that neural network (knowledge/mapping) better, for example, you observe more cause -> effect that can be added to the neural network as training data; the neural network, in effect, was located as model of knoweldge and it was updated/improved (learning) based on your observation.

Understanding (perception), generally, in psychology, is said to represent a top-down and a bottom-up process. The bottom is the sensory reality or the stimuli itself that triggered the attention to deal with it. The top is the knowledge of cause-and-effect that is consulted in processing the stimuli for the benefit of gaining and better understanding of the nature of that stimulus and what it will mean once understanding/perception of it has been completed. Interestingly enough, as knowledge is, in my opinion, higher conceptual internal representations of cause-and-effect.

I theorise that expectation, in general, is the prediction of consulting and using our internal cause-and-effect models (our knowledge) to predict/infer the outcome based on what the model suggests based on what it knows about cause-and-effect. Prediction is a part of understanding. A top-down process might be expectations about what will happen or result based on a stimulus. If the expected outcome is not met, this suggests the model needs to improve. A bottom process is creating knowledge (or consulting existing knowledge to understand) a sensory stimulus. 

In humans, the brain and the mind and the internal representations of knowledge of cause and effect, are the realm of the cognitive 'top' realm. This draws on and relies on existing knowledge to make sense of and understand what is sensed. You might suggest that a belief is a result of understanding.

Cause and effect

In my mind, knowledge is the mapping of cause to effect. Understanding or perception is finding the model of causality. 

A cause is thus in many ways simply an observation. What comes with that observation is data. What do you do with that data to map it to its effect is the process of understanding.

In psychology, the Gestalt theory of perception, in my mind, is simply representing observational data, which may be complex in nature (a scene), into simpler constituent cause-and-effect models.

For example, the idea of grouping similar colours, attributes, shapes, and patterns of some observational/sensory stimuli (data), is simply a very rudimentary means to create a cause-and-effect models that can suggest that those shapes, colours might be in themselves be a means to make a knowledge mapping (i.e it's a bottom-up process) such that colours (cause) might mean and outcome, such as danger (effect). In this way, cause-and-effect is fundamental to how we represent our understanding.   

It might be that elementary patterns, groups, similarities, or familiarieties are inherent cause-and-effect models that we build up on in a layered fashion to build up more and more complex, hierarchical cause-and-effect models from observational/sensory data/stimuli. The notion of being round or similar or consistent (Gestalt principles), are simpler cause-and-effect model of observed data that can be used to build higher meaning, cause and effect models, such as the red circles or patterns in the scene of observational data is what means 'dangerous'.