The experience sampling method (ESM) is a means of describing experiences. This method is a field-driven approach that aims to allows participants to more easily fill in questionnaires or surveys (relevant to their experiences) while they are actually situated within the context or field of the experiences that the questions are targeted towards, and this usually, “…involves sampling participants’ experiences in natural environments, in real time (or close to it), and on multiple measurement occasions” (Meers et al., 2020).
ESM usually defines experience as the collection of contextual experience information through surveys and questionnaires; however, our research aims to base experiences on observational data (S-R links and relevant CSIs) collected by agents. However, like ESM, the underlying goal is the same, which is to obtain descriptive information about raw sensory experiences from situational contexts (the environment), as proposed by Lewin’s Field theory, but to be more generic and applicable to all gathered sensory data.
While there needs to be research into determining how best to compose this sensory data into contextualised experience, the initial approach is that it would be a composition of S-R links, other environmental context and CSIs, that is translated into a usable and cohesive representation of experience.
To implement such an ESM, a possible approach is to process S-R links and CSIs through a processing pipeline which uses methods to attempt to filter, validate, aggregate, and contextualise the raw observational (sensory) data into experience. Figure 9 shows how the identification of observations precedes the increasingly higher level of abstraction that results from processing observations.
Figure 9 illustrates how the reasoning process might be defined by producing abstractions.
Our research proposes to analyse agent-based observations in a similar manner that clinical psychology uses to describe and rationalise experiences, namely through:
- Eidetic Reduction
- Eidetic Variation
- Phenomenological Reduction
For example, Eidetic Reduction is a consultation and deliberation process, usually performed by humans to determine the underlying essence of something, such as experiences or observations. This is done to try to remove any unneeded context that may occur in one or multiple variances of that experience, but keep the relevant context. What remains after the irrelevant variances have been removed is said to be the essence, which, once obtained, would allow one to define the unique identification traits of the experience. The essence of anything is typically an abstraction formed by simplifying and choosing to disregard information. This might be useful in trying to identify and discern different situations from similar observational experiences.
Defining situations
One example of a further abstraction from experience (which is itself an abstraction based on observation) is the identification and formation of situational boundaries (situations). Situations can provide context such that different situations can be responded to differently, such that learning and a specific rule creation might be specifically targeted to a unique situation, and not apply in others. In forming situations, time and duration of observations are likely to be a factor when determining when one situation ends and another begins, and this research would explore this.
Generally, Eidetic Reduction includes various approaches and considerations, including:
- Profiling, which is the need to recognise the same thing that is experienced or manifested differently each time
- Detecting essence, which is the detection of essence by looking for the underlying essence, devoid of unneeded and generally irrelevant information or context
- Seeking Universals, which is the process of utilising a collection of several instances of the same situation to identify to remove/reduce things that are not general or always inherent to identify so-called Universals in situations. This could be achieved through variations of situations in the simulation environment.
- Identifying Objects of awareness/consciousness, which is the process of attempting to find the objects that have been included in the subject’s experience (or in our case, observations), and can extend to looking for meaning in those objects.
The last technique might be a useful way to define observations as about various objects that make up the context of the agent’s situation, particularly in light of Husserl's suggestion that, “..any experience we have includes three elements: a subject, a predicate and an object” (Deurzen, 2015). This supports the perhaps obvious idea that before reasoning can occur, identification of conceptual objects must occur.
A conceptual illustration that aims to implement a sensory-based system is illustrated in Figure 10, which outlines a possible approach.
The opposite of Eidetic reduction, is Eidetic Variation which specifically varies instances of the same thing, to establish universals (things that do not change), and this is usually applied, like Eidetic reduction, to non-material objects such as as ideas, acts, conceptual simulations such as a theoretical situation - or as this research proposes, variances provided by similar experiences observed in a simulated virtual reality. This could be used to learn about a situation by varying it until the fundamental characteristics or rules emerge, much like what happens in back-propagation (machine learning).
Phenomenological Reductions, which form part of the discipline of Phenomenology, are also a study of conscious phenomena, and are a “… methodical study of the process of human awareness and the experiences” (Deurzen, 2015). Here, a conscious phenomenon might be an arbitrary description of a human experience or, as proposed, one that is observed by an agent, for example.
Likewise, Phenomenological reductions follow similar but different prescriptive methods to Eidetic Reduction:
- Epoche, which is removing biases, assumptions and any prejudices when interpreting an experience (noise). This is similar to seeking essence.
- Horizontalization which is identifying the limits or unknowns that may need to be explored. This could help determine what to explore or look for next, in terms of acquiring knowledge. This effectively identifies gaps
These methods and approaches could be combined with Eidetic reduction to refine and clarify, define and refine experiential agent observations. Naturally, an agent is not human; however, there are aspects of this approach that might apply to an artificial intelligence entity describing and clarifying its observations to identify what to do about them.
Both Eidetic and Phenomenological reductions are often used in therapeutic consultations between a Therapist who is listening (applying reductions) to observations that the patient (subject) is describing about their experience from memory (recollection). Figure 11 illustrates Phenomenological observation used in such a setting, where the goal is for the patient (Client) to gain an underlying understanding of their experiences, through dialogue with the Therapist to help obtain the essence of the meanings behind the client’s experiences.
One could conceivably, in this respect, replace the Client with the agent (to make observations about its experiences), and the Therapist with the thinking or reasoning cognitive construct in the agent that is trying to make sense of its observed experiences (forming abstractions). This would be done by reducing irrelevant situational complexity to find an experience’s unique, identifiable core. An interesting aspect of this approach is the continual looping feedback mechanism (like a game loop) where the experiential observations made by the client and assessments of these observations, made primarily by the therapist, are validated with the client, effectively providing training data for defining experiences.
Using a framework centred around Eidetic and Phenomenological reductions to analyse awareness and experiences that an agent observes might provide the basis for a structured and methodical means to process the situational complexities experienced in the agent's reality.
In practice, Phenomenology, like Eidetic reduction, operates on cognitive, conceptual constructs such as ideas and experiences, and one might suggest that anything that is to be reasoned about (or processed), whether its derived from a physical perceptions or not, can be thought of as a conceptual simulation at some point merely by thinking about it, i.e cognitively simulating it. In this way, collecting virtual sensory observations which are then reduced to software ideas/models and which ultimately need to be processed in order to establish reason and quantify any experience, might also be seen as being equivalent to conceptual simulation.
In this way, agent observations could ultimately manifest as cognitive simulations modelled as software models, which then arguably apply to being modelled using Phenomenology as a structural and rigorous formal basis on which to quantify experiences. This would enable a systematic approach to agent observation in order to make sense of experienced situations and the associated situational contexts that occur within the virtual environment.
References
Meers, K. et al. (2020) ‘mobileQ: A free user-friendly application for collecting experience sampling data’, Behavior Research Methods, 52(4), pp. 1510–1515. doi: 10.3758/s13428-019-01330-1.
Deurzen, E. van (2015) ‘Structural Existential Analysis (SEA): A Phenomenological Method for Therapeutic Work’, Journal of contemporary psychotherapy, 45(1), pp. 59–68. doi: 10.1007/s10879-014-9282-z.