Fear and the intolerance of the unknown appear to significantly influence human inclination to explore (Carleton, 2016). Similarly, and perhaps inherently related, is motivation and emotion, which appear to be cognitive constructs that appear to be inherent in human experiences and which influence their behaviour.

This suggests that evaluating stimuli and determining how they affect the agent's sensitivity to fear is likely an important aspect in processing observations. In this way, fear also appears to be an abstraction derived from experience, likely related to value/need/priority.

Need

Maslow’s hierarchy of needs model theorises, “…that human needs are arranged in a hierarchy”, and so suggests that individuals (humans at least) manifest motivation based on their own personal hierarchy of needs (Koltko-Rivera, 2006). This suggests that needs, and a mechanism for establishing and evaluating needs, might represent a foundational basis that predicates experiential value, or the value of arbitrary experiences and how this could influence the agent's exploratory behaviour. In this respect, need, like fear, is itself apparently an abstraction based on evaluative observation.

The theory suggests that the more important the need is, the closer it is to the bottom of the hierarchy and that these represent the basic needs that must be satisfied first before moving up the hierarchy. Specifically, “…that humans are driven by innate needs for survival, safety, love and belonging, esteem, and self-realisation, in that order” (Abulof, 2017).

This suggests that the physical needs, i.e those that are more likely to be evaluated and perceived through physical perception (survival/pain/hunger), are more important and have a higher priority, and therefore subsume other needs that appear higher up, which would be lower priority.

While this is based on humans, establishing and positioning where in a hierarchy of needs an individual or agent is, might be useful in establishing and evaluating the value of an arbitrary experience (referred previously as experiential value), and thus may help inform their behaviour accordingly.

An interesting and perhaps tangential diversion is the possible emergence of needs and synthesis of goals purely based on the value of witnessed experiences. That is, establishing a new goal dynamically without prior knowledge of that goal, perhaps based purely on the evaluation of established underlying needs. More research would be required if this were to be pursued.

Emotion

David Hume argued that emotional (an abstraction) and instinctive physical drives are what govern human behaviour, and that there are inherent forms of understanding already present in human nature which influence decisions and responses, and which, “…are not learned, nor are they the result of deliberation or will” (Hill, 2020).

The James-Lange theory of emotion suggests that emotions are in response to the physical perceptions which are then codified as emotions, and that, “…rather than causing bodily and visceral responses, an emotion is itself a perception of these specific reactions” (Simon Blackburn, 2016). In this way, emotions appear to be a precursor to action (behaviour).

Furthermore, emotions appears to be intrinsically linked to environmental stimuli and rationalizing the unknown, for example, Schacter and Singer’s theory of emotion suggests, “…that individuals who are in a state of physiological arousal for which they have no explanation (no suitable representative abstraction) will label that state as an emotion that is appropriate to the situation in which they find themselves. (‘Schachter-Singer’s theory of emotion’, 2006).

This is an interesting idea considering the evaluation of new and unknown stimuli and may suggest that emotion can be used to inform behaviour in experiences that are specifically new and unexpected.

Generally, “…mismatched expectations are often associated with an emotional response” (Van De Poll and Swinderen, 2021), which seems to indicate a relationship between emotion and expectation, or, as Singer suggests, perhaps the failure to find an expectation.

It is possible that the reason why mismatched expectations produce an emotional response, and perhaps vice versa, is that, like expectation, emotions represent an associated validation or failure in codification of a prior, costly knowledge-formation process (formation of abstractions). Additionally, high-value needs/priorities may be more closely aligned with a specific associated emotion than lower priorities or needs, and so it may be possible for a model to simulate the association between such high-priority S-R links with specific emotion. This would require more research.

In summary, in addition to collecting experiences through observations based on stimuli and responses theory, the observational data can be used to inform cognitive processes (formation of abstractions) in the agent, as well as being used for latent analysis.

References

Carleton, R. N. (2016) ‘Into the unknown: A review and synthesis of contemporary models involving uncertainty’, Journal of anxiety disorders, 39(Journal Article), pp. 30–43. doi: 10.1016/j.janxdis.2016.02.007.

Koltko-Rivera, M. E. (2006) ‘Rediscovering the Later Version of Maslow’s Hierarchy of Needs: Self-Transcendence and Opportunities for Theory, Research, and Unification’, Review of general psychology, 10(4), pp. 302–317. doi: 10.1037/1089-2680.10.4.302.

Abulof, U. (2017) ‘Introduction: Why We Need Maslow in the Twenty-First Century’, Society (New Brunswick), 54(6), pp. 508–509. doi: 10.1007/s12115-017-0198-6.

Hill, J. (2020) ‘The Role of Instinct in David Hume’s Conception of Human Reason’, Journal of Scottish philosophy, 18(3), pp. 273–288. doi: 10.3366/jsp.2020.0277.

Simon Blackburn (2016) ‘James-Lange theory of emotion’. Oxford University Press.

‘Schachter-singer’s theory of emotion’ (2006). 

Van De Poll, M. N. and Swinderen, B. van (2021) ‘Balancing Prediction and Surprise: A Role for Active Sleep at the Dawn of Consciousness?’, Frontiers in systems neuroscience, 15, pp. 768762–768762. doi: 10.3389/fnsys.2021.768762.

In addition to the research’s primary objectives, which are derived from autonomous observational exploration, this model could also allow for more directed, intervening and controlled exploration, one which is encouraged through reinforcement learning.

Based on Pavlov’s work, B.F Skinner showed that S-R links can be re-enforced by pleasant or unpleasant environmental consequences, and that, “…behavior is principally controlled by schedules of reinforcement”, where a positive re-enforcement was the delivery of a physical reward, while negative re-enforcement was not getting a reward or the avoidance of unpleasant consequences/outcomes/reactions (Morgan, 2010).

This showed that very complex forms of learning could be built up by reinforcing to produce small changes in behaviour, and that, “…encoding and automatization of such associations are crucial for the achievement of more complex interactions within our environment” (Allenmark, Moutsopoulou and Waszak, 2015). This concept is similar to how machine learning works through evolving and testing variations.

It might then be possible, through controllable simulation of stimuli within the virtual environment, to influence the collected S-R links. This might encourage certain types of associations to form, which might help control the formation of knowledge in specific simulations, perhaps to understand how this might affect the learning and teaching of an artificial entity.

D.O. Hebb suggested that learning occurs through “repeated stimulation” (Ghassemzadeh, Posner and Rothbart, 2013), where the connection to this knowledge theoretically grows larger and is then more accessible (and is then more evident in resultant behaviour).

Whether through self-exploration or controlled exploration, a repeated stimulus and its response could be rewarded either directly by an experimenter, or indirectly and dynamically in self-exploration (e.g perhaps based on situational frequency).

Through simulation, one could emulate such rewarded responses by possibly increasing the count or value of that S-R link, i.e assign a higher value to the association. This could be used to help build a picture to determine if a produced reaction is right or more or less appropriate, particularly where behaviour or knowledge for the stimulus is not yet established.

In this way, it might be possible to establish or determine that a response contributes towards a goal, and then collecting S-R links that contribute towards that goal could form the basis for learning how to behave or what to do to achieve a goal. This is not itself novel, as multi-agent simulations often use this approach in adapting to maximise their progress towards reaching a goal. This, however, could establish a basis for simulating goal-oriented learning (or test such models in a virtual environment), which might help to gather observations relevant to understanding the nature of behaviour in the face of changing circumstances that affect those needs or goals. Indeed, detecting change or assessing the severity or influence of changing circumstances could be novel could generalise and detect it properly.

An agent might utilise both directed (reinforcement) and self-exploration as a means of evaluating new stimuli for the purpose of evaluating resultant responses against its needs or goals. While not novel, it is useful.

An autonomous agent might simulate self-exploration by trying to perform different actions (e.g., to self-experiment to understand new causality, events and relationships that form as a result), which are likely informed by previous observations and learning, while a directed approach (reinforcement) would likely influence produced responses (or aim to witness them) by explicitly and prescriptively rewarding behaviour.

In autonomous exploration, it might be possible (and novel) to derive goals by grouping responses that satisfy a need/goal, and in this way, it may be possible that agents could tell what it takes to achieve goals, similarly to how back-propagation aims to reduce the error function, by rewarding S-R links that tend towards realising a goal.

Autonomous exploration would rely on gradually realising favourable S-R links through multiple experiences, while directed exploration and explicit rewards would identify favourable S-R links more expediently; however, this would make the training biased on a specific S-R link that the experimenter felt was relevant (introducing bias), instead of it being derived from multiple variances experienced over time by an autonomous agent.

The implication of this goal-oriented or self-rewarding behaviour is possibly the ability to document which behaviours contribute towards achieving a goal or tendency. This could be very useful for understanding why patterns form, and not merely that they do, and specifically which determined behaviours or observations are specifically relevant.

References

Morgan, D. L. (2010) ‘Schedules of Reinforcement at 50: A Retrospective Appreciation’, The Psychological record, 60(1), pp. 151–172. doi: 10.1007/BF03395699.

Allenmark, F., Moutsopoulou, K. and Waszak, F. (2015) ‘A new look on S–R associations: How S and R link’, Acta Psychologica, 160, pp. 161–169. doi: 10.1016/j.actpsy.2015.07.016.

Ghassemzadeh, H., Posner, M. I. and Rothbart, M. K. (2013) ‘Contributions of Hebb and Vygotsky to an Integrated Science of Mind’, Journal of the history of the neurosciences, 22(3), pp. 292–306. doi: 10.1080/0964704X.2012.761071.

Forming Expectations

As the previous research by Pavlov showed, meeting any expectation, from a learning perspective, appears to be an important mechanism which can reinforce prior learning of cause and effect (stimulus and response). Through exploration, we proposed to utilise this mechanism to reinforce or redefine prior situational learning. In this way, expectations are a high-level abstraction about causality (events) and their results/outcomes, while causality itself is an abstraction of experiences.

For example, by using the Stimulus-response model, after a certain threshold of repeated S-R link occurrences (behavior in response to a stimulus), the produced response could become automatic, such that the detection of the stimulus becomes to mean that a fixed, expected response, which in humans, “…allows for faster, more efficient behavior when the same response is required to the same stimulus in the future”, likely cutting down on the processing required to determine or derive this behavior from its knowledge base (Allenmark, Moutsopoulou and Waszak, 2015).

One could simulate this by deleting re-occurring duplicate S-R link sequences (or rather stop measuring them) once some occurrence threshold is met, and then replace any cognitive processing associated with the S-R link sequences with producing a now fixed and expected learnt response.

This may be a simulation of a trivial mechanism, but it might inform the evolution, codification and emergence of more complex behaviour, including detecting important milestones (or importance in general) or consistency in situations. In our research, expectation could be created from observed causality that is established by stimulus-response sequences in the simulation.

Equally, a mismatched expectation could be used signal that the current knowledge is incomplete and requires more experience to properly inform it.

Incidentally, this may have other interesting outcomes such as organically simulating fixed and growth mind-sets in artificial learning entities, as the willingness to resist behavioral certainty or expectations (fixed mindset) for the potential benefit of gaining new, and potentially interesting outcomes, such as new and diverse learning opportunities which naturally comes with arresting ones expectations (growth mindsets), namely through exploring. The alternative (fixed mindsets) forgoes this cognitive processing burden of establishing reason and produces behaviour/responses based on established fixed expectations, perhaps limiting it from new learning in the process, and which is more closely modelled by traditional rule-based AI, based, for example, on finite state machines and static behavioural trees, etc.

Expectations influence behaviour

As an example of the importance expectations play in human behavioural learning in general, and as a consequence, the ability to form them from observations, is the phenomenon where one sees what one expects to see, without evaluating other available data. For example, while driving, we often notice pedestrians but might not even register a cyclist, or even notice a familiar person at an unfamiliar and unexpected location.

A popular example of this is a video demonstration (Dothetest, 2008) showing a Basketball game being played, and after having watched the short clip, the viewer is asked if they had seen the dancing bear. In many cases, they did not and are entertainingly surprised after re-watching the video (now expecting the dancing bear) that the dancing bear is not only readily viewable, but that it should have been obvious that it was there, as it occurs directly in the centre of view during the game.

Another example of the subtlety of expectation in influencing human behavior, is called priming, where showing first a series of numbers (priming our expectations) then showing an ambiguous figure (which is similar but is not a number), results in our immediate intuition to think, or expect that the unknown figure is a number, when in fact it might equally be shaped as many other things. This can therefore influence the behaviour that occurs after priming.

I speculate that expectations may serve as a short-cut to limit the overhead of cognitive processing, and as a means to calibrate the ineffectiveness of learning (or classification of experience), forming a kind of error detection function. It may thus be possible or useful to model and simulate priming behaviour by modelling the formation of expectations in the face of experiencing unknown and/or ambiguous situations to better understand how this might affect learning.

The identification and detection of underlying and otherwise unknown expectations from within experiences might be the precursor to the establishment of rules, which seem inextricably linked.

One model that appears to use fixed expectations to establish a fixed association between a stimulus and response without needing the stimuli to recur is called ‘One-Trial Learning’ (Feng and Sun, 2019). This might be crucial in evaluating any new and unknown. One-Trial learning occurs when a fixed response is generated in response to a single experience, usually resulting in a bad or undesirable situation, like when eating food which causes one to convulse and throw up, or when we experience pain. The response to these situations is a decisive and fixed one.

Goal-orientated pursuit

The implication of this is that this seems to suggest that there is a specific evaluation of the value of an encountered experience, i.e it possesses what our research classifies as an ‘experiential value’. Moreover, this suggests that behaviour is likely based on the degree of alignment to a set of priorities, where high-ranking priorities are more likely to trigger One-Trial learning. For example, reactions to experiences that compromise physical safety are high-priority goals, which strongly influence specific behavioural reactions.

The establishment of an experiential value measurement might thus help inform how to learn to react in response to new, and previously unknown stimuli, but which is recognised to be affecting a high-value goal, such as survival. A discussion of how one might value arbitrary experiences is discussed later.

One-Trail learning might be simulated by either hard-coding triggered behaviour to known, high-priority perceptions that are experienced, eg, getting hurt, which is more akin to how pre-determined AI works, or more interestingly, the model might dynamically assign a higher value to sensory data associated with that high-priority experience (which might be predefined or possibly even self-emerge). In this case, such one-time learning behaviour is triggered so that no subsequent cognitive processing is required to define the expected behaviour in response to it, as it's already been rapidly determined.

Either way, dealing with expectations while interpreting experiences is likely an important part of rationalising about reality.

Using expectations to make Sense of the unknown

Another aspect of this research is dealing with the unknown object that is experienced, and how it is rationalised or made sense of. What is the process of making sense?

Interestingly, expectation again appears to be a crucial component of this process. For example, research has demonstrated that ‘amodal’ completion helps make sense of something that is not perceptually present (unknown) because cognitively we allow what is missing or unknown to fit or be extrapolated into matching (or completing) an already known expectation (Ekroll, Sayim and Wagemans, 2017).

Figure 13) illustrates amodal completion, where the shape A represents a known expectation, and shape B is then amodally completed to align and define it according to the existing expectation, and in doing so, to make sense of unknown B.

In this way, it seems that knowledge may well be the collection of expectations, possibly informed by having witnessed expectations from observations of cause-and-effect, much like stimuli causing responses (S-R links) is represented.

Forming and identifying expectations

If knowledge (reasoning) can be generalized (at least) as a collection of expectations which are derived from the experience of observed causality, then the observation of an unknown thing (or situation) either needs to correspond or extrapolate to an existing expectation (from existing knowledge), or it needs to form a new expectation - which appears to be the creation of new knowledge. This model may underlie a means to simulate, trigger and identify the need for the formation of new knowledge based on perception and expectation.

It is in this way, that a simulation environment (such as proposed in our research) which allows for the establishment of causality through perceptual observation might allow for the construction of knowledge as expectations derived from sensory data, and in doing so, that the unknown might be reasoned about as the degree in which its observation fits the properties of known expectations (amodal completion).

These properties might include the causal relationship, such as S-R links and the associated CSIs. Matching S-R links and CSIs might indicate a means to make sense of the unknown or make sense of situations where properties of known expectations match. The degree of match might indicate the confidence in the understanding. Confidence in this respect appears to be the rate of identification of known expectations.

Understanding itself might then be defined as the ability to make ‘sense’, which in turn is the ability and degree to match known expectations (or knowledge), which in turn is derived from observations.

References

dothetest (2008) ‘Test Your Awareness: Do The Test’. Available at: https://www.youtube.com/watch?v=Ahg6qcgoay4 (Accessed: 13 August 2022).

Feng, N. and Sun, B. (2019) ‘On simulating one-trial learning using morphological neural networks’, Cognitive systems research, 53, pp. 61–70. doi: 10.1016/j.cogsys.2018.05.003.

Ekroll, V., Sayim, B. and Wagemans, J. (2017) ‘The Other Side of Magic: The Psychology of Perceiving Hidden Things’, Perspectives on psychological science, 12(1), pp. 91–106. doi: 10.1177/1745691616654676.