Cognitive behaviourism researcher E.C. Tolman showed that behavioural conditioning alone, i.e learning purely through stimulus-response, is not enough to inform learning, and that cognitive mechanisms play a role.

Through experiments, it was shown that when a learning agent, such as a rat, was put into a maze, it solved the maze faster next time if it knew there was a reward at the end of it, specifically that, “…rats do learn to expect goals in specific locations” (Tolman, Ritchie and Kalish, 1992).

Physical mechanisms that support the formation of abstractions

This indicates that there exists a prior or remembered knowledge or expectation (see above) that is likely embedded in a cognitive structure that aided subsequent learning. This is learning beyond merely reacting only to short-term stimulus (S-R links).

Having a means to draw on past observational experience and knowledge is likely to be important, as is the association of circumstantial information (discussed previously), and perhaps more importantly, is the timely and accurate retrieval of knowledge given any particular situation (in real-time), i.e, situation-detection.

Memory, for example, appears to be a cognitive construct in which, it is theorised, stored information moves from short-term to long-term memory through rehearsal, and it is suggested that there are several types of memory, such as Episodic, Semantic, Autobiographical, Flash-back memory, etc. Memory is likely constantly changing as new experiences are experienced. Through simulation, it might be necessary to discern when this change occurs.

The prior research by Tolman et. al (Tolman, Ritchie and Kalish, 1992) shows that locational context, i.e where you were at the time of the event, can impact memory, suggesting that location is a property or relationship of/with circumstance (CSI), and this could be stored and utilised in synthesising responses.

To this end, an interesting idea is the notion of cognitively placing an agent back in a prior state temporarily, such that the experience retrieval mechanisms can be optimised to help remember contextual information, i.e CSIs about a similar past situation. This would simulate thinking back on past experiences, for example.

The subsequent formation of knowledge from sensory experience, may be enhanced through the integration of existing 3rd-party cognitive architectures (Kotseruba and Tsotsos, 2018) such as SOAR (Laird, 2022), Pogamut (Gemrot et al., 2009) and others, which provide cognitive models such as episodic and semantic memory, and which could utilize perceptional input that is sensed and captured by the virtual agents. This could aid in the selection and learning of situations and behaviour, and contribute to the rationalisation of the experienced unknown. This means that there might be less need in this research to create cognitive architectures as there is in defining the abstract data that would be processed by them.

It's not clear yet if these existing cognitive models would allow for the importation of a custom format of experience that would be collected by the agents, and which is based on S-R links and CSIs, such that this research could utilise this existing capability. A benefit of this would be that they are likely to provide more robust cognitive mechanisms than any that were re-engineered in this research.

Tangentially, one might simulate cognitive reflection as being a process of producing new S-R links derived from existing S-R links and evaluating them as a whole, and evaluating their relationships, i.e context. In this respect, reflection appears to produce abstractions from other abstractions.

References

Tolman, E. C., Ritchie, B. F. and Kalish, D. (1992) ‘Studies in Spatial Learning. I. Orientation and the Short-Cut’, Journal of experimental psychology. General, 121(4), pp. 429–434. doi: 10.1037/0096-3445.121.4.429.

Laird, J. E. (2022) ‘Introduction to Soar’. arXiv. doi: 10.48550/arXiv.2205.03854.

Kotseruba, I. and Tsotsos, J. K. (2018) ‘40 years of cognitive architectures: Core cognitive abilities and practical applications’, The Artificial intelligence review, 53(1), pp. 17–94. doi: 10.1007/s10462-018-9646-y.

Gemrot, J. et al. (2009) ‘Pogamut 3 Can Assist Developers in Building AI (Not Only) for Their Videogame Agents’, in Dignum, F. et al. (eds) Agents for Games and Simulations: Trends in Techniques, Concepts and Design. Berlin, Heidelberg: Springer (Lecture Notes in Computer Science), pp. 1–15. doi: 10.1007/978-3-642-11198-3_1.

An approach to situation detection is to represent the world objects in a hierarchical scene graph, and model sensory events as influences on those objects. This can allow for the identification of effects of causality and the influence it has on multiple related objects, such as the propagation of effects to their dependencies, e.g pushing a box with a pen in it, will also move the pen relative to the box. In this way, when a stimulus affects an object, the resulting change/response can be observed in its dependencies.

This should make it possible to identify which objects should be actively observed without checking all of them in the scene, eg, pushing the box means only the box and its dependent might need to be checked, saving computation time. While this is useful, there is likely still a need to be able to determine when objects that are still important, i.e make up the situation, but have not actually changed in response to the stimuli (universals).

The basic premise is that the scene graph is checked for differences caused by stimuli, and these changes can be used to detect situational boundaries. An illustration of this approach is outlined in Figure 12.

Another possibility is that this hierarchical representation might serve as a means to aggregate, group and generalise the effects that multiple changes in the scene have. This could help analyse consistency or help derive boundaries by forming level of detail calculations, where a lower level of detail might ignore smaller, frequent or consistent changes and focus on bigger changes in order to generalise a situation boundary at that level. This is essentially the formation of abstractions from details.

It would also be important to determine the level of granularity or resolution of the time-based observational window, as this is likely to affect how situational boundaries are captured, for example.

A stimulus may be defined as a class of relationships which exists between two parties (a sender and a receiver), specifically where the sender initiates the relationship and the receiver is primarily subject to it. Circumstance may then be defined as a particular response to a stimulus, and a situation as a set of coordinated circumstances that can characterise behaviour.

This leads to defining observation as the witnessing or collection of circumstances and situations.

In this model, a circumstance is analogous to a stimulus-response pair or S-R link, and a situation may be defined by multiple, sequenced or simultaneous circumstances.

With this model, it is easy to define a known stimulus, circumstance and situation by identifying the sender, receiver, response, and the pattern that they occur in. However, is it possible to define yet-to-be-known circumstances and situations purely from observations and then simulate this as behaviour in artificial entities by producing both known and learnt (previously unknown) circumstances and situations?

Circumstance: Contextual Situational Information

Circumstances can be very rich, full of interrelated properties and co-existing facts. Kurt Lewin suggested through his Field theory that situational context, i.e the environmental or ‘space’ influence, contributes significantly to understanding behaviour that occurs within it (Burnes and Cooke, 2013).

This suggests that experiential learning should be based on more than a collection of S-R associations and that other contextual circumstantial information about the environment and situation is important. Arguably richer S-R links might be gathered in context, which can be provided by environmental simulations.

It is in this way that a simulation environment that elicits notions of perceptual reality to an inhabiting agent may assist in providing the surrounding context in which S-R links are experienced. This situational context can then be used to help define, identify collections of discernible situational circumstances in order to better understand experiences within it and identify situational boundaries. This may require the formation of a model such as that presented in the previous section for defining circumstances (and also storing and querying it).

Modelling and defining circumstances might be achieved by collecting what this research defines as circumstantial situational information or CSIs, i.e contextual information from the environment or constituents of situations. This, for example, might be stored and then be used as a lookup query along with any sensory-based knowledge when forming new or identifying existing situational knowledge.

A primary consideration would need to be spent determining how much and which information is collected, and what would constitute or affect what one defines as being relevant (and unique) when discerning boundaries, particularly in real-time.

Indeed, failures derived from the inability to reliably identify and establish relevant context are a principal concern for safety, reliance and trust in artificial intelligence (Denning and Arquilla, 2022).

Despite the many theories of knowledge, situational knowledge formation is likely to result from a latent knowledge-formation process (probably systematic, albeit complex) from which an understanding of any situational circumstance can be derived. Such resultant understanding is likely to be the generalised deconstruction of dynamic and complex situational stimuli and circumstances into discernible situational boundaries from which actionable situational rules/knowledge can be created.

A possibility, too, is the ability to create behavioural rules that are based on observational experiences. These rules, being based on observation-based training, could develop rules that are less fragile as they are less dependent on specific contexts having been derived from generations of generalisation informed by many variances experienced during explorations. This might be considered a form of dynamic training based on varying experiences as opposed to static training data.

Generalization

As part of the Little Albert experiment performed by J.B Watson, it was shown that a generalised stimulus-response association (learning) is established such that the perception of similar stimuli (but not the same) produced the same learnt response in Little Albert. In the experiment, the same response was produced when either the white rat (subject of the experiment) or other, “…white furry objects such as dogs, rabbits, fur coats, cotton, and wool…” were perceived, suggesting that generalisation of detected stimuli is a fundamental characteristic, even in infants (Guercio, 2018).

This suggests that a learned response could happen when circumstances are similar but not the same as the original situation, i.e the context is similar. This has implications in any potential software model that looks up circumstantial information to determine its next situational response, as it would need to look beyond the specific and detect similar or general, i.e, fuzzier cases that derive from it.

Generally, the problem of context is an open concern and is particularly relevant in light of failures found in even contemporary AI such as Artificial Neural Networks (ANNs) where research shows that, “…a frequent feature of these failures is the machines were being used in a context different from what they were designed or trained for”, often because they are unable to generalize and are based on fixed signatures (Denning and Arquilla, 2022).

Relevant questions posed by that research are, “…can the machine detect there has been a change in the environment…”, and, “…can a machine read context in real time and adapt its recommendations?”, and also highlighting the ever-present need to be able to, “…tell when a rule is relevant to the situation at hand”. This evidently highlights the importance of change detection within environmental situations. These, it argues, are inherently difficult questions that AI has been trying to solve ever since its inception.

A possible approach to generalising experiences is by storing or processing not only S-R links, but also any associated contextual situational information (CSI) that identifies circumstances in observed situations. The retrieval mechanism for such a model would need to be able to find circumstantial data that is generalizable to that specific circumstantial information, in the same way infants do. Efficiency is likely to be a factor of success.

While contemporary machine learning approaches based on the provision of static training data, and therefore the inherent possibility and oftentimes likelihood of specific training data that has a limited context, i.e bias, our proposed research explores how training data that can by synthesized by exploration of the domain in which it is relevant, by observing variances and similarities in it, might help to gain a richer contextual and generalized understanding to base its learning on. This effectively means dynamically creating training data by experiencing more variations of its domain. This could allow for a better approximation of algorithms or rules that can be used to describe it. This in itself is not novel (exploration), but reasoning about the effects of exploration could be.

References

Burnes, B. and Cooke, B. (2013) ‘Kurt Lewin’s Field Theory: A Review and Re-evaluation’, International journal of management reviews : IJMR, 15(4), pp. 408–425. doi: 10.1111/j.1468-2370.2012.00348.x.

Denning, P. J. and Arquilla, J. (2022) ‘The context problem in artificial intelligence’, Communications of the ACM, 65(12), pp. 18–21. doi: 10.1145/3567605.

Guercio, J. M. (2018) ‘The Importance of a Deeper Knowledge of the History and Theoretical Foundations of Behavior Analysis: 1863-1960’, Behavior analysis (Washington, D.C.), 18(1), pp. 4–15. doi: 10.1037/bar0000123.