Cognitive behaviourism researcher E.C. Tolman showed that behavioural conditioning alone, i.e learning purely through stimulus-response, is not enough to inform learning, and that cognitive mechanisms play a role.
Through experiments, it was shown that when a learning agent, such as a rat, was put into a maze, it solved the maze faster next time if it knew there was a reward at the end of it, specifically that, “…rats do learn to expect goals in specific locations” (Tolman, Ritchie and Kalish, 1992).
Physical mechanisms that support the formation of abstractions
This indicates that there exists a prior or remembered knowledge or expectation (see above) that is likely embedded in a cognitive structure that aided subsequent learning. This is learning beyond merely reacting only to short-term stimulus (S-R links).
Having a means to draw on past observational experience and knowledge is likely to be important, as is the association of circumstantial information (discussed previously), and perhaps more importantly, is the timely and accurate retrieval of knowledge given any particular situation (in real-time), i.e, situation-detection.
Memory, for example, appears to be a cognitive construct in which, it is theorised, stored information moves from short-term to long-term memory through rehearsal, and it is suggested that there are several types of memory, such as Episodic, Semantic, Autobiographical, Flash-back memory, etc. Memory is likely constantly changing as new experiences are experienced. Through simulation, it might be necessary to discern when this change occurs.
The prior research by Tolman et. al (Tolman, Ritchie and Kalish, 1992) shows that locational context, i.e where you were at the time of the event, can impact memory, suggesting that location is a property or relationship of/with circumstance (CSI), and this could be stored and utilised in synthesising responses.
To this end, an interesting idea is the notion of cognitively placing an agent back in a prior state temporarily, such that the experience retrieval mechanisms can be optimised to help remember contextual information, i.e CSIs about a similar past situation. This would simulate thinking back on past experiences, for example.
The subsequent formation of knowledge from sensory experience, may be enhanced through the integration of existing 3rd-party cognitive architectures (Kotseruba and Tsotsos, 2018) such as SOAR (Laird, 2022), Pogamut (Gemrot et al., 2009) and others, which provide cognitive models such as episodic and semantic memory, and which could utilize perceptional input that is sensed and captured by the virtual agents. This could aid in the selection and learning of situations and behaviour, and contribute to the rationalisation of the experienced unknown. This means that there might be less need in this research to create cognitive architectures as there is in defining the abstract data that would be processed by them.
It's not clear yet if these existing cognitive models would allow for the importation of a custom format of experience that would be collected by the agents, and which is based on S-R links and CSIs, such that this research could utilise this existing capability. A benefit of this would be that they are likely to provide more robust cognitive mechanisms than any that were re-engineered in this research.
Tangentially, one might simulate cognitive reflection as being a process of producing new S-R links derived from existing S-R links and evaluating them as a whole, and evaluating their relationships, i.e context. In this respect, reflection appears to produce abstractions from other abstractions.
References
Tolman, E. C., Ritchie, B. F. and Kalish, D. (1992) ‘Studies in Spatial Learning. I. Orientation and the Short-Cut’, Journal of experimental psychology. General, 121(4), pp. 429–434. doi: 10.1037/0096-3445.121.4.429.
Laird, J. E. (2022) ‘Introduction to Soar’. arXiv. doi: 10.48550/arXiv.2205.03854.
Kotseruba, I. and Tsotsos, J. K. (2018) ‘40 years of cognitive architectures: Core cognitive abilities and practical applications’, The Artificial intelligence review, 53(1), pp. 17–94. doi: 10.1007/s10462-018-9646-y.
Gemrot, J. et al. (2009) ‘Pogamut 3 Can Assist Developers in Building AI (Not Only) for Their Videogame Agents’, in Dignum, F. et al. (eds) Agents for Games and Simulations: Trends in Techniques, Concepts and Design. Berlin, Heidelberg: Springer (Lecture Notes in Computer Science), pp. 1–15. doi: 10.1007/978-3-642-11198-3_1.