1. bookVolume 2 (2010): Issue 2 (December 2010)
    Cognitive Architectures, Model Comparison, and AGI, Editors: Christian Lebiere, Cleotilde Gonzalez and Walter Warwick
Journal Details
License
Format
Journal
eISSN
1946-0163
First Published
23 Nov 2011
Publication timeframe
2 times per year
Languages
English
access type Open Access

Editorial: Cognitive Architectures, Model Comparison and AGI

Journal Details
License
Format
Journal
eISSN
1946-0163
First Published
23 Nov 2011
Publication timeframe
2 times per year
Languages
English
Editorial: Cognitive Architectures, Model Comparison and AGI

Cognitive Science and Artificial Intelligence share compatible goals of understanding and possibly generating broadly intelligent behavior. In order to determine if progress is made, it is essential to be able to evaluate the behavior of complex computational models, especially those built on general cognitive architectures, and compare it to benchmarks of intelligent behavior such as human performance. Significant methodological challenges arise, however, when trying to extend approaches used to compare model and human performance from tightly controlled laboratory tasks to complex tasks involving more open-ended behavior. This paper describes a model comparison challenge built around a dynamic control task, the Dynamic Stocks and Flows. We present and discuss distinct approaches to evaluating performance and comparing models. Lessons drawn from this challenge are discussed in light of the challenge of using cognitive architectures to achieve Artificial General Intelligence.

Keywords

Anderson, J. R. & Lebiere, C. L. (2003). The Newell test for a theory of cognition. Behavioral & Brain Sciences 26, 587-637.10.1017/S0140525X0300013XSearch in Google Scholar

Anderson, J. R., & Lebiere, C. (1998). The atomic components of thought. Hillsdale, NJ: Lawrence Erlbaum Associates.Search in Google Scholar

Curran, T. & Keele, S.W. (1993). Attentional and nonattentional forms of sequence learning. Journal of Experimental Psychology: Learning, Memory and Cognition, 19, 189-202.Search in Google Scholar

Dutt, V., & Gonzalez, C. (2007). Slope of inflow impacts dynamic decision making. In Proceedings of the 25th International Conference of the System Dynamics Society (pp. 79). Boston, MA: System Dynamics Society.Search in Google Scholar

Foyle, D. & Hooey, B. (2008). Human Performance Modeling in Aviation. Mahwah, NJ: Erlbaum.Search in Google Scholar

Gluck, K, & Pew, R. (2005). Modeling Human Behavior with Integrated Cognitive Architectures. Mahwah, NJ: Erlbaum.Search in Google Scholar

Gonzalez, C., & Dutt, V. (2007). Learning to control a dynamic task: A system dynamics cognitive model of the slope effect. In Lewis, Polk, & Laird (Eds.), 8th International Conference on Cognitive Modeling (pp. 61-66). Ann Arbor, MI.Search in Google Scholar

Kaminka, G. A., & Burghart, C. R. (2007). Evaluating Architectures for Intelligence. Technical Report WS-07-04. AAAI Press, Menlo Park, California.Search in Google Scholar

Lebiere, C., Gonzalez, C., Dutt, V. & Warwick, W. (2009). Increasing Generalization Requirements for Cognitive Models: Comparing Models of Open-ended Behavior in Dynamic Decision-Making. In Proceedings of the 9th International Conference on Cognitive Modeling. Manchester, England.Search in Google Scholar

Lebiere, C., Gonzalez, C., & Warwick, W. (2009). A Comparative Approach to Understanding General Intelligence: Predicting Cognitive Performance in an Open-ended Dynamic Task. In Proceedings of the Second Artificial General Intelligence Conference (AGI-09). Amsterdam-Paris: Atlantis Press.Search in Google Scholar

Lebiere, C., & Wray, R. (2006). Between a Rock and a Hard Place: Cognitive Science Principles Meet AI-Hard Problems. Technical Report SS-06-02. AAAI Press, Menlo Park, California.Search in Google Scholar

Newell, A. N. (1973). You can't play 20 questions with nature and win: Projective comments on papers in this symposium. In W. G. Chase (Ed.), Visual information processing. New York: Academic Press.Search in Google Scholar

Stewart, D. (1994). Interview with Herbert Simon. Omni Magazine, June 1994.Search in Google Scholar

Warwick, W. (2009) Comparing the Comparisons. In Proceedings of the Behavior Representation in Modeling and Simulation (BRIMS-09). Sundance, Utah.Search in Google Scholar

Recommended articles from Trend MD

Plan your remote conference with Sciendo