<abstract xml:lang="en" xmlns="http://www.w3.org/1999/xhtml"><p>The equivalence test (Stewart and West, 2007; Stewart, 2007) is a statistical measure for evaluating the similarity between a model and the system being modelled. It is designed to avoid over-fitting and to generate an easily interpretable summary of the quality of a model. We apply the equivalence test to two tasks: Repeated Binary Choice (Erev et al., 2010) and Dynamic Stocks and Flows (Gonzalez and Dutt, 2007). In the first case, we find a broad range of statistically equivalent models (and win a prediction competition) while identifying particular aspects of the task that are not yet adequately captured. In the second case, we re-evaluate results from the Dynamic Stocks and Flows challenge, demonstrating how our method emphasizes the breadth of coverage of a model and how it can be used for comparing different models. We argue that the explanatory power of models hinges on numerical similarity to empirical data over a broad set of measures.</p></abstract>

The equivalence test (Stewart and West, 2007; Stewart, 2007) is a statistical measure for evaluating the similarity between a model and the system being modelled. It is designed to avoid over-fitting and to generate an easily interpretable summary of the quality of a model. We apply the equivalence test to two tasks: Repeated Binary Choice (Erev et al., 2010) and Dynamic Stocks and Flows (Gonzalez and Dutt, 2007). In the first case, we find a broad range of statistically equivalent models (and win a prediction competition) while identifying particular aspects of the task that are not yet adequately captured. In the second case, we re-evaluate results from the Dynamic Stocks and Flows challenge, demonstrating how our method emphasizes the breadth of coverage of a model and how it can be used for comparing different models. We argue that the explanatory power of models hinges on numerical similarity to empirical data over a broad set of measures.

Testing for Equivalence: A Methodology for Computational Cognitive Modelling

Centre for Theoretical Neuroscience, University of Waterloo, 200 University Avenue West Waterloo, ON N2L 3G1, Canada

Institute of Cognitive Science, Carleton University, 1125 Colonel By Drive Ottawa, ON K1S 5B6, Canada

Journal of Artificial General Intelligence

{"article-title":"Testing for Equivalence: A Methodology for Computational Cognitive Modelling"}

The equivalence test (Stewart and West, 2007; Stewart, 2007) is a statistical measure for evaluating the similarity between a model and the system being...

Testing for Equivalence: A Methodology for Computational Cognitive Modelling

Publié en ligne: 23 nov. 2011

Pages: 69 - 87

DOI: https://doi.org/10.2478/v10229-011-0010-8

Mots clés
equivalence testing, cognitive modelling, model comparison, dynamic stocks and flows, repeated binary choice

This content is open access.

Testing for Equivalence: A Methodology for Computational Cognitive Modelling

Publié en ligne: 23 nov. 2011

Pages: 69 - 87

DOI: https://doi.org/10.2478/v10229-011-0010-8

Mots clésequivalence testing, cognitive modelling, model comparison, dynamic stocks and flows, repeated binary choice

This content is open access.

Mots clés
equivalence testing, cognitive modelling, model comparison, dynamic stocks and flows, repeated binary choice