<abstract xmlns="http://www.w3.org/1999/xhtml">

<p>Learning to perform abstract reasoning often requires decomposing the task in question into intermediate subgoals that are not specified upfront, but need to be autonomously devised by the learner. In Raven progressive matrices (RPMs), the task is to choose one of the available answers given a context, where both the context and answers are composite images featuring multiple objects in various spatial arrangements. As this high-level goal is the only guidance available, learning to solve RPMs is challenging. In this study, we propose a deep learning architecture based on the transformer blueprint which, rather than directly making the above choice, addresses the subgoal of predicting the visual properties of individual objects and their arrangements. The multidimensional predictions obtained in this way are then directly juxtaposed to choose the answer. We consider a few ways in which the model parses the visual input into tokens and several regimes of masking parts of the input in self-supervised training. In experimental assessment, the models not only outperform state-of-the-art methods but also provide interesting insights and partial explanations about the inference. The design of the method also makes it immune to biases that are known to be present in some RPM benchmarks.</p>
</abstract>

Learning to perform abstract reasoning often requires decomposing the task in question into intermediate subgoals that are not specified upfront, but need to be autonomously devised by the learner. In Raven progressive matrices (RPMs), the task is to choose one of the available answers given a context, where both the context and answers are composite images featuring multiple objects in various spatial arrangements. As this high-level goal is the only guidance available, learning to solve RPMs is challenging. In this study, we propose a deep learning architecture based on the transformer blueprint which, rather than directly making the above choice, addresses the subgoal of predicting the visual properties of individual objects and their arrangements. The multidimensional predictions obtained in this way are then directly juxtaposed to choose the answer. We consider a few ways in which the model parses the visual input into tokens and several regimes of masking parts of the input in self-supervised training. In experimental assessment, the models not only outperform state-of-the-art methods but also provide interesting insights and partial explanations about the inference. The design of the method also makes it immune to biases that are known to be present in some RPM benchmarks.

Learning Abstract Visual Reasoning Via Task Decomposition: A Case Study in Raven Progressive Matrices

jakub.k.kwiatkowski@doctorate.put.poznan.pl

International Journal of Applied Mathematics and Computer Science

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

Learning to perform abstract reasoning often requires decomposing the task in question into intermediate subgoals that are not specified upfront, but need to...

Learning Abstract Visual Reasoning Via Task Decomposition: A Case Study in Raven Progressive Matrices

Data publikacji: 25 cze 2024

Zakres stron: 309 - 321

Otrzymano: 11 sie 2023

Przyjęty: 05 lut 2024

DOI: https://doi.org/10.61822/amcs-2024-0022

Słowa kluczowe
abstract visual reasoning, Raven progressive matrices, machine learning, problem decomposition

© 2024 Jakub Kwiatkowski et al., published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

Learning Abstract Visual Reasoning Via Task Decomposition: A Case Study in Raven Progressive Matrices

Data publikacji: 25 cze 2024

Zakres stron: 309 - 321

Otrzymano: 11 sie 2023

Przyjęty: 05 lut 2024

DOI: https://doi.org/10.61822/amcs-2024-0022

Słowa kluczoweabstract visual reasoning, Raven progressive matrices, machine learning, problem decomposition

© 2024 Jakub Kwiatkowski et al., published by Sciendo

This work is licensed under the Creative Commons Attribution-NonCommercial-NoDerivatives 3.0 License.

Słowa kluczowe
abstract visual reasoning, Raven progressive matrices, machine learning, problem decomposition