Published Online: Feb 02, 2022
Page range: 335 - 360
Received: Oct 28, 2021
Accepted: Dec 14, 2021
DOI: https://doi.org/10.2478/ausi-2021-0016
© 2021 Kristóf Szabados et al., published by Sciendo
This work is licensed under the Creative Commons Attribution 4.0 International License.
Context
It is crucial to understand how reproducible the measurement results in the scientific publications are, as reproducibility is one of the cornerstones of engineering.
Objective
The goal of this study is to investigate the scientific publications presented at the premier technical debt conferences by understanding how reproducible the reported findings are.
Method
We conducted a systematic literature review of 135 unique papers published at the “International Workshop on Managing Technical Debt” and the “International Conference on Managing Technical Debt”, the premier scientific conference series on technical debt.
Results
Only 44 of the investigated 135 papers presented numerical evidence and only 5 papers listed the tools, the availability of the tools, and the version of the tools used. For the rest of the papers additional information would have been needed for the potential reproducibility. One of the published papers even referred to a pornographic site as a source of a toolset for empirical research.