1. bookVolume 50 (2017): Issue 3 (August 2017)
Journal Details
First Published
17 Oct 2008
Publication timeframe
4 times per year
access type Open Access

Organizational Learning Supported by Machine Learning Models Coupled with General Explanation Methods: A Case of B2B Sales Forecasting

Published Online: 22 Aug 2017
Page range: 217 - 233
Received: 15 May 2017
Accepted: 20 Jul 2017
Journal Details
First Published
17 Oct 2008
Publication timeframe
4 times per year

Background and Purpose: The process of business to business (B2B) sales forecasting is a complex decision-making process. There are many approaches to support this process, but mainly it is still based on the subjective judgment of a decision-maker. The problem of B2B sales forecasting can be modeled as a classification problem. However, top performing machine learning (ML) models are black boxes and do not support transparent reasoning. The purpose of this research is to develop an organizational model using ML model coupled with general explanation methods. The goal is to support the decision-maker in the process of B2B sales forecasting.


Argyris, C. & Schön, D. (1996). Organizational Learning II: Theory, Method and Practice. Addison Wesley.Search in Google Scholar

Armstrong, J. S., Green, K. C. & Graefe, A. (2015). Golden Rule of Forecasting: Be conservative. Journal of Business Research, 68 (8), 1717-1731, http://dx.doi.org/10.1016/j.jbusres.2015.03.031Search in Google Scholar

Avison, D., & Fitzgerald, G. (2006). Methodologies for Developing Information Systems : A Historical Perspective. The Past and Future of Information Systems, 27–38, https://doi.org/10.1007/978-0-387-34732-5_3Search in Google Scholar

Bohanec, M. (2016). Anonimized data set for B2B sales history. Retrieved 15.07.2017 from http://www.salvirt.com/research/b2bdatasetSearch in Google Scholar

Bohanec, M., Kljajić Borštnar, M. & Robnik-Šikonja, M. (2015a). Feature subset selection for B2B sales forecasting. In: 13th International Symposium on Operational Research, Bled, Slovenia, 285-290.Search in Google Scholar

Bohanec, M., Kljajić Borštnar, M. & Robnik-Šikonja, M. (2015b). Machine learning data set analysis with visual simulation. In: InterSymp 2015, Baden-Baden, Germany, 16-20.Search in Google Scholar

Bohanec, M., Kljajić Borštnar, M. & Robnik-Šikonja, M. (2016a). Nabor atributov za opisovanje medorganizacijske prodaje [A collection of attributes describing business to business (B2B) sales]. Uporabna informatika, XXIV (2), 74-80.Search in Google Scholar

Bohanec, M., Kljajić Borštnar, M. & Robnik-Šikonja, M. (2016b). Sample size for identification of important attributes in B2B sales. In: 16th International Conference on Operational Research, Osijek, Croatia, 133.Search in Google Scholar

Bohanec, M., Kljajić Borštnar, M. & Robnik-Šikonja, M. (2017a). Explaining Machine Learning Predictions. Expert Systems with Applications, 71, 416-428.Search in Google Scholar

Bohanec, M., Robnik-Šikonja, M., & Kljajić Borštnar, M. (2017). Decision-making framework with double-loop learning through interpretable black-box machine learning models. Industrial Management & Data Systems, 117(7), in print (July, 2017b), http://dx.doi.org/10.1108/IMDS-09-2016-0409Search in Google Scholar

Breiman L. (2001), Random Forests, Machine Learning Journal, 45, 5–32.Search in Google Scholar

Brynjolfsson, E., Hitt, L. M. & Kim, H. H. (2011). Strength in numbers: How does data-driven decision-making affect firm performance? Retrieved 11 May 2017 from http://ebusiness.mit.edu/research/papers/2011.12_Brynjolfsson_Hitt_Kim_Strength%20in%20Numbers_302.pdfSearch in Google Scholar

Caruana, R. and Niculescu-Mizil, A. (2006). An empirical comparison of supervised learning algorithms. In Proceedings of the 23rd International Conference on Machine Learning (pp. 161–168). New York, NY, USA: ACM.Search in Google Scholar

Davis, D. F. & Mentzer, J. T. (2007). Organizational factors in sales forecasting management. International Journal of Forecasting, 23 (3), 475-495, http://dx.doi.org/10.1016/j.ijforecast.2007.02.005Search in Google Scholar

D’Haen, J., & Van der Poel, D. (2013), Model-supported business-to-business prospect prediction based on an iterative customer acquisition framework, Industrial Marketing Management 42, 544–551, http://dx.doi.org/10.1016/j.indmarman.2013.03.006Search in Google Scholar

Duran, R. E. (2008). Probabilistic sales forecasting for small and medium-size business operations. In B. Prasad (Ed.), Soft Computing Applications in Business, 129–146, Springer Berlin Heidelberg, http://dx.doi.org/10.1007/978-3-540-79005-1_8Search in Google Scholar

Fildes, R., Goodwin, P., & Lawrence, M. (2006). The design features of forecasting support systems and their effectiveness. Decision Support Systems, 42(1), 351–361, https://doi.org/10.1016/j.dss.2005.01.003Search in Google Scholar

Fleischmann, K. R., & Wallace, W. A. (2005). A covenant with transparency. Communications of the ACM, 48(5), 93–97, https://doi.org/10.1145/1060710.1060715Search in Google Scholar

Goodwin, P., Fildes, R., Lawrence, M., & Stephens, G. (2011). Restrictiveness and guidance in support systems. Omega, 39(3), 242–253, https://doi.org/10.1016/j.omega.2010.07.001Search in Google Scholar

Grossler, A., Maier, F. H., & Milling, P. M. (2000). Enhancing Learning Capabilities by Providing Transparency in Business Simulators. Simulation & Gaming, 31(2), 257–278, https://doi.org/10.1177/104687810003100209Search in Google Scholar

Hevner, A. R., March, S. T., Park, J., & Ram, S. (2004). Design Science in Information Systems Research. Management Information Systems Quarterly, 28(1), 75–105. https://doi.org/10.2307/25148625Search in Google Scholar

Huber, G.P. (1991). Organizational Learning: The Contributing Processes and the Literatures. Organization Science, 2(1), Special Issue: Organizational Learning: Papers in Honor of (and by) James G. March, 88-115, https://doi.org/10.1287/orsc.2.1.88Search in Google Scholar

Ingram, T. N., LaForge, R. W., Avila, R. A., Schwepker Jr, C. H. & Williams, M. R. (2012). Sales Management: Analysis and Decision Making. ME Sharpe.Search in Google Scholar

Kerkkänen, A. & Huiskonen, J. (2007). Analysing inaccurate judgmental sales forecasts. European J. Industrial Engineering, 1 (4), 355-369, https://doi.org/10.1504/EJIE.2007.015387Search in Google Scholar

Kljajić, M., & Farr, J. V. (2010). The Role of Systems Engineering in the Development of Information Systems, In M. Hunter (ed.), Strategic Information Systems: Concepts, methodologies, tools, and applications, Hershey, PA, Information Science Reference. 369–381.Search in Google Scholar

Kljajić Borštnar, M., Kljajić, M., Škraba, A., Kofjač, D., & Rajkovič, V. (2011). The relevance of facilitation in group decision making supported by a simulation model. System Dynamics Review, 27(3), 270–293, https://doi.org/10.1002/sdr.460Search in Google Scholar

Kuchinke, K. P. (2000). The role of feedback in management training settings. Human Resource Development Quarterly, 11 (4), 381–401, http://dx.doi.org/10.1002/1532-1096(200024)11:4%3C381::AID-HRDQ5%3E3.0.CO;2-3Search in Google Scholar

Lawrence, M., Goodwin, P., O’Connor, M. & Önkal, D. (2006). Judgmental forecasting: A review of progress over the last 25 years. International Journal of Forecasting, 22 (3), 493-518, http://dx.doi.org/10.1016/j.ijforecast.2006.03.007Search in Google Scholar

Lilien, G. L. (2016). The B2B knowledge gap. International Journal of Research in Marketing, 33, 543–556, http://dx.doi.org/10.1016/j.ijresmar.2016.01.003Search in Google Scholar

Lodato, M. W. (2006). Integrated sales process management: A methodology for improving sales effectiveness in the 21st century. AuthorHouse.Search in Google Scholar

McAfee, A. & Brynjolfsson, E. (2012). Big data. The management revolution. Harvard Business Review, 90 (10), 61–67.Search in Google Scholar

McCarthy, T. M., Davis, D. F., Golicic, S. L. & Mentzer, J. T. (2006). The evolution of sales forecasting management: a 20-year longitudinal study of forecasting practices. Journal of Forecasting, 25 (5), 303-324, http://dx.doi.org/10.1002/for.989Search in Google Scholar

Merkert, J., Mueller, M., & Hubl, M. (2015), A Survey of the Application of Machine Learning in Decision Support Systems, 23rd European Conference on Information Systems (ECIS) 2015 Completed Research Papers, Münster, Germany, 26.-29.05.2015.Search in Google Scholar

Monat, J. (2011). Industrial sales lead conversion modeling. Marketing Intelligence and Planning, 29, 178-194, http://dx.doi.org/10.1108/02634501111117610Search in Google Scholar

Provost, F. & Fawcett, T. (2013). Data science and its relationship to big data and data-driven decision making. Big Data, 1 (1), 51–59, http://dx.doi.org/10.1089/big.2013.1508Search in Google Scholar

Robnik-Šikonja, M. & Kononenko, I. (2008). Explaining classifications for individual instances. IEEE Transactions on Knowledge and Data Engineering, 20 (5), 589-600, http://dx.doi.org/10.1109/TKDE.2007.190734Search in Google Scholar

Sein, M., Henfridsson, O., Purao, S., Rossi, M., & Lindgren, R. (2011). Action Design Research. Management Information Systems Quarterly, 35(1). Retrieved 12.01.2017 from http://aisel.aisnet.org/misq/vol35/iss1/5Search in Google Scholar

Senge P.M & Sterman J.D (1992). Systems thinking and organizational learning—acting locally and thinking globally in the organization of the future. European Journal of Operational Research, 59(1), 137–150, http://dx.doi.org/10.1016/0377-2217(92)90011-WSearch in Google Scholar

Simon, H. (1960). The new science of management decision. Prentice-Hall.Search in Google Scholar

Simon, H. A. (1991). Organizations and Markets. Journal of Economic Perspectives, 5(2), 25–44. Retrieved 31.7.2017 from http://people.ds.cam.ac.uk/mb65/mst-ir/documents/simon-1991.pdfSearch in Google Scholar

Sterman, J. D. (1994). Learning in and about complex systems. System Dynamics Review, 10(2–3 (Special Issue: Systems thinkers, systems thinking), 291–330. https://doi.org/10.1002/sdr.4260100214Search in Google Scholar

Söhnchen, F. & Albers, S. (2010). Pipeline management for the acquisition of industrial projects. Industrial Marketing Management, 39 (8), 1356-1364.Search in Google Scholar

Škraba A, Kljajić M, & Kljajić Borštnar M. (2007). The role of information feedback in the management group decision-making process applying system dynamics models. Group Decision and Negotiation, 16, 77–95.Search in Google Scholar

Štrumbelj, E., Kononenko, I. & Robnik-Šikonja, M. (2009). Explaining instance classifications with interactions of subsets of feature values. Data & Knowledge Engineering, 68(10), 886–904, https://doi.org/10.1016/j.datak.2009.01.004Search in Google Scholar

Verikas, A., Gelzinis, A., & Bacauskiene, M. (2011). Mining data with random forests: A survey and results of new tests, Pattern Recognition, 44(2), 330–349, http://dx.doi.org/10.1016/j.patcog.2010.08.011Search in Google Scholar

Wirth, R., & Hipp, J. (n.d.). CRISP-DM: Towards a Standard Process Model for Data Mining. Retrieved 12.2.20117 from https://www.scribd.com/document/76859286/CRISP-DM-Towards-a-Standard-Process-Model-for-DataSearch in Google Scholar

Yan, J., Zhang, C., Zha, H., Gong, M., Sun, C., Huang, J., Chu, S., & Yang, X. (2015), “On machine learning towards predictive sales pipeline analytics”, In: Twenty-Ninth AAAI Conference on Artificial Intelligence, 1945–1951.Search in Google Scholar

Recommended articles from Trend MD

Plan your remote conference with Sciendo