Explainability of Artificial Intelligence Models: Technical Foundations and Legal Principles

[1] Ablameyko M. et al. (2022), ‘Legal aspects of e-commerce cooperation between Eurasian economic union and Vietnam’, In: Journal of Science and Technology – Binh Duong University 5.2 Search in Google Scholar

[2] Alarie B., Niblett A., and Yoon A. H. (2016), ‘Using machine learning to predict outcomes in tax law’, In: Can. Bus. LJ 58, p. 231 Search in Google Scholar

[3] Almuslim I. and Inkpen D., ‘Legal Judgment Prediction for Canadian Appeal Cases’, In: 2022 7th International Conference on Data Science and Machine Learning Applications (CDMA). IEEE. 2022, pp. 163–168 Search in Google Scholar

[4] Arnold M. et al. (2019), ‘FactSheets: Increasing trust in AI services through supplier’s declarations of conformity’, In: IBM Journal of Research and Development 63.4/5. Retrieved from: https://aifs360.mybluemix.net/introduction [accessed on 17 December 2022] Search in Google Scholar

[5] Baracaldo N. et al. (2022), ‘Towards an Accountable and Reproducible Federated Learning: A FactSheets Approach’, In: arXiv preprint arXiv:2202.12443. Retrieved from: https://arxiv.org/pdf/2202.12443.pdf [accessed on 17 December 2022] Search in Google Scholar

[6] Barbosa S. et al. (2016), ‘Averaging gone wrong: Using time-aware analyses to better understand behavior’, In: Proceedings of the 25th International Conference on World Wide Web., pp. 829–841. Retrieved from: https://dl.acm.org/doi/pdf/10.1145/2872427.2883083 [accessed on 27 July 2022] Search in Google Scholar

[7] Benk M. and Ferrario A. (2020), ‘Explaining Interpretable Machine Learning: Theory, Methods and Applications’, In: Methods and Applications (December 11, 2020). Retrieved from: https://www.researchgate.net/profile/Andrea-Ferrario7/publication/348678581_Explaining_Interpretable_Machine_Learning_Theory_Methods_and_Applications/links/600ab71e299bf14088b21f03/Explaining-Interpretable-Machine-Learning-Theory-Methods-and-Applications.pdf [accessed on 25 October 2022] Search in Google Scholar

[8] Bhandari A. (2020), ‘Feature Scaling for Machine Learning: Understanding the Difference Between Normalization vs. Standardization’. Retrieved from: https://www.analyticsvidhya.com/blog/2020/04/feature-scaling-machine-learning-normalization-standardization/ [accessed on 17 December 2022] Search in Google Scholar

[9] Bhattacharya A. (2022), Applied Machine Learning Explainability Techniques: Make ML models explainable and trustworthy for practical applications using LIME, SHAP, and more. Packt Publishing Search in Google Scholar

[10] Borghesi A., Baldo F., and Milano M. (2020), ‘Improving deep learning models via constraint-based domain knowledge: a brief survey’, In: arXiv preprint arXiv:2005.10691. Retrieved from: https://arxiv.org/pdf/2005.10691.pdf [accessed on 25 July 2022] Search in Google Scholar

[11] Bui T. H. and Nguyen V. P. (2022), ‘The impact of artificial intelligence and digital economy on vietnam’s legal system’, In: International Journal for the Semiotics of Law-Revue internationale deŚemiotique juridique, pp. 1–21 Search in Google Scholar

[12] Cameron A., Pham T., and Atherton J. (2018), ‘Vietnam Today–first report of the Vietnam’s Future Digital Economy Project’, In: Canberra: CSIRO Search in Google Scholar

[13] Carvalho D. V, Pereira E. M., and Cardoso J. S. (2019), ‘Machine learning interpretability: A survey on methods and metrics’, In: Electronics 8.8. p. 832. Retrieved from: https://pdfs.semanticscholar.org/232d/e9d0c1947ec757bb6644f27d203e488b1aaf.pdf [accessed on 27 July 2022] Search in Google Scholar

[14] Chao P.-J. et al. (2021), ‘Knowledge of and competence in artificial intelligence: Perspectives of Vietnamese digital-native students’, In: IEEE Access 9, pp. 75751–75760 Search in Google Scholar

[15] Dandl S. et al. (2020), ‘Multiobjective counterfactual explanations’, In: International Conference on Parallel Problem Solving from Nature. Springer, pp. 448–469 Search in Google Scholar

[16] Dahan S. et al. (2020), ‘Predicting Employment Notice Period with Machine Learning: Promises and Limitations’, In: McGill Law Journal/Revue de droit de McGill 65.4, pp. 711–753 Search in Google Scholar

[17] Dong G. and Liu H. (2018), Feature Engineering for Machine Learning and Data Analytics, CRC Press Search in Google Scholar

[18] Edwards L. and Michael V. (2017), ‘Slave to the algorithm: Why a right to an explanation is probably not the remedy you are looking for’, Duke L. & Tech. Rev. 16, p. 18. Retrieved from: https://discovery.ucl.ac.uk/id/eprint/1574817/1/Veale_slavetothealgorithm_published.pdf [accessed on 27 July 2022] Search in Google Scholar

[19] Elite Data Science (2022), ‘Best Practices for Feature Engineering’. Retrieved from: https://elitedatascience.com/feature-engineering-best-practices [accessed 17 December 2022] Search in Google Scholar

[20] Erasmus A., Brunet T. DP, and Fisher E. (2021), ‘What is interpretability?’, In: Philosophy & Technology 34.4, pp. 833–862. Retrieved from: https://link.springer.com/article/10.1007/s13347-020-00435-2 [accessed on 27 July 2022] Search in Google Scholar

[21] Fadel S. (2022), Explainable Machine Learning, Game Theory, and Shapley Values: A Technical Review. Statistics Canada, Oct. 7. Retrieved from: https://www.statcan.gc.ca/en/data-science/network/explainable-learning [accessed on 10 July 2022] Search in Google Scholar

[22] Finale D.-V. and Kim B. (2017), ‘Towards a rigorous science of interpretable machine learning’, In: arXiv preprint arXiv:1702.08608. Retrieved from: https://arxiv.org/pdf/1702.08608.pdf [accessed on 27 July 2022] Search in Google Scholar

[23] Gaon A. and Stedman I. (2018), ‘A call to action: Moving forward with the governance of artificial intelligence in Canada’, In: Alta. L. Rev. 56. Retrieved from: http://albertalawreview.com/index.php/ALR/article/download/2547/2514 [accessed on 27 July 2022] Search in Google Scholar

[24] Ghasemi M. et al. (2021), ‘The Application of Machine Learning to a General Risk– Need Assessment Instrument in the Prediction of Criminal Recidivism’, In: Criminal Justice and Behavior 48.4, pp. 518–538. Retrieved from: https://journals.sagepub.com/doi/pdf/10.1177/0093854820969753 [accessed on 27 July 2022] Search in Google Scholar

[25] Gill J. K. (2020), ‘Health insurance fraud detection’, In:. https://era.library.ualberta.ca/items/e68678e1-1021-4e4c-8fa2-54455deb9fd0 [accessed on 27 July 2022] Search in Google Scholar

[26] Gilpin L. H et al. (2018), ‘Explaining explanations: An overview of interpretability of machine learning’, In: IEEE 5th International Conference on data science and advanced analytics (DSAA), pp. 80–89. Retrieved from: https://arxiv.org/pdf/1806.00069.pdf [accessed on 27 July 2022] Search in Google Scholar

[27] Goldstein A. et al. (2015), ‘Peeking inside the black box: Visualizing statistical learning with plots of individual conditional expectation’, In: journal of Computational and Graphical Statistics 24.1, pp. 44–65. Retrieved from: https://arxiv.org/pdf/1309.6392.pdf [accessed on 27 July 2022] Search in Google Scholar

[28] Graziani M. et al. (2022), ‘A global taxonomy of interpretable AI: unifying the terminology for the technical and social sciences’, In: Artificial Intelligence Review, pp. 132. Retrieved from: https://link.springer.com/content/pdf/10.1007/s10462-022-10256-8.pdf [accessed on 27 July 2022] Search in Google Scholar

[29] Kapoor S. and Narayanan A. (2022), ‘Leakage and the Reproducibility Crisis in ML-based Science’, In: arXiv preprint arXiv:2207.07048. Retrieved from: https://arxiv.org/pdf/2207.07048.pdf [accessed on 25 July 2022] Search in Google Scholar

[30] Lehr D. and Ohm P. (2017), ‘Playing with the data: what legal scholars should learn about machine learning’, In: UCDL Rev. 51, p. 653. Retrieved from: https://lawreview.law.ucdavis.edu/issues/51/2/Symposium/51-2_Lehr_Ohm.pdf [accessed on 25 July 2022] Search in Google Scholar

[31] Lipton P. (2009), ‘Understanding Without Explanation’, In: Scientific understanding: Philosophical perspectives Search in Google Scholar

[32] Lokanan M. E. and Sharma K. (2022), ‘Fraud prediction using machine learning: The case of investment advisors in Canada’, In: Machine Learning with Applications 8. p. 100-269. Retrieved from: https://www.sciencedirect.com/science/article/pii/S2666827022000111/pdfft?isDTMRedir=true&download=true [accessed on 27 July 2022] Search in Google Scholar

[33] Lundberg S. M. and Lee S.-I. (2017), ‘A Unified Approach to Interpreting Model Predictions’, In: Advances in Neural Information Processing Systems. Ed. by I. Guyon et al. Vol. 30. Curran Associates, Inc.. Retrieved from: https://proceedings.neurips.cc/paper/2017/file/8a20a8621978632d76c43dfd28b67767-Paper.pdf. [accessed on 27 July 2022] Search in Google Scholar

[34] Marcinkevics R. and Vogt J. E. (2020), ‘Interpretability and explainability: A machine learning zoo mini-tour’, In: arXiv preprint arXiv:2012.01805. Retrieved from: https://arxiv.org/pdf/2012.01805.pdf [accessed on 27 July 2022] Search in Google Scholar

[35] Masis S. (2021), Interpretable Machine Learning with Python: Learn to build interpretable high-performance models with hands-on real-world examples, Packt Publishing Ltd Search in Google Scholar

[36] Mehrabi N. et al. (2021), ‘A survey on bias and fairness in machine learning’, In: ACM Computing Surveys (CSUR) 54.6, pp. 1–35. Retrieved from: https://arxiv.org/pdf/1908.09635.pdf [accessed on 27 July 2022] Search in Google Scholar

[37] Melis D. A. and Jaakkola T. (2018), ‘Towards robust interpretability with self-explaining neural networks’, In: Advances in neural information processing systems 31. Retrieved from: https://proceedings.neurips.cc/paper/2018/file/3e9f0fc9b2f89e043bc6233994dfcf76-Paper.pdf [accessed on 27 July 2022] Search in Google Scholar

[38] MOJ & HCMUL, Minutes of the Conference ‘Legal Responsibility in Artificial Intelligence Application: International Practices and Experiences for Vietnam’, Ministry of Justice and Ho Chi Minh City University of Law, HCMC 11/12/2022 Search in Google Scholar

[39] Molnar C. (2020), Interpretable machine learning. Retrieved from: https://christophm.github.io/interpretable-ml-book/index.html [accessed on 27 July 2022] Search in Google Scholar

[40] Murdoch W. J. et al. (2019), ‘Definitions, methods, and applications in interpretable machine learning’, In: Proceedings of the National Academy of Sciences 116.44, pp. 22071–22080. Retrieved from: https://www.pnas.org/doi/pdf/10.1073/pnas.1900654116 [accessed on 27 July 2022] Search in Google Scholar

[41] Nalbandian L. (2021), ‘Using Machine-Learning to Triage Canada’s Temporary Resident Visa Applications’, In: Ryerson Centre for Immigration and Settlement (RCIS) and the CERC in Migration and Integration. Retrieved from: https://www.torontomu.ca/content/dam/centre-for-immigration-and-settlementtmcis/publications/workingpapers/2021_9_Nalbandian_Lucia_Using_Machine_Learning_to_Triage_Canadas_Temporary_Resident_Visa_Applications.pdf [accessed on 27 July 2022] Search in Google Scholar

[42] Northcutt C. G., Athalye A., and Mueller J. (2021), ‘Pervasive Label Errors in Test Sets Destabilize Machine Learning Benchmarks’, In: arXiv preprint arXiv:2103.14749. Retrieved from: https://arxiv.org/pdf/2103.14749.pdf [accessed on 27 July 2022] Search in Google Scholar

[43] Lapuschkin S. et al. (2019), ‘Unmasking Clever Hans predictors and assessing what machines really learn’, In: Nature communications 10.1, pp. 1–8. Retrieved from: https://www.nature.com/articles/s41467-019-08987-4 [accessed on 25 July 2022] Search in Google Scholar

[44] Parentoni L. (2022), ‘What should we reasonably expect from artificial intelligence?’, In: Publication pending at time of review. Retrieved from: https://www.researchgate.net/profile/Leonardo-Parentoni/publication/361988480_What_should_we_reasonably_expect_from_artificial_intelligence/links/62d0198e953dfc1e93ff7c45/What-should-we-reasonably-expect-from-artificial-intelligence.pdf [accessed on 25 July 2022] Search in Google Scholar

[45] Price II. and Nicholson W. (2017), ‘Artificial intelligence in health care: applications and legal issues’. Retrieved from: https://repository.law.umich.edu/cgi/viewcontent.cgi?article=2932&context=articles [accessed on 27 July 2022] Search in Google Scholar

[46] PwC (2017), ‘Sizing the prize: PwC’s Global Artificial Intelligence Study: Exploiting the AI Revolution’. Retrieved from: https://www.pwc.com/gx/en/issues/data-and-analytics/publications/artificial-intelligence-study.html [accessed on 17 December 2022] Search in Google Scholar

[47] Ribeiro M. T., Singh S., and Guestrin C. (2018), ‘Anchors: High-precision model-agnostic explanations’, In: Proceedings of the AAAI conference on artificial intelligence, Vol. 32. Retrieved from: https://ojs.aaai.org/index.php/AAAI/article/view/11491/11350 [accessed on 27 July 2022] Search in Google Scholar

[48] Ribeiro M. T., Singh S., and Guestrin C. (2016), ‘Why should I trust you? - Explaining the predictions of any classifier’, In: Proceedings of the 22nd ACM SIGKDD international conference on knowledge discovery and data mining, pp. 1135–1144. Retrieved from: https://arxiv.org/pdf/1602.04938.pdf [accessed on 27 July 2022] Search in Google Scholar

[49] Roscher R. et al. (2020), ‘Explainable machine learning for scientific insights and discoveries’, In: IEEE Access 8, pp. 42200–42216. Retrieved from: https://ieeexplore.ieee.org/stamp/stamp.jsp?arnumber=9007737 [accessed on 25 July 2022] Search in Google Scholar

[50] Roth A. E. (1988), ‘Introduction to the Shapley value’, In: The Shapley value, pp. 1–27. Retrieved from: http://library.fa.ru/files/Roth2.pdf#page=9 [accessed on 27 July 2022] Search in Google Scholar

[51] Rudin C. (2019), ‘Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead’, In: Nature Machine Intelligence 1.5, pp. 206–215. Retrieved from: https://www.ncbi.nlm.nih.gov/pmc/articles/PMC9122117/ [accessed on 27 July 2022] Search in Google Scholar

[52] Rueden L. V. et al. (2019), ‘Informed Machine Learning–A Taxonomy and Survey of Integrating Knowledge into Learning Systems’, In: arXiv preprint arXiv:1903.12394. Retrieved from: https://arxiv.org/pdf/1903.12394.pdf [accessed on 25 October 2022] Search in Google Scholar

[53] Russell S. and Norvig P. (2021), Artificial intelligence: a modern approach, 4th Edition. Pearson Search in Google Scholar

[54] Scherer M. U. (2015), ‘Regulating artificial intelligence systems: Risks, challenges, competencies, and strategies’, In: Harv. JL & Tech. 29, p. 353. Retrieved from: https://euro.ecom.cmu.edu/program/law/08-732/AI/Scherer.pdf [accessed on 25 October 2022] Search in Google Scholar

[55] Suresh H. and Guttag J. (2021), ‘A framework for understanding sources of harm throughout the machine learning life cycle’, In: Equity and access in algorithms, mechanisms, and optimization, pp. 1–9. Retrieved from: https://dl.acm.org/doi/fullHtml/10.1145/3465416.3483305 [accessed on 27 July 2022] Search in Google Scholar

[56] Tran D. M. et al. (2022), ‘Digital Health Policy and Programs for Hospital Care in Vietnam: Scoping Review’, In: Journal of medical Internet research 24.2 Search in Google Scholar

[57] Treasury Board of Canada (2021), Directive on Automated Decision-Making, Government of Canada, Apr. 21. Retrieved from: https://www.tbs-sct.canada.ca/pol/doc-eng.aspx?id=32592 [accessed on 07/25/2022] Search in Google Scholar

[58] Vincent J. (2016), ‘Twitter taught Microsoft’s AI chatbot to be a racist asshole in less than a day’, Tayandyou (Twitter). Retrieved from: https://www.theverge.com/2016/3/24/11297050/tay-microsoft-chatbot-racist [accessed on 17 December 2022] Search in Google Scholar

[59] Watson D. S. and Floridi L. (2021), ‘The Explanation Game: a Formal Framework for Interpretable Machine Learning’, In: Ethics, Governance, and Policies in Artificial Intelligence. Springer, pp. 185–219. Retrieved from: https://link.springer.com/content/pdf/10.1007/s11229-020-02629-9.pdf [accessed on 27 July 2022] Search in Google Scholar