Benchmarking 24 Large Language Models for Automated Multiple-Choice Question Generation in Latvian

[1] A. M. Olney, “Generating multiple-choice questions from a textbook: LLMs match human performance on most metrics,” in Proceedings of the Workshop on Empowering Education with LLMs – the Next-Gen Interface and Content Generation, Tokyo, Japan, Jul. 2023. https://ceurws.org/Vol-3487/paper7.pdf Search in Google Scholar

[2] A. Säuberli and S. Clematide, “Automatic generation and evaluation of reading comprehension test items with large language models,” arXiv preprint arXiv:2404.07720, Apr. 2024. https://doi.org/10.48550/arXiv.2404.07720 Search in Google Scholar

[3] D. de-Fitero-Dominguez, A. Garcia-Cabot, and E. Garcia-Lopez, “Automated multiple-choice question generation in Spanish using neural language models,” Neural Computing and Applications, vol. 36, no. 29, pp. 18223–18235, Jul. 2024. https://doi.org/10.1007/s00521-024-10076-7 Search in Google Scholar

[4] Y. S. Kıyak and E. Emekli, “ChatGPT prompts for generating multiple-choice questions in medical education and evidence on their validity: A literature review,” Postgraduate Medical Journal, vol. 100, no. 1189, pp. 858–865, Nov. 2024. https://doi.org/10.1093/postmj/qgae065 Search in Google Scholar

[5] C. Grévisse, M.A.S. Pavlou, and J.G. Schneider, “Docimological quality analysis of LLM-generated multiple choice questions in computer science and medicine,” SN Computer Science, vol. 5, Jun. 2024, Art. no. 636. https://doi.org/10.1007/s42979-024-02963-6 Search in Google Scholar

[6] S. S. Mucciaccia, T. M. Paixão, F. Mutz, A. F. De Souza, C. S. Badue, and T. Oliveira-Santos, “Automatic multiple-choice question generation and evaluation systems based on LLM: A study case with university resolutions,” in Proceedings of the 31st International Conference on Computational Linguistics, 2025, pp. 2246–2260. Search in Google Scholar

[7] R. Darģis, G. Bārzdiņš, I. Skadiņa, and B. Saulite. “Evaluating open-source LLMs in low-resource languages: Insights from Latvian high school exams,” in Proceedings of the 4th International Conference on Natural Language Processing for Digital Humanities, Miami, USA, Nov. 2024, pp. 289–293. https://doi.org/10.18653/v1/2024.nlp4dh-1.28 Search in Google Scholar

[8] N. A. Etori, A. Kanepajs, K. Lu, and R. Karisa, “LAG-MMLU: Benchmarking frontier LLM understanding in Latvian and Giriama,” in Proceedings of the Joint 25th Nordic Conference on Computational Linguistics and 11th Baltic Conference on Human Language Technologies (NoDaLiDa/Baltic-HLT 2025), Tallinn, Estonia, 2025. Search in Google Scholar

[9] Matisāne L, “Civilā aizsardzība.” Valsts izglītības satura centrs, 2011. [Online] Available: https://registri.visc.gov.lv/vispizglitiba/saturs/dokumenti/metmat/civila_aizsardziba.pdf. Accessed on: Oct. 7, 2024. Search in Google Scholar

[10] Z. Frīde, B. Gudriķe, V. Hausmanis, J. Kalniņš, I. Treimane, V. Vāvere, and V. Vecgrāvis, Latviešu literatūras vēsture. Latvijas Zinātņu akadēmijas Literatūras, folkloras un mākslas institūts, Apgāds Zvaigzne ABC, 1998. Search in Google Scholar