Dynamic domain analysis for predicting concept drift in engineering AI-enabled software
Murtuza Shahzad
Orcid profile, , , and 
Article Category: Research Papers
Published Online: May 07, 2025
Page range: 124 - 151
Received: Nov 15, 2024
Accepted: Mar 11, 2025
DOI: https://doi.org/10.2478/jdis-2025-0020
Keywords
© 2025 Murtuza Shahzad et al., published by Sciendo
This work is licensed under the Creative Commons Attribution 4.0 International License.
Purpose
This research addresses the challenge of concept drift in AI-enabled software, particularly within autonomous vehicle systems where concept drift in object recognition (like pedestrian detection) can lead to misclassifications and safety risks. This study introduces a proactive framework to detect early signs of domain-specific concept drift by leveraging domain analysis and natural language processing techniques. This method is designed to help maintain the relevance of domain knowledge and prevent potential failures in AI systems due to evolving concept definitions.
Design/methodology/approach
The proposed framework integrates natural language processing and image analysis to continuously update and monitor key domain concepts against evolving external data sources, such as social media and news. By identifying terms and features closely associated with core concepts, the system anticipates and flags significant changes. This was tested in the automotive domain on the pedestrian concept, where the framework was evaluated for its capacity to detect shifts in the recognition of pedestrians, particularly during events like Halloween and specific car accidents.
Findings
The framework demonstrated an ability to detect shifts in the domain concept of pedestrians, as evidenced by contextual changes around major events. While it successfully identified pedestrian-related drift, the system’s accuracy varied when overlapping with larger social events. The results indicate the model’s potential to foresee relevant shifts before they impact autonomous systems, although further refinement is needed to handle high-impact concurrent events.
Research limitations
This study focused on detecting concept drift in the pedestrian domain within autonomous vehicles, with results varying across domains. To assess generalizability, we tested the framework for airplane-related incidents and demonstrated adaptability. However, unpredictable events and data biases from social media and news may obscure domain-specific drifts. Further evaluation across diverse applications is needed to enhance robustness in evolving AI environments.
Practical implications
The proactive detection of concept drift has significant implications for AI-driven domains, especially in safety-critical applications like autonomous driving. By identifying early signs of drift, this framework provides actionable insights for AI system updates, potentially reducing misclassification risks and enhancing public safety. Moreover, it enables timely interventions, reducing costly and laborintensive retraining requirements by focusing only on the relevant aspects of evolving concepts. This method offers a streamlined approach for maintaining AI system performance in environments where domain knowledge rapidly changes.
Originality/value
This study contributes a novel domain-agnostic framework that combines natural language processing with image analysis to predict concept drift early. This unique approach, which is focused on real-time data sources, offers an effective and scalable solution for addressing the evolving nature of domain-specific concepts in AI applications.