Citation:

Voolla Sandeep kumar, 2026. Load-Aware Dispatch Framework for Concurrent ETL and ML Model Execution in Enterprise Systems  Volume 6 Issue 2: 269-277.

Abstract:

Enterprise systems are moving an ETL workload and machine learning (ML) model execution to the same computational substrate, but research on the treatment of dispatch control in both workload categories has been disjointed. This review examines recent literature relevant to load-aware dispatch for the concurrent execution of ETL workloads and ML models in enterprise systems. It focuses on orchestration logic, lifecycle management, resource competition, data freshness, latency protection, and operational governance. The review concludes that the recent journal literature has a lot to offer in regard to MLOps architecture, quality assurance, deployment automation, and intelligent data preparation, but does not discuss in detail the joint ETL-ML dispatch under common enterprise load. Practical experience shows repeatedly that the breakdown of coordination is possible at the border of data pipelines, model pipelines, and infrastructure schedulers. Some of the most prominent gaps include the poor cross-layer observability, the inability to support mixed I/O- and latency-sensitive workloads, the inability to support freshness-latency trade-offs in policies and the inability to treat the queue isolation, workload classification, and feedback control in a production environment. A topic-specific conceptual framework is thus created to relate data-plane dynamics, model-serving constraints, and dispatch policy choice. The field is important since the enterprise value is more and more relying on the stable coexistence of constantly updated streams of data and constantly used predictive services.

References:

[1] Ashmore, R., Calinescu, R., & Paterson, C. (2021). Assuring the machine learning lifecycle: Desiderata, methods, and challenges. ACM Computing Surveys, 54(5), 1–39.

[2] Lwakatare, L. E., Raj, A., Bosch, J., Olsson, H. H., Crnkovic, I., & Ågren, S. M. (2020). Large-scale machine learning systems in real-world industrial settings: A review of challenges and solutions. Information and Software Technology, 127, 106368.

[3] Paleyes, A., Urma, R.-G., & Lawrence, N. D. (2022). Challenges in deploying machine learning: A survey of case studies. ACM Computing Surveys, 55(6), 1–29.

[4] Shahin, M., Ali Babar, M., & Zhu, L. (2022). Continuous integration, delivery and deployment of machine learning models: A systematic review. Journal of Systems and Software, 189, 111331.

[5] Kreuzberger, D., Kühl, N., & Hirschl, S. (2023). Machine learning operations (MLOps): Overview, definition, and architecture. IEEE Access, 11, 31866–31879.

[6] Giray, G. (2021). A software engineering perspective on engineering machine learning systems: State of the art and challenges. Journal of Systems and Software, 180, 111031.

[7] Braiek, H. B., & Khomh, F. (2020). On testing machine learning programs. Journal of Systems and Software, 164, 110542.

[8] Serban, A., van der Blom, K., Hoos, H. H., & Visser, J. (2021). Adoption and effects of software engineering best practices in machine learning. Empirical Software Engineering, 26(5), 1–44.

[9] Martínez-Fernández, S., Franch, X., Jedlitschka, A., Oriol, M., & Trendowicz, A. (2022). Software engineering for AI-based systems: A survey. ACM Transactions on Software Engineering and Methodology, 31(2), 1–59.

[10] Bilalli, B., Abelló, A., Aluja-Banet, T., & Wrembel, R. (2022). Intelligent data preparation: A survey. The VLDB Journal, 31(4), 853–885.

[11] He, X., Zhao, K., & Chu, X. (2021). AutoML: A survey of the state-of-the-art. Knowledge-Based Systems, 212, 106622.

[12] Zöller, M.-A., & Huber, M. F. (2021). Benchmark and survey of automated machine learning frameworks. Journal of Artificial Intelligence Research, 70, 409–472.

[13] Casalicchio, E., & Perciballi, V. (2020). Container orchestration: A survey. Systems, 8(4), 43.

[14] Studer, S., Bui, T. B., Drescher, C., Hanuschkin, A., Winkler, L., Peters, S., & Müller, K.-R. (2021). Towards CRISP-ML(Q): A machine learning process model with quality assurance methodology. Machine Learning and Knowledge Extraction, 3(2), 392–413.

[15] Tamburri, D. A. (2020). Sustainable MLOps: Trends and challenges. Journal of Data and Information Quality, 12(3), 1–19.

Keywords:

Concurrent ETL-ML Execution, Enterprise Systems, Load-Aware Dispatch, Mlops, Resource Orchestration .