Citation:

Yeshwanth Macha, Sunij Kumar Pulichikkunnu, 2024. "A Survey of DevOps Practices for Machine Learning and Artificial Intelligence Workflows in Modern Software Development", ESP Journal of Engineering & Technology Advancements  4(3): 200-208.

Abstract:

The rapid pace of the evolution of digital technologies has altered the software development industry, and Artificial Intelligence (AI) and Machine Learning (ML) have become the innovators of several industries. Compared to conventional software systems, AI/ML processes require complex data pipelines, continuous model training, validation, and deployment, necessitating the integration of development practices with strong operational practices. MLOps, as the application of DevOps principles to AI/ML systems, provides a formalized approach to automating, monitoring, and controlling these processes while ensuring scalability, reliability, and maintainability. This paper summarizes potential DevOps practices that can support AI/ML processes in contemporary software development and discusses how continuous integration and delivery, infrastructure as code, containerization, automated testing, and version control contribute to successful model lifecycle management. It also highlights the importance of cross-functional cooperation, monitoring, and governance to address challenges such as data and model drift, reproducibility issues, and operational inefficiencies. The study reports key enablers and gaps by integrating current practices and methods in organizations, offering guidance to researchers and practitioners to adopt effective, scalable, and responsible AI/ML systems. The findings reinforce the role of DevOps practices in balancing development and operations to build reliable, data-driven applications in dynamic environments.

References:

[1] M. Steidl, M. Felderer, and R. Ramler, “The pipeline for the continuous development of artificial intelligence models—Current state of research and practice,” J. Syst. Softw., vol. 199, 2023, doi: 10.1016/j.jss.2023.111615.

[2] D. E. Rzig, F. Hassan, and M. Kessentini, “An empirical study on ML DevOps adoption trends, efforts, and benefits analysis,” Inf. Softw. Technol., 2022, doi: 10.1016/j.infsof.2022.107037.

[3] G. Modalavalasa, “The Role of DevOps in Streamlining Software Delivery: Key Practices for Seamless CI/CD,” Int. J. Adv. Res. Sci. Commun. Technol., vol. 1, no. 12, pp. 258–267, Jan. 2021, doi: 10.48175/IJARSCT-8978C.

[4] C. Watson, N. Cooper, D. N. Palacio, K. Moran, and D. Poshyvanyk, “A Systematic Literature Review on the Use of Deep Learning in Software Engineering Research,” ACM Trans. Softw. Eng. Methodol., vol. 31, no. 2, pp. 1–58, Apr. 2022, doi: 10.1145/3485275.

[5] F. H. Alshammari, “Trends in Intelligent and AI-Based Software Engineering Processes: A Deep Learning-Based Software Process Model Recommendation Method,” Comput. Intell. Neurosci., vol. 2022, pp. 1–11, Oct. 2022, doi: 10.1155/2022/1960684.

[6] A. Goyal, “Driving Continuous Improvement in Engineering Projects with AI-Enhanced Agile Testing and Machine Learning,” Int. J. Adv. Res. Sci. Commun. Technol., vol. 3, no. 3, pp. 1320–1331, Jul. 2023, doi: 10.48175/IJARSCT-14000T.

[7] Z. Kotti, R. Galanopoulou, and D. Spinellis, “Machine Learning for Software Engineering: A Tertiary Study,” ACM Comput. Surv., vol. 55, no. 12, 2023, doi: 10.1145/3572905.

[8] A. R. Munappy, J. Bosch, H. H. Olsson, A. Arpteg, and B. Brinne, “Data management for production quality deep learning models: Challenges and solutions,” J. Syst. Softw., vol. 191, p. 111359, Sep. 2022, doi: 10.1016/j.jss.2022.111359.

[9] N. Lokiny, “The Role of AI and Machine Learning in DevOps Automation,” vol. 7, no. 2, pp. 328–333, 2020.

[10] R. Miñón, J. Diaz-De-arcaya, A. I. Torre-Bastida, and P. Hartlieb, “Pangea: An MLOps Tool for Automatically Generating Infrastructure and Deploying Analytic Pipelines in Edge, Fog and Cloud Layers,” Sensors, 2022, doi: 10.3390/s22124425.

[11] A. R. Bilipelli, “End-to-End Predictive Analytics Pipeline of Sales Forecasting in Python for Business Decision Support Systems,” Int. J. Curr. Eng. Technol., vol. 12, no. 6, pp. 819–827, 2022.

[12] G. Sriraman and S. R., “A machine learning approach to predict DevOps readiness and adaptation in a heterogeneous IT environment,” Front. Comput. Sci., vol. 5, Oct. 2023, doi: 10.3389/fcomp.2023.1214722.

[13] S. Tatineni, “Applying DevOps Practices for Quality and Reliability Improvement in Cloud-Based Systems.” 2023. doi: 10.13140/RG.2.2.25688.88326.

[14] A. Goyal, “Optimising Software Lifecycle Management through Predictive Maintenance : Insights and Best Practices,” Int. J. Sci. Res. Arch., vol. 07, no. 02, pp. 693–702, 2022.

[15] F. M. A. Erich, C. Amrit, and M. Daneva, “A qualitative study of DevOps usage in practice,” J. Softw. Evol. Process, vol. 29, no. 6, Jun. 2017, doi: 10.1002/smr.1885.

[16] D. Kreuzberger, N. Kühl, and S. Hirschl, “Machine Learning Operations (MLOps): Overview, Definition, and Architecture,” IEEE Access, vol. 11, pp. 31866–31879, 2023, doi: 10.1109/ACCESS.2023.3262138.

[17] V. Shah, “Next-Gen Emergency Communication Using LowPower Wide-Area and Software-Defined WANS,” Int. J. Adv. Res. Sci. Commun. Technol., vol. 2, no. 1, pp. 600–609, 2022, doi: 10.48175/IJARSCT-8349M.

[18] S. Amershi et al., “Software Engineering for Machine Learning Applications,” Icse, vol. 2020, pp. 1–10, 2019.

[19] Abhishek and P. Khare, “Cloud Security Challenges: Implementing Best Practices for Secure SaaS Application Development,” Int. J. Curr. Eng. Technol., vol. 11, no. 06, pp. 669–676, Nov. 2021, doi: 10.14741/ijcet/v.11.6.11.

[20] L. Faubel, K. Schmid, and H. Eichelberger, “MLOps Challenges in Industry 4.0,” SN Comput. Sci., 2023, doi: 10.1007/s42979-023-02282-2.

[21] G. Modalavalasa, “Towards Sustainable Development Based on Machine Learning Models for Accurate and Efficient Flood Prediction,” vol. 8, no. 2, pp. 940–944, 2021.

[22] R. Amaro, R. Pereira, and M. M. Da Silva, “Capabilities and Practices in DevOps: A Multivocal Literature Review,” IEEE Trans. Softw. Eng., 2023, doi: 10.1109/TSE.2022.3166626.

[23] E. Hechler, M. Oberhofer, and T. Schaeck, Deploying AI in the Enterprise. Berkeley, CA: Apress, 2020. doi: 10.1007/978-1-4842-6206-1.

[24] H. P. Kapadia, “AI Enhanced Web Accessibility Features,” Int. J. Res. Anal. Rev., vol. 8, no. 4, pp. 476–483, 2021.

[25] B. Erdenebat, B. Bud, T. Batsuren, and T. Kozsik, “Multi-Project Multi-Environment Approach—An Enhancement to Existing DevOps and Continuous Integration and Continuous Deployment Tools,” Computers, vol. 12, no. 12, p. 254, Dec. 2023, doi: 10.3390/computers12120254.

[26] P. P. Hanzelik, A. Kummer, and J. Abonyi, “Edge-Computing and Machine-Learning-Based Framework for Software Sensor Development,” Sensors, vol. 22, no. 11, p. 4268, Jun. 2022, doi: 10.3390/s22114268.

[27] N. Jha and R. Popli, “Artificial intelligence for software testing-perspectives and practices,” in Proceedings - 2021 4th International Conference on Computational Intelligence and Communication Technologies, CCICT 2021, 2021. doi: 10.1109/CCICT53244.2021.00075.

[28] S. Shafiq, A. Mashkoor, C. Mayr-Dorn, and A. Egyed, “A Literature Review of Using Machine Learning in Software Development Life Cycle Stages,” IEEE Access, vol. 9, pp. 140896–140920, 2021, doi: 10.1109/ACCESS.2021.3119746.

[29] G. Lorenzoni, P. Alencar, N. Nascimento, and D. Cowan, “Machine Learning Model Development from a Software Engineering Perspective: A Systematic Literature Review,” arvix, 2021.

[30] E. Nascimento, A. Nguyen-Duc, I. Sundbø, and T. Conte, “Software engineering for artificial intelligence and machine learning software: A systematic literature review,” arvix.org, 2020, doi: /10.48550/arXiv.2011.03751.

[31] M. Barenkamp, J. Rebstadt, and O. Thomas, “Applications of AI in Classical Software Engineering,” AI Perspect., vol. 2, no. 1, Dec. 2020, doi: 10.1186/s42467-020-00005-4.

Keywords:

DevOps, MLOps, Continuous Integration, Automation, Software Development, Artificial Intelligence, DevOps Practices.