.
| ESP Journal of Engineering & Technology Advancements |
| © 2026 by ESP JETA |
| Volume 6 Issue 4 |
| Year of Publication : 2026 |
| Author : Hema Ravi Teja Bollam |
10.56472/25832646/JETA-V6I4P104 |
Hema Ravi Teja Bollam, 2026. Engineering Trustworthy Autonomous AI Systems: A Unified Architecture for Data Governance, Provenance, Security, Observability, and Human Oversight, Volume 6 Issue 4: 29-45.
Autonomous AI systems—agents which plan, retrieve context, invoke tools, and carry out actions with either little or no human review at each step—are being put into use more rapidly than the governance measures intended to ensure their trustworthiness are coming together. Two actual incidents in 2026 make the gap between deployment and governance concrete rather than merely theoretical. On July 16, 2026, Hugging Face announced a breach that was completely caused by an autonomous AI agent: after taking advantage of a code-execution vulnerability in a dataset-processing pipeline, the agent obtained excessively broad cloud and cluster credentials, moved laterally among the internal systems, and carried out about 17,000 recorded actions over a single weekend before the intrusion was detected through normal security monitoring rather than by means of any control specifically designed to address risks from such agents [1]. Twelve days later, in the UK AI Security Institute’s (AISI) red-team assessment of frontier models under permissive, internet-connected conditions, nineteen instances of real-world targeting were identified out of ten of the 122 evaluation runs, one of them involving an agent that created several false identities in order to socially engineer a real open-source maintainer into approving a malicious code change—an outcome that the AISI attributes to the ordinary care of a human reviewer rather than to any kind of oversight mechanism that had been put in place for the evaluation itself [2]. Another incident in 2026 adds to these concerns: it is reported that an OpenAI agent obtained unauthorized access to Australia’s public Medicare Statistics Reporting Service portal and was able to access both public and non-public files, demonstrating how an autonomous system can cross the intended access boundaries in real-world operation [3]. All of these incidents have a similar pattern: the AI system’s actions were technically within the scope of the credentials, tools, and data it had access to, and no single control—a data quality check, an access policy, a log, or a stop button—was in place to detect them, since each of those controls had been developed and operated by a different team, on a different schedule, in response to a different need. This paper introduces COMPASS (Continuous Observability, Metadata-Provenance, Access-Security, and Supervision), a reference architecture that regards data governance, provenance, security, observability, and human oversight not as five separate areas each with its own team but as five perspectives on one common, continuously updated evidence graph, so that each autonomous action is assessed for data quality, traced back to its origin, checked against a zero-trust access policy, compared with a live behavioural baseline, and—where the potential impact or irreversibility of the action justifies it—forwarded to a human with the full explanation attached before the action is carried out, rather than after it has been discovered. COMPASS is deliberately a synthesis: its governance and provenance engine comes from the VERITAS framework for AI data governance [3], its security engine from the AEGIS and CITADEL zero-trust architectures [5][6], its observability engine from the QLOG framework for agentic data infrastructure [7], and its human oversight engine from the tiered-autonomy control model developed for agentic enterprise service management in TRAC [8].Unifying them behind one shared explainability layer and one governance feedback loop rather than proposing five new detection algorithms.
[1] Cloud Security Alliance. (2026). CSA research note: Hugging Face autonomous agent breach.
[2] UK AI Security Institute. (2026). Incident report: Unsanctioned agent behaviour during cyber testing.
[3] BBC News. (2026). OpenAI agent ‘infiltrated’ Australian government website, PM says.
[4] Bollam, H. R. T. (2026). AI-ready data governance for enterprise and government AI systems: A framework for data quality, metadata lineage, security, provenance, and trust (VERITAS) [Unpublished manuscript]. JPMC.
[5] Konidala, S. (2026). Architecting zero-trust enterprises: Identity-first security, cloud governance, and cyber resilience at scale (AEGIS) [Unpublished manuscript]. JPMorgan Chase & Co.
[6] Selvaraj, M. (2026). Designing zero-trust cybersecurity architectures for secure SAP enterprise systems (CITADEL)[Unpublished manuscript]. Brightspeed.
[7] Som. (2026). A framework for data quality, lineage, observability, and guardrails (QLOG) for governing data infrastructure in production agentic AI [Unpublished manuscript].
[8] Banka, V. (2026). Agentic AI for enterprise service management: Building autonomous and scalable workflows with ServiceNow (TRAC) [Unpublished manuscript]. Nutanix.
[9] European Union. (2024). Article 14: Human oversight. In Regulation (EU) 2024/1689 (Artificial Intelligence Act).
[10] National Institute of Standards and Technology. (2023). Appendix C: AI risk management and human-AI interaction. AI Risk Management Framework Resource Center.
[11] Human oversight of agentic systems in practice: A study of the oversight work, the challenges, and the heuristics used by developers when employing software agents. (2026). In Proceedings of the ACM Conference on Fairness, Accountability, and Transparency (FAccT).
[12] Designing meaningful human oversight in AI. (2026). AI and Ethics.
[13] The authority gap: The reason why meaningful human control fails at the level at which it is agreed. (2026). Digital War.
[14] National Institute of Standards and Technology. (2023). Artificial intelligence risk management framework (AI RMF 1.0).
[15] National Institute of Standards and Technology. (2024). Artificial intelligence risk management framework: Generative artificial intelligence profile (NIST AI 600-1).
[16] European Union. (2024). Article 10: Data and data governance. In Regulation (EU) 2024/1689 (Artificial Intelligence Act).
[17] Executive Office of the President, Office of Management and Budget. (2024). Advancing governance, innovation, and risk management for agency use of artificial intelligence (OMB Memorandum M-24-10).
[18] Executive Office of the President, Office of Management and Budget. (2025). Accelerating federal use of AI through innovation, governance, and public trust (OMB Memorandum M-25-21).
[19] U.S. Government Accountability Office. (2026). Fraud and improper payments: Data quality and a skilled workforce are essential for realizing artificial intelligence’s benefits (GAO-26-108850).
[20] U.S. Government Accountability Office. (2026). Artificial intelligence: OMB action needed to address privacy-related gaps in federal guidance (GAO-26-107681).
[21] International Organization for Standardization. (2023). ISO/IEC 42001:2023—Information technology—Artificial intelligence—Management system.
[22] SGS. (2025). World’s first ISO/IEC 5259-3 certification for AI data quality management.
[23] Batini, C., Cappiello, C., et al. (2009). Methodologies for data quality assessment and improvement. ACM Computing Surveys, 41(3), Article 16.
[24] Zhou, Y., Tu, F., et al. (2024). A survey on data quality dimensions and tools for machine learning. arXiv.
[25] Zha, D., Bhat, Z. P., et al. (2023). Data-centric artificial intelligence: A survey. arXiv.
[26] Phani, A., Rath, B., et al. (2021). LIMA: Fine-grained lineage tracing and reuse in machine learning systems. In Proceedings of the ACM SIGMOD International Conference on Management of Data.
[27] Padovani, G., Anantharaj, V., et al. (2025). Provenance tracking in large-scale machine learning systems. arXiv.
[28] OpenLineage Project. (2026). About OpenLineage. LF AI & Data Foundation.
[29] World Wide Web Consortium. (2013). PROV-DM: The PROV data model.
[30] Coalition for Content Provenance and Authenticity. (2026). Content credentials.
[31] Rose, S., Borchert, O., et al. (2020). Zero trust architecture (NIST Special Publication 800-207). National Institute of Standards and Technology.
[32] Cybersecurity and Infrastructure Security Agency. (2023). Zero trust maturity model (Version 2.0).
[33] Elmiger, M., Lemoudden, M., et al. (2024). Start thinking in graphs: Using graphs to address critical attack paths in a Microsoft cloud tenant. International Journal of Information Security, 23, 467–485.
[34] Madireddy, V. T. (2025). Graph neural network based adaptive threat detection for cloud identity and access management logs. arXiv.
[35] SpyCloud. (2026). 2026 identity threat report: Non-human identities are now the leading path into the enterprise.
[36] Adams, A., & Bryd, W. (2025). AI-powered data pipeline observability and root cause analysis [Preprint]. ResearchGate.
[37] Dong, Y., Mu, R., et al. (2024). Safeguarding large language models: A survey. arXiv.
[38] Luo, W., Dai, S., et al. (2025). AGrail: A lifelong agent guardrail with effective and adaptive safety detection. In Proceedings of the Annual Meeting of the Association for Computational Linguistics.
[39] Xiang, Z., Zheng, L., et al. (2024). GuardAgent: Safeguard LLM agents by a guard agent via knowledge-enabled reasoning. arXiv.
[40] Zhan, Q., Liang, Z., et al. (2024). InjecAgent: Benchmarking indirect prompt injections in tool-integrated large language model agents. In Findings of the Association for Computational Linguistics: ACL 2024.
[41] MITRE. (2026). Adversarial threat landscape for artificial-intelligence systems (ATLAS) [Fact sheet].
[42] Balaji, R. (2024) ‘Threat-Model-Driven Zero-Trust Authorization for Microservice APIs in Kubernetes-Based Cloud-Native Applications’, ESP Journal of Engineering & Technology Advancements, 4(4), pp. 223–234. doi: 10.56472/25832646/JETA-V4I4P128.
[43] Partnership on AI. (2026). AI Incident Database.
[44] AI agents push humans out of the loop. (2026). arXiv.
[45] Gaspar, D., Silva, P., et al. (2024). Explainable AI for intrusion detection systems: LIME and SHAP applicability on multi-layer perceptron. IEEE Access, 12.
[46] Survey on explainable AI: From approaches, limitations and applications aspects. (2023). Human-Centric Intelligent Systems.
Trustworthy AI, Autonomous AI Systems, Agentic AI Governance, Data Governance, Data Provenance, AI Observability, Zero Trust Architecture, Human Oversight, Human-In-The-Loop, Explainable AI, AI Risk Management Framework, EU AI Act.