검색 상세

Bounded Autonomy of Enterprise AI Agents : A Governance-centric Design Study

  • 박영진 (YoungJin Park, 아주대학교 일반대학원)

초록/요약

The diffusion of large language model （LLM）-based AI agents is expanding the autonomy and operational scope of AI systems in enterprise environments. However, existing research has focused primarily on AI performance, automation levels, and explainability, while the question of how AI agent autonomy should be governed and controlled within organizational contexts has received comparatively little attention. Enterprise environments require security, accountability, data control, human oversight, and operational stability to be addressed concurrently. Therefore, a governance-centered approach grounded in bounded autonomy, rather than full autonomy, is necessary. To this end, this study integrates the theoretical perspectives of bounded autonomy, sociotechnical envelopment, and human-AI delegation, conducting a two-study Design Science Research （DSR） program that combines analysis of an actual enterprise AI agent deployment with evaluation by twelve practitioners. Study 1 identified governance artifacts based on a Salesforce Agentforce deployment, and Study 2 applied Gioia methodology and DEMATEL to validate the practical relevance of these artifacts and to examine the causal relationships and structural priorities among them. The results show that enterprise AI agents function not as fully autonomous systems but as sociotechnical actors operating within a governance-inscribed bounded autonomy structure. Governance artifacts, including Prompt Governance, Permission Control, Retrieval Governance, Auditability, Human-in-the-Loop （HITL）, and Output Validation, did not operate in isolation but formed an interconnected causal structure, within which antecedent mechanisms such as permission control and input validation had a significant effect on organizational trustworthiness and operational stability. This study yields three interrelated outputs. First, Study 1 and Round 1 of Study 2 （Gioia methodology） produced five governance design principles （DP1 through DP5） as prescriptive knowledge specifying what governance an enterprise AI agent must embed. Second, the same data yielded the Recursive Trust Calibration （RTC） Model, a four-stage cyclical process showing how these design principles operate in sequence, from explainability to bounded autonomy, organizational conformance, and continuous verification, with the final stage recursively recalibrating the first. Third, Round 2 of Study 2 （DEMATEL） established the causal priorities among governance artifacts, indicating which mechanisms must be established first for the others to function effectively. This study contributes theoretically by reconceptualizing enterprise AI agent autonomy from a governance-centered perspective of bounded autonomy, as opposed to a technical capability-centered view, and by extending Lee and See's （2004） concept of calibrated trust from the individual operator level to a structural, recursive process at the organizational governance level through the RTC Model. It also advances design knowledge for enterprise AI governance by integrating human- AI delegation structures with a sociotechnical governance perspective, and contributes to the literature by conducting DSR using cases from a commercial AI platform. From a practical standpoint, this study demonstrates the need for a governance-first approach in the design and operation of enterprise AI agents, and provides concrete design principles, a process model of trust formation, and priority- based governance guidelines for building accountable and controllable AI agent systems.

more

목차

1. Introduction 1
2. Theoretical background 6
2.1. AI Agents and the Challenge of Autonomy in Enterprise Contexts 6
2.2. Bounded Rationality and Bounded Autonomy 8
2.3. Sociotechnical Envelopment and Governance-by-Design 9
2.4. Human-AI Delegation and Governance 11
2.5. Integrated Theoretical Perspective for Enterprise AI Agent Governance 14
3. Research Model 15
3.1. Research Design Overview 18
3.2. Study 1: Field Case Study of an Enterprise AI Agent Deployment 19
3.3. Study 2: Expert-Based Evaluation and Refinement of Governance Artifacts 21
3.4. Integration of Study 1 and Study 2. 27
3.5. Research Rigor and Validity 28
3.6. Summary of Research Method 29
4. Research Result 30
4.1. Qualitative Field Study: Field Case and Agentforce KPI Evidence. 31
4.1.1. Field Case Context and System Overview 31
4.1.2. Governance Artifacts for Bounded Autonomy 32
4.1.3. Governance-aligned Research Variables 34
4.1.4. KPI-Based Field Evidence 36
4.2. Emergent Governance Mechanisms from Gioia-Based Qualitative Analysis 39
4.2.1. Qualitative Data Structure Based on the Gioia Methodology and Derivation of Emergent Governance Dimensions 39
4.2.2. Alignment Analysis Between Established Field-Derived Artifact and Emergent Practitioner-Derived Artifact 42
4.2.3. Newly Emerged Governance Artifacts Derived from Operational Environment Contexts 46
4.2.4. Deriving Governance Design Principles for Bounded Autonomy in Enterprise AI Agents 49
4.3. DEMATEL Results and Causal Structure 54
4.3.1. Cause Factors 56
4.3.2. Effect Factors 58
4.3.3. Managerial Implications 59
4.4. Integrated Interpretation of Research Results from a DSR Perspective 60
5. Discussion 63
5.1. Theoretical Implications 64
5.1.1. From Autonomous AI to Governance-Embedded Bounded Autonomy 64
5.1.2. Extending Human-AI Delegation Perspectives to Enterprise AI Agents 65
5.1.3. Enterprise AI Governance as a Sociotechnical Governance Ecosystem 66
5.1.4. From Technical Architecture to Design Knowledge in DSR 67
5.1.5. From Governance Patterns to a Process Theory of Trust 68
5.1.6. Contingency Conditions in Applying the Governance Framework 70
5.2. Practical Implications 71
5.3. Limitations and future research 74
6. Conclusion 76
References 79

more
반출 Meta View 목록