Introduction
AI Governance is becoming an important issue as businesses move from simple AI tools to systems that can plan tasks, use software, make recommendations, and take actions with limited human involvement. These systems are often described as autonomous or agentic AI systems.
An autonomous AI system may interact with customers, analyse information, use business applications, initiate workflows, or make decisions based on defined rules. Because the system can act with greater independence, businesses need clear controls around responsibility, data, security, contracts, and human oversight.
India is actively developing its AI governance framework. In April 2026, MeitY constituted the AI Governance and Economic Group (AIGEG). Its terms of reference include coordinating AI policy across ministries and regulators, reviewing mechanisms to hold firms accountable for compliance with local laws, studying emerging AI risks and regulatory gaps, and developing India’s approach to AI governance.
IndiaAI’s responsible-AI work also identifies human oversight, responsibility, accountability, transparency, privacy, safety, monitoring, due diligence, and auditability as important governance considerations.
At the same time, India’s current framework does not treat an AI system as an independent legal person. Existing legal responsibilities continue to matter, while policymakers are examining how those laws should adapt to increasingly autonomous systems. India’s earlier AI governance material specifically identifies accountability for AI decisions as a challenge and discusses the need to consider how existing legal frameworks should respond to autonomous AI.
This guide explains AI Governance for Autonomous AI Systems in India, including accountability, human oversight, data protection, cybersecurity, AI contracts, intellectual property, decision-making, monitoring, incident response, and important 2026 considerations.
Why AI Governance Matters
Autonomous AI systems can perform tasks with less direct human intervention. This creates new questions about who controls the system and who is responsible for its actions.
For example, an AI system may:
- Approve or reject a workflow.
- Select a supplier.
- Respond to customers.
- Analyse applications.
- Generate business documents.
- Access company systems.
- Trigger automated actions.
- Make recommendations that influence important decisions.
Good AI Governance can help businesses:
- Define clear responsibility.
- Set limits on AI actions.
- Maintain human oversight.
- Protect personal and confidential data.
- Manage cybersecurity risks.
- Review third-party AI providers.
- Monitor AI decisions.
- Maintain proper records.
- Respond to AI-related incidents.
- Demonstrate responsible AI practices.
IndiaAI’s responsible-AI materials specifically connect accountability with human oversight, impact assessment, monitoring, due diligence, and auditability.
Key Areas of AI Governance
1. Identify Autonomous AI Systems
The first step in AI Governance is to identify where autonomous or agentic AI is being used.
Businesses should create an inventory of:
- AI agents.
- Generative AI systems.
- Automated decision systems.
- AI-powered workflows.
- AI assistants.
- Third-party AI services.
- AI APIs.
- Software agents.
For each system, businesses should record its purpose, owner, data access, connected systems, level of autonomy, and potential business impact.
This makes it easier to understand where AI creates material legal or operational risk.
2. Define the Scope of AI Authority
An autonomous AI system should have clearly defined limits.
Businesses should specify:
- What the system can do.
- What information it can access.
- Which systems it can use.
- Which actions it can take.
- Which actions require approval.
- What spending limits apply.
- When the system must stop.
For example, an AI agent may be allowed to prepare a purchase order but may require human approval before making the payment.
Clear authority limits can reduce the risk of unwanted or unauthorised actions.
3. Human Oversight and Intervention
Human oversight is a central part of responsible AI Governance.
Businesses should identify situations where a human must review or approve an AI action.
Human intervention may be appropriate for:
- High-value transactions.
- Employment decisions.
- Financial decisions.
- Legal decisions.
- Sensitive customer matters.
- Security incidents.
- High-impact automated decisions.
India’s responsible-AI framework identifies human oversight and determination as a key principle and connects it with legal responsibility.
Businesses should also provide a practical way to stop or override an AI system when necessary.
4. Accountability for AI Decisions
An AI system may involve many people and technologies.
Responsibility can involve:
- The business owner.
- The AI developer.
- The technology vendor.
- The system integrator.
- The data provider.
- The person who approved deployment.
- The person who monitors the system.
Businesses should therefore maintain clear responsibility at each stage.
India’s AI governance discussions have specifically identified accountability for AI decisions as a challenge because decisions can be influenced by many stages of an AI system’s lifecycle.
A clear accountability structure can make it easier to investigate failures and respond to complaints.
5. AI Decision-Making
Autonomous AI can influence important business decisions.
Examples include:
- Credit assessments.
- Recruitment.
- Fraud detection.
- Customer support.
- Insurance assessments.
- Product recommendations.
- Business approvals.
Companies should assess whether an AI decision could significantly affect a person or business.
They should also consider:
- Accuracy.
- Bias.
- Explainability.
- Human review.
- Appeals.
- Error correction.
India’s responsible-AI work highlights concerns around opaque decision-making, bias, incorrect outcomes, accountability, and the difficulty of auditing AI systems.
6. Data Protection and Privacy
Autonomous AI systems may process large amounts of information.
This can include:
- Customer data.
- Employee data.
- Financial information.
- Business records.
- Personal information.
- Confidential documents.
Businesses should know:
- What data the AI can access.
- Why the data is being used.
- How long it is retained.
- Who can access it.
- Whether a third-party AI provider receives it.
MeitY’s current policy pages include the Digital Personal Data Protection Rules, 2025, which form part of India’s developing data-protection framework.
Businesses should therefore review autonomous AI systems that process personal data against the requirements applicable to their activities.
7. Cybersecurity and AI System Security
Autonomous systems can create new cybersecurity concerns because they may connect to tools, databases, APIs, and business systems.
Security controls should cover:
- Authentication.
- Authorisation.
- API security.
- Credential management.
- System isolation.
- Logging.
- Vulnerability management.
- Monitoring.
Businesses should also consider what happens if an AI agent is compromised.
An attacker who gains control of an autonomous AI system could potentially influence its actions or access connected systems.
8. AI Agents and Connected Systems
An autonomous AI system may not operate alone.
It may connect to:
- Email.
- Databases.
- CRM systems.
- Payment platforms.
- Cloud services.
- Internal applications.
- External APIs.
Businesses should apply the principle of least privilege.
An AI agent should receive only the permissions it needs.
Companies should also review whether each connection is necessary and whether the action can be reversed when something goes wrong.
9. AI Contracts and Vendor Responsibility
Many businesses use third-party AI providers.
Contracts should clearly address:
- AI system functionality.
- Data processing.
- Security.
- Confidentiality.
- Intellectual property.
- Service levels.
- Incident notification.
- Audit rights.
- Liability.
- Indemnity.
- Business continuity.
- Data deletion.
- Termination.
Businesses should avoid relying only on the vendor’s standard terms.
The contract should explain who is responsible when an autonomous AI system produces an incorrect or harmful result.
10. Intellectual Property and AI Outputs
Autonomous AI systems may generate:
- Text.
- Images.
- Software.
- Reports.
- Designs.
- Marketing content.
- Business documents.
Businesses should determine how intellectual-property rights are handled.
The review should consider:
- Input data.
- Third-party content.
- Software licences.
- AI outputs.
- Ownership terms.
- Confidential information.
- Open-source components.
This becomes especially important when the AI is integrated into products or commercial services.
11. Transparency and Explainability
Users may need to know when AI is making or influencing an important decision.
Businesses should consider:
- AI disclosures.
- User notices.
- Decision explanations.
- Model limitations.
- Internal documentation.
- Escalation procedures.
India’s responsible-AI framework identifies transparency and explainability as important principles. It notes that people may need to understand when decisions are based on AI and may need access to explanations appropriate to the context.
Clear communication can also help businesses manage customer and employee expectations.
12. AI Monitoring and Audit Trails
Autonomous AI should be monitored after deployment.
Companies should track:
- AI actions.
- User instructions.
- System decisions.
- Errors.
- Exceptions.
- Security events.
- Human approvals.
- System changes.
A proper audit trail can help the business understand what happened when an AI system produces an unexpected result.
India’s responsible-AI materials specifically identify monitoring, due diligence, and auditability as important accountability mechanisms.
13. Incident Response and AI Failures
Companies should prepare for AI-related incidents.
Examples include:
- Incorrect automated decisions.
- Unauthorised actions.
- Data leakage.
- Security compromise.
- Harmful AI outputs.
- Unapproved system changes.
- Vendor failures.
An AI incident-response plan should explain:
- Who receives the report.
- Who can stop the system.
- How evidence is preserved.
- How affected users are handled.
- When management is informed.
- Whether legal or regulatory reporting applies.
- How the system is restored.
14. AI Governance Policies
Companies should establish written policies for autonomous AI systems.
A policy can address:
- Approved AI uses.
- Restricted uses.
- Prohibited uses.
- Human-approval requirements.
- Data-access rules.
- Security controls.
- Vendor requirements.
- Monitoring.
- Incident reporting.
- Employee responsibilities.
A policy also helps create consistency when different business teams use AI systems.
15. AI Governance Across the Business
AI governance should not sit with one team alone.
The following functions may need to work together:
- Legal.
- Compliance.
- Technology.
- Cybersecurity.
- Data protection.
- Procurement.
- Risk management.
- Business teams.
A cross-functional structure can help businesses identify legal, technical, financial, and operational risks before deployment.
MeitY’s 2026 AIGEG terms of reference similarly emphasise coordination across ministries, departments, and sectoral regulators and oversight of cross-sector AI governance issues.
Common AI Governance Risks
Businesses may face AI Governance risks due to:
- Unclear AI ownership.
- Excessive AI authority.
- Weak human oversight.
- Unauthorised AI actions.
- Poor data controls.
- Inadequate cybersecurity.
- Unclear vendor responsibility.
- Weak AI contracts.
- Poor audit trails.
- Unexplained AI decisions.
- Bias or unfair outcomes.
- Insufficient monitoring.
- Incomplete AI policies.
- Poor incident-response processes.
- Failure to review AI systems after major changes.
These risks can create legal, financial, operational, privacy, cybersecurity, and reputational problems.
Best Practices for AI Governance
Indian businesses should consider the following practices:
- Maintain an inventory of autonomous AI systems.
- Classify systems according to risk.
- Define clear AI authority and limits.
- Require human approval for higher-risk actions.
- Apply least-privilege access.
- Protect personal and confidential data.
- Review AI vendors and contracts.
- Test AI systems before deployment.
- Monitor AI activity continuously.
- Maintain detailed audit trails.
- Establish AI incident-response procedures.
- Review AI-generated outputs.
- Update AI policies regularly.
- Train employees on responsible AI use.
- Conduct periodic AI governance reviews.
The aim should be simple: know what the AI can do, control what it is allowed to do, monitor its actions, and keep clear responsibility with human and legal entities.
2026 AI Governance Considerations
AI Governance is particularly relevant in 2026 because India is actively building structures for AI governance while businesses are using increasingly capable AI systems.
In April 2026, MeitY constituted the AI Governance and Economic Group (AIGEG). Its responsibilities include reviewing existing mechanisms, issuing guidelines to ensure firms are accountable for compliance with local laws, studying emerging AI risks and regulatory gaps, and overseeing AI governance initiatives across the public and private sectors.
MeitY’s 2026 documents also continue to emphasise responsible AI principles, governance, standards, monitoring, and applicable national and international guidelines.
Businesses should therefore focus on:
- AI system inventories.
- Risk classification.
- Human oversight.
- Decision accountability.
- Data protection.
- Cybersecurity.
- Vendor management.
- AI contracts.
- Monitoring and auditability.
- Incident response.
- AI policy development.
- Regulatory-change monitoring.
The legal position around autonomous AI liability is still developing. Earlier Indian AI governance work noted that existing laws address many AI-related harms but may need adaptation for AI-specific circumstances and clearer accountability mechanisms.
Businesses should therefore avoid assuming that an autonomous AI system can carry legal responsibility on its own. Instead, organisations should identify the human, corporate, contractual, and operational responsibilities surrounding the system.
How Derecho Consulting Can Help
Derecho Consulting can help Indian businesses build AI Governance frameworks for autonomous and agentic AI systems through AI governance assessments, policy development, legal-risk reviews, vendor due diligence, technology-contract reviews, data-protection assessments, cybersecurity governance, accountability frameworks, and AI compliance documentation.
A proactive approach can help businesses:
- Identify autonomous AI systems.
- Define AI authority and limits.
- Establish human-oversight processes.
- Review AI vendor contracts.
- Assess privacy and data risks.
- Strengthen AI policies.
- Develop audit and monitoring processes.
- Establish AI incident-response procedures.
- Review accountability structures.
- Prepare for regulatory developments.
Derecho Consulting can also support periodic reviews as AI technology and India’s regulatory framework continue to evolve.
Conclusion
Legal Governance of Autonomous AI Systems in India is becoming an important issue as AI systems move from providing information to performing increasingly complex tasks.
Autonomous AI can support business efficiency and innovation. However, it can also create new questions about authority, accountability, privacy, cybersecurity, contracts, intellectual property, and decision-making.
A strong AI Governance framework should clearly define what an AI system can do, what information it can access, when human approval is required, and who is responsible for its actions.
By maintaining AI inventories, defining risk levels, limiting system permissions, protecting data, reviewing vendors, monitoring AI activity, and maintaining clear audit trails, businesses can build stronger controls around autonomous systems.
India’s establishment of the AIGEG in 2026 shows that AI governance is becoming an active policy area, while the country’s responsible-AI work continues to emphasise accountability, human oversight, transparency, privacy, safety, monitoring, due diligence, and auditability.
A proactive approach to AI Governance can help Indian businesses manage autonomous AI systems more responsibly, reduce legal and operational risks, and support sustainable AI adoption in 2026 and beyond.