← Courses
AI Project Scaffolding
← Module 2
Module 3 of 8
Module 4 →
Intro
Scenario
Lesson
Context
Lab Debate ~25 min
Intro

Should the Machine Own This?

2 min read

Every scaffolding decision is also a question about authority. When you write a CLAUDE.md that says "AI can commit directly to feature branches," you've decided the machine owns that decision. When you write "all commits require human review before merge," you've kept that ownership with people.

These aren't technical choices. They're governance choices. They determine what the AI is authorized to do without asking, what requires human approval, and what should never be delegated at all. Get them wrong and the scaffolding works technically while failing organizationally — the AI ships things it shouldn't, or humans can't effectively review things it does.

This module puts those decisions in front of you explicitly. Three real scenarios. Three cases where the right answer isn't obvious. You'll take a position, defend it, and have it challenged — not to find the right answer, but to develop the judgment that produces a defensible one.

Portfolio artifact
Debate
A written defense of AI autonomy boundaries — three position papers arguing where AI should and shouldn't act without human approval, grounded in EU AI Act Art.14 and NIST GOVERN requirements.
  • Apply EU AI Act Article 14 human oversight requirements to scaffolding decisions
  • Distinguish between AI actions that are low-stakes-reversible and those that require human approval
  • Argue a clear position on AI autonomy boundaries under pressure
  • Identify the conditions under which "AI can do this" becomes an acceptable answer
  • Write workflow rules that are grounded in governance principles, not just convenience
Scenario

The Autonomy Meeting

3 min read

A startup engineering team is writing their AI governance policy. They've been using AI coding tools for eight months. The question on the table is straightforward to ask and hard to answer: what should the AI be allowed to do on its own?

The team lead wants to be specific. Not a list of principles — actual rules that would go in the CLAUDE.md and be enforced. The debate has split into three camps.

One engineer argues for maximum autonomy. The AI should be able to create files, write tests, commit to feature branches, and open PRs without asking. Every interruption slows down the work. The overhead of constant approval is the reason AI tools don't deliver on their potential. "If we don't trust it enough to let it work, why are we using it?"

Another engineer argues for strict oversight. Every AI output should be reviewed before it enters the codebase. No exceptions. The AI is a tool, not a colleague. "You wouldn't merge a PR without reading it. Why would you let the AI commit without reading it?"

A third engineer is trying to find a middle path, but can't articulate the principle that would let them say "yes" to some actions and "no" to others. The conversation keeps looping. "It depends" is not a governance policy.

The team needs a principled framework — one that can draw a line, explain why the line is there, and hold up when someone challenges it. That framework is what this module teaches.

Lesson

The Reversibility Framework

3 min read

The question is not "do we trust the AI?" It's "what happens when it's wrong?" Every decision about AI autonomy is really a decision about error recovery. The right framework is about reversibility, scope, and accountability — not capability.

Can the action be undone cleanly if the AI made the wrong call? A file created in a feature branch can be deleted. A commit pushed to origin can be reverted. A database migration run in production cannot be trivially undone. A message sent to a customer cannot be unsent. The more irreversible the action, the stronger the argument for human approval before it happens.

How many things does this action affect, and how visibly? Writing a new utility function affects one file and is easily reviewed. Refactoring the authentication middleware affects every authenticated route and requires careful review by someone who understands the full system. High-scope actions require human judgment that the AI lacks — it can execute the refactor but can't fully model the blast radius.

If something goes wrong, is there a record of what the AI was told, what it did, and who approved it? An AI commit with no review creates an accountability gap — the human who ran the session is responsible for the output, but there's no record of what instructions were given or what decisions the AI made. Accountability surface is not about blame — it's about learning and preventing recurrence.

EU AI Act — Article 14: Human Oversight

Article 14 requires that high-risk AI systems enable human oversight — specifically that humans can understand what the system is doing, intervene where necessary, and that the system doesn't prevent override. For developer tools, "high-risk" is context-dependent, but the principle applies: scaffolding that makes AI output invisible to review violates the spirit of Art.14. Workflow rules that require human review before merge are a practical implementation of this requirement.

NIST AI RMF — GOVERN Function

GOVERN requires that AI use is bounded by policies that define accountability. The autonomy question is a GOVERN question: who is accountable when AI output causes a problem? If the answer is "whoever ran the session" without any documented policy, accountability is ambiguous. Clear AI autonomy rules in CLAUDE.md make accountability explicit before something goes wrong.

EU AI Act — Article 17: Quality Management

Article 17 requires post-market monitoring for high-risk systems — meaning organizations must be able to detect when AI output quality degrades. Without human review checkpoints, quality degradation is invisible until a problem surfaces in production. Regular human review is not just governance overhead — it's quality signal that the AI cannot generate for itself.

Context

Drawing the Line

2 min read

Three tests that resolve most autonomy debates. You'll apply them to each scenario in the lab.

The Worst-Case Test

If this AI action produces the worst plausible output, what happens? Describe it specifically. If the worst case is "we have to rewrite the function" — that's recoverable, autonomy is probably acceptable. If the worst case is "the auth system has a silent vulnerability in production for a week before anyone notices" — autonomy is not acceptable for this action.

The Visibility Test

Will a human see this output before it has real-world effects? Reading a PR diff counts. Watching the CI run counts. If the output goes directly from AI to production with no human in the loop at any point, the visibility test fails — regardless of how confident you are in the AI's quality on this type of task.

The Explanation Test

If someone asks "why did this happen?", is there an audit trail that explains the AI's instructions and reasoning? Not for every line of code — for every significant decision. If there's no record, accountability is impossible and learning from mistakes requires memory that organizations don't have.

You'll apply all three tests to each of the three scenarios in the lab and defend a position on where the autonomy line should be drawn.

⚔ Debate Lab
Autonomy Boundaries
~25 minutes · 3 scenarios
What you're doing
Three real scenarios where a team must decide how much autonomy to grant AI. For each: take a position, defend it against challenge, and specify the exact workflow rule you'd write in CLAUDE.md.
Roles
You — Decision-MakerTake a position on each autonomy question. Your reasoning must be grounded in the three tests and governance requirements.
🎯
AI — ChallengerI'll push back on every position — both sides. Too permissive? I'll describe the failure mode. Too restrictive? I'll show the cost.
Three scenarios
AI auto-committing tests · AI modifying shared utilities · AI opening PRs without description review
Apply to each scenario
Worst-case test — describe the specific failure
Visibility test — will a human see this before real-world effect?
Explanation test — is there an audit trail?
EU AI Act Art.14 — is human oversight preserved?
NIST GOVERN — is accountability documented?
Shift + Enter for a new line
✓ Module Complete
You've completed Module 3 of 8. Your autonomy position papers are your third portfolio artifact.
Next Module →