AI Code Review and Quality Management Strategy: AI-Assisted Development Guidelines from the Amazon Case

• Introduction
• Analyzing the Quality Risks of AI Code
  • 1. Edge Case Omission
  • 2. Security Blind Spots
  • 3. Performance Anti-patterns
  • 4. Context Ignorance
  • 5. Hallucinated Code
• The Amazon AI Code Review Policy
  • Policy Background: The March 5, 2026 Outage
  • Core Policy Content
  • How the Policy Is Implemented
• AI Code Review Checklist
  • Functional Correctness
• Comparing AI Code Review Tools
• Building an AI Code Review Pipeline
  • Architecture Overview
  • GitHub Actions Pipeline Configuration
  • Custom AI Review Script
• Semantic Diff Analysis
• Integrating CI/CD Quality Gates
  • Multi-stage Quality Gate Architecture
  • AI Hallucination Detection Script
• Metric-driven Quality Management
  • Core Metric Definitions
  • Prometheus Metric Collection
  • Grafana Dashboard Configuration
• Failure Cases and Lessons
  • Case 1: Cascade Failure from Over-reliance on AI (Amazon, March 2026)
  • Case 2: A Security Vulnerability Generated by AI
  • Case 3: License Contamination
• AI Code Governance at the Organization Level
  • Roles and Responsibilities Matrix (RACI)
  • Phased Adoption Roadmap
• Linking with DORA Metrics
• Recovery Procedure (AI Code Related Incidents)
• Conclusion
• References

Introduction

On March 5, 2026, the Amazon shopping service suffered a large-scale outage that lasted about six hours. The post-mortem traced the cause to an edge case bug that went undetected in code generated by an AI coding assistant. The incident sent a shock through the entire industry, and Amazon announced a new policy immediately: code that junior and mid-level engineers generate or modify with an AI assistant must obtain a senior engineer sign-off before it is deployed to production.

The incident ran as the top story on Hacker News and set off an industry-wide discussion about quality management in AI-assisted development. In 2026, with adoption of AI coding tools such as GitHub Copilot, Amazon CodeWhisperer and Cursor past 80%, we have to answer a fundamental question: how can we trust code that an AI generated?

This article takes the Amazon policy change as its starting point, analyzes the quality risks of AI code systematically, and covers building a review pipeline at the organization level, integrating CI/CD quality gates, and practical operational strategy.

Analyzing the Quality Risks of AI Code

Code generated by an AI coding assistant can look perfectly normal on the surface while carrying subtle defects. This is called the "plausible but subtly wrong" pattern, and it is hard to catch in an existing code review process.

1. Edge Case Omission

An AI model generates the "most plausible" code based on the distribution of its training data. It therefore handles the happy path well, but frequently omits handling for boundary conditions and exceptional situations.

# AI-generated code: works fine in the common case
def calculate_discount(price, discount_percent):
    return price * (1 - discount_percent / 100)

# Problem: edge cases are not handled
# - discount_percent is 100 or greater (negative price)
# - price is negative
# - discount_percent is None/NaN

# Code hardened by a senior engineer
def calculate_discount(price, discount_percent):
    if price is None or price < 0:
        raise ValueError(f"Invalid price: {price}")
    if discount_percent is None or not (0 <= discount_percent <= 100):
        raise ValueError(f"Invalid discount: {discount_percent}")
    return round(price * (1 - discount_percent / 100), 2)

2. Security Blind Spots

AI models often fail to fully understand security context. They can generate code that contains vulnerabilities such as SQL injection, XSS and SSRF.

// AI-generated code: vulnerable to SQL injection
public List<User> findUsers(String name) {
    String sql = "SELECT * FROM users WHERE name = '" + name + "'";
    return jdbcTemplate.query(sql, new UserRowMapper());
}

// Fixed after senior review: use a prepared statement
public List<User> findUsers(String name) {
    String sql = "SELECT * FROM users WHERE name = ?";
    return jdbcTemplate.query(sql, new UserRowMapper(), name);
}

3. Performance Anti-patterns

Even when the AI generates functionally correct code, it often fails to account for performance on large data volumes or under high concurrency.

// AI-generated code: N+1 query problem
async function getOrdersWithProducts(userId) {
  const orders = await db.orders.findMany({ where: { userId } })
  for (const order of orders) {
    order.products = await db.products.findMany({
      where: { orderId: order.id },
    })
  }
  return orders
}

// Optimized code: use a JOIN or include
async function getOrdersWithProducts(userId) {
  return db.orders.findMany({
    where: { userId },
    include: { products: true },
  })
}

4. Context Ignorance

The AI does not know a project's architectural conventions, a team's coding conventions, or domain-specific business rules. As a result it can generate code that is technically correct but inappropriate in the context of the project.

5. Hallucinated Code

This is the phenomenon of confidently generating APIs that do not exist, library methods that are no longer used, and incorrect configuration values. Such code passes compilation and basic tests, but can trigger unexpected behavior at runtime.

The Amazon AI Code Review Policy

Policy Background: The March 5, 2026 Outage

The six-hour Amazon shopping service outage unfolded as follows.

1. A junior developer used an AI assistant to modify the shopping cart discount calculation logic
2. The AI-generated code passed every unit test
3. A peer at the same level approved it in code review (no senior review took place)
4. After the production deploy, a specific combination of promotions triggered an infinite loop
5. The entire shopping cart service became unresponsive
6. The circuit breaker fired, but a cascade failure still spread to dependent services

Core Policy Content

The main points of the AI-assisted development policy Amazon announced are as follows.

How the Policy Is Implemented

Amazon introduced the following automation into its internal code review system.

# .amazon/ai-review-policy.yaml (example structure)
ai_code_review:
  enabled: true
  detection:
    # Automatic detection of AI-generated code
    copilot_telemetry: true
    codewhisperer_metadata: true
    commit_message_pattern: 'ai-assisted|copilot|codewhisperer'
  approval_rules:
    junior_mid:
      required_approvers:
        min_level: L6
        count: 1
      test_coverage_threshold: 90
      mandatory_checklist: true
    senior:
      required_approvers:
        min_level: L6
        count: 1 # self-review allowed
      test_coverage_threshold: 85
      mandatory_checklist: true
  notifications:
    slack_channel: '#ai-code-review'
    escalation_timeout: 24h

AI Code Review Checklist

This is the systematic checklist a senior engineer uses when reviewing AI-generated code.

Functional Correctness

## AI Code Review Checklist

### 1. Functional Correctness

- [ ] Does it match the business requirements exactly
- [ ] Are all edge cases handled (null, empty, boundary)
- [ ] Is error handling appropriate (exception types, recovery strategy)
- [ ] Have concurrency issues been considered (race condition, deadlock)

### 2. Security

- [ ] Is input validation sufficient (injection, XSS)
- [ ] Is the authentication/authorization logic correct
- [ ] Is sensitive data handled properly (logging, masking)
- [ ] Are encryption/hashing implemented correctly

### 3. Performance

- [ ] Is the code free of N+1 query problems
- [ ] Are there no unnecessary memory allocations
- [ ] Is the caching strategy appropriate
- [ ] Is index usage optimized

### 4. Maintainability

- [ ] Does it follow the project coding conventions
- [ ] Are the dependencies appropriate (no unnecessary libraries added)
- [ ] Are the tests sufficient (unit, integration, E2E)
- [ ] Is there anything that needs documentation

### 5. AI-specific Verification

- [ ] Are there no APIs/libraries the AI hallucinated
- [ ] Were no deprecated patterns used
- [ ] Are there no license problems (copied GPL code and the like)
- [ ] Is there no unnecessary code the AI inserted

Comparing AI Code Review Tools

This section compares the main AI code review tools currently available on the market.

Building an AI Code Review Pipeline

Architecture Overview

An AI code review pipeline is made up of the following stages.

1. Change detection: determine whether a PR contains AI-generated code when it is opened
2. Static analysis: apply the existing linters plus AI-specific rules
3. LLM-based review: semantic analysis and generation of review comments
4. Semantic diff: analysis of semantic code changes
5. Approval gate: wait for senior engineer approval
6. Deployability decision: deploy once every gate has passed

GitHub Actions Pipeline Configuration

# .github/workflows/ai-code-review.yml
name: AI Code Review Pipeline

on:
  pull_request:
    types: [opened, synchronize, reopened]

permissions:
  contents: read
  pull-requests: write

jobs:
  detect-ai-code:
    runs-on: ubuntu-latest
    outputs:
      is_ai_assisted: ${{ steps.detect.outputs.ai_assisted }}
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Detect AI-assisted changes
        id: detect
        run: |
          # Check AI tool metadata
          AI_MARKERS=$(git log --format="%B" origin/main..HEAD | \
            grep -ciE "copilot|codewhisperer|cursor|ai-assisted" || true)

          if [ "$AI_MARKERS" -gt 0 ]; then
            echo "ai_assisted=true" >> "$GITHUB_OUTPUT"
          else
            echo "ai_assisted=false" >> "$GITHUB_OUTPUT"
          fi

  static-analysis:
    runs-on: ubuntu-latest
    needs: detect-ai-code
    steps:
      - uses: actions/checkout@v4

      - name: Run ESLint with AI-specific rules
        run: npx eslint . --config .eslintrc.ai.json --format json > eslint-report.json

      - name: Run Semgrep security scan
        uses: returntocorp/semgrep-action@v1
        with:
          config: >-
            p/owasp-top-ten
            p/javascript
            p/typescript

      - name: Upload analysis results
        uses: actions/upload-artifact@v4
        with:
          name: static-analysis
          path: |
            eslint-report.json
            semgrep-results.json

  llm-review:
    runs-on: ubuntu-latest
    needs: [detect-ai-code, static-analysis]
    if: needs.detect-ai-code.outputs.is_ai_assisted == 'true'
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: Run CodeRabbit review
        uses: coderabbit-ai/coderabbit-action@v2
        with:
          token: ${{ secrets.CODERABBIT_TOKEN }}
          review_level: comprehensive

      - name: Run custom LLM review
        env:
          OPENAI_API_KEY: ${{ secrets.OPENAI_API_KEY }}
        run: |
          python scripts/ai_review.py \
            --diff "$(git diff origin/main...HEAD)" \
            --checklist .github/ai-review-checklist.md \
            --output review-comments.json

  coverage-gate:
    runs-on: ubuntu-latest
    needs: detect-ai-code
    if: needs.detect-ai-code.outputs.is_ai_assisted == 'true'
    steps:
      - uses: actions/checkout@v4

      - name: Run tests with coverage
        run: |
          npm ci
          npm run test:coverage -- --reporter=json > coverage.json

      - name: Check AI code coverage threshold
        run: |
          COVERAGE=$(jq '.total.lines.pct' coverage.json)
          THRESHOLD=90
          echo "Coverage: ${COVERAGE}%, Threshold: ${THRESHOLD}%"
          if (( $(echo "$COVERAGE < $THRESHOLD" | bc -l) )); then
            echo "FAIL: AI-assisted code coverage ${COVERAGE}% < ${THRESHOLD}%"
            exit 1
          fi

  senior-approval:
    runs-on: ubuntu-latest
    needs: [llm-review, coverage-gate]
    if: needs.detect-ai-code.outputs.is_ai_assisted == 'true'
    steps:
      - name: Require senior approval
        uses: actions/github-script@v7
        with:
          script: |
            const reviews = await github.rest.pulls.listReviews({
              owner: context.repo.owner,
              repo: context.repo.repo,
              pull_number: context.issue.number,
            });

            const seniorTeamMembers = ['senior-eng-1', 'senior-eng-2', 'tech-lead'];
            const seniorApproval = reviews.data.some(
              review => review.state === 'APPROVED' &&
                        seniorTeamMembers.includes(review.user.login)
            );

            if (!seniorApproval) {
              core.setFailed('AI-assisted code requires senior engineer approval');
            }

Custom AI Review Script

#!/usr/bin/env python3
"""AI code review automation script"""

import json
import sys
from dataclasses import dataclass
from openai import OpenAI


@dataclass
class ReviewComment:
    file: str
    line: int
    severity: str  # critical, warning, info
    category: str  # security, performance, correctness, style
    message: str
    suggestion: str


REVIEW_PROMPT = """You are a senior software engineer.
Review the following code changes. Focus in particular on the problems that occur frequently in AI-generated code:

1. Edge case omission
2. Security vulnerabilities (SQL injection, XSS, SSRF, etc.)
3. Performance anti-patterns (N+1, unnecessary allocations, etc.)
4. Concurrency issues (race condition, deadlock)
5. Hallucinated code (calls to APIs that do not exist)
6. Inadequate error handling

Return the review result in JSON format.
"""


def review_diff(diff_content: str, checklist_path: str) -> list[ReviewComment]:
    client = OpenAI()

    with open(checklist_path) as f:
        checklist = f.read()

    response = client.chat.completions.create(
        model="gpt-4o",
        messages=[
            {"role": "system", "content": REVIEW_PROMPT},
            {"role": "user", "content": f"Checklist:\n{checklist}\n\nCode changes:\n{diff_content}"},
        ],
        response_format={"type": "json_object"},
        temperature=0.1,
    )

    result = json.loads(response.choices[0].message.content)
    return [ReviewComment(**comment) for comment in result.get("comments", [])]


def main():
    import argparse

    parser = argparse.ArgumentParser()
    parser.add_argument("--diff", required=True)
    parser.add_argument("--checklist", required=True)
    parser.add_argument("--output", required=True)
    args = parser.parse_args()

    comments = review_diff(args.diff, args.checklist)

    critical_count = sum(1 for c in comments if c.severity == "critical")

    with open(args.output, "w") as f:
        json.dump([vars(c) for c in comments], f, indent=2, ensure_ascii=False)

    print(f"Review complete: {len(comments)} comments ({critical_count} critical)")

    if critical_count > 0:
        print("CRITICAL issues were found. A senior review is required.")
        sys.exit(1)


if __name__ == "__main__":
    main()

Semantic Diff Analysis

A traditional text-based diff makes it hard to grasp the semantic changes in AI-generated code. A semantic diff analyzes the meaning of a code change at the AST (Abstract Syntax Tree) level.

"""Semantic diff analysis example, based on the Python AST"""

import ast
import difflib
from dataclasses import dataclass


@dataclass
class SemanticChange:
    change_type: str  # added, removed, modified, moved
    entity_type: str  # function, class, variable, import
    name: str
    risk_level: str   # low, medium, high
    description: str


def analyze_semantic_diff(old_source: str, new_source: str) -> list[SemanticChange]:
    old_tree = ast.parse(old_source)
    new_tree = ast.parse(new_source)

    old_functions = {
        node.name: ast.dump(node)
        for node in ast.walk(old_tree)
        if isinstance(node, ast.FunctionDef)
    }
    new_functions = {
        node.name: ast.dump(node)
        for node in ast.walk(new_tree)
        if isinstance(node, ast.FunctionDef)
    }

    changes = []

    # Newly added functions
    for name in set(new_functions) - set(old_functions):
        changes.append(SemanticChange(
            change_type="added",
            entity_type="function",
            name=name,
            risk_level="medium",
            description=f"New function '{name}' was added - AI-generated code needs verification",
        ))

    # Modified functions
    for name in set(old_functions) & set(new_functions):
        if old_functions[name] != new_functions[name]:
            changes.append(SemanticChange(
                change_type="modified",
                entity_type="function",
                name=name,
                risk_level="high",
                description=f"Logic of function '{name}' changed - edge cases need verification",
            ))

    # Deleted functions
    for name in set(old_functions) - set(new_functions):
        changes.append(SemanticChange(
            change_type="removed",
            entity_type="function",
            name=name,
            risk_level="high",
            description=f"Function '{name}' was deleted - dependencies need checking",
        ))

    return changes

Integrating CI/CD Quality Gates

Multi-stage Quality Gate Architecture

When integrating AI code review into a CI/CD pipeline, configure multi-stage gates so that the strength of verification rises step by step.

# .github/workflows/quality-gates.yml
name: Multi-stage Quality Gates

on:
  pull_request:
    branches: [main, release/*]

jobs:
  # Gate 1: basic lint and type check (all code)
  gate-1-lint:
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm run lint
      - run: npm run typecheck

  # Gate 2: test coverage (higher threshold for AI code)
  gate-2-test:
    needs: gate-1-lint
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
      - run: npm ci
      - run: npm run test:coverage
      - name: Enforce coverage thresholds
        run: |
          node -e "
            const report = require('./coverage/coverage-summary.json');
            const total = report.total;
            const threshold = process.env.AI_ASSISTED === 'true' ? 90 : 80;
            if (total.lines.pct < threshold) {
              console.error('Coverage ' + total.lines.pct + '% < ' + threshold + '%');
              process.exit(1);
            }
          "

  # Gate 3: security scan (SAST + Dependency)
  gate-3-security:
    needs: gate-1-lint
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4

      - name: Semgrep SAST
        uses: returntocorp/semgrep-action@v1
        with:
          config: p/owasp-top-ten

      - name: Dependency audit
        run: npm audit --audit-level=high

      - name: Trivy filesystem scan
        uses: aquasecurity/trivy-action@master
        with:
          scan-type: fs
          severity: CRITICAL,HIGH

  # Gate 4: AI-specific review (AI code only)
  gate-4-ai-review:
    needs: [gate-2-test, gate-3-security]
    runs-on: ubuntu-latest
    steps:
      - uses: actions/checkout@v4
        with:
          fetch-depth: 0

      - name: AI hallucination check
        run: |
          python scripts/check_hallucinations.py \
            --diff "$(git diff origin/main...HEAD)" \
            --package-lock package-lock.json

      - name: Semantic diff analysis
        run: |
          python scripts/semantic_diff.py \
            --base origin/main \
            --head HEAD \
            --report semantic-diff-report.json

  # Gate 5: senior approval (AI code + core services)
  gate-5-approval:
    needs: gate-4-ai-review
    runs-on: ubuntu-latest
    environment: production
    steps:
      - name: Verify senior approval
        uses: actions/github-script@v7
        with:
          script: |
            // List of senior engineers (managed in CODEOWNERS)
            const response = await github.rest.pulls.listReviews({
              owner: context.repo.owner,
              repo: context.repo.repo,
              pull_number: context.issue.number,
            });
            const approved = response.data.filter(r => r.state === 'APPROVED');
            if (approved.length === 0) {
              core.setFailed('Senior engineer approval required');
            }

AI Hallucination Detection Script

"""AI hallucinated code detection: find references to packages/APIs that do not exist"""

import json
import re
import subprocess


def check_import_hallucinations(diff_content: str, package_lock_path: str) -> list[dict]:
    """Check whether newly added imports exist among the actually installed packages"""
    issues = []

    # Extract newly added imports from the diff
    added_imports = re.findall(
        r"^\+.*(?:import|require)\s*\(?['\"]([^'\"]+)['\"]",
        diff_content,
        re.MULTILINE,
    )

    # Extract the list of installed packages from package-lock.json
    with open(package_lock_path) as f:
        lock_data = json.load(f)

    installed = set(lock_data.get("packages", {}).keys())
    installed_names = set()
    for pkg in installed:
        # Extract the name from the node_modules/package-name form
        name = pkg.replace("node_modules/", "")
        if name:
            installed_names.add(name)

    for imp in added_imports:
        # Handle scoped packages and subpaths
        base_package = imp.split("/")[0]
        if imp.startswith("@"):
            base_package = "/".join(imp.split("/")[:2])

        # Skip relative paths
        if base_package.startswith("."):
            continue

        # Skip Node.js built-in modules
        builtin_modules = [
            "fs", "path", "os", "http", "https", "crypto",
            "stream", "url", "util", "events", "child_process",
            "buffer", "querystring", "assert", "net",
        ]
        if base_package in builtin_modules:
            continue

        if base_package not in installed_names:
            issues.append({
                "type": "hallucinated_import",
                "package": base_package,
                "severity": "critical",
                "message": f"Package '{base_package}' is not installed. "
                           f"The AI may have referenced a package that does not exist.",
            })

    return issues

Metric-driven Quality Management

These are the core metrics for tracking the quality of AI code quantitatively, together with how to build the dashboard.

Core Metric Definitions

Prometheus Metric Collection

# prometheus/ai-code-review-rules.yml
groups:
  - name: ai_code_review
    rules:
      - record: ai_code:defect_rate:ratio
        expr: |
          sum(rate(code_defects_total{source="ai_assisted"}[7d]))
          /
          sum(rate(code_changes_total{source="ai_assisted"}[7d]))

      - record: ai_code:review_turnaround:p95
        expr: |
          histogram_quantile(0.95,
            sum(rate(pr_review_duration_seconds_bucket{ai_assisted="true"}[7d]))
            by (le)
          )

      - alert: AICodeDefectRateHigh
        expr: ai_code:defect_rate:ratio > 0.05
        for: 7d
        labels:
          severity: warning
        annotations:
          summary: 'The defect rate of AI-generated code has exceeded 5%'
          description: 'AI code defect rate over the last 7 days: {{ $value | humanizePercentage }}'

      - alert: SeniorReviewBacklogHigh
        expr: ai_code:pending_senior_reviews > 10
        for: 2h
        labels:
          severity: warning
        annotations:
          summary: 'The senior review queue has exceeded 10 items'

Grafana Dashboard Configuration

{
  "dashboard": {
    "title": "AI Code Quality Dashboard",
    "panels": [
      {
        "title": "AI vs Manual Code Defect Rate (7d rolling)",
        "type": "timeseries",
        "targets": [
          {
            "expr": "ai_code:defect_rate:ratio",
            "legendFormat": "AI-assisted"
          },
          {
            "expr": "manual_code:defect_rate:ratio",
            "legendFormat": "Manual"
          }
        ]
      },
      {
        "title": "Review Turnaround Time (p95)",
        "type": "gauge",
        "targets": [
          {
            "expr": "ai_code:review_turnaround:p95 / 3600",
            "legendFormat": "Hours"
          }
        ],
        "fieldConfig": {
          "defaults": {
            "thresholds": {
              "steps": [
                { "value": 0, "color": "green" },
                { "value": 4, "color": "yellow" },
                { "value": 8, "color": "red" }
              ]
            }
          }
        }
      },
      {
        "title": "Senior Review Queue Depth",
        "type": "stat",
        "targets": [
          {
            "expr": "ai_code:pending_senior_reviews"
          }
        ]
      }
    ]
  }
}

Failure Cases and Lessons

Case 1: Cascade Failure from Over-reliance on AI (Amazon, March 2026)

Situation: a junior developer used an AI assistant to modify the discount calculation logic. The AI-generated code looked correct on the surface and passed every existing test. It was missing handling, however, for the case where three specific promotions apply at the same time.

Impact: about six hours of Amazon shopping service outage. Estimated revenue loss of about 300 million dollars.

Lessons:

• Make senior review mandatory for AI-generated code
• Adopt tooling that automatically generates edge case tests
• Restrict AI use in business-critical areas such as promotion logic

Case 2: A Security Vulnerability Generated by AI

Situation: when an AI assistant generated user authentication code, it omitted the algorithm specification in JWT token verification. As a result, code vulnerable to the "alg: none" attack was deployed.

// Vulnerable code generated by AI
const decoded = jwt.verify(token, secret)

// Correct code: specify the algorithm
const decoded = jwt.verify(token, secret, { algorithms: ['HS256'] })

Lessons:

• Apply dedicated SAST rules to security-critical code as a requirement
• Strengthen automated security review for code involving JWT, encryption and authentication

Case 3: License Contamination

Situation: the AI copied GPL-licensed code verbatim out of its training data and included it in a commercial project. After the code was deployed to production, a license scan found it and an emergency patch was needed.

Lessons:

• Include license scanning in the CI/CD pipeline as a requirement
• Adopt a code similarity check tool for AI-generated code

AI Code Governance at the Organization Level

Roles and Responsibilities Matrix (RACI)

(R: Responsible, A: Accountable, C: Consulted, I: Informed)

Phased Adoption Roadmap

Phase 1 (weeks 1-2): foundations
├── Introduce the AI code detection mechanism
├── Roll out the basic checklist
└── Define the senior review process

Phase 2 (weeks 3-4): automation
├── Configure CI/CD quality gates
├── Integrate LLM-based review tools
├── Enforce coverage thresholds
└── Build the security scan pipeline

Phase 3 (weeks 5-8): optimization
├── Build the metrics dashboard
├── Tune false positives
├── Deepen semantic diff analysis
└── Set per-team custom rules

Phase 4 (weeks 9-12): maturity
├── Link with DORA metrics
├── Automate AI code quality reports
├── Establish organization-wide governance
└── Share best practices with other teams

Linking with DORA Metrics

Track the effect the AI code review process has on the overall software delivery performance of the team using DORA metrics.

Recovery Procedure (AI Code Related Incidents)

This is the recovery procedure to follow when a production incident is caused by AI-generated code.

#!/bin/bash
# AI code incident recovery runbook

# 1. Immediate rollback
echo "Step 1: run the production rollback"
kubectl rollout undo deployment/affected-service -n production

# 2. Check the blast radius
echo "Step 2: determine the blast radius"
kubectl logs -l app=affected-service -n production --since=1h | \
  grep -c "ERROR"

# 3. Trace the change history of AI-generated code
echo "Step 3: check the AI code change history"
git log --oneline --all --grep="ai-assisted" --since="7 days ago"

# 4. Check the review history of that PR
echo "Step 4: check the review history"
gh pr list --state merged --label "ai-assisted" --json number,title,mergedAt

# 5. Record the incident timeline
echo "Step 5: start recording the incident timeline"
cat <<'TEMPLATE'
## Incident timeline
- Detected at:
- Rolled back at:
- Recovery confirmed at:
- Root cause:
- AI tool:
- Review history:
- Prevention measures:
TEMPLATE

Conclusion

The six-hour Amazon outage shows the practical challenge we face in 2026, now that AI-assisted development has become routine. AI coding tools improve development productivity dramatically, but at the same time they introduce a new type of risk.

The point is not to ban the use of AI tools but to build a systematic quality management process. Making senior engineer review mandatory as the Amazon policy does, integrating multi-stage quality gates into the CI/CD pipeline, and improving continuously on the basis of metrics is the right direction.

The core principles of AI code review can be summarized as follows.

1. Trust but Verify: do not reject AI-generated code out of hand, but always put it through a verification process.
2. Automate what can be automated: integrate static analysis, security scans and coverage checks into CI/CD and automate them.
3. Do not replace human judgment: business logic, architecture decisions and security design must be reviewed by an experienced engineer.
4. Measure and improve: track AI code quality continuously on the basis of metrics and improve the process.

References

• Amazon AI Code Review Policy (March 2026) - Announcement of the Amazon AI code review policy
• DORA Metrics - State of DevOps Report - Framework for measuring software delivery performance
• Google Engineering Practices - Code Review - Code review best practices from Google
• CodeRabbit Documentation - Official documentation for the AI code review tool
• Amazon CodeGuru Reviewer - AWS CodeGuru Reviewer guide
• Semgrep Rules Registry - Registry of static analysis rules
• OWASP Code Review Guide - OWASP code review guide
• GitHub Copilot Documentation - Official GitHub Copilot documentation