Infrastructure Documentation That Actually Gets Read: A Contractor's Approach

The documentation problem in infrastructure work is not that engineers don't know they should write it. It's that the documentation produced under deadline pressure is almost never the documentation that gets used. Runbooks that take 45 minutes to follow during an incident. Architecture diagrams that were accurate when the system was deployed and haven't been updated since the first refactor. README files that explain how to run the application on the author's laptop in 2023.

As a contractor, I hand off infrastructure to clients and their teams regularly. The quality of that handoff determines whether they call me six months later with a billing crisis — which they wouldn't have if the runbook had been clearer — or whether the handoff is genuinely clean and they have what they need to operate the system without me.

This is the documentation approach I've converged on after several handoffs that didn't work well and several that did.

---

Why Most Infrastructure Documentation Fails

Three patterns appear in almost every documentation failure I've inherited:

Documentation is written after implementation, not during it. The engineer who built the system writes documentation from memory. The implicit decisions — "I chose this subnet range because...", "this Lambda function needs the execution role attached to the VPC because..." — are lost because they were clear in the moment and seemed too obvious to write down. Two years later, nobody knows why the architecture is the way it is, and changing it requires reverse-engineering the original constraints.

Documentation is organized around the author, not the reader. A README that explains what the system does is written from the perspective of someone who already knows how it works. The reader who inherits the system is asking different questions: What do I do when X breaks? How do I deploy a change? Where are the credentials? How do I know if the system is healthy? Architecture overviews that read like design documents don't answer operational questions.

Documentation is not tested. A runbook that has never been followed end-to-end by someone other than its author is a hypothesis, not a procedure. The steps that seem obvious to the author are the steps most likely to fail for the reader. Testing documentation is the single most effective quality check available, and it is almost never done.

---

The Documents I Write for Every Infrastructure Engagement

I maintain a standard set of documents for every engagement. Not all of them are delivered as part of the SOW — some are internal, some are client-facing — but the structure is consistent enough that I'm rarely starting from scratch.

1. The Architecture Overview (For Humans)

Not a diagram with service icons connected by arrows. An architecture overview written in prose that answers: What does this system do? What are its major components? Why were they built this way, and what alternatives were rejected? What are the current known limitations and the expected evolution path?

The diagram is included, but it's a supporting element, not the document itself. A diagram without context answers "what exists" but not "why it exists" or "what to do about it."

I write architecture overviews in a format I call Why Before What: I explain the problem being solved before I describe the solution. This means the document retains value even when the implementation changes — the problem context helps the next engineer evaluate whether the current solution still makes sense.

For a GCP-hosted analytics pipeline, the architecture overview would open not with "this system uses Pub/Sub for event ingestion and Dataflow for processing" but with: "The business requirement is near-real-time aggregation of checkout events for the fraud detection model, which needs an event-level view within 90 seconds of transaction. Batch ETL was rejected because of the latency requirement; message queue alternatives to Pub/Sub were rejected because of the volume profile (~50,000 events/peak hour) and the existing GCP stack. The streaming pipeline design reflects the 90-second SLA..."

When the fraud detection team asks in 18 months whether they can switch to a batch pipeline because the latency requirement has relaxed, that context is already documented.

2. The Operations Runbook

The runbook is for the operator responding to an alert at 2am — or in my case, often for a US-based client engineer responding to an alert at 2am their time while I'm unavailable. It must be:

• Fast to navigate. Top-level sections are alert names or symptom descriptions. Someone responding to a PagerDuty alert should be able to find the relevant section in under 30 seconds.
• Specific, not general. "Check the logs" is not a runbook step. "Navigate to Cloud Logging → query: resource.type="cloud_run_revision" AND severity>=ERROR AND timestamp>"2025-01-01T00:00:00Z" → look for ENOMEM error messages, which indicate the container is hitting memory limits" is a runbook step.
• Tested. I require that at least one person other than me has followed the runbook before I consider it complete. Usually this is the client's on-call engineer doing a dry run during the handoff period.
• Linked, not embedded. Where a runbook step requires following another process (e.g., "open a change request before modifying the firewall rules"), I link to the other process rather than duplicating it. Duplicated instructions drift out of sync.

The format I use for each runbook entry:

## Alert: [Alert Name]

**Severity:** P1 / P2 / P3
**Typical cause:** [1-2 sentences on what usually triggers this]
**Impact if unresolved:** [What breaks and how fast]

### Immediate steps (first 5 minutes)
1. [Specific step with exact navigation path or command]
2. [Specific step]
3. [Decision point: if X, go to section Y; if Z, go to section W]

### Resolution steps
1. [Step]
2. [Step]

### Escalation
- Contact [Name/Role] if not resolved in [timeframe]
- Link to incident communication template

### Post-incident
- [ ] Update this runbook if the steps were inaccurate
- [ ] Open postmortem doc if P1

The checkbox at the bottom is not bureaucratic overhead. It's the mechanism by which the runbook improves over time. Every incident is a runbook test.

3. The Expected Monthly Burn Document

This is the document I wish every contractor I inherited work from had left behind: a clear statement of expected monthly cloud costs with explanations.

Format:

## Expected Monthly Infrastructure Cost

**Baseline (current traffic):** $X/month ± $Y

| Service | Monthly Cost | Cost Driver | Scales With |
|---------|-------------|-------------|-------------|
| GKE (3x e2-standard-2) | $180 | Node count | Team size decision |
| Cloud SQL (db-standard-2) | $95 | Instance class | Not traffic-dependent |
| Cloud Storage | $12 | Data volume | Data growth rate |
| Pub/Sub | $4 | Message volume | Transaction volume |
| Cloud CDN | $25 | Traffic | User traffic |

**What to watch:**
- Cloud CDN scales linearly with traffic — 2× traffic = 2× CDN cost
- Cloud SQL is fixed regardless of traffic
- Pub/Sub cost spikes indicate message volume anomalies — investigate before assuming normal growth

**Billing alert thresholds:**
- $X: normal
- $X+20%: investigate before month-end
- $X+40%: escalate immediately

The client who has this document can look at their monthly bill and understand it. The client without it is calling me.

4. The Credential and Access Inventory

A document that answers: Where are the credentials? Who has access to what? How is access provisioned for new team members? Where are the service account keys, and which applications use them?

This document is security-sensitive and should not live in a public repository. I recommend a shared password manager (1Password, Bitwarden) entry or a restricted Google Doc with audit logging enabled.

The credential inventory I deliver includes:

• A table of all service accounts: purpose, permissions, which application uses it, rotation schedule
• IAM role assignments: who has what role on which project
• External credentials: any API keys for third-party services, where they're stored, rotation procedure
• Access provisioning procedure: step-by-step for adding a new team member or contractor

Systems without documented credential inventories develop unknown access over time. Service accounts accumulate; people who've left the company retain access because nobody checked. The document makes the invisible visible.

---

Documentation During, Not After

The habit change that made the most practical difference in my documentation quality: I write the architecture overview and the initial runbook skeleton before I begin implementation, not after.

The pre-implementation architecture document is a design document — it describes what I plan to build and why. It gets reviewed by the client before I write code. This produces two things: a better architecture (because explaining it in writing before building it surfaces gaps), and a documentation artifact that's already 60% complete by the time I deploy.

The runbook skeleton is a list of the operational scenarios I anticipate: what breaks if the database goes down, what happens if the deployment pipeline fails, how to roll back a bad release. Writing it pre-implementation forces me to design for operability — if writing a runbook step takes me 20 minutes because the procedure is complicated, that's a signal to simplify the system, not to write a more complicated runbook.

---

What to Include in a Handoff

A clean infrastructure handoff includes, at minimum:

1. Architecture overview — current state, design decisions, known limitations
2. Runbooks — one entry per alert that currently fires or could fire in the next six months
3. Expected monthly burn document
4. Credential and access inventory
5. Change management procedure — how to deploy safely, what requires a change window, what can be done without review
6. Contact list — who to call for what: the client's cloud support tier, the relevant service teams, any third-party vendors with SLAs

The handoff is complete when someone who didn't build the system can operate it — not perfectly, but safely — without calling me for the first thirty days. That's the standard I hold the documentation to, and it's different from "documentation exists."

---

A Note on Documentation for Asynchronous Teams

Working across 13-hour time zones from Seoul makes high-quality documentation a survival skill, not a nice-to-have. When a US-based client engineer encounters a problem at 2pm their time, I'm asleep. If the runbook answers their question, they resolve it. If it doesn't, the incident waits for my morning, the client is frustrated, and the engagement quality suffers.

This constraint forced me to write better documentation than I would have if I were available during all client working hours. The runbook has to be complete enough that a competent engineer who's never touched the system can follow it. That standard is harder than "complete enough that I can follow it" — but it's also the standard that makes the documentation genuinely useful beyond its author.

---

Resources

• Google Cloud — architecture framework documentation
• AWS — Well-Architected Tool and documentation patterns
• The Site Reliability Engineering book — Chapter 8: On-Call
• Divio documentation system — a useful framework for classifying and writing different types of technical documentation

---

Documentation Review as Part of the Engagement Cycle

Documentation quality degrades between engagements. A runbook that was accurate when I wrote it in March may be wrong by October if the infrastructure evolved without documentation updates. I build documentation review into every ongoing engagement:

At the start of a renewal or follow-on engagement: I read the existing documentation before looking at the current state of the infrastructure. If the documentation and the infrastructure disagree, I update the documentation before beginning new work. The discrepancy is itself useful information about what changed since the last engagement.

After every incident: I update the relevant runbook section within 48 hours of a resolved incident. The most common update: "The steps in section 3 are correct, but step 3b takes 15 minutes, not 5 — add a timeout expectation." Small updates like this make the runbook measurably better over time without requiring a documentation overhaul.

At handoff: A final documentation review is part of every engagement close. I read every document I've produced and ask: if I walked away today and never came back, what would be ambiguous? What assumption is documented nowhere? What would the on-call engineer not know to check? Anything in those categories gets written before the engagement ends.

The practical measure of documentation quality is the first 30 days after handoff. If I receive no questions that should have been answerable from the documentation, the documentation was complete. If I receive three calls in week one because the runbook didn't cover a common scenario, the documentation had gaps — regardless of how comprehensive it looked when I delivered it.

---

Tools and Format Decisions

Documentation tool choices matter for longevity. I've inherited documentation in tools that were abandoned (Confluence instances nobody has admin credentials for), in formats that don't export cleanly (Notion databases with complex relational structures that don't translate to plain text), and in systems the current team doesn't use (a JIRA wiki attached to a project nobody works in anymore).

My default for contractor-delivered documentation: Markdown in a Git repository. Reasons:

• Plain text survives tool migrations indefinitely
• Version history in Git shows what changed and when, without a separate audit trail
• Diff-able, so reviews are easy
• Renderable in GitHub, GitLab, any modern code platform, and locally without any special tool

For clients with strong Confluence or Notion preferences, I write in their preferred tool — but I export a Markdown copy to the infrastructure repository as a backup. The export takes 5 minutes; the insurance it provides against tool migrations is worth it.

Architecture diagrams I draw in Mermaid (embedded in Markdown) or draw.io (.drawio files committed alongside the documentation). Both formats are open and renderable without vendor-specific tools. Lucidchart-only or Miro-only diagrams are locked to subscriptions that may lapse.

The test I apply to any documentation tool decision: can someone with a standard development environment read and update this documentation in five years without a special subscription? If no, I choose differently.