HUMANSREADCODE

AI‑Augmented Architecture Option Exploration

Mon, 02 Feb 2026 00:00:00 +0000

Introduction: what AI changes (and what it doesn’t)

AI changes the cost of exploring options. It does not change who is accountable for the decision.

If you’re the architect, tech lead, or staff engineer, “AI‑assisted” is not a new accountability model; it’s a faster way to:

Surface plausible architectures you might not have considered.
Make tradeoffs explicit earlier (and with less friction).
Leave a stronger evidence trail for future teams.

But it only works if you treat AI as a generator and critic, not an authority. The workflow below is designed to be:

Tool‑agnostic (no vendor lock‑in).
Enterprise‑safe (no secrets in prompts).
Compatible with real constraints (compliance, integration, uptime, budgets).

TL;DR: quick start

If you only do three things:

Assemble a Context Pack (constraints + current state + decision criteria).
Generate 3–6 concrete options (at least one “boring” option), each with risks and a 2‑week validation experiment.
Choose deliberately using a tradeoff matrix, then capture the key decisions in one or more ADRs with measurable fitness functions.

The full flow is: Context Pack → clarify → drivers → architecture characteristics → options + risks → tradeoffs + mitigations → transition architectures → ADRs + fitness/enforcement + feedback.

Scenario: modernizing policy/quote pricing

You’ve inherited a policy/quote pricing capability that grew over a decade:

A monolith handles quoting, rating, underwriting rules, discounts, and taxes.
Multiple channels depend on it: call center, partner portal, direct‑to‑consumer.
Release cadence is slow, and pricing changes require “big bang” deployments.
The business wants faster experimentation (A/B tests, segmented offers) without breaking regulatory guardrails.

Constraints (the ones that always show up in the real world):

Regulatory/compliance: you must be able to explain how a price was calculated.
Auditability: reproduce a quote months later.
Availability: quoting is revenue critical; you need graceful degradation.
Integration: downstream policy issuance and billing expect current interfaces.
Performance: partners time out; call center productivity suffers.
Scalability: quote/policy workloads spike (campaigns, renewals, partner traffic) and must scale predictably.
Data sensitivity: prompts cannot include PII, secrets, or proprietary pricing tables.

You want to explore multiple viable target architectures (not just “microservices!”), choose deliberately, and write down why.

The “Context Pack”: minimum artifacts before asking AI for options

Option exploration fails when the prompt is missing the constraints that matter.

Before you ask AI to generate architectures, assemble a lightweight Context Pack:

Domain + goals (success, non‑goals).
Constraints (regulatory, SLOs, performance budgets, data handling).
Enterprise decision constraints (approved stacks, platform rules, security model).
Current state (simple diagrams, critical flows, known pain points).
Decision criteria (what decides, and how you’ll measure it).

If you want a copy/paste template, see the Appendix: Context Pack checklist.

Stop conditions: don’t proceed without these

You cannot name the top 3 constraints that would invalidate an option.
You cannot describe the most important runtime failure mode.
You cannot explain what “auditability” means for this domain (what must be reproducible).

If any stop condition triggers, do not ask for architectures yet. Ask for missing artifacts.

The 6‑stage workflow

This is the workflow I use when I want AI to accelerate thinking without replacing judgment.

Stage 1 — Clarify (make constraints explicit)

Goal: turn fuzzy intent into a crisp problem statement and a bounded search space.

Inputs: your Context Pack.

Prompts that work well (tool‑agnostic and safe):

“Summarize the problem in 5 bullets, then list 10 clarifying questions that would change the architecture choice.”
“Restate the constraints as testable statements (e.g., ‘must reproduce quote for 7 years’).”

Outputs:

A validated problem statement.
A list of constraints you can point at in a decision review.

Guardrail: do not accept new constraints invented by the model. Every constraint must map to a known source (policy, stakeholder, incident, contract).

Stage 2 — From business drivers to architecture characteristics

Goal: use the key business drivers to reveal the driving architecture characteristics (explicit) and the implicit ones that will still constrain you.

This stage is where you turn “the business wants faster experimentation” into qualities you can actually design and test against.

Prompts that work well:

“Extract the key business drivers from the Context Pack, then derive the driving architecture characteristics that follow from them.”
“List implicit architecture characteristics that are not stated, but must be true for this domain (e.g., auditability, determinism, explainability). Mark each as: given vs inferred, and state the evidence.”
“Rewrite each characteristic as a testable statement (SLO, retention rule, determinism requirement, operability constraint).”

Outputs:

A short list of key business drivers.
A ranked list of architecture characteristics split into driving (decision makers) vs implicit (still mandatory).
A first‑pass measurement idea for each characteristic (even if it’s approximate).

Note: if you previously used “transition complexity” as a criterion, it’s usually better expressed as the architecture characteristic evolvability (how safely and incrementally the system can change over time).

Guardrail: if a characteristic is inferred, label it as inferred and require an evidence link (regulation, contract, incident history, stakeholder statement).

Stage 3 — Generate permutations (candidate styles → options → isomorphism → risks)

Goal: take the driving + implicit architecture characteristics and deliberately create optionality around candidate architecture styles that fit the problem domain.

Select candidate architecture styles (or combinations)

Examples (pick what actually fits your domain and org):

Modular monolith + façade (strangler)
Layered / hexagonal with a rules boundary
Domain services (coarse‑grained services, not “microservices by default”)
Event‑driven / async integration for decoupling
Event sourcing or append‑only audit log for replay/audit

Create 3–6 concrete options

For each candidate style (or combination), ask for an option that includes:

Core components and responsibilities
Data ownership and audit/replay story
Failure modes and degradation rules
A first‑pass transition hypothesis from current state (incremental steps and major dependencies)

Apply domain‑to‑architecture isomorphism

For each option, validate that the architecture shape matches the domain shape:

Where do underwriting rules, pricing/rating, discounts, taxes, and approvals live?
What is the “unit of audit” (quote, rating run, rule set version), and how is it reproduced?

Align across the intersections (constraints that often kill options)

For each option, do a quick pass across architecture intersections that add real‑world constraints:

Implementation (libraries, language/runtime constraints)
Infrastructure (platform, runtime, deployment, scaling)
Data topologies (stores, retention, lineage)
Engineering practices (testing strategy, release controls)
Team topologies (ownership boundaries, on‑call reality)
Systems integration (contracts, sync/async, backwards compatibility)
Business environment (regulatory approvals, audit workflows)
Enterprise alignment (standards, shared platforms)
Generative AI (what context is needed, what must not leave the boundary)

Where permitted, MCP Servers can be a practical way to safely access internal enterprise and business‑unit decision documents (standards, policies, platform guidance) as you evaluate these intersections. This helps you distinguish “violates a hard constraint” from “needs an exception” early, before you invest in deeper analysis.

Perform a first‑pass risk analysis per option

Have AI enumerate:

Top risks (technical, operational, organizational)
Assumptions that must be true
The cheapest experiment that would validate/invalidate the option (≤ 2 weeks)

Stage 4 — Explore tradeoffs (scoring + risk comparison + mitigations)

Goal: make the decision legible. You’re not optimizing for a perfect score; you’re optimizing for a decision you can defend.

Tradeoff matrix template (headings)

Criterion	Weight (1–5)	Notes / evidence
Auditability / replay	5	What must be reproducible?
Performance (p95)	4	Partner timeout budget; throughput assumptions
Scalability	4	Peak load, bursts, and predictable degradation
Evolvability	4	Incremental change, parallel run, cutovers, reversibility, “stop‑anytime” states
Operational complexity	4	On‑call, deployments, runbooks
Change velocity	3	Time to ship pricing change
Cost to operate	3	Infra + people
Blast radius	4	Failure containment
Integration risk	4	Contracts, downstream dependencies
Security / data handling	5	PII boundaries

Scoring rubric (keep it honest)

Use a simple qualitative scale, but always include “why”:

5 = clearly meets the need with margin; known patterns; low uncertainty
3 = plausible but requires work or introduces notable risk
1 = likely fails the requirement or creates unacceptable risk

AI can help here by forcing completeness:

“Fill out the tradeoff table. For each score, write the strongest argument against that score (steelman the critique).”

Risk comparison and mitigation prompts that work well:

“Compare the risk profiles across options. Which risks are unique vs shared, and which are existential?”
“For the top 3 risks in each option, propose mitigations and the earliest point in the migration where each mitigation must exist.”

This stage is where many “clever” architectures die in a good way.

A worked mini‑example (abridged)

To make the workflow more concrete, here’s what “3 options + a tradeoff slice” can look like.

Option A — Modular monolith + façade (strangler)

Keep a single deployable for now, but carve out a pricing boundary (clear module APIs).
Add a façade in front of the monolith to create a seam for incremental replacement.
Add an append‑only audit log of “pricing runs” (inputs + rule version + outputs).

Option B — Domain services (coarse‑grained)

Split into a small number of services aligned to stable domain boundaries (e.g., pricing, quote orchestration).
Use contract tests and versioned APIs to protect channels and downstream systems.
Centralize auditability via a shared audit store or per‑service append‑only logs.

Option C — Event‑driven integration + audit log

Keep synchronous quote calls where latency demands it, but decouple downstream effects via events.
Use an append‑only audit log for replay/reproduction.
Avoid “event sourcing everywhere” unless the org is ready for the operational complexity.

Tradeoff matrix (5‑criteria slice):

Criterion	Weight (1–5)	A	B	C	Notes / evidence
Auditability / replay	5	4	4	5	C is strongest if audit log is first‑class
Performance (p95)	4	5	4	4	A is simplest path to hit latency budgets
Evolvability	4	4	5	3	B enables parallel evolution; C adds more moving parts
Operational complexity	4	3	3	1	C risks overload without on‑call maturity
Integration risk	4	4	3	3	A preserves existing contracts; B/C require more interface work

The goal is not to “win the table”. The goal is to make the decisive tradeoffs legible and testable.

Stage 5 — Explore transition architectures (current → target, incrementally)

Goal: now that you have a current state and a target direction, explore incremental transition architectures (stepping‑stone states) that keep the system evolvable and safe at every stage.

This is distinct from picking a target. It’s about proving you can get there without a big bang, and that each intermediate state is operationally viable if you need to pause (a “stop‑anytime” state).

Transition architecture prompts that work well:

“Given the current architecture and the chosen target direction, propose 3 incremental transition paths. Each path must define stepping‑stone architectures, cutover points, and rollback strategy.”
“For each transition step, list: what changes, what stays stable, how contracts remain compatible, and what the system looks like if you stop here for 6 months.”
“Identify the key seams for strangling (facades, anti‑corruption layers, event capture, parallel run) and the fitness functions that must hold at each step.”

Outputs:

2–3 viable transition architectures (stepping‑stone states), not just a single migration plan.
The “stop‑anytime” architecture for each step (so the design stays open to evolution).
A small set of per‑step fitness functions (what must not regress while you transition).

Stage 6 — Hone in on the target architecture (NFRs, fitness/enforcement, ADRs, feedback)

Goal: converge on a target architecture direction, then capture the decision and the quality gates that will keep it true over time.

ADRs: expect more than one

Option exploration typically produces multiple ADRs. At minimum you’ll usually want:

One ADR for the target architecture direction.
One ADR for the transition strategy (stepping‑stone states, cutovers, rollback).
(Often) one ADR per cross‑cutting decision that changes constraints (data/audit approach, integration patterns, deployment model, etc.).

ADR outline (tailored to option exploration)

Title: Decision on pricing architecture direction (Option X)

Status: Proposed / Accepted / Superseded
Context: key constraints, drivers, and what forced the decision
Decision: what we chose and the boundaries (what’s in/out)
Options considered: short summaries of the top alternatives
Tradeoffs: the 3–5 decisive tradeoffs (with evidence links)
Consequences: what gets harder, what we must invest in (ops, tooling, training)
Migration plan: incremental steps, cutover strategy, rollback strategy
Open questions: explicitly tracked unknowns

Fitness functions (measurable, not aspirational)

Define a small set of fitness functions (measurable quality constraints that you check continuously) that must improve (or must not regress):

Quote performance p95 ≤ 400ms for partner channel
Quote throughput scales to peak volume within the cost envelope
“Reproduce quote” workflow completes with matching premium within tolerance
Pricing change lead time ≤ 7 days (from request → production)
Error budget policy and degradation behavior defined and tested

AI is helpful here to convert words into tests:

“Rewrite these quality attributes as measurable fitness functions with owners and measurement frequency.”

Where you can, also define lightweight enforcement functions (the checks that make the fitness functions real), such as:

CI checks for contract compatibility, performance budgets, and security scanning
Runtime checks/dashboards for SLOs and replay/audit workflows
Release controls for pricing rule changes (approval workflow, versioning, rollback)

Finally, establish a feedback loop:

Decide what telemetry and operational signals will cause you to revisit the ADR(s).
Define who reviews it, how often, and what triggers escalation (error budget breach, audit failure, unacceptable performance).

Failure modes and guardrails

AI‑assisted option exploration fails in predictable ways.

Failure mode: novelty bias

The model prefers interesting architectures (event sourcing everywhere!) over boring ones that fit your org.

Guardrails:

Require at least one “boring option” (e.g., modular monolith + façade).
Score operational complexity with a high weight.

Failure mode: hallucinated constraints

The model invents rules you didn’t state (“must be multi‑region active‑active”).

Guardrails:

Every constraint in the output must trace to the Context Pack.
Have a human reviewer mark each constraint as “given” vs “assumed”.

Failure mode: overconfidence in estimates

AI writes confident performance, cost, and migration timelines.

Guardrails:

Treat all numbers as placeholders until measured.
Force the model to provide ranges and uncertainty.

Failure mode: decision laundering

Teams hide behind “the AI said” to avoid hard conversations.

Guardrails:

Write ADRs in first person plural: “we chose… because…”
Include the strongest argument against the chosen option.

Guardrail: adversarial analysis (critique each stage)

Option exploration gets safer and more accurate when you deliberately critique the output of each stage using adversarial analysis. The goal is not to “win an argument” with the model; it’s to surface blind spots early.

How to do it:

Use an alternative model (or at least a fresh chat) as a critic.
Use multiple personas to force different failure modes to show up: security, architecture, engineering, operations, data, compliance/audit, enterprise platform.
Require the critic to cite which input artifact/constraint each claim depends on (Context Pack vs assumption).

Prompt pack: see the Appendix: Adversarial review prompts.

To avoid endless cycles of improvement:

Timebox adversarial reviews (e.g., 30–60 minutes per stage).
Cap review rounds (e.g., max 2 critique iterations per stage).
Use an explicit stop rule: only iterate if a critique surfaces (1) a violated constraint, (2) an unmitigated existential risk, or (3) a missing artifact required for governance.
Track deltas: capture changes as a short list (“what changed and why”) so you don’t re-litigate the same points.

Adoption in enterprise reality

In large organizations, option exploration is as much about governance and evidence as it is about diagrams.

Use criticality tiers

Not every system needs the same rigor. Define tiers (example):

Tier 0 (revenue/regulatory critical): full evidence bundle, ADR required, explicit fitness functions, architecture review.
Tier 1 (important): tradeoff matrix + ADR light.
Tier 2 (low risk): lightweight decision notes.

Progressive enforcement

Start with a small number of teams and make the workflow easy to adopt:

Provide Context Pack and ADR templates.
Offer a “review clinic” for the first 3–5 decisions.
Require evidence bundles only for Tier 0 initially.

The evidence bundle

What reviewers actually need is not more narrative; it’s better artifacts:

Context Pack (bounded and safe)
Options (with diagrams)
Transition architectures (stepping‑stone states and stop‑anytime designs)
Tradeoff matrix (with weights)
Risks + experiments
ADR(s) + fitness functions

AI can accelerate drafting every piece, but humans must validate each claim.

Conclusion: faster exploration, better accountability

AI makes it cheaper to explore architecture options and harder to pretend we didn’t consider alternatives.

If you adopt the workflow above, you get outcomes that are faster, defensible, and executable.

That’s the arc I care about in this series: quality → context → execution → feedback. AI helps in every stage, but only if we keep accountability where it belongs.

Appendix: templates and prompt pack

Context Pack checklist (copy/paste)

Domain + goals

One‑paragraph problem statement (what is broken, what success looks like)
Top 3 business outcomes (e.g., weekly pricing changes, partner performance, audit readiness)
Non‑goals (what you are explicitly not trying to solve right now)

Constraints

Regulatory/compliance constraints (explainability, retention, approvals)
Availability target (SLOs) and degradation rules
Performance budget (p50/p95) and throughput expectations
Scalability expectations (peak load, burst behavior, scaling limits, cost envelope)
Data classification (what can/can’t leave the boundary)
Budget and team constraints (people, skills, on‑call maturity)

Decision constraints (enterprise / business unit)

Compute strategy / technology constraints (cloud/on‑prem, regions, runtimes, managed services)
Security constraints (identity model, encryption standards, network segmentation, key management)
Technology constraints (approved stacks, platforms, vendors, lifecycle/end‑of‑support rules)
Patterns / practices constraints (integration patterns, deployment standards, SDLC controls)
Quality constraints (availability tiering, performance budgets, DR/RTO/RPO, observability requirements)

If you have internal decision documents (enterprise architecture standards, platform playbooks, security policies), link them here.

Where permitted, it can be useful to use MCP Servers to safely access these documents during option evaluation so your options and target architecture align with enterprise and business‑unit constraints.

Current state

C4‑ish diagram(s): context + container (keep it simple)
Critical flows: quote, re‑quote, bind, endorsement, renewal
Known pain points and incident history (top 5)
External dependencies (policy admin system, document generation, payments, CRM)

Decision criteria

The quality attributes that decide outcomes (e.g., auditability, evolvability, performance, scalability)
How you’ll measure them (even approximately)

Adversarial review prompts (copy/paste)

“Act as a security reviewer. Identify the top 10 security and data‑handling risks in this stage output, including identity, secrets, encryption, tenancy, and audit trails. For each risk, propose a mitigation and where it belongs (design‑time vs build‑time vs runtime).”
“Act as an operations/SRE reviewer. Where will this fail at 2am? List the top failure modes, required telemetry, runbooks, and the minimum viable operability baseline for each option.”
“Act as an enterprise architect. Which enterprise/business‑unit constraints are violated or likely to require exceptions? Identify the exact decision constraints impacted and what evidence you would need to approve an exception.”
“Act as a domain/audit reviewer. Can we reproduce a quote exactly months later? Identify what must be versioned, retained, and replayable for auditability and explainability.”
“Act as a skeptical staff engineer. Find contradictions, missing interfaces, and migration steps that assume ‘magic rewrites’. Propose the smallest next step that reduces risk.”
“Steelman the strongest case against the preferred option and against the preferred transition path. What would make you reject it in a review?”

Architecting Context: How Foundational Architecture Practices Power AI-Augmented Delivery

Fri, 31 Oct 2025 00:00:00 +0000

Introduction

When we talk about AI-augmented delivery, that clarity becomes essential. Large language models and coding agents can’t intuit your intent — they need structured, high-quality context to operate safely and consistently.

That’s where solid architecture practices come in. The methods defined by Neal Ford and Mark Richards in Fundamentals of Software Architecture provide exactly the kind of disciplined thinking that fuels Context Engineering and Specification-Driven Development (SDD).

Let’s break that down.

1. Architecture Creates Context

Every time we make an architectural decision, we’re not just shaping a system — we’re creating context. Each artifact, model, and record we produce explains how the system works, why we made certain tradeoffs, and what rules or constraints apply.

That context becomes a reusable source of truth for both humans and AI systems.

Good architecture doesn’t just describe; it defines the decision space. It gives structure to reasoning — and that’s what context engineering depends on.

2. The Architecture Context Cycle

The cycle starts with business intent and ends with operational feedback. Each step creates artifacts that capture decisions, tradeoffs, and constraints — the building blocks of high-quality context.

Step 1. Understand the Key Business Drivers

Start with why. Every architecture decision should connect back to the business drivers — performance, agility, cost, security, compliance, or growth. These drivers become your north star and define what “good” looks like for the system.

Document them clearly. They’ll influence every step that follows and form the foundation of your architecture context.

Step 2. Identify the Driving Architecture Characteristics

From the business drivers, extract the key architecture characteristics. Things like scalability, availability, and modifiability aren’t just buzzwords — they are the measurable qualities that shape the architecture.

These characteristics turn abstract goals into something concrete and testable. They’re also critical inputs for AI agents — they describe what matters most when generating or evaluating solutions.

Step 3. Select Candidate Architecture Styles

Now explore the options. Based on those characteristics, select architecture styles that fit — microservices, modular monolith, event-driven, layered, hexagonal, or combinations.

Each style has strengths and tradeoffs. Document them. You’re building a map of possibilities that defines the constraints and degrees of freedom in your design. This becomes valuable context for humans and machines alike.

Step 4. Apply Domain-to-Architecture Isomorphism

This is where the real matching happens. Take the problem domain and map it to each candidate architecture. You’re looking for isomorphism — where the shape of your domain aligns with the shape of the architecture.

This alignment tells you which style fits naturally and where friction exists. Capturing this analysis as an artifact provides rich, instructive context for anyone (human or AI) who needs to understand why you made that call.

Step 5. Align Across the Nine Intersections of Architecture

Neal Ford and Mark Richards identify nine intersections where architecture meets the rest of the ecosystem:

Implementation
Infrastructure
Data topologies
Engineering practices
Team topologies
Systems integration
Business environment
Enterprise alignment
Generative AI

Each intersection adds constraints and dependencies — and each one adds context. When documented, these connections help AI systems understand how architecture fits within organisational and operational realities.

Step 6. Perform an Architecture Risk Analysis

Every architecture choice carries risk. Assess it, document it, and define mitigation strategies. These risk artifacts become part of the knowledge base for future decisions and for AI systems that need to understand tradeoffs.

When context engineering references these risks, AI agents can make safer, more informed recommendations.

Step 7. Identify NFRs, Fitness Functions, and Enforcement Functions

This step defines how quality is measured and maintained. Non-functional requirements (NFRs), fitness functions, and enforcement functions turn architectural intent into measurable constraints.

They set the quality gates. In Context Engineering, they become guardrails — standards that AI systems can apply automatically to ensure consistency and compliance.

Step 8. Document Decisions in Architecture Decision Records

Every major decision should live in an Architecture Decision Record (ADR). An ADR tells you what decision was made, why it was made, and what alternatives were considered.

They’re short, structured, and easy to read — which makes them perfect for both people and LLMs. When included in the architecture context, ADRs give AI systems traceable reasoning and historical perspective.

Step 9. Establish Feedback Loops

Architecture isn’t a one-and-done process. Use telemetry, operational insights, and incident reviews to feed back into your architecture decisions.

This creates a living context — one that reflects the real-world performance and evolution of your systems. AI-assisted feedback loops can even help detect drift, monitor adherence, and flag emerging risks.

3. Architecture Artifacts as Context Assets

Each step in this cycle produces something useful — and together, they form a high-quality, interconnected knowledge system:

Business drivers define purpose.
Architecture characteristics define priorities.
Styles and isomorphism define structure.
Intersections define dependencies.
Risks and NFRs define guardrails.
ADRs define reasoning.
Feedback defines evolution.

When structured and stored properly — Markdown, YAML, or Mermaid — these artifacts become the context substrate that Context Engineering and Specification-Driven Development rely on.

4. The Role of C4 Diagrams

C4 diagrams bring structure to the visual side of context. They move from broad to detailed, giving both humans and AI a layered understanding of the system.

Level 1: Context — What the system is and who it interacts with.
Level 2: Container — Major deployable units and responsibilities.
Level 3: Component — Key modules and internal flows.
Level 4: Code — The implementation detail.

Captured as code (Mermaid or Structurizr DSL), these diagrams are machine-readable. That means they can be included directly in your context pipeline to guide code generation, architecture validation, and SDD workflows.

5. From Architecture to AI-Augmented Delivery

Architecture produces the clearest, richest form of context an organisation can have. When we feed that context into AI-augmented workflows, we bridge strategy and execution.

Here’s the chain of context: Business Drivers → Characteristics → Styles → Domain Mapping → Intersections → Risks → NFRs → ADRs → Feedback.

That chain doesn’t just describe the system — it teaches it. Each link provides a layer of understanding that Context Engineering and SDD can use to generate reliable, explainable, and compliant outputs.

6. Architecture as Context Infrastructure

Architecture is no longer just about design decisions. It’s about building the context infrastructure that allows AI systems and humans to collaborate safely and effectively.

When we apply these practices consistently, we create architectures that are understandable by machines and trustworthy by design.

That’s how we move from architecture as craft to architecture as context — the foundation of AI-augmented modernisation. What Context Infrastructure Really Means

When we talk about context infrastructure, we’re not talking about servers, networks, or pipelines. We’re talking about the architecture of knowledge — the structured, reusable foundation that makes our systems understandable by both humans and machines.

It’s everything that defines intent and meaning: business drivers, architecture characteristics, styles, NFRs, fitness and enforcement functions, ADRs, standards, diagrams, and the feedback loops that connect them. Captured in consistent, machine-readable formats like Markdown, YAML, or DSLs, these artifacts form a living system of knowledge that defines why things exist, how they work, and what good looks like.

This is what allows AI systems to reason safely within enterprise boundaries. It turns architecture from static documentation into an active system of context — continuously accessible, verifiable, extensible, and executable. That’s the real foundation of AI-augmented delivery.

7. AI as a Partner in the Architecture Process

AI isn’t replacing architects — it’s extending their reach. At each step of this process, AI can act as a force multiplier:

Business Drivers — Summarise drivers from documentation, highlight inconsistencies, and suggest clarifying questions and missing context.
Architecture Characteristics — Suggest architecture characteristics appropriate for the stated key business drivers.
Candidate Styles — Propose architecture styles that align with the stated drivers and characteristics.
Domain-to-Architecture Mapping — Generate architecture diagrams as code, validate the fit between the problem domain and architecture components.
Intersections — Surface dependency conflicts across infrastructure, data, team topologies, and constraints.
Risk Analysis — Identify potential failure points, gaps, or performance risks based on prior patterns.
NFRs and Fitness Functions — Grade coverage, suggest measurable metrics, and flag gaps.
ADRs — Review language for clarity, completeness, and rationale consistency.
Feedback Loops — Analyse telemetry to detect drift and recommend architectural refinements.

AI can also grade and quality gate the outputs of each step — checking for structure, completeness, clarity, and alignment with established standards. By embedding these checks into the workflow, we build a continuous assurance layer that reinforces good practice instead of inspecting for it later.

It’s a feedback loop: better architecture creates better context for AI, and better AI augments the quality of architecture. That’s how we raise the floor on architectural excellence while freeing humans to focus on the creative, strategic, and system-level thinking we excel at.

From Quality to Context to Execution: Engineering the Next Stage of AI-Augmented Delivery

Tue, 14 Oct 2025 00:00:00 +0000

Introduction

This is where Context Engineering comes in: the discipline of designing, curating, and maintaining structured, high-quality context across the SDLC so AI systems can operate effectively within enterprise boundaries.

1. From Built-In Quality to High-Quality Context

Every artifact we create through the SDLC — from requirements to architecture, from test cases to runbooks — contributes to the context fabric of our organisation. When quality is built in at each step, that fabric becomes strong, consistent, and richly descriptive — the perfect substrate for AI augmentation.

Within this context are:

High-quality requirements — acceptance criteria, use cases, and user journeys
Architecture artifacts — key business drivers, architecture characteristics, risk analysis, NFRs, fitness functions, and diagrams
Enterprise constraints — standards, decisions, patterns, tooling, technology, security
Operational knowledge — runbooks, observability frameworks, and lessons learned
Consistent engineering practices — standards, quality targets, templates, automation, and metrics

Together, these form a shared system of understanding — the living context that explains not only what we are building, but how and why we build it that way.

Every time we work with an AI system, we must ensure that this context — from enduring architectural principles to the most recent design decisions — is available and relevant. Mechanisms such as RAG (Retrieval-Augmented Generation) and MCP (Model Context Protocol) allow us to dynamically load and update this information, ensuring every interaction starts well-informed and stays aligned as work evolves.

And here’s where it gets interesting: The same LLMs that depend on high-quality context can also help us maintain and improve it.

AI can assist in:

Reviewing and grading artifacts for completeness or consistency
Detecting drift or outdated information
Suggesting missing dependencies or unclear logic
Generating structured documentation and linking related artifacts automatically

This creates a reinforcing feedback loop — humans produce and refine context, LLMs enhance and maintain it, and together they raise the quality bar over time.

2. Why Context Matters: LLMs Are New Hires

Every time you start a new session with a large language model, you’re effectively onboarding a brand-new engineer.

The model understands coding — but it has no idea how your business operates, what your architectural patterns look like, how your teams collaborate, or what “good” means in your organisation.

We don’t have the luxury of letting an LLM shadow a senior engineer for months, observe pull requests, attend design reviews, or absorb the organisation’s nuances organically. Instead, we must deliver that understanding through context — structured, high-quality, and relevant every time.

That’s why Context Engineering is essential. Each new interaction with an AI agent requires us to re-establish a shared understanding of standards, goals, and constraints. The richer and more structured the context, the faster the “onboarding” — and the better the outcomes.

The side benefit? Humans thrive on this same structure. Well-formed context makes engineers more productive, decisions more traceable, and knowledge more durable.

3. From Context to Code: Specification-Driven Development

Once we’ve built up high-quality context, we can start turning that context into action through Specification-Driven Development (SDD) — a workflow that transforms structured context into executable specifications for coding agents.

Using tools like Spec Kit, we can orchestrate this process step by step:

Constitution – Load foundational context, standards, and objectives into the workflow to ground subsequent steps.
High-level specifications – Define what to build (requirements, user stories, and goals).
Clarifications – Resolve ambiguity and fill gaps before technical planning.
Technical plans – Create detailed implementation plans aligned to the chosen tech stack.
Tasks – Generate actionable implementation tasks.
Artifact analysis – Check consistency and coverage across generated artifacts.
Implementation – Have the coding agent execute the tasks to build the feature according to plan.

Each of these artifacts is stored within the repository — versioned, reviewable, and auditable — giving both humans and coding agents shared visibility and traceability. Furthermore, these artifacts can be refined and tuned by humans and agents at each step in the SDD process.

By curating these executable specifications, we create a clear, structured foundation that allows multiple coding agents to work in parallel — each implementing different tasks marked “parallel” within a specification, or on separate specifications within the same codebase. This parallelisation becomes practical and safe when the codebase is modular, well-structured, and covered by robust tests.

Where systems are not yet ready for this level of parallelised automation, SDD provides a path forward. We can use the same specification-driven workflow to incrementally refactor and improve codebases — step by step — until they reach the maturity required for safe, concurrent AI collaboration.

In this way, SDD is both a delivery framework and a modernisation engine. It helps us scale the work of development safely across human and AI contributors while continuously improving the technical foundation underneath.

This approach eliminates brittle “long prompt” interactions and replaces them with a repeatable, inspectable workflow, where each stage can be reviewed, refined, and reused.

4. The New Skill: Context Engineering

The emerging skill in AI-augmented delivery isn’t prompting — it’s context engineering.

Prompting is transient; context endures. Prompting says “do this”; context says “here’s how we do things here.”

In this new paradigm, the role of the engineer evolves from coder to context architect — someone who builds and maintains the structured environment that allows AI agents to work effectively and safely within enterprise constraints.

And while this helps AI coding agents perform better, it also strengthens human teams — creating a shared foundation of clarity, standards, and alignment across the delivery ecosystem.

Conclusion

Every engineer — human or AI — performs best when given clarity, context, and structure.

By combining built-in quality with Context Engineering and Specification-Driven Development, we create an environment where AI coding agents and human engineers work together — safely, efficiently, and at scale.

We can’t teach LLMs our culture through osmosis — we must encode it through context. That’s how we transform modernisation from an experiment into an engineered system of continuous understanding.

AI-Augmented Quality Across the SDLC: Building It In, Not Testing It In

Sat, 27 Sep 2025 00:00:00 +0000

Introduction

Too often, enterprises treat quality as a reactionary step during testing. By then, it’s too late – defects are costly, drift is entrenched, and risks multiply. The better path is to treat quality as a continuous discipline, woven through requirements, architecture, build, and operations.

Here’s how quality can – and must – be designed into every stage of the lifecycle, with AI serving as an augmentation layer to scale and strengthen the practice.

1. Requirements: Building clarity from the start

High-quality requirements are the foundation of everything that follows. Yet requirements are often rushed, inconsistent, or ambiguous. To build quality in, we must:

Ask the right clarifying questions up front.
Capture requirements in standardised templates.
Define clear, testable acceptance criteria.

AI strengthens this process by:

Suggesting questions we may not have thought of.
Highlighting ambiguities, contradictions, or missing edge cases.
Grading requirements against quality standards.
Enhancing acceptance criteria for completeness and clarity.

The outcome: requirements that are sharper, more consistent, and ready to drive quality downstream.

2. Architecture & Design: Guardrails for sustainable solutions

Architecture decisions shape the system’s long-term health. Quality at this stage means ensuring that:

Critical business needs are mapped to the right architecture characteristics.
Candidate styles are identified through those characteristics.
The most appropriate architecture style(s) is chosen for the given problem domain.
Risks are identified, with mitigations designed in.
Decisions are captured throughout in Architecture Decision Records (ADRs) capturing the “why”.
Designs are expressed as living artifacts (e.g., C4 diagrams in mermaid syntax).
Fitness functions continuously measure the adherence of the implementation to the architecture design.

AI can augment every step here:

Suggesting architecture options and trade-offs.
Detecting drift between intended and actual designs.
Rapidly producing architecture diagrams and design models.
Grading outputs against standardised practices.
Architecture artifacts feed context engineering.

By building quality into architecture, we reduce risk, improve alignment, and create strong feedback loops as systems evolve.

3. Build & Test: Raising the engineering baseline

The build and test phases benefit enormously from high-quality context established upstream. Instead of scrambling to “test in” quality, we now have rich context to leverage where we can:

Use AI to suggest additional test cases, edge cases, and coverage improvements.
Apply standardised engineering practices (with AI verifying compliance).
Leverage the testing pyramid effectively to minimise costly duplication.
Focus humans on high-value work in the problem space, while AI enforces the basics.
Enable developers to parallelise certain tasks which would in the past be done sequentially.

This is where context engineering and Specification-Driven Development (SDD) shines – combining high-quality context with engineering standards so AI can guide teams toward compliant, consistent, and valuable outputs.

4. Operations: Predict failures, detect anomalies

Quality does not end at release – it extends into how systems behave in production. Operations is where organisations often pay the price for weak upstream practices, but with AI augmentation we can move from reactive firefighting to proactive assurance.

AI enables operations to:

Predict failures before they occur, allowing preventive action.
Detect anomalies in real time, spotting drift or unusual behavior early.
Correlate signals across monitoring, logging, and observability data.
Feed insights back into design and build, creating a closed-loop lifecycle.

This shifts operations from a cost center into a learning system that strengthens quality everywhere else.

Conclusion: Quality is not a phase

The lesson is clear: quality cannot be tested in – it must be built in.

When quality is designed into requirements, architecture, build, and operations – and when AI augments human expertise at each stage – modernisation becomes safer, more resilient, and more scalable.

This is how we lower transformation risk, empower engineers to innovate, and create a foundation where AI augmentation thrives.

Modernisation Guardrails & AI

Sun, 07 Sep 2025 00:00:00 +0000

Introduction

Without these standards applied, measured, and enforced across the SDLC — from requirements and analysis to architecture, design, build, test, security, release, and operations — enterprises risk fragmentation, drift, and poor outcomes.

The Path Forward

The path forward is clear:

Define standards - set consistent expectations across the SDLC.
Enable easy adoption - provide frameworks, tooling, and practices for easy uptake.
Measure adherence - track metrics, targets, and thresholds across the SDLC.
Ensure with care - apply governance functions within criticality and organisational constraints.

Quality Gates as More Than Checklists

When these foundations are in place, quality gates become more than checklists. They provide:

Guardrails that ensure the right activities happen, at the right stage, at the right quality.
Observability and trust — not just detecting drift, but building confidence in the process.
The freedom for engineers to innovate, knowing the basics are safeguarded.

Where AI Comes In

And here’s where human-directed AI comes in. Quality gates grounded in consistent standards create the perfect substrate for AI augmentation. AI can:

Grade outputs against standards, ensuring consistency and reliability.
Alert on drift or non-adherence, helping teams course-correct early.
Automate repetitive compliance checks, freeing humans for higher-value work.
Leverage high quality context in context engineering to produce smarter, more relevant AI-driven decisions.

Conclusion

That’s how modernisation, foundations, and AI intersect: quality gates as the bridge from standards to safe AI augmentation.

Modernisation & AI

Sat, 23 Aug 2025 00:00:00 +0000

Introduction

Now, with the rise of AI and LLMs, we’re at an inflection point. The reality is that AI initiatives will struggle — or fail — without the solid architecture, engineering, and standards that modernisation provides.

Where Modernisation and AI Converge

Where modernisation and AI converge, we can industrialise the SDLC:

Built-in governance: Architecture standards, engineering standards, fitness functions, and quality gates — enforced by AI.
Quality everywhere: Embedding automated quality measures across the SDLC ensures quality is built into the product and issues are caught early and continuously, not late in the cycle.
More time in the problem space: Engineers spend less time battling compliance, and more time planning early, iterating rapidly, and solving meaningful business problems.
Context engineering: High-quality requirements, data, and standardised practices give AI the knowledge it needs to reason safely about complex systems.

The Future of Enterprise Delivery

The next evolution of enterprise delivery is AI-augmented modernisation and the industrialised SDLC.

I’ll be sharing patterns and lessons as I explore this frontier — bridging architecture, engineering, quality, and AI to create sustainable, scalable delivery.

About

Sat, 21 Dec 2024 00:00:00 +0000

Karl Kyck

Hi, I’m Karl Kyck, a software architect specialising in managing risk and complexity in large enterprise software systems. I focus on delivering optimal solutions that drive real business value through sustainable software engineering practices.

HUMANSREADCODE is dedicated to promoting sustainable software engineering practices and demonstrating how this approach enables the creation of effective, value-driven solutions. This blog explores topics such as software architecture, engineering practices, organisational structure, delivery methodologies, and product-thinking. By integrating these elements, we aim to help you build the right product, at the right time, using the right methods.

You can follow me on:

Privacy

Sat, 21 Dec 2024 00:00:00 +0000

We collect usage data using google analytics for the purposes of visualizing the traffic on this website.

All data is anonymized and we do not sell or share any data with third parties.

If you choose to deny, no info will be collected whatsoever.

The Known and Unknown Framework

Sat, 12 Aug 2023 00:00:00 +0000

Introduction

In the complex and ever-evolving landscape of software development, operating in the gray, with it’s many uncertainties and ambiguities, is a reality that every software engineer faces. It is in this realm of uncertainty, the “Known and Unknown Framework” emerges as a guiding beacon, offering a structured approach to achieve continuous clarity of purpose. This framework empowers developers to identify the facts upon which their work can be firmly grounded, driving successful project outcomes.

The Known and Unknown Framework

At its core, the Known and Unknown Framework is a powerful tool that helps teams navigate the intricacies of the gray area by classifying information into distinct categories: known knowns, known unknowns, unknown knowns, and unknown unknowns. Let’s delve into each of these categories to understand their significance and how they contribute to achieving clarity of purpose.

Known Knowns: The Foundation of Solid Work

Known knowns represent the bedrock of factual information upon which sound decisions and work can be built. These are the certainties, the established truths that provide a reliable starting point for any software development endeavor. For example, well-documented specifications, existing codebase details, and proven algorithms fall into this category. Leveraging known knowns allows developers to undertake tasks with confidence, knowing they are on a solid foundation.

Known Unknowns: Igniting the Quest for Answers

Known unknowns are the questions that arise when faced with uncertainties. They represent gaps in understanding that, once addressed, can transform ambiguity into clarity. Embracing known unknowns is key to proactive problem-solving and ensuring that no stone is left unturned. These questions become a roadmap for further exploration, research, and discovery, enabling developers to convert uncertainties into known knowns.

Unknown Knowns: Harnessing Intuition

Sometimes, software development engineers possess a deep-rooted intuition that guides their decisions. These are the unknown knowns, the unspoken hunches that hold untapped potential. By acknowledging and converting these intuitive insights into known unknowns (questions), developers can subject them to scrutiny, leading to a deeper understanding of their value and relevance. This transformation ensures that intuition is not lost but rather refined and channeled toward concrete outcomes.

Unknown Unknowns: Embracing the Journey of Discovery

The realm of the unknown unknowns is where groundbreaking discoveries lie in wait. These are the surprises, the unforeseen challenges, and the unexplored opportunities that emerge as development progresses. Engaging in deliberate discovery sessions and research efforts helps developers unveil these hidden treasures, gradually transforming them into known unknowns and even known knowns. By proactively seeking the unknown, developers can expand their horizons and stay ahead in a rapidly evolving field.

Putting the Framework into Action

To harness the power of the Known and Unknown Framework, consider the following step-by-step process:

Assumptions go into the Known Unknowns section – assumptions need to be turned into facts, so start there.
Populate the Known Knowns and Known Unknowns Quadrants: Work with what you know and what you don’t know. This establishes a clear starting point for your project.
Leverage Known Knowns for Solid Work: Utilize facts as building blocks for tasks that require certainty. These established facts become the pillars upon which your work stands.
Iterative Clarity Building: Continuously identify new known knowns, turn known unknowns into known knowns, and prioritize tasks based on this evolving clarity. This iterative process ensures that your project remains aligned with emerging insights.
Explore the Unknowns: Use brainstorming sessions to uncover unknown knowns, use discovery and research activities to unveil unknown unknowns. Turn these into known unknowns and continue the cycle.

Conclusion

In the dynamic world of software development, achieving continuous clarity of purpose amid uncertainties is a formidable challenge. The Known and Unknown Framework emerges as a beacon of guidance, offering a structured approach to categorize and leverage information effectively. By harnessing the power of known knowns, known unknowns, unknown knowns, and unknown unknowns, software development engineers can navigate the gray area with confidence, making informed decisions, and ultimately delivering successful outcomes. Remember, assumptions are the silent assassins that must be eliminated – this framework equips you to replace assumptions with solid facts, paving the way for excellence in your projects.

Text generated from my summarised list of bullet points by ChatGPT, August 12, 2023, OpenAI, https://chat.openai.com. Edited for style and content.

Automation testing as an upfront software design technique

Thu, 10 Aug 2023 22:00:00 +0000

Credit: The Simpsons/20th Century Animation

Effective software development testing practices play a pivotal role in crafting robust, reliable, and high-quality software systems. While testing is traditionally seen as a validation mechanism, it’s increasingly being recognized as a potent tool for upfront software design.

By incorporating testing practices into the early stages of software development, teams can not only catch defects early but also shape the architecture, behavior, and interactions of the system. This proactive approach not only enhances software quality but also fosters a more streamlined development process.

In this article, we will delve into various testing practices that serve as upfront software design techniques, highlighting their significance in shaping software systems from inception.

Behavour-Driven Development (BDD)

Behavour-Driven Development (BDD) stands as a testament to the philosophy that upfront design of a system’s behavior is crucial for success. BDD shifts the focus from writing traditional requirements to defining the expected behavior of the system through executable specifications.

This promotes a shared understanding among stakeholders and helps to align development efforts with the desired outcomes. BDD encourages collaboration between developers, testers, and business representatives, fostering a holistic view of the software’s functionality right from the outset.

Having these conversations upfront, clarifying requirements, and implementing the executable specifications as tests before implementing code, avoids costly mistakes that can be made due to the premature implementation of code, when a full understanding of the requirements has not been reached.

Integration Testing

Integration testing extends the upfront design of a system’s behavior to its interactions with external dependencies. By simulating the integration points with other systems, services, or components, integration testing ensures that the software’s interactions are well-defined, reliable, and consistent.

Taking a test-first approach to integration testing makes us consider the nuances of how the application will interact with external dependencies, plan accordingly, and design the behavior of how the application will interact with external dependencies before a line of code has been written.

This practice aids in early identification of compatibility issues, communication protocols, and data exchange mechanisms, enabling a smoother integration process and reducing late-stage surprises.

Consumer-Driven Contract Testing

Consumer-Driven Contract Testing emphasizes the upfront design of how an application interacts with external services. Adopting a contract-first approach, this practice defines expectations and interfaces between systems from the outset.

By ensuring that both the consumer and provider of a service adhere to a predefined contract, this approach promotes compatibility, reduces integration challenges, and enables smoother interactions.

The process of consumer-driven contract testing demands that the consumer design upfront, the behavior of how the application will interact with the external service. The resultant contract, with the expected request and response, must be agreed by both parties before it can be used for the purposes of testing and implementing code.

Furthermore, this approach creates an automated communication channel and feedback loop between consumer and provider, automatically raising alerts should incompatibilities arise during development. The relevant parties are alerted to issues before they impact running systems – allowing those parties to intervene and make informed decisions on how to proceed.

Test-Driven Development (TDD)

Test-Driven Development (TDD) embodies the idea of upfront design at the code level. With TDD, developers design and write tests before writing the actual code. This approach enforces a clear understanding of the desired functionality, leading to well-structured, modular, and maintainable code. TDD’s red-green-refactor cycle encourages developers to iterate through design decisions, making the design process an integral part of writing code.

Performance Testing

Credit: Atlassian

Performance testing embraces the upfront consideration and design of a system’s responsiveness under varying load profiles. By simulating real-world usage scenarios and stress conditions, performance testing aids in uncovering bottlenecks, resource constraints, and scalability concerns early in the development lifecycle. This proactive approach allows for architectural adjustments and optimizations that can significantly impact the system’s overall performance.

The upfront consideration and design of how the application will behave under various load profiles requires the research and identification of those load profiles, establishing SLAs, and even considering customer happiness indicators: SLOs and SLIs.

End-to-End Testing

End-to-End testing focuses on the upfront design of a system’s intercommunication across distributed components. By simulating user interactions and data flow through the entire software stack, end-to-end testing uncovers integration issues, data inconsistencies, and communication failures. This practice ensures that the system’s components collaborate seamlessly, fulfilling business requirements cohesively.

With upfront planning, a smoke test with a very limited range of paths can be defined, such that it uncovers the intricacies of how each component within the overall system needs to be interconnected to allow data to flow through that system. COME BACK TO THIS

Security Testing

Security testing advocates for the upfront design of a system’s resistance to potential attacks. By simulating various attack vectors and vulnerabilities, security testing helps identify weak points and design flaws that could lead to security breaches. This proactive stance ensures that the software is fortified against potential threats, safeguarding sensitive data and user privacy.

Certain practices such as Threat Modelling push teams to think upfront of how their application may be vulnerable to attack and to take mitigating action, making appropriate architectural decisions prior to the implementation of code.

Those decisions can then be captured as security tests that validate the application is resilient to these attacks. Teams can take a test-driven approach, writing the security tests before writing code, then writing code that ensures the tests pass.

Conclusion

Incorporating testing practices as upfront software design techniques not only enhances the quality and reliability of software systems but also contributes to a more efficient and collaborative development process.

From defining behavior through BDD to ensuring security resilience through security testing, these practices collectively empower software teams to shape their creations proactively, minimizing rework, improving architectural decisions, and ultimately delivering software that meets and exceeds user expectations.

By embracing these testing practices as integral components of upfront design, software development can navigate challenges with foresight and finesse, ensuring a robust foundation for successful products.

Text generated from my summarised list of bullet points by ChatGPT, August 10, 2023, OpenAI, https://chat.openai.com. Edited for style and content.

Self Testing Code - Changing Code with Confidence

Sun, 06 Aug 2023 22:00:00 +0000

Test Driven Development

In the ever-evolving landscape of software development, where change is the only constant, the need for robust testing practices has never been more critical. Martin Fowler’s insightful exploration of self-testing code has illuminated a path that not only enhances the quality of our code but empowers developers with a newfound confidence to shape and mould systems with agility.

Self-testing code is “the practice of writing comprehensive automated tests in conjunction with the functional software.”

At the heart of self-testing code lies a profound benefit: the confidence to implement changes. In traditional development cycles, every alteration, no matter how minor, can lead to a cascade of unintended consequences. This uncertainty can stifle progress and hinder innovation. However, by cultivating a culture of self-testing code, we unlock greater confidence that our changes will not break existing functionality. This confidence liberates us to experiment, optimize, and innovate, transforming the development process into a journey of continuous improvement rather than a daunting tightrope walk.

One renowned technique that embodies the spirit of self-testing code is Test-Driven Development (TDD). TDD is more than just a testing approach; it’s a design philosophy that guides us to build code with a clear purpose and a well-defined structure. By writing tests before the actual code, TDD encourages a thoughtful and intentional approach to software development. This, in turn, naturally leads to the creation of self-testing code. TDD’s iterative cycle of red-green-refactor not only enforces test coverage but also shapes the architecture of our applications, resulting in a harmonious blend of robustness and design.

With this early focus on design, TDD prompts us to envision the end result before we start coding, driving us to think critically about interfaces, interactions, and system boundaries. This proactive design aspect of TDD sets the stage for cleaner, more maintainable codebases. Therefore, while self-testing code is a prominent outcome of TDD, its latent design benefits are equally, if not more, beneficial.

It’s important to clarify that the journey to self-testing code isn’t dictated by the sequence of writing tests. Whether tests are crafted before or after the code implementation is a tactical choice. What truly matters is the presence of these tests in the end. The essence of self-testing code revolves around having a safety net in place that ensures the reliability and stability of our systems. The path we take to arrive at these tests is secondary to the profound advantage they bring.

By embracing this paradigm, we equip ourselves with the tools to confidently move forward, knowing that our changes are built on a solid foundation. Whether we embark on this journey through TDD or other means, the destination remains the same—a realm where code is not only tested but is inherently self-testing, and where we can make changes confidently with speed and reliability.

Serving Static Content using AWS CloudFront and AWS S3

Mon, 08 Mar 2021 00:00:00 +0000

AWS CloudFront can be leveraged for caching and routing to APIs, CMSes, etc. It’s a nice cheap setup with quite a lot of flexibility and integration with AWS services. If you want something light and quick you can check out AWS Amplify or Netflify.

This tutorial covers creating the base for this stack by serving static content using AWS CloudFront (the CDN), S3 (storage for the static content). The S3 bucket will be private with serverside encryption enabled by default. AWS Route 53 (DNS service) will be configured to have requests sent to a custom domain routed to the CloudFront distribution created in this tutorial.

The source code for this tutorial can be found here.

What if I don’t want a custom domain

If no custom domain is required, and hitting the CloudFront distribution domain directly is adequate for your needs, then follow these instructions instead.

Costs

Warning! You will incur costs by following this tutorial.

Service	Cost
Domain name	.com goes for around €8 for one year (€11 to renew) with Namecheap
AWS Route 53 hosted zone	1 x €0.50 per month
AWS CloudFront	Charged on data transfer out & http/https requests
AWS S3	You pay for storage, requests, and retrievals - Caching will make retrieval minimal

Step 1: Register a Domain Name

You can use AWS Route 53 to register a domain name by following these instructions. Alternatively you can use your favourite domain registrar to register a domain. My favourite is Namecheap due to their low prices, customer support, and tooling.

This tutorial will use Route 53 to alias your domain name and the subdomain www to route traffic to the CloudFront distribution e.g.

example.com
www.example.com

Step 2: Create a Hosted Zone

If you decided to use AWS Route 53 to register a domain name then a Route 53 hosted zone will already have been created for you. Take note of the Hosted Zone Id. Now go straight to step 4.

Otherwise, if you decided to use a domain registrar other than AWS Route 53 then create an AWS Route 53 hosted zone by using the following CloudFormation template:

AWSTemplateFormatVersion: '2010-09-09'

Parameters:
  DomainName:
    Type: String

Resources:
  HostedZone:
    Type: AWS::Route53::HostedZone
    Properties:
      Name: !Ref DomainName

Outputs:
  HostedZoneId:
    Description: The Hosted Zone Id
    Value: !Ref HostedZone
    Export:
      Name: !Sub ${AWS::StackName}-HostedZoneId
  HostedZoneNameServers:
    Description: The Hosted Zone Name Servers
    Value: !Join
      - ', '
      - !GetAtt HostedZone.NameServers
    Export:
      Name: !Sub ${AWS::StackName}-HostedZoneNameServers

Use the following AWS CLI command to create the hosted zone:

aws cloudformation deploy \
--template-file hosted-zone.yaml \
--stack-name mystaticwebsite-hosted-zone \
--parameter-overrides \
DomainName=<your fully qualified domain name>

Run the following AWS CLI command to get the hosted zone id and the hosted zone nameservers:

aws cloudformation describe-stacks \
--stack-name mystaticwebsite-hosted-zone

Look for the "Outputs" JSON element in the output of the above command. It should look like this:

"Outputs": [
    {
        "OutputKey": "HostedZoneId",
        "OutputValue": "<this will be your hosted zone id>",
        "Description": "The Hosted Zone Id",
        "ExportName": "mystaticwebsite-hosted-zone-HostedZoneId"
    },
    {
        "OutputKey": "HostedZoneNameServers",
        "OutputValue": "<this will be a comma separated list of the nameservers associated with your hosted zone>",
        "Description": "The Hosted Zone Name Servers",
        "ExportName": "mystaticwebsite-hosted-zone-HostedZoneNameServers"
    }
],

Take note of the hosted zone id, and the comma separated list of hosted zone name servers.

Step 3: Set Route 53 as the DNS service for the domain

To successfully have traffic to the domain routed to the CloudFront distribution, Route 53 needs to be set as the DNS service for the domain. To achieve this, the nameservers for the domain need to be set to the nameservers for the Route 53 hosted zone created in Step 2. The nameservers for the Route 53 hosted zone can be found in the comma separated list of the "OutputValue" in the output from the CloudFormation describe CLI command in Step 2.

The domain registrar used to register the domain will have a console that can be used to update the nameservers for the domain. For example these are the instructions for Namecheap

Step 4: Create the Certificate

With the creation of the Route 53 hosted zone, and with Route 53 as the DNS service for the domain, the certificate for the static website and can be created and automatically validated by AWS Route 53. The following CloudFormation template creates the certificate using AWS ACM, however it must be created in the us-east-1 region otherwise it cannot be used with CloudFront:

AWSTemplateFormatVersion: '2010-09-09'

Parameters:
  DomainName:
    Type: String
  HostedZoneId:
    Type: String

Resources:
  Certificate:
    Type: AWS::CertificateManager::Certificate
    Properties:
      DomainName: !Ref DomainName
      SubjectAlternativeNames:
        - !Ref DomainName
        - !Sub www.${DomainName}
      DomainValidationOptions:
        - DomainName: !Ref DomainName
          HostedZoneId: !Ref HostedZoneId
        - DomainName: !Sub www.${DomainName}
          HostedZoneId: !Ref HostedZoneId
      ValidationMethod: DNS

Outputs:
  CertificateArn:
    Description: The Certificate ARN
    Value: !Ref Certificate
    Export:
      Name: !Sub ${AWS::StackName}-CertificateArn

This is the AWS CLI command to create the stack:

aws cloudformation deploy \
--template-file acm-certificate.yaml \
--stack-name mystaticwebsite-acm-certificate \
--region us-east-1 \
--parameter-overrides \
DomainName=<your fully qualified domain name> \
HostedZoneId=<hosted zone id>

Notice the region is set to us-east-1. This is necessary otherwise the certificate won’t work with CloudFront.

Once the stack is created, run the following command to get the certificate ARN, this is required when creating the CloudFront distribution.

aws cloudformation describe-stacks \
--region us-east-1 \
--stack-name mystaticwebsite-acm-certificate

The Outputs JSON element will look like this:

"Outputs": [
    {
        "OutputKey": "CertificateArn",
        "OutputValue": "arn:aws:acm:us-east-1:111111111111:certificate/b8d0e2c9-daf7-42e8-a59b-3693bc299c32",
        "Description": "The Certificate ARN",
        "ExportName": "mystaticwebsite-acm-certificate-CertificateArn"
    }
],

Take note of the ARN which will look like this: arn:aws:acm:us-east-1:111111111111:certificate/b8d0e2c9-daf7-42e8-a59b-3693bc299c32

Step 5: Create The CloudFront Distribution

The following CloudFormation template creates the Origin Access Identity, static resources S3 bucket, and CloudFront distribution:

AWSTemplateFormatVersion: '2010-09-09'

Parameters:
  DomainName:
    Type: String
  CertificateArn:
    Type: String

Resources:
  OriginAccessIdentity:
    Type: AWS::CloudFront::CloudFrontOriginAccessIdentity
    Properties:
      CloudFrontOriginAccessIdentityConfig:
        Comment: !Sub ${AWS::StackName}-s3-origin-oai

  StaticResourcesBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketEncryption:
        ServerSideEncryptionConfiguration:
          - ServerSideEncryptionByDefault:
              SSEAlgorithm: AES256
      PublicAccessBlockConfiguration:
        BlockPublicAcls: true
        BlockPublicPolicy: true
        IgnorePublicAcls: true
        RestrictPublicBuckets: true

  StaticResourcesBucketPolicy:
    Type: AWS::S3::BucketPolicy
    Properties:
      Bucket: !Ref StaticResourcesBucket
      PolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              AWS: !Sub arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity ${OriginAccessIdentity}
            Action: s3:GetObject
            Resource: !Sub arn:aws:s3:::${StaticResourcesBucket}/*

  Distribution:
    Type: AWS::CloudFront::Distribution
    Properties:
      DistributionConfig:
        Aliases:
          - !Ref DomainName
          - !Sub www.${DomainName}
        Origins:
          - DomainName: !Sub ${StaticResourcesBucket}.s3.${AWS::Region}.amazonaws.com
            Id: S3Origin
            S3OriginConfig:
              OriginAccessIdentity: !Sub origin-access-identity/cloudfront/${OriginAccessIdentity}
        Enabled: true
        DefaultRootObject: index.html
        DefaultCacheBehavior:
          AllowedMethods:
            - DELETE
            - GET
            - HEAD
            - OPTIONS
            - PATCH
            - POST
            - PUT
          TargetOriginId: S3Origin
          ForwardedValues:
            QueryString: false
            Cookies:
              Forward: none
          ViewerProtocolPolicy: redirect-to-https
          CachePolicyId: 658327ea-f89d-4fab-a63d-7e88639e58f6
        PriceClass: PriceClass_All
        ViewerCertificate:
          AcmCertificateArn: !Ref CertificateArn
          SslSupportMethod: sni-only

Outputs:
  DistributionId:
    Description: CloudFront Distribution Id
    Value: !Ref Distribution
    Export:
      Name: !Sub ${AWS::StackName}-DistributionId
  DistributionDomainName:
    Description: CloudFront Distribution Domain Name
    Value: !GetAtt Distribution.DomainName
    Export:
      Name: !Sub ${AWS::StackName}-DistributionDomainName
  StaticResourcesBucketName:
    Description: Static Resources Bucket Name
    Value: !Ref StaticResourcesBucket
    Export:
      Name: !Sub ${AWS::StackName}-StaticResourcesBucketName

Run the following command to create the CloudFormation stack:

aws cloudformation deploy \
--template-file cloudfront-distribution.yaml \
--stack-name mystaticwebsite-cloudfront-distribution \
--parameter-overrides \
DomainName=<your domain name> \
CertificateArn=<certificate arn>

Origin Access Identity

The Origin Access Identity allows CloudFront to read from the S3 bucket without having to make the S3 bucket public.

S3 Bucket

The static content is stored in the S3 bucket from which CloudFront will serve the content. My preference is to encrypt all data whether it’s at rest or in-flight, so here serverside encryption is enabled by default. It’s data that will be publicly accessible through CloudFront, so the default AWS S3 key would appear to be adequate here despite it being a shared key across AWS accounts.

The S3 bucket is explicitly configured as private with the PublicAccessBlockConfiguration.

S3 Bucket Policy

It is necessary to give s3:GetObject permission to the Origin Access Identity so that CloudFront can request items from the S3 bucket. This means that the bucket can be kept private but that CloudFront can still access the static content within.

Cache Policy

The cache policy id for the default cache behaviour is set to the managed cache policy 658327ea-f89d-4fab-a63d-7e88639e58f6

Managed-CachingOptimized - which means caching is enabled and CloudFront cache invalidation is required after making changes to static content files.

Try It Out

It should now be possible test the CloudFront distribution. Upload an index.html file to the S3 bucket then send an HTTP request to the CloudFront URL. The name of the S3 bucket and CloudFront URL are required. Run the following command to get the CloudFront distribution stack outputs:

aws cloudformation describe-stacks \
--stack-name mystaticwebsite-cloudfront-distribution

The output should look like the following:

"Outputs": [
    {
        "OutputKey": "DistributionId",
        "OutputValue": "<distribution Id>",
        "Description": "CloudFront Distribution Id",
        "ExportName": "mystaticwebsite-cloudfront-distribution-DistributionId"
    },
    {
        "OutputKey": "DistributionDomainName",
        "OutputValue": "d1111111111111.cloudfront.net",
        "Description": "CloudFront Distribution Domain Name",
        "ExportName": "mystaticwebsite-cloudfront-distribution-DistributionDomainName"
    },
    {
        "OutputKey": "StaticResourcesBucketName",
        "OutputValue": "mystaticwebsite-cloudfront-staticresourcesbucket-1ab0a0a0a9abc",
        "Description": "Static Resources Bucket Name",
        "ExportName": "mystaticwebsite-cloudfront-distribution-StaticResourcesBucketName"
    }
],

Take note of the OutputValue of "OutputKey": "DistributionDomainName" and "OutputKey": "StaticResourcesBucketName".

Run the following command to create a file name index.html and upload it to the S3 bucket:

echo 'My Static Content' > index.html && \
aws s3 cp index.html s3://<static resources bucket name>

With a browser, request the CloudFront URL taken from the stack output to see the content returned from CloudFront. Alternatively the following cURL command can be used:

curl https://<cloudfront distribution domain>

The response should look like this:

My Static Content

Step 6: Route Traffic To The CloudFront Distribution

A further step is required to have requests to the domain routed to the CloudFront distribution. It requires Route 53 alias records to be created:

IPV4 alias record (A) pointing the root domain to the CloudFront distribution
IPV4 alias record (A) pointing the www subdomain to the CloudFront distribution
IPV6 alias record (AAAA) pointing the root domain to the CloudFront distribution
IPV6 alias record (AAAA) pointing the www subdomain to the CloudFront distribution

The following CloudFormation template sets up the appropriate records:

AWSTemplateFormatVersion: '2010-09-09'

Parameters:
  DomainName:
    Type: String
  HostedZoneId:
    Type: String
  DistributionDomainName:
    Type: String

Resources:
  HostedZoneRecordSetGroup:
    Type: AWS::Route53::RecordSetGroup
    Properties:
      HostedZoneId: !Ref HostedZoneId
      RecordSets:
        - Name: !Ref DomainName
          Type: A
          AliasTarget:
            HostedZoneId: Z2FDTNDATAQYW2
            DNSName: !Ref DistributionDomainName
        - Name: !Sub www.${DomainName}
          Type: A
          AliasTarget:
            HostedZoneId: Z2FDTNDATAQYW2
            DNSName: !Ref DistributionDomainName
        - Name: !Ref DomainName
          Type: AAAA
          AliasTarget:
            HostedZoneId: Z2FDTNDATAQYW2
            DNSName: !Ref DistributionDomainName
        - Name: !Sub www.${DomainName}
          Type: AAAA
          AliasTarget:
            HostedZoneId: Z2FDTNDATAQYW2
            DNSName: !Ref DistributionDomainName

Run the following command to create the CloudFormation stack:

aws cloudformation deploy \
--template-file mystaticwebsite-record-set-group.yaml \
--stack-name mystaticwebsite-record-set-group \
--parameter-overrides \
DomainName=<your domain name> \
HostedZoneId=<hosted zone id> \
DistributionDomainName=<cloudfront distribution domain name>

Try it Out

Navigate to the domain name in a browser or run the following command:

curl https://<your domain name>

The following should be returned:

My Static Content

No domain no problem

If no domain is required, and hitting the CloudFront distribution domain directly is adequate for your needs, then this single CloudFormation template will suffice:

AWSTemplateFormatVersion: '2010-09-09'

Resources:
  OriginAccessIdentity:
    Type: AWS::CloudFront::CloudFrontOriginAccessIdentity
    Properties:
      CloudFrontOriginAccessIdentityConfig:
        Comment: !Sub ${AWS::StackName}-s3-origin-oai

  StaticResourcesBucket:
    Type: AWS::S3::Bucket
    Properties:
      BucketName: !Sub ${AWS::StackName}-static-resources
      BucketEncryption:
        ServerSideEncryptionConfiguration:
          - ServerSideEncryptionByDefault:
              SSEAlgorithm: AES256
      PublicAccessBlockConfiguration:
        BlockPublicAcls: true
        BlockPublicPolicy: true
        IgnorePublicAcls: true
        RestrictPublicBuckets: true

  StaticResourcesBucketPolicy:
    Type: AWS::S3::BucketPolicy
    Properties:
      Bucket: !Ref StaticResourcesBucket
      PolicyDocument:
        Statement:
          - Effect: Allow
            Principal:
              AWS: !Sub arn:aws:iam::cloudfront:user/CloudFront Origin Access Identity ${OriginAccessIdentity}
            Action: s3:GetObject
            Resource: !Sub arn:aws:s3:::${StaticResourcesBucket}/*

  Distribution:
    Type: AWS::CloudFront::Distribution
    Properties:
      DistributionConfig:
        Origins:
          - DomainName: !Sub ${StaticResourcesBucket}.s3.${AWS::Region}.amazonaws.com
            Id: S3Origin
            S3OriginConfig:
              OriginAccessIdentity: !Sub origin-access-identity/cloudfront/${OriginAccessIdentity}
        Enabled: true
        DefaultRootObject: index.html
        DefaultCacheBehavior:
          AllowedMethods:
            - DELETE
            - GET
            - HEAD
            - OPTIONS
            - PATCH
            - POST
            - PUT
          TargetOriginId: S3Origin
          ForwardedValues:
            QueryString: false
            Cookies:
              Forward: none
          ViewerProtocolPolicy: redirect-to-https
          CachePolicyId: 658327ea-f89d-4fab-a63d-7e88639e58f6
        PriceClass: PriceClass_All

Outputs:
  DistributionId:
    Description: CloudFront Distribution Id
    Value: !Ref Distribution
    Export:
      Name: !Sub ${AWS::StackName}-DistributionId
  DistributionDomainName:
    Description: CloudFront Distribution Domain Name
    Value: !GetAtt Distribution.DomainName
    Export:
      Name: !Sub ${AWS::StackName}-DistributionDomainName
  StaticResourcesBucketName:
    Description: Static Resources Bucket Name
    Value: !Ref StaticResourcesBucket
    Export:
      Name: !Sub ${AWS::StackName}-StaticResourcesBucketName

Run the following command to create the CloudFormation stack:

aws cloudformation deploy \
--template-file cloudfront-distribution-no-custom-domain.yaml \
--stack-name mystaticwebsite-cloudfront-distribution

Run the following command to retrieve the CloudFront distribution domain:

aws cloudformation describe-stacks \
--stack-name mystaticwebsite-cloudfront-distribution

The output should look like the following:

"Outputs": [
    {
        "OutputKey": "DistributionId",
        "OutputValue": "<distribution Id>",
        "Description": "CloudFront Distribution Id",
        "ExportName": "mystaticwebsite-cloudfront-distribution-DistributionId"
    },
    {
        "OutputKey": "DistributionDomainName",
        "OutputValue": "d1111111111111.cloudfront.net",
        "Description": "CloudFront Distribution Domain Name",
        "ExportName": "mystaticwebsite-cloudfront-distribution-DistributionDomainName"
    },
    {
        "OutputKey": "StaticResourcesBucketName",
        "OutputValue": "mystaticwebsite-cloudfront-staticresourcesbucket-1ab0a0a0a9abc",
        "Description": "Static Resources Bucket Name",
        "ExportName": "mystaticwebsite-cloudfront-distribution-StaticResourcesBucketName"
    }
],

Take note of the OutputValue of "OutputKey": "DistributionDomainName" and "OutputKey": "StaticResourcesBucketName".

Run the following command to create a file name index.html and upload it to the S3 bucket:

echo 'My Static Content' > index.html && \
aws s3 cp index.html s3://<static resources bucket name>

With a browser, request the CloudFront URL taken from the stack output to see the content returned from CloudFront. Alternatively the following cURL command can be used:

curl https://<cloudfront distribution domain>

The response should look like this:

My Static Content

Safer Deployments with CodeDeploy Canary Deployments

Wed, 30 Dec 2020 00:00:00 +0000

Without a good rollout and rollback strategy, there is greater risk of releasing breaking changes or broken software that impacts all users for an extended period of time. This can erode confidence in your releases and customers’ confidence in your products.

Canary deployments can help minimise this risk by first routing a small percentage of traffic to the new version for a configured amount of time, before routing the remaining traffic to the new version. If any errors are detected during the initial routing then all traffic is routed back to the previous version.

AWS CodeDeploy provides native support for canary deployments of Lambdas. The AWS Serverless Application Model (SAM) provides abstractions to more easily configure CodeDeploy canary deployments of Lambdas using CloudFormation.

This blog post describes how to implement Lambda canary deployments using CodeDeploy and SAM, with the added bonus of a pre-traffic automated test Lambda for smoke testing the new version. CloudWatch Alarms trigger automatic rollback on increased error detection during the initial traffic shifting phase of the deployment.

CodeDeploy requires Lambda Versions and a Lambda Alias to provide support for canary deployments. A Lambda Version is an immutable snapshot of a Lambda at a point in time, and a Lambda Alias routes traffic to a specific Lambda Version. CodeDeploy begins a canary deployment by creating a new Lambda Version, then routes a certain percentage of traffic to that Lambda Version. If no errors are detected during the deployment timeframe, which is configurable, CodeDeploy will point the Lambda Alias to the new Lambda Version, thereby shifting 100% traffic to the new version.

Example Serverless Application

Here’s a CloudFormation stack with a simple example of this in action. An API Gateway RESTful API backed by two Lambdas uses CodeDeploy for canary deployments. Test Lambdas are defined to run pre and post-traffic tests against the new versions of ExampleAFunction and ExampleBFunction. CloudWatch Alarms for the alias and current version of ExampleALambda and ExampleBLambda are also defined. The code for this example can be found here.

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Globals:
  Function:
    Runtime: nodejs12.x
    MemorySize: 128
    Timeout: 30

Resources:

  ExampleApi:
    Type: AWS::Serverless::Api
    Properties:
      StageName: live

  ExampleAFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: index.handler
      InlineCode: |
        exports.handler = (event, context, callback) => {
        	callback(
        		null,
        		{
        			statusCode: 200,
        			body: JSON.stringify({
        				message: 'Hello World A'
        			})
        		});
        };        
      AutoPublishAlias: live
      Events:
        ExampleApiEvent:
          Type: Api
          Properties:
            RestApiId: !Ref ExampleApi
            Path: /example/a
            Method: get
      DeploymentPreference:
        Type: Canary10Percent5Minutes
        Alarms:
          - !Ref ExampleAAliasErrorMetricGreaterThanZeroAlarm
          - !Ref ExampleALatestVersionErrorMetricGreaterThanZeroAlarm
        Hooks:
          PreTraffic: !Ref PreTrafficLambdaFunction
          PostTraffic: !Ref PostTrafficLambdaFunction

  ExampleAAliasErrorMetricGreaterThanZeroAlarm:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmDescription: Lambda Function Error > 0
      ComparisonOperator: GreaterThanThreshold
      Dimensions:
        - Name: Resource
          Value: !Sub ${ExampleAFunction}:live
        - Name: FunctionName
          Value: !Ref ExampleAFunction
      EvaluationPeriods: 2
      MetricName: Errors
      Namespace: AWS/Lambda
      Period: 60
      Statistic: Sum
      Threshold: 0

  ExampleALatestVersionErrorMetricGreaterThanZeroAlarm:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmDescription: Lambda Function Error > 0
      ComparisonOperator: GreaterThanThreshold
      Dimensions:
        - Name: Resource
          Value: !Sub ${ExampleAFunction}:live
        - Name: FunctionName
          Value: !Ref ExampleAFunction
        - Name: ExecutedVersion
          Value:
            Fn::GetAtt:
              - ExampleAFunction.Version
              - Version
      EvaluationPeriods: 2
      MetricName: Errors
      Namespace: AWS/Lambda
      Period: 60
      Statistic: Sum
      Threshold: 0

  ExampleBFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: index.handler
      InlineCode: |
        exports.handler = (event, context, callback) => {
        	callback(
        		null,
        		{
        			statusCode: 200,
        			body: JSON.stringify({
        				message: 'Hello World B'
        			})
        		});
        };        
      AutoPublishAlias: live
      Events:
        ExampleAApiEvent:
          Type: Api
          Properties:
            RestApiId: !Ref ExampleApi
            Path: /example/b
            Method: get
      DeploymentPreference:
        Type: Canary10Percent5Minutes
        Alarms:
          - !Ref ExampleBAliasErrorMetricGreaterThanZeroAlarm
          - !Ref ExampleBLatestVersionErrorMetricGreaterThanZeroAlarm
        Hooks:
          PreTraffic: !Ref PreTrafficLambdaFunction
          PostTraffic: !Ref PostTrafficLambdaFunction

  ExampleBAliasErrorMetricGreaterThanZeroAlarm:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmDescription: Lambda Function Error > 0
      ComparisonOperator: GreaterThanThreshold
      Dimensions:
        - Name: Resource
          Value: !Sub ${ExampleBFunction}:live
        - Name: FunctionName
          Value: !Ref ExampleBFunction
      EvaluationPeriods: 2
      MetricName: Errors
      Namespace: AWS/Lambda
      Period: 60
      Statistic: Sum
      Threshold: 0

  ExampleBLatestVersionErrorMetricGreaterThanZeroAlarm:
    Type: AWS::CloudWatch::Alarm
    Properties:
      AlarmDescription: Lambda Function Error > 0
      ComparisonOperator: GreaterThanThreshold
      Dimensions:
        - Name: Resource
          Value: !Sub ${ExampleBFunction}:live
        - Name: FunctionName
          Value: !Ref ExampleBFunction
        - Name: ExecutedVersion
          Value:
            Fn::GetAtt:
              - ExampleBFunction.Version
              - Version
      EvaluationPeriods: 2
      MetricName: Errors
      Namespace: AWS/Lambda
      Period: 60
      Statistic: Sum
      Threshold: 0

  PreTrafficLambdaFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: index.handler
      InlineCode: |
        "use strict";

        const AWS = require("aws-sdk");
        const codedeploy = new AWS.CodeDeploy();

        exports.handler = (event, context, callback) => {

          console.log("Entering PreTraffic hook.");

          // Read the DeploymentId and LifecycleEventHookExecutionId from the event payload
          const deploymentId = event.DeploymentId;
          const lifecycleEventHookExecutionId = event.LifecycleEventHookExecutionId;
          var validationTestResult = "Failed";

          // Perform PreTraffic validation tests here. Set the test result
          // to "Succeeded" for this tutorial.
          console.log("This is where PreTraffic validation tests happen.")
          validationTestResult = "Succeeded";

          // Complete the PreTraffic hook by sending CodeDeploy the validation status
          const params = {
            deploymentId: deploymentId,
            lifecycleEventHookExecutionId: lifecycleEventHookExecutionId,
            status: validationTestResult // status can be 'Succeeded' or 'Failed'
          };

          // Pass AWS CodeDeploy the prepared validation test results.
          codedeploy.putLifecycleEventHookExecutionStatus(params, (err, data) => {
           if (err) {
             // Validation failed.
             console.log('PreTraffic validation tests failed');
             console.log(err, err.stack);
             callback("CodeDeploy Status update failed");
           } else {
             // Validation succeeded.
             console.log("PreTraffic validation tests succeeded");
             callback(null, "PreTraffic validation tests succeeded");
           }
          });
        }        
      Policies:
        - Version: 2012-10-17
          Statement:
            - Effect: Allow
              Action:
                - codedeploy:PutLifecycleEventHookExecutionStatus
              Resource: !Sub arn:${AWS::Partition}:codedeploy:${AWS::Region}:${AWS::AccountId}:deploymentgroup:${ServerlessDeploymentApplication}/*
        - Version: 2012-10-17
          Statement:
            - Effect: Allow
              Action:
                - lambda:InvokeFunction
              Resource:
                - !GetAtt ExampleAFunction.Arn
                - !GetAtt ExampleBFunction.Arn
      FunctionName: CodeDeployHook_preTrafficHook
      Environment:
        Variables:
          ExampleAFunctionCurrentVersion: !Ref ExampleAFunction.Version
          ExampleBFunctionCurrentVersion: !Ref ExampleBFunction.Version

  PostTrafficLambdaFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: index.handler
      InlineCode: |
        "use strict";

        const AWS = require("aws-sdk");
        const codedeploy = new AWS.CodeDeploy();

        exports.handler = (event, context, callback) => {

          console.log("Entering PostTraffic hook.");

          // Read the DeploymentId and LifecycleEventHookExecutionId from the event payload
          const deploymentId = event.DeploymentId;
          const lifecycleEventHookExecutionId = event.LifecycleEventHookExecutionId;
          var validationTestResult = "Failed";

          // Perform PostTraffic validation tests here. Set the test result
          // to "Succeeded" for this tutorial.
          console.log("This is where PostTraffic validation tests happen.")
          validationTestResult = "Succeeded";

          // Complete the PostTraffic hook by sending CodeDeploy the validation status
          const params = {
            deploymentId: deploymentId,
            lifecycleEventHookExecutionId: lifecycleEventHookExecutionId,
            status: validationTestResult // status can be 'Succeeded' or 'Failed'
          };

          // Pass AWS CodeDeploy the prepared validation test results.
          codedeploy.putLifecycleEventHookExecutionStatus(params, (err, data) => {
           if (err) {
             // Validation failed.
             console.log('PostTraffic validation tests failed');
             console.log(err, err.stack);
             callback("CodeDeploy Status update failed");
           } else {
             // Validation succeeded.
             console.log("PostTraffic validation tests succeeded");
             callback(null, "PostTraffic validation tests succeeded");
           }
          });
        }        
      Policies:
        - Version: 2012-10-17
          Statement:
            - Effect: Allow
              Action:
                - codedeploy:PutLifecycleEventHookExecutionStatus
              Resource: !Sub arn:${AWS::Partition}:codedeploy:${AWS::Region}:${AWS::AccountId}:deploymentgroup:${ServerlessDeploymentApplication}/*
        - Version: 2012-10-17
          Statement:
            - Effect: Allow
              Action:
                - lambda:InvokeFunction
              Resource:
                - !GetAtt ExampleAFunction.Arn
                - !GetAtt ExampleBFunction.Arn
      FunctionName: CodeDeployHook_postTrafficHook
      Environment:
        Variables:
          ExampleAFunctionCurrentVersion: !Ref ExampleAFunction.Version
          ExampleBFunctionCurrentVersion: !Ref ExampleBFunction.Version

SAM Reduces Boilerplate

Just a few lines in the CloudFormation applies the CodeDeploy canary configuration courtesy of the SAM transformation:

DeploymentPreference:
  Type: Canary10Percent5Minutes
    Alarms:
      - !Ref ExampleAAliasErrorMetricGreaterThanZeroAlarm
      - !Ref ExampleALatestVersionErrorMetricGreaterThanZeroAlarm
    Hooks:
      PreTraffic: !Ref PreTrafficLambdaFunction
      PostTraffic: !Ref PostTrafficLambdaFunction

SAM transforms the CloudFormation to create the following resources:

CodeDeploy Application
CodeDeploy DeploymentGroup per Lambda
CodeDeployServiceRole
Lambda Alias with an UpdatePolicy applying CodeDeploy Application, Deployment Group, and pre/post-traffic hooks

Pre/Post-Traffic Tests

Pre- traffic tests against the new version, validating service contracts and using known test scenarios, provide greater confidence in a deployment being successful. Additional tests can be added over time as the system evolves and more weaknesses are revealed. If these tests fail then no traffic is shifted to the new version and no customers are affected.

The example includes a post-traffic test Lambda for some post-traffic shifting smoke testing should the need arise.

Ideally these tests can be run in parallel to minimise execution time and provide rapid feedback. Tests in non-production environments can be more thorough and numerous, employing test data and mocked boundaries to exercise known scenarios that may not be possible to test in production.

CloudWatch Alarms Trigger Rollback

During initial traffic shifting, CloudWatch Alarms monitor error rates of the new version and if triggered will fail the deployment and automatically rollback to the previous version. Only a small number of requests will be negatively impacted before rollback occurs and traffic is shifted back to the previous version. If all goes well, the remainder of the traffic is shifted to the new version.

Phased Rollouts

This strategy encourages you to adopt a phased rollout strategy that introduces no breaking changes, and therefore requires no downtime for your applications. Thought has to go into architecting and coding your applications so that multiple versions can co-exist. For example how will database, contract, or configuration changes be handled and rolled back?

Business Case

There needs to be business cases behind the architectural design decisions that you make; in this case not all solutions require 100% uptime, with phased rollout of changes. It may not even be possible in certain circumstances. However, when services like AWS CodeDeploy make it so easy to apply these strategies it’s almost more work not to adopt this strategy.

AWS Well-Architected Best Practice

This strategy is described as a best practice by the Serverless Application Lens of the AWS Well-Architected Framework, falling within the Operational Excellence Pillar. The ability to build applications that run with 0% downtime is a valuable, marketable skill. Operation excellence is a bar that continues to rise and with expectations on developers only increasing it makes sense to keep your skills sharp.

With Lambdas, you pay for what you use, so previous versions, now unused, will not incur costs and can be retired as necessary. Adopting this strategy helps you optimise costs.

Caveats

CodeDeploy canary deployments use request based traffic shifting, not user based traffic shifting so some or all of your users could be impacted, albeit a much smaller number of requests.

CloudWatch Alarms gather metrics against the alias and not the version so it may lead to false positives against the new version should the previous version begin to fail.

The automatic cleanup of old versions is encouraged so as not to run out of Lambda storage space, which typically occurs at exactly the wrong moment, and can prevent you from deploying. An EventBridge Scheduled Event can be configured to periodically run a Lambda that removes older versions.

When using SAM to apply a CodeDeploy deployment configuration to a Lambda, pre/post traffic hook Lambda names must start with: CodeDeployHook_

Charges Apply!

Charges apply so be sure to remove all resources created after experimentation.

References

AWS Well-Architected Framework: Serverless lens

Serverless Application Model (SAM): AWS::Serverless::Function (DeploymentPreference)

AWS Serverless Application Model Developer Guide: Deploying serverless applications gradually

Burning Monk (guest post for Lumigo): AWS Lambda Canary Deployment

Encrypt DynamoDB with a Customer Managed KMS Key

Thu, 22 Oct 2020 00:00:00 +0000

Customer managed CMKs can be fully controlled (key rotation, key policies, IAM policies, etc). The examples use CloudFormation to create the CMK, DynamoDB table, and test Lambda.

Charges Apply!

Charges apply so be sure to remove all resources created after experimentation.

TL;DR

Use a Customer managed customer master key (CMK)
Add a condition to the CMK key policy to allow DynamoDB to use it
Configure a DynamoDB table with SSE using the CMK
1 CMK is ~$1.00 per month + $0.03 per 10,000 requests

Keep it secret, keep it safe

DynamoDB is an AWS managed key-value and document database. Server side encryption is enabled by default and uses an AWS owned customer master key (CMK).

There are three types of CMK that can be used with DynamoDB to encrypt data at rest:

AWS owned CMK (shared across multiple accounts, cannot be managed, charges do not apply)
AWS managed CMK (shared across a single account, cannot be managed, charges apply)
Customer managed CMK (can be fully managed, charges apply)

Use a Customer managed CMK for the greatest level of control. There are several advantages to using a Customer managed CMK for DynamoDB SSE.

Full control over these keys means fine grained control over key rotation, key and IAM policies, etc
Reduces blast radius when using one Customer managed CMK per table

CloudFormation

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Resources:

  DynamoDbTableKmsKey:
    Type: AWS::KMS::Key
    Properties:
      EnableKeyRotation: true
      KeyPolicy:
        Version: '2012-10-17'
        Statement:
          - Effect: Allow
            Principal:
              AWS: !Sub arn:aws:iam::${AWS::AccountId}:root
            Action: kms:*
            Resource: '*'
          - Effect: Allow
            Principal:
              AWS: '*'
            Action:
              - kms:Encrypt
              - kms:Decrypt
            Resource: '*'
            Condition:
              StringEquals:
                kms:CallerAccount: !Ref AWS::AccountId
                kms:ViaService: !Sub dynamodb.${AWS::Region}.amazonaws.com

  DynamoDbTable:
    Type: AWS::DynamoDB::Table
    Properties:
      AttributeDefinitions:
        - AttributeName: id
          AttributeType: S
      KeySchema:
        - AttributeName: id
          KeyType: HASH
      BillingMode: PAY_PER_REQUEST
      SSESpecification:
        KMSMasterKeyId: !Ref DynamoDbTableKmsKey
        SSEEnabled: true
        SSEType: KMS

  DynamoDbWriteFunction:
    Type: AWS::Serverless::Function
    Properties:
      InlineCode: !Sub
        - |
          def handler(event, context):
            import boto3
            dynamodb = boto3.client('dynamodb')
            id = event['id']
            return dynamodb.put_item(TableName='${TableName}', Item={'id':{'S':f'{id}'}})          
        - TableName: !Ref DynamoDbTable
      Handler: index.handler
      Runtime: python3.7
      Policies:
        Statement:
          - Effect: Allow
            Action:
              - dynamodb:PutItem
            Resource: !GetAtt DynamoDbTable.Arn

Specify a CMK key policy condition to allow the DynamoDB service to use the CMK for server side encryption and only from the same account
Reference the KMS CMK above when defining the DynamoDB table
Use the Lambda for inserting data

Deploying to AWS

aws cloudformation deploy --stack-name dynamodb-sse-with-kms-cmk --template-file sam.yaml --capabilities CAPABILITY_IAM

Invoke the test Lambda using the AWS CLI

aws lambda invoke \
--function-name <lambda name goes here> \
--payload '{ "id": "1" }' \
response.json

Delete the CloudFormation Stack

aws cloudformation delete-stack --stack-name dynamodb-sse-with-kms-cmk

Split CloudFormation Stacks

Sun, 14 Jul 2019 22:00:00 +0000

Placing all your resources into a single stack is going to create pain for you later on. Moving and renaming resources between stacks can be quite tricky. Once a resource is managed by a CloudFormation stack it’s not possible to remove that resource without deleting it, and it’s not possible to move a resource from one stack to another. Splitting up your resources across multiple stacks can really help. Here are some suggestions on how you can achieve that.

Resources that change together stay together

Your application may consist of an API Gateway with multiple Lambdas and SNS/SQS resources for internal processing. RDS databases with clustering. VPCs with Subnets, Security Groups, etc. These resources are logically grouped and belong in a dedicated stack.

Stateful Resources

Stateful resources such as RDS, S3, DynamoDB, etc should get their own stack. The only way to move or rename these resources is to define a new resource with a different name and copy the data across. By mixing these resources with more volatile resources you risk having to do this. Best to avoid such a scenario.

Configuration Resources

Configuration Resources such as ParameterStore and SecretsManager belong in their own stack. Changes to these resources should not require a redeployment of the application that uses them.

Application Boundaries

Resources that make up the boundary of an application such as API Gateway, SNS, Kinesis, etc should get their own stack. Other applications access these resources through a “public” contract and minimising disruption to these resources is important.

Shared Resources

Any resources shared by multiple applications should get their own stack. VPCs, Subnets, KMS, S3, etc. VPCs cannot be deleted if they are using by any resources. Removing a KMS key means losing access to data, and moving an S3 bucket means creating a new one and copying the data over to the new bucket.

Conclusion

These are some guidelines that have come in handy over time. Comment below with your experiences and recommendations.

How to use OpenAPI with AWS API Gateway

Wed, 29 May 2019 00:00:00 +0000

Use an OpenAPI specification to define API Gateway APIs. This is good practice because:

The specification forces you to think about the design upfront before touching code which is expensive to correct later
This contract first approach allows frontend, backend, and test developers to work in parallel
Endpoints can be mocked for developers to test against
Serverless Application Model (SAM) uses the specification to configure the API Gateway API, this makes it living documentation, evolving alongside code and infrastructure.

TL;DR

Create an OpenAPI specification with API Gateway Extensions to OpenAPI
Copy the file to an S3 bucket
Define an API Gateway (AWS::Serverless::Api) resource in your SAM template
Use the AWS::Include transform to include and transform the OpenAPI specification from S3
Assign the AWS::Include transform function to the DefinitionBody property of the API Gateway resource
Package and deploy your SAM application

Source Code

The source code and instructions to build and deploy this example to AWS can be found here: https://github.com/karlkyck/api-gateway-openapi. Running this example on AWS will incur costs so be sure to delete the CloudFormation stacks when you are finished experimenting.

OpenAPI Definition

Start by creating your OpenAPI specification in a dedicated file.

The following OpenAPI specification defines one RESTful endpoint with two operations on that endpoint.

---
openapi: 3.0.0
info:
  title: API Gateway OpenAPI Example
  version: 1.0.0
 
paths:
  /api/posts:
    get:
      summary: List Posts
      operationId: listPosts
      requestBody:
        required: true
        content:
          application/json:
            schema:
              '$ref': '#/components/schemas/CreatePostRequestBody'
      responses:
        '200':
          description: Retrieve the list of Posts
          content:
            application/json:
              schema:
                '$ref': '#/components/schemas/ListPostsResponseBody'
      x-amazon-apigateway-integration:
        uri:
          Fn::Sub: arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${ListPostsFunction.Arn}/invocations
        httpMethod: POST
        type: aws_proxy
    post:
      summary: Create a new Post
      operationId: createPost
      responses:
        '200':
          description: Success
          content:
            application/json:
              schema:
                '$ref': '#/components/schemas/Post'
      x-amazon-apigateway-integration:
        uri:
          Fn::Sub: arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${CreatePostFunction.Arn}/invocations
        httpMethod: POST
        type: aws_proxy
 
components:
  schemas:
    BasePost:
      type: object
      required:
        - title
        - description
        - publishedDate
        - content
      properties:
        title:
          type: string
        description:
          type: string
        publishedDate:
          type: string
          format: date-time
        content:
          type: string
    Post:
      allOf:
        - $ref: '#/components/schemas/BasePost'
        - type: object
          required:
            - id
            - createdDate
            - updatedDate
          properties:
            id:
              type: string
            createdDate:
              type: string
              format: date-time
            updatedDate:
              type: string
              format: date-time
    CreatePostRequestBody:
      allOf:
        - $ref: '#/components/schemas/BasePost'
    ListPostsResponseBody:
      type: array
      items:
        $ref: '#/components/schemas/Post'

The two x-amazon-apigateway-integration API Gateway Extensions to OpenAPI references allow us to refer to the relevant Lambda function in our SAM template.

By using CloudFormation Intrinsic Functions in your OpenAPI specification such as the Fn::Sub (variable substitution) function calls in the example above, you can refer to the relevant Lambdas backing your API in your SAM template.

SAM Definition

The following SAM template creates an API Gateway API and two Lambdas, each with an API Gateway event defined.

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
 
Parameters:
  DeploymentBucket:
    Type: String
 
Globals:
  Function:
    Runtime: nodejs8.10
    Timeout: 180
 
Resources:
 
  ApiGatewayApi:
    Type: AWS::Serverless::Api
    Properties:
      StageName: live
      EndpointConfiguration: REGIONAL
      DefinitionBody:
        'Fn::Transform':
          Name: AWS::Include
          Parameters:
            Location: !Sub s3://${DeploymentBucket}/openapi.yaml
 
  ListPostsFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: ListPostsHandler.handler
      CodeUri: ./dist
      Events:
        ApiGatewayApiEvent:
          Type: Api
          Properties:
            RestApiId: !Ref ApiGatewayApi
            Path: /api/posts
            Method: get
 
  CreatePostFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: CreatePostHandler.handler
      CodeUri: ./dist
      Events:
        ApiGatewayApiEvent:
          Type: Api
          Properties:
            RestApiId: !Ref ApiGatewayApi
            Path: /api/posts
            Method: post

The important part to focus on here is the DefinitionBody parameter of the ApiGatewayApi resource. We are using the AWS::Include transform to include the OpenAPI specification from its S3 location.

The AWS::Include transform allows you to use CloudFormation Intrinsic Functions in your OpenAPI specification such as the Fn::Sub (variable substitution) function calls in the example OpenAPI specification.

Deployment

When deploying your SAM project CloudFormation will load the OpenAPI specification from the specified S3 location performing an AWS::Include transform. Intrinsic Function calls within the OpenAPI specification will be executed, thereby transforming your OpenAPI specification accordingly.

API Documentation

The API documentation is now linked to the API Gateway API. The OpenAPI specification is also bound to your SAM template by way of references to Lambda functions and other resources and parameters. Therefore the specification and SAM template evolve together.

The OpenAPI specification is also made available in the AWS console for other developers and teams to export and consume.

Safe Deployments with API Gateway and Lambda AutoPublishAlias

Wed, 29 May 2019 00:00:00 +0000

TL;DR

Use the Serverless Application Model (SAM) AutoPublishAlias parameter to have CloudFormation automatically version and alias your Lambdas upon deployment
Reference your Lambda Alias from your OpenAPI definition
Deploy

Source Code

The source code and instructions to build and deploy this example to AWS can be found here: https://github.com/karlkyck/api-gateway-lambda-autopublishalias. Running this example on AWS will incur costs so be sure to delete the CloudFormation stacks when you are finished experimenting.

SAM (Serverless Application Model) Template

Begin by defining your SAM template. By specifying the AutoPublishAlias parameter in our Globals section applies the AutoPublishAlias parameter to each of the functions defined in the template.

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31
 
Parameters:
  DeploymentBucket:
    Type: String
 
Globals:
  Function:
    Runtime: nodejs10.x
    Timeout: 180
    AutoPublishAlias: live
 
Resources:
 
  ApiGatewayApi:
    Type: AWS::Serverless::Api
    Properties:
      StageName: live
      EndpointConfiguration: REGIONAL
      DefinitionBody:
        'Fn::Transform':
          Name: AWS::Include
          Parameters:
            Location: !Sub s3://${DeploymentBucket}/openapi.yaml
 
  ListPostsFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: ListPostsHandler.handler
      CodeUri: ./dist
      Events:
        ApiGatewayApiEvent:
          Type: Api
          Properties:
            RestApiId: !Ref ApiGatewayApi
            Path: /api/posts
            Method: get

In the above CloudFormation, live is specified as the Alias name for our newly deployed Lambda. When CloudFormation deploys this Lambda for the first time it will automatically create a new Lambda Version of 1 and a Lambda Alias called live. CloudFormation will point the Lambda Alias live to the newly created Lambda Version 1.

When there is an update to the Lambda code, upon deployment CloudFormation will create a new Lambda Version using the previous Lambda Version number incremented by 1. CloudFormation will then point the Lambda Alias ‘live’ to this new Lambda Version.

The SAM template defines the API Gateway resource that references an OpenAPI specification. The OpenAPI specification describes the API and also links API endpoints to Lambda resources within the SAM template. See how to use OpenAPI with API Gateway for more details.

OpenAPI Specification

The key here is to specify the Lambda Alias when making reference to a Lambda from the OpenAPI specification. If the Lambda Alias is left out when making reference to a Lambda function the API call will not work.

---
openapi: 3.0.0
info:
  title: API Gateway Lambda AutoPublishAlias Example
  version: 1.0.0
 
paths:
  /api/posts:
    get:
      summary: List Posts
      operationId: listPosts
      responses:
        '200':
          description: Retrieve the list of Posts
          content:
            application/json:
              schema:
                '$ref': '#/components/schemas/ListPostsResponseBody'
      x-amazon-apigateway-integration:
        uri:
          Fn::Sub: arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${ListPostsFunction.Arn}:live/invocations
        httpMethod: POST
        type: aws_proxy
 
components:
  schemas:
    BasePost:
      type: object
      required:
        - title
        - description
        - publishedDate
        - content
      properties:
        title:
          type: string
        description:
          type: string
        publishedDate:
          type: string
          format: date-time
        content:
          type: string
    Post:
      allOf:
        - $ref: '#/components/schemas/BasePost'
        - type: object
          required:
            - id
            - createdDate
            - updatedDate
          properties:
            id:
              type: string
            createdDate:
              type: string
              format: date-time
            updatedDate:
              type: string
              format: date-time
    ListPostsResponseBody:
      type: array
      items:
        $ref: '#/components/schemas/Post'

The important line is:

Fn::Sub: arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${ListPostsFunction.Arn}:**live**/invocations

Wrapping Up

Upon deployment the API Gateway endpoint will point to a Lambda Alias. If in an emergency a rollback is required the Lambda Alias can be repointed to a previous version. This can be done manually through the AWS Console. There is a way to automate this process which will be covered in a separate blog post.

References

IP Filtering with AWS API Gateway Resource Policies

Fri, 03 May 2019 00:00:00 +0000

TL;DR

Use an OpenAPI specification with the x-amazon-apigateway-policy API Gateway Extension to OpenAPI to apply an API Gateway Resource Policy when deploying:

---
openapi: 3.0.3

...

x-amazon-apigateway-policy:
  Version: '2012-10-17'
  Statement:
    - Effect: Allow
      Principal: '*'
      Action: execute-api:Invoke
      Resource:
        - execute-api:/*/*/*
      Condition:
        IpAddress:
          'aws:SourceIp':
            Ref: AllowedIpsList

...

Where AllowedIpsList is a comma separated list of IPs or CIDR blocks defined as a parameter in your CloudFormation template.

Rationale

When it comes to accessing and consuming resources on the cloud a least privileged approach is best. IP restriction on your API Gateway APIs can help.

It is possible to apply an API Gateway Resource Policy to an API Gateway API during deployment via CloudFormation.

CloudFormation Template

Your API Gateway API definition and reference to your OpenAPI specification is defined in your SAM (Serverless Application Model) template.

AWSTemplateFormatVersion: '2010-09-09'
Transform: AWS::Serverless-2016-10-31

Parameters:
  AllowedIpsList:
    Type: List<String>

Globals:
  Function:
    Runtime: nodejs12.x
    Timeout: 30
    AutoPublishAlias: live

Resources:

  ApiGatewayApi:
    Type: AWS::Serverless::Api
    Properties:
      StageName: live
      EndpointConfiguration: REGIONAL
      DefinitionBody:
        openapi: 3.0.3
        info:
          title: API Gateway IP Filtering Example API
          version: 1.0.0

        x-amazon-apigateway-policy:
          Version: '2012-10-17'
          Statement:
            - Effect: Allow
              Principal: '*'
              Action: execute-api:Invoke
              Resource:
                - execute-api:/*/*/*
              Condition:
                IpAddress:
                  'aws:SourceIp':
                    Ref: AllowedIpsList

        paths:
          /api/example:
            get:
              summary: Example API Endpoint
              operationId: example
              responses:
                '200':
                  description: Success
              x-amazon-apigateway-integration:
                uri:
                  Fn::Sub: arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${ExampleFunction.Arn}:live/invocations
                httpMethod: POST
                type: aws_proxy

  ExampleFunction:
    Type: AWS::Serverless::Function
    Properties:
      Handler: index.handler
      InlineCode: |
        exports.handler = (event, context, callback) => {
        	callback(
        		null,
        		{
        			statusCode: 200,
        			body: JSON.stringify({
        				message: 'Hello World'
        			})
        		});
        };        
      Events:
        ApiGatewayApiEvent:
          Type: Api
          Properties:
            RestApiId: !Ref ApiGatewayApi
            Path: /api/example
            Method: get

The DefinitionBody property of the ApiGatewayApi allows you to specify an OpenAPI specification definition which is transformed and applied by CloudFormation upon deployment.

The list of allowed IPs is passed to the SAM template as the AllowedIpsList parameter. This parameter is referenced from within the OpenAPI specification using the Ref function.

OpenAPI Specification

The OpenAPI specification, embedded in the CloudFormation template above, contains the definition of your API. The API Gateway Resource Policy is declared in the specification as an API Gateway Extension to OpenAPI.

---
openapi: 3.0.3
info:
  title: API Gateway IP Filtering Example API
  version: 1.0.0

x-amazon-apigateway-policy:
  Version: '2012-10-17'
  Statement:
    - Effect: Allow
      Principal: '*'
      Action: execute-api:Invoke
      Resource:
        - execute-api:/*/*/*
      Condition:
        IpAddress:
          'aws:SourceIp':
            Ref: AllowedIpsList

paths:
  /api/example:
    get:
      summary: Example API Endpoint
      operationId: example
      responses:
        '200':
          description: Success
      x-amazon-apigateway-integration:
        uri:
          Fn::Sub: arn:aws:apigateway:${AWS::Region}:lambda:path/2015-03-31/functions/${ExampleFunction.Arn}:live/invocations
        httpMethod: POST
        type: aws_proxy

Here we see the Ref function referring to the AllowedIpsList parameter.

This API Gateway Resource Policy is allowing all invocations where the source IP is in the list of IPs supplied.

Source Code

The source code and instructions to build and deploy this example to AWS can be found here: https://github.com/karlkyck/api-gateway-ip-filtering. Running this example on AWS will incur costs so be sure to delete the CloudFormation stacks when you are finished experimenting.

Conclusion

This is a simple, cheap way to allow access to your API Gateway APIs only from certain IP addresses. AWS WAF is an alternative, but it incurs costs.

Non-blocking Webapps with Spring Boot & CompletableFuture

Wed, 19 Jul 2017 22:00:00 +0000

CompletableFuture, introduced in Java 8, provides an easy way to write asynchronous, non-blocking, multithreaded code. Since Spring 4.2 it’s now possible to have Controllers, Services, and Repositories return CompletableFuture from non-private methods annotated with @Async. In this blog post we will see how we can take advantage of this to write non-blocking, asynchronous code across multiple layers within our application.

You can find the source code for the sample project for this post here

Interface Driven Multilayered Architecture

Our application will consist of 3 layers of responsibility with each layer performing a specific role in the application:

The Controller layer for presentation (JSON)
The Service layer for business logic
The Repository layer for persistence

Using a multilayered architecture pattern allows us to write modular code. With the Interface Driven Design architecture pattern we can establish a well known contract at the boundary of each module. This allows us to encapsulate the implementation details of each module and prevent those implementation details from leaking and causing coupling between modules.

With this approach we can substitute the implementation of a module for an alternative implementation using the same well known contract as defined by that module’s interface e.g. an RDBMS implementation for a MongoDB implementation, or a mock implementation.

This approach makes it much easier unit test our code by being able to mock each module by virtue of it’s interface.

AsyncConfiguration

To achieve non-blocking withCompletableFuturein our application we will need to configure ThreadPoolTaskExecutor instances for each layer within our application. The reason for this is to ensure that one layer cannot starve another of threads and cause deadlock.

A bounded thread pool is better for performance as spawning a new thread for each request can be costly. It is also useful to bound a thread pool as it makes you to consider the nature of your application and the resource and tuning requirements it will need in production.

We make use of custom configuration properties, ApplicationProperties, to configure each ThreadPoolExecutor.

@Configuration
@EnableAsync
public class AsyncConfiguration implements AsyncConfigurer {
 
    ...
 
    @Override
    @Bean(name = TASK_EXECUTOR_DEFAULT)
    public Executor getAsyncExecutor() {
        return newTaskExecutor(TASK_EXECUTOR_NAME_PREFIX_DEFAULT);
    }
 
    @Bean(name = TASK_EXECUTOR_REPOSITORY)
    public Executor getRepositoryAsyncExecutor() {
        return newTaskExecutor(TASK_EXECUTOR_NAME_PREFIX_REPOSITORY);
    }
 
    @Bean(name = TASK_EXECUTOR_SERVICE)
    public Executor getServiceAsyncExecutor() {
        return newTaskExecutor(TASK_EXECUTOR_NAME_PREFIX_SERVICE);
    }
 
    @Bean(name = TASK_EXECUTOR_CONTROLLER)
    public Executor getControllerAsyncExecutor() {
        return newTaskExecutor(TASK_EXECUTOR_NAME_PREFIX_CONTROLLER);
    }
 
    @Override
    public AsyncUncaughtExceptionHandler getAsyncUncaughtExceptionHandler() {
        return new SimpleAsyncUncaughtExceptionHandler();
    }
 
    private Executor newTaskExecutor(final String taskExecutorNamePrefix) {
        final ApplicationProperties.Async asyncProperties = applicationProperties.getAsync();
        final ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(asyncProperties.getCorePoolSize());
        executor.setMaxPoolSize(asyncProperties.getMaxPoolSize());
        executor.setQueueCapacity(asyncProperties.getQueueCapacity());
        executor.setThreadNamePrefix(taskExecutorNamePrefix);
        return executor;
    }
}

Repository Layer

Here is our repository interface. As we are using MongoDB for this example we extend MongoRepository:

public interface UserRepository extends MongoRepository<User, String> {
 
    @Async(AsyncConfiguration.TASK_EXECUTOR_REPOSITORY)
    CompletableFuture<Page<User>> findAllBy(final Pageable pageable);
 
    @Async(AsyncConfiguration.TASK_EXECUTOR_REPOSITORY)
    CompletableFuture<User> findOneById(final String id);
}

We specify our User domain model as the type this Repository will manage. The id type for the User entity is of type String.

Spring Data allows us to create query methods, methods whose very signature defines a database query. When Spring Data creates a proxy bean for a Repository it will use the query method signatures to implement the query you wish to execute. The findByconvention allows us to define query methods and return CompletableFuture instances from our query methods thus making them asynchronous and non-blocking.

Check out my blog post on creating asynchronous query methods with Spring Data and CompletableFuture.

Find All Users

With the following query method we can run a query to find all users in our database:

CompletableFuture<Page<User>> findAllBy(final Pageable pageable);

Spring Data supports pagination out of the box and provides the Page type that represents a page of entity objects. The CompletableFuture<Page<User>> being returned by our method is yielding a page of User entity objects. To specify the pagination criteria to use in the query the Pageable parameter is used. Pageable is another type provided by Spring for this very purpose.

Find One User

The query method to find one user should return only one result so we use the prefix findOne. We are also searching by the id of the user so we specify that by naming the method findOneById. The parameter id allows us to pass the id of the user we are searching for. To make the method asynchronous we give it a return type of CompletableFuture<User>.

CompletableFuture<User> findOneById(final String id);

If no user is found theCompletableFuture will yield a null result. Spring Data supports returning Optional from standard query methods, but unfortunately there is no support for returning CompletableFuture<Optional>.

Service Layer

Now to create our service layer. First we create an interface to define the contract for our service. Other than providing a contract on how you expose the functionality of your service, interface driven development is great for testing.

public interface UserService {
 
    CompletableFuture<Page<User>> findAll(final Pageable pageable);
 
    CompletableFuture<Optional<User>> findOneById(final String id);
}

The implementation for our service is very simple:

@Service
public class UserServiceImpl implements UserService {
 
    private final UserRepository userRepository;
 
    public UserServiceImpl(final UserRepository userRepository) {
        this.userRepository = userRepository;
    }
 
    @Override
    @Async(AsyncConfiguration.TASK_EXECUTOR_SERVICE)
    public CompletableFuture<Page<User>> findAll(final Pageable pageable) {
        return userRepository.findAllBy(pageable);
    }
 
    @Override
    @Async(AsyncConfiguration.TASK_EXECUTOR_SERVICE)
    public CompletableFuture<Optional<User>> findOneById(final String id) {
        return userRepository
                .findOneById(id)
                .thenApply(Optional::ofNullable);
    }
}

To let Spring and maintainers of your code (that includes you) know that we are creating a bean that will be used as a service we annotate the class with @Service. @Service doesn’t provide any additional behaviour over @Component but it may do so at some point in the future and it helps to be explicit in your intentions for the bean.

We are also using the recommended injection method of constructor based injection. If there is only one constructor in the bean then Spring doesn’t require the constructor be annotated with @Inject.

To enable asynchronous execution using Spring we annotate our method implementations with @Async and also provide the name of the executor we want the work to be dispatched to. Here we are using the TASK_EXECUTOR_SERVICE executor.

In the method findOneById we are transforming our response object of User to Optional<User> as the requested user may not exist. By wrapping the potentially null returned value in Optional we are explicitly stating that the returned element may or may not be present.

RestController

Here’s what our RESTful controller looks like:

@RestController
@RequestMapping(value = UserController.REQUEST_PATH_API_USERS)
class UserController {
 
    static final String REQUEST_PATH_API_USERS = "/api/users";
    private static final Logger log = LoggerFactory.getLogger(UserController.class);
    private static final String REQUEST_PATH_API_USERS_INDIVIDUAL_USER = "/{userId}";
 
    private final UserService userService;
 
    public UserController(final UserService userService) {
        this.userService = userService;
    }
 
    @Async(AsyncConfiguration.TASK_EXECUTOR_CONTROLLER)
    @GetMapping(produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
    public CompletableFuture<ResponseEntity> getUsers(final Pageable paging) {
        return userService
                .findAll(paging)
                .<ResponseEntity>thenApply(ResponseEntity::ok)
                .exceptionally(handleGetUsersFailure);
    }
 
    @Async(AsyncConfiguration.TASK_EXECUTOR_CONTROLLER)
    @GetMapping(value = REQUEST_PATH_API_USERS_INDIVIDUAL_USER,
                produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
    public CompletableFuture<ResponseEntity> getUser(@PathVariable final String userId) {
        return userService
                .findOneById(userId)
                .thenApply(mapMaybeUserToResponse)
                .exceptionally(handleGetUserFailure.apply(userId));
    }
 
    private static Function<Throwable, ResponseEntity> handleGetUsersFailure = throwable -> {
        log.error("Unable to retrieve users", throwable);
        return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).build();
    };
 
    private static Function<Optional<User>, ResponseEntity> mapMaybeUserToResponse = maybeUser -> maybeUser
            .<ResponseEntity>map(ResponseEntity::ok)
            .orElse(ResponseEntity.notFound().build());
 
    private static Function<String, Function<Throwable, ResponseEntity>> handleGetUserFailure = userId -> throwable -> {
        log.error(String.format("Unable to retrieve user for id: %s", userId), throwable);
        return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).build();
    };
}

To make our request methods asynchronous and non-blocking we annotate them with the @Async annotation and specify the executor we want to dispatch the work on.

Get All Users

Here’s the method we use to return all users when an HTTP GET request is received:

@Async(AsyncConfiguration.TASK_EXECUTOR_CONTROLLER)
@GetMapping(produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
public CompletableFuture<ResponseEntity> getUsers(final Pageable paging) {
    return userService
            .findAll(paging)
            .<ResponseEntity>thenApply(ResponseEntity::ok)
            .exceptionally(handleGetUsersFailure);
}
 
private static Function<Throwable, ResponseEntity> handleGetUsersFailure = throwable -> {
    log.error("Unable to retrieve users", throwable);
    return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).build();
};

The return type from our method is CompletableFuture<ResponseEntity> and we annotate the method with @Async and specify the task executor we want to dispatch the work on. In this case we are using the TASK_EXECUTOR_CONTROLLER task executor.

It is possible to specify the type returned in your ResponseEntity however in order to return error codes and error information of our own leaving this specificity out is necessary.

The method takes a Pageable parameter. This is a type provided by Spring and allows the caller to specify paging parameters e.g.

http://localhost:8080/api/users?page=2&size=10

With this URL we are requesting the users resource and specifying the page offset and the number of results to return.

Our service and repository methods for finding all users take this Pageable parameter so all we need to do is pass it on.

When we invoke our service method findAll we get a CompleteableFuture<Page<User>> back. Our job is to map this type to the expected type CompletableFuture<ResponseEntity>. By using the method thenApply onCompletableFuturewe can do something with the return value in this case Page<User>. All we need to do is wrap the Page<User> object in a ResponseEntity and give it a HTTP response code of 200 (OK). Spring provides a handy way of instantiating just such a ResponseEntity with the ResponseEntity.ok() method.

.<ResponseEntity>thenApply(ResponseEntity::ok)

We have to explicitly define the return type when invoking the thenApply method to tell Java that we are creating a ResponseEntity<Page<User>> object but we want it to be treated as a plain ResponseEntity object. This helps us when we define our error recovery:

.exceptionally(handleGetUsersFailure);

By specifying error handling code to be run in the event of an exception we can create an appropriate response to the client making the HTTP request. In this case we want to recover from any error and return a status code of 500:

ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).build();

Get One User

This is the method we use to retrieve one user:

@Async(AsyncConfiguration.TASK_EXECUTOR_CONTROLLER)
@GetMapping(value = REQUEST_PATH_API_USERS_INDIVIDUAL_USER,
            produces = MediaType.APPLICATION_JSON_UTF8_VALUE)
public CompletableFuture<ResponseEntity> getUser(@PathVariable final String userId) {
    return userService
            .findOneById(userId)
            .thenApply(mapMaybeUserToResponse)
            .exceptionally(handleGetUserFailure.apply(userId));
}
 
private static Function<Optional<User>, ResponseEntity> mapMaybeUserToResponse = maybeUser -> maybeUser
        .<ResponseEntity>map(ResponseEntity::ok)
        .orElse(ResponseEntity.notFound().build());
 
private static Function<String, Function<Throwable, ResponseEntity>> handleGetUserFailure = userId -> throwable -> {
    log.error(String.format("Unable to retrieve user for id: %s", userId), throwable);
    return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).build();
};

Again, to make this method asynchronous and non-blocking we annotate it with @Async and return CompletableFuture<ResponseEntity>.

We need to bind the {userId} path variable to the userId parameter. This is done with @PathVariable. Issuing a GET request to the following URL will invoke this method:

http://localhost:8080/api/users/1234

As with the getAllUsers method we need to transform the CompletableFuture<Optional<User>> response we get from the UserService to ResponseEntity. To do this we again use the CompletableFuture.thenApplymethod. This time we create a pure function to transform the response:

private static Function<Optional<User>, ResponseEntity> mapMaybeUserToResponse = maybeUser -> maybeUser
        .<ResponseEntity>map(ResponseEntity::ok)
        .orElse(ResponseEntity.notFound().build());

In the case where the user exists we are mapping the Optional<User> parameter to a ResponseEntity with the User object as the body and a status code of 200 (OK). If the Optional<User> is empty we instead return a ResponseEntity with no body and a status code of 404 (Not Found).

If there an exception is thrown we recover by sending a response with a status code of 500 (Internal Server Error):

private static Function<String, Function<Throwable, ResponseEntity>> handleGetUserFailure = userId -> throwable -> {
    log.error(String.format("Unable to retrieve user for id: %s", userId), throwable);
    return ResponseEntity.status(HttpStatus.INTERNAL_SERVER_ERROR).build();
};

Wrapping Up

By using this technique you can split up your application into separate layers and have each layer perform tasks asynchronously. This separation of concerns makes it easier for you to conceptualise your logic, combine and recombine your code and logic, and make it easier to tune, and test your code from unit to integration to system testing.

Take a look at this code in action by cloning the source code.

Asynchronous Query Methods with Spring Data & Completable Future

Thu, 08 Jun 2017 00:00:00 +0000

CompletableFuture, introduced in Java 8, provides an easy way to write asynchronous, non-blocking, multithreaded code. Since Spring 4.2 it’s now possible to write asynchronous code by returning CompletableFuture from non-private methods annotated with @Async.

Spring Data has taken advantage of this advancement and now allows you to to write non-blocking, asynchronous Repository queries using CompletableFuture. This is done by returning CompletableFuture from a query method and annotating the method with the @Async annotation.

When invoked the query method will return immediately with the actual query execution taking place in a task submitted to a Spring TaskExecutor.

You can find the source code for this blog post here.

Repository

Creating asynchronous query methods in a Repository is as simple as returning CompletableFuture from your query method and annotating it with @Async:

public interface UserRepository extends MongoRepository<User, String> {
 
    @Async(AsyncConfiguration.TASK_EXECUTOR_REPOSITORY)
    CompletableFuture<Page<User>> findAllBy(final Pageable pageable);
 
    @Async(AsyncConfiguration.TASK_EXECUTOR_REPOSITORY)
    CompletableFuture<User> findOneById(final String id);
}

Spring even allows you to specify the TaskExecutor to run the query on. In this case the query methods are using a TaskExecutor configured in the custom AsyncConfiguration file.

Task Executor

When configuring a TaskExecutor to use with asynchronous query method it is often advantageous to use a bounded ThreadPoolTaskExecutor.

Here’s an example of how to create a ThreadPoolTaskExecutor, the source code for which can be found here:

@Configuration
@EnableAsync
public class AsyncConfiguration implements AsyncConfigurer {
 
    private static final String TASK_EXECUTOR_DEFAULT = "taskExecutor";
    private static final String TASK_EXECUTOR_NAME_PREFIX_DEFAULT = "taskExecutor-";
    private static final String TASK_EXECUTOR_NAME_PREFIX_REPOSITORY = "serviceTaskExecutor-";
    ...
 
    public static final String TASK_EXECUTOR_REPOSITORY = "repositoryTaskExecutor";
    ...
 
    private final ApplicationProperties applicationProperties;
 
    public AsyncConfiguration(final ApplicationProperties applicationProperties) {
        this.applicationProperties = applicationProperties;
    }
 
    @Override
    @Bean(name = TASK_EXECUTOR_DEFAULT)
    public Executor getAsyncExecutor() {
        return newTaskExecutor(TASK_EXECUTOR_NAME_PREFIX_DEFAULT);
    }
 
    @Bean(name = TASK_EXECUTOR_REPOSITORY)
    public Executor getRepositoryAsyncExecutor() {
        return newTaskExecutor(TASK_EXECUTOR_NAME_PREFIX_REPOSITORY);
    }
     
    ...
 
    @Override
    public AsyncUncaughtExceptionHandler getAsyncUncaughtExceptionHandler() {
        return new SimpleAsyncUncaughtExceptionHandler();
    }
 
    private Executor newTaskExecutor(final String taskExecutorNamePrefix) {
        final ApplicationProperties.Async asyncProperties = applicationProperties.getAsync();
        final ThreadPoolTaskExecutor executor = new ThreadPoolTaskExecutor();
        executor.setCorePoolSize(asyncProperties.getCorePoolSize());
        executor.setMaxPoolSize(asyncProperties.getMaxPoolSize());
        executor.setQueueCapacity(asyncProperties.getQueueCapacity());
        executor.setThreadNamePrefix(taskExecutorNamePrefix);
        return executor;
    }
}

The thread pool configuration parameters are being supplied by the custom configuration properties singleton that is injected into the constructor.

The SimpleAsyncUncaughtExceptionHandler simply logs any exceptions that occur during the execution of a task.

Hopefully this gives you an idea of the powerful asynchronous features Spring with its support for CompletableFuture brings.

Self contained projects with Gradle Wrapper

Sat, 03 Jun 2017 22:00:00 +0000

The Gradle Wrapper is a handy way of bundling a Gradle runtime with your project. That way you provide a specific version of Gradle to be used with your project and Gradle does not have to be installed separately. This is very useful for anyone who clones your repo and wants to build your project.

To generate a Gradle Wrapper for your project you will need to have Gradle installed. This is necessary to first generate the Gradle Wrapper but from then on the Wrapper can be used to run Gradle commands against your project without having to have Gradle installed.

You can find a sample project using the Gradle Wrapper here

Install Gradle

Begin by installing Gradle. This is a simple as

Ensuring you have Java 7 or higher
Downloading a Gradle distribution
Unzipping the Gradle distribution to a location of your choice
Adding the <gradle-install-path/bin folder to your PATH

Official instructions can be found here

Generate the Gradle Wrapper

To generate a Gradle Wrapper for your project and make it self contained execute the following command in the root folder of the project:

gradle wrapper --gradle-version 3.5

Be sure to set the –gradle-version flag to the version of Gradle you wish to use with your project.

Add Generated Files to VCS

Now that the Gradle Wrapper has been generated for you project you will see the following new files:

<your project folder>/
  gradlew
  gradlew.bat
  gradle/wrapper/
    gradle-wrapper.jar
    gradle-wrapper.properties

All of the above files must be checked into your version control system.

Add SHA-256 Checksum Verification

For added security and to ensure the integrity of the Gradle distribution downloaded by the Gradle Wrapper you can configure the SHA-256 checksum verification. To do this you will need to generate a SHA-256 hash of the target distribution:

> shasum -a 256 gradle-3.5-all.zip 
d84bf6b6113da081d0082bcb63bd8547824c6967fe68704d1e3a6fde822b7212  gradle-3.5-all.zip

Add the hash sum to your project’s gradle-wrapper.properties file with the distributionSha256Sum property:

distributionSha256Sum=d84bf6b6113da081d0082bcb63bd8547824c6967fe68704d1e3a6fde822b7212

Run the Gradle Wrapper

Run the Gradle Wrapper to make sure everything is working by executing the following command:

./gradlew clean

The Gradle Wrapper should download the target Gradle distribution, verify the integrity of the file using the SHA-256 checksum and then run the specified command.

Performance Testing with ab and Docker

Mon, 29 May 2017 22:00:00 +0000

Whenever possible I use official Docker images to run applications, services, and tools so I don’t have to install and maintain a myriad of different applications on my development machine. The Apache HTTP server benchmarking tool ab is a handy utility you can use to quickly performance test your HTTP services. It comes bundled as part of the Apache HTTP server installation, but why bother installing it when you can just use the official Docker image.

Docker Run

To run the latest Docker image for the Apache HTTP server on linux:

docker run -dit --name httpd-ab -v /var/www/html:/usr/local/apache2/htdocs/ httpd

This will pull down the latest official Apache HTTP server Docker image, start the httpdserver, and name the container httpd-ab. The volume -v /var/www/html:/usr/local/apache2/htdocs/ is used to serve HTML from the /var/www/html directory on the host machine.

Docker Exec

You can access the ab command line tool on the running container by running the following command on linux:

docker exec -it httpd-ab bash

This will drop you into a bash session running on the container. You can now run the abcommand to performance test your application.

Example Command

Here’s an example command you can run to performance test a server:

ab -c 350 -n 20000 example.com/

This command will execute 20000 HTTP GET requests to example.com maxing out at 350 simultaneous requests.

Search

Mon, 01 Jan 0001 00:00:00 +0000

HUMANSREADCODE

AI‑Augmented Architecture Option Exploration

Introduction: what AI changes (and what it doesn’t)

TL;DR: quick start

Scenario: modernizing policy/quote pricing

The “Context Pack”: minimum artifacts before asking AI for options

Stop conditions: don’t proceed without these

The 6‑stage workflow

Stage 1 — Clarify (make constraints explicit)

Stage 2 — From business drivers to architecture characteristics

Stage 3 — Generate permutations (candidate styles → options → isomorphism → risks)

Stage 4 — Explore tradeoffs (scoring + risk comparison + mitigations)

Tradeoff matrix template (headings)

Scoring rubric (keep it honest)

A worked mini‑example (abridged)

Stage 5 — Explore transition architectures (current → target, incrementally)

Stage 6 — Hone in on the target architecture (NFRs, fitness/enforcement, ADRs, feedback)

ADRs: expect more than one

ADR outline (tailored to option exploration)

Fitness functions (measurable, not aspirational)

Failure modes and guardrails

Failure mode: novelty bias

Failure mode: hallucinated constraints

Failure mode: overconfidence in estimates

Failure mode: decision laundering

Guardrail: adversarial analysis (critique each stage)

Adoption in enterprise reality

Use criticality tiers

Progressive enforcement

The evidence bundle

Conclusion: faster exploration, better accountability

Appendix: templates and prompt pack

Context Pack checklist (copy/paste)

Adversarial review prompts (copy/paste)

Architecting Context: How Foundational Architecture Practices Power AI-Augmented Delivery

Introduction

1. Architecture Creates Context

2. The Architecture Context Cycle

Step 1. Understand the Key Business Drivers

Step 2. Identify the Driving Architecture Characteristics

Step 3. Select Candidate Architecture Styles

Step 4. Apply Domain-to-Architecture Isomorphism

Step 5. Align Across the Nine Intersections of Architecture

Step 6. Perform an Architecture Risk Analysis

Step 7. Identify NFRs, Fitness Functions, and Enforcement Functions

Step 8. Document Decisions in Architecture Decision Records

Step 9. Establish Feedback Loops

3. Architecture Artifacts as Context Assets

4. The Role of C4 Diagrams

5. From Architecture to AI-Augmented Delivery

6. Architecture as Context Infrastructure

7. AI as a Partner in the Architecture Process

From Quality to Context to Execution: Engineering the Next Stage of AI-Augmented Delivery

Introduction

1. From Built-In Quality to High-Quality Context

2. Why Context Matters: LLMs Are New Hires

3. From Context to Code: Specification-Driven Development

4. The New Skill: Context Engineering

Conclusion

AI-Augmented Quality Across the SDLC: Building It In, Not Testing It In

Introduction

1. Requirements: Building clarity from the start

2. Architecture & Design: Guardrails for sustainable solutions

3. Build & Test: Raising the engineering baseline

4. Operations: Predict failures, detect anomalies

Conclusion: Quality is not a phase

Modernisation Guardrails & AI

Introduction

The Path Forward

Quality Gates as More Than Checklists

Where AI Comes In

Conclusion

Modernisation & AI

Introduction

Where Modernisation and AI Converge

The Future of Enterprise Delivery

About

Archives

Privacy

The Known and Unknown Framework