- ask-claude v2.4.0 -> v2.6.1: Added mandatory disagreement scan (Step 3.5) with web-reference demand + internal consistency check. Updated callouts for the paste-vs-path rule (now HARD RULE, not just example). - ask-hermes v4.1.1 -> v4.2.0: Added mandatory disagreement scan mirroring ask-claude. - ask-dev v1.0.0 -> v1.1.0: Added mandatory disagreement scan mirroring ask-claude. - create-plan v1.1.0 (new): Multi-agent plan-build dispatcher. Trigger phrases, 10-Q brief script (min as needed, max 10), curator dispatch, capture/handoff format, all 6 failure modes. Companion references: curator-framing-prompt.md, deep-research-dispatch-template.md, ask-claude-validation-template.md, structured-brief-10q.md. 5 rounds of Claude validation on the design plan; final SHIP from ask-hermes peer review.
112 lines
5.5 KiB
Markdown
112 lines
5.5 KiB
Markdown
# Plan Validation Prompt Pattern
|
|
|
|
The most effective prompt shape for using `ask-claude` to validate a plan. Proven against Opus 4.8 on real plan files (workshopped Jul 6 2026 against `plan_skill_plan.md`).
|
|
|
|
## Why This Pattern
|
|
|
|
Generic "review this plan" prompts fail three ways:
|
|
1. Claude proposes new features instead of finding flaws.
|
|
2. Claude's checks are vibes ("this looks incomplete") instead of evidence-grounded.
|
|
3. Claude's review isn't constrained to checkable-from-text-only properties, so it hallucinates verifications.
|
|
|
|
The pattern below solves all three.
|
|
|
|
## The Template
|
|
|
|
```
|
|
Validate this plan. Read it in full first.
|
|
|
|
CRITICAL CONSTRAINT: Do NOT propose new features, new architecture, or new complexity.
|
|
Your job is to find FLAWS in what's already proposed — not to add more. If you find
|
|
a problem, say what's wrong and suggest the SIMPLEST fix.
|
|
|
|
CONTEXT (why this plan exists):
|
|
- <one paragraph: the goal, what triggered the plan, what skills/tools it reuses>
|
|
|
|
VALIDATE:
|
|
1. <dimension 1 — must be checkable from plan text alone>
|
|
2. <dimension 2>
|
|
3. ...
|
|
|
|
Report format:
|
|
- Release-blockers (must fix before building)
|
|
- Medium (should fix)
|
|
- Low (nice to have)
|
|
- Pass-criteria met / not met
|
|
|
|
Then: give 2-3 concrete suggestions for how to make this plan better ACHIEVE the
|
|
operator's goal. Be specific.
|
|
|
|
Do not rewrite the plan. Just report findings and suggestions.
|
|
|
|
==== ARTIFACT BEGINS ====
|
|
|
|
<paste full artifact here>
|
|
|
|
==== ARTIFACT ENDS ====
|
|
```
|
|
|
|
## Why It Works
|
|
|
|
| Ingredient | Purpose |
|
|
|---|---|
|
|
| `CRITICAL CONSTRAINT: Do NOT propose new features` | Kills Claude's feature-creep reflex (his default mode is thoroughness → overengineering). |
|
|
| `say what's wrong and suggest the SIMPLEST fix` | Forces fixes to be small. Big fixes = new complexity = rejected. |
|
|
| Numbered VALIDATE list with checkable dimensions | Forces structured findings. If a dimension is uncheckable from the artifact text alone, drop it. |
|
|
| Pass-criteria met / not met | Binary verdict the dispatcher can act on. |
|
|
| `2-3 concrete suggestions to ACHIEVE the goal` | Lets Claude contribute value (refinements) without breaking the no-new-features rule. |
|
|
| Pasted artifact with markers | Avoids the "file not found" wasted turn (see SKILL.md Pitfalls). |
|
|
|
|
## Dimensions That Work (Checkable from Text)
|
|
|
|
- **Completeness** — does the plan cover all phases / all sub-tasks of the topic?
|
|
- **Internal consistency** — do the steps in §3 actually produce the outputs claimed in §2?
|
|
- **TDD discipline** — is there a failing test before every code-producing task?
|
|
- **Bite-size** — are all tasks 2-5 minutes of focused work?
|
|
- **Bite-size inverse** — which task is suspiciously large and should be split?
|
|
- **Output contract** — are the file paths, names, and structures consistent across all sections?
|
|
- **Trigger / dispatch chain** — does the plan's "use skill X" actually do what's claimed? (e.g., "uses `ask-hermes`" — does it say with what prompt, with what resume rule?)
|
|
- **Session management** — same-topic = resume, new-topic = fresh. Does the plan enforce this?
|
|
|
|
## Dimensions That DO NOT Work (Uncheckable from Text)
|
|
|
|
These belong to the *dispatcher* (which has web + filesystem), NOT to Claude:
|
|
|
|
- "Are versions current as of 2026?" → Claude has no live web access in print mode.
|
|
- "Do all file paths exist in the project?" → Claude has no filesystem access to the operator's project.
|
|
- "Does this conflict with installed packages?" → Claude has no access to the operator's env.
|
|
|
|
If you ask Claude these, it will hallucinate. Either drop them from the prompt or pass the evidence inline (e.g., "Here is the official v0.27.0 release page contents: [paste]").
|
|
|
|
## When To Use
|
|
|
|
- Plan for a new skill, integration, infrastructure change, profile setup
|
|
- Plan for a multi-file feature build
|
|
- Plan for a research deliverable that needs structural review
|
|
- ANY plan you're about to ask the user to approve
|
|
|
|
## When NOT To Use
|
|
|
|
- Single-step, trivial changes (one file, one config edit) — overkill
|
|
- Research-only outputs (use the `deep-research` validate-fix cycle instead — see `deep-research` skill)
|
|
- Plans that are themselves research findings (Claude should be reading the source URLs, not just the synthesis)
|
|
|
|
## Example: Real Validation Run
|
|
|
|
Session: 2026-07-06, validating `plan_skill_plan.md`.
|
|
|
|
**What worked:**
|
|
- 3 release-blockers caught: (1) Phase 1 interactive Q&A contradicts "curator runs autonomously", (2) Claude template asked Claude to check things it structurally couldn't, (3) output filename contract contradictory.
|
|
- 6 medium findings: deep-research sync vs async, who writes pre-plan.md, session-id capture fragility, trigger-phrase collision, --no-research breaks checklist, no Claude-down fallback.
|
|
- 4 low findings: rename `_pre-plan.md`, weak self-review, missing validation persistence, false "reproducible" claim.
|
|
- 3 actionable suggestions, including the highest-leverage one: "front-load the brief into the dispatcher" — solved B1 cleanly by changing where the Q&A lives.
|
|
|
|
**What didn't:**
|
|
- None. Cost $0.32, caught real defects. Beats running the plan and discovering them at runtime.
|
|
|
|
## Variations
|
|
|
|
**For research-heavy plans** (where the primary risk is factual error, not structure): use the `deep-research` skill's validate-fix cycle instead — dispatch `ask-hermes` to web-verify each claim, then feed the report back to `deep-research --resume`.
|
|
|
|
**For plans you wrote yourself** (high self-review blindness): prepend `Be especially skeptical of the actor assignments and the open questions — those are where author bias lives.`
|