ask-claude v2.6.1, ask-hermes v4.2.0, ask-dev v1.1.0, create-plan v1.1.0
- ask-claude v2.4.0 -> v2.6.1: Added mandatory disagreement scan (Step 3.5) with web-reference demand + internal consistency check. Updated callouts for the paste-vs-path rule (now HARD RULE, not just example). - ask-hermes v4.1.1 -> v4.2.0: Added mandatory disagreement scan mirroring ask-claude. - ask-dev v1.0.0 -> v1.1.0: Added mandatory disagreement scan mirroring ask-claude. - create-plan v1.1.0 (new): Multi-agent plan-build dispatcher. Trigger phrases, 10-Q brief script (min as needed, max 10), curator dispatch, capture/handoff format, all 6 failure modes. Companion references: curator-framing-prompt.md, deep-research-dispatch-template.md, ask-claude-validation-template.md, structured-brief-10q.md. 5 rounds of Claude validation on the design plan; final SHIP from ask-hermes peer review.
This commit is contained in:
Hermes Agent
156
ask-claude/references/peer-response-protocol.md
Normal file
156
ask-claude/references/peer-response-protocol.md
Normal file
@@ -0,0 +1,156 @@
|
||||
# Peer Response Protocol — Disagreement Scan, Web-Reference Mandate, Cross-Doc Desync
|
||||
|
||||
This reference documents the three cross-cutting patterns that emerged from the
|
||||
v2.6.0 ask-claude update. They apply to ANY peer/consultant dispatch
|
||||
(ask-claude, ask-hermes, ask-dev, ask-claude-as-curator, future peers) and to
|
||||
ANY document-edit workflow where multiple sections describe the same thing.
|
||||
|
||||
## 1. Disagreement Scan (MANDATORY before applying)
|
||||
|
||||
Source: ask-claude v2.6.0 Step 3.5, ask-hermes v4.2.0, ask-dev v1.1.0 (all
|
||||
updated in the same session).
|
||||
|
||||
**Rule:** For every peer response, before applying or moving on, walk through
|
||||
the peer's findings and classify each one:
|
||||
|
||||
- **Agree + will apply** → no action
|
||||
- **Agree + will skip** → state explicitly WHY you are skipping. Do not
|
||||
silently drop. The "why" matters — operator can re-prioritize if the
|
||||
rationale is wrong.
|
||||
- **Disagree** → push back via `--resume` (ask-hermes/ask-dev) or a follow-up
|
||||
turn (ask-claude). Either send the pushback OR state why you decided not to.
|
||||
- **Disagreed silently dropped: should be EMPTY.** If non-empty, that's a
|
||||
bug — surface it.
|
||||
|
||||
**Output format (emit every time):**
|
||||
```
|
||||
[Peer's findings as-is]
|
||||
|
||||
## My disagreement scan
|
||||
- **Agreed and applied:** [list]
|
||||
- **Agreed but skipped:** [item] — [reason]
|
||||
- **Disagreed and pushed back:** [item] — [what I said]
|
||||
- **Web references / live tests requested:** [items where I demanded a URL or run]
|
||||
- **Disagreed silently dropped:** (should be empty — if not, justify)
|
||||
```
|
||||
|
||||
The "disagreed silently dropped" line is the tripwire. Empty is the only
|
||||
acceptable state. Non-empty means the agent is rationalizing acceptance
|
||||
without doing the work.
|
||||
|
||||
## 2. Web-Reference Mandate (for unverified concrete claims)
|
||||
|
||||
Source: ask-claude v2.6.0, ask-hermes v4.2.0 (already had canonical form), ask-dev v1.1.0.
|
||||
|
||||
**Rule:** For any peer claim that is a concrete factual statement
|
||||
(tool version, API behavior, library status, system state assertion,
|
||||
"best practice" recommendation) AND where the peer did NOT cite a source URL
|
||||
inline OR run a live command to verify, send a follow-up demanding the
|
||||
verification.
|
||||
|
||||
**Pattern:**
|
||||
> "Show me the command output that supports X. I want a URL or a live test,
|
||||
> not reasoning."
|
||||
|
||||
Claude has web search (mcp_searxng_searxng_web_search). Hermes peers have
|
||||
filesystem + terminal + web search. **No concrete factual claim is acceptable
|
||||
without a source.**
|
||||
|
||||
**Applies to:**
|
||||
- Tool versions, port numbers, env var names, config keys
|
||||
- "X is broken" / "Y is down" system state assertions
|
||||
- Library APIs, package availability, install commands
|
||||
- Architecture pattern recommendations
|
||||
- "Best practice" assertions
|
||||
- Anything the plan or system will depend on
|
||||
|
||||
## 3. Cross-Document Desync (the recurring edit pattern)
|
||||
|
||||
Source: observed across 5 rounds of plan validation in a single session
|
||||
(plan_skill_plan.md v1 → v5). The pattern fires every time.
|
||||
|
||||
**The problem:** When a value (count, name, path, number) changes in a
|
||||
document with multiple sections describing the same thing (prose + diagram +
|
||||
checklist + cross-references), the edit almost always lands in the prose
|
||||
section and gets missed in the diagram and checklist.
|
||||
|
||||
**Symptom:** Sections describe the same construct with different values. The
|
||||
section that gets READ CAREFULLY (prose) is correct; the section that gets
|
||||
SCANNED (diagram, checklist) is stale.
|
||||
|
||||
**Meta-pattern from the workshop (5 rounds of validation, 4 had desync):**
|
||||
the first round almost always misses 2-3 cross-references. The second round
|
||||
catches 1-2 more. The third+ rounds usually surface real design issues
|
||||
that the desync had been hiding. **Treat the first round of "all clean"
|
||||
findings as suspicious** — re-grep for the changed value across the whole
|
||||
document before declaring done.
|
||||
|
||||
**Mitigation (MANDATORY before finalizing any doc edit):**
|
||||
1. Identify the value being changed (count, name, path, etc.)
|
||||
2. `grep -nE "<old_value>" <file>` to find every occurrence
|
||||
3. Update ALL occurrences, not just the obvious one
|
||||
4. Re-grep with the new value to confirm zero stragglers
|
||||
5. For cross-section references (e.g., "see §6 for the 9 pitfalls" when
|
||||
§6 now has 11), grep the literal cross-reference too — they are
|
||||
often hardcoded numbers that don't update with the section they point
|
||||
at
|
||||
|
||||
**This applies to:**
|
||||
- Plan documents (sections, diagrams, checklists, cross-references)
|
||||
- Skill files (description, examples, pitfall counts, version strings)
|
||||
- Config files (any field referenced in multiple sections)
|
||||
- Any markdown with both ASCII diagrams AND prose describing the diagram
|
||||
|
||||
**The lesson generalizes:** if a value is mentioned in 3+ places, expect
|
||||
to miss at least one on each edit. Treat the grep as a pre-commit hook.
|
||||
|
||||
**Pre-finalize workflow (run this every time, no exceptions):**
|
||||
```bash
|
||||
# 1. Identify the value you just changed
|
||||
OLD_VALUE="<the value before your edit>"
|
||||
NEW_VALUE="<the value after your edit>"
|
||||
|
||||
# 2. Find any stragglers of the old value
|
||||
grep -nE "$OLD_VALUE" <file> || echo "OK: no stragglers of $OLD_VALUE"
|
||||
|
||||
# 3. Confirm the new value is in the expected places
|
||||
grep -nE "$NEW_VALUE" <file>
|
||||
|
||||
# 4. Find any cross-section references that hardcode the count/number
|
||||
grep -nE "§[0-9].*\b$OLD_VALUE\b" <file> || echo "OK: no §X cross-refs to $OLD_VALUE"
|
||||
```
|
||||
|
||||
Run this even when "the edit is in one obvious place." Multi-place edits
|
||||
hide.
|
||||
|
||||
## 4. Pre-Build Verification Gate
|
||||
|
||||
Source: plan_skill_plan.md §7, surfaced when Claude flagged session-id
|
||||
line-1 capture as a first-run failure risk that "five rounds of counting-fixes
|
||||
never touched."
|
||||
|
||||
**The problem:** Plans can be internally consistent on design while
|
||||
containing hidden assumptions about the runtime environment (positional
|
||||
output parsing, banner formats, version-specific CLI flags, etc.). These
|
||||
don't show up in design review.
|
||||
|
||||
**Mitigation:** Before declaring any plan "buildable," list the
|
||||
runtime-assumption items and empirically verify each one against the
|
||||
actual deployed environment. Common categories:
|
||||
|
||||
- Positional parsing assumptions (line 1 of output, JSON field order)
|
||||
- CLI flag assumptions (`--max-turns 600` works, `--yolo` works, etc.)
|
||||
- Path assumptions (config files at expected locations, skills registered)
|
||||
- Version assumptions (skill X is at version Y, has feature Z)
|
||||
|
||||
**Output: a verified-items checklist in the plan's §7.** Each item is
|
||||
either confirmed (with the actual command output) or marked as
|
||||
"out of scope" (with the reason).
|
||||
|
||||
## When to apply this protocol
|
||||
|
||||
- After EVERY ask-claude, ask-hermes, ask-dev, or any peer response —
|
||||
emit the disagreement scan output.
|
||||
- After EVERY multi-section doc edit — grep for the old value, then re-grep.
|
||||
- Before declaring any plan "buildable" — verify the runtime-assumption
|
||||
checklist empirically.
|
||||
Reference in New Issue
Block a user