Files

155 lines
12 KiB
Markdown
Raw Permalink Normal View History

---
name: skill-quality-review
description: "Quality review of skill documents (SKILL.md, references/, templates/, scripts/). Produces a verdict (Approved / Approved with fixes / Needs revision), severity-tiered findings, and a verification checklist. Triggers on 'review this skill', 'audit the skill', 'code quality review of SKILL.md', 'is this skill well-written', 'check the skill for issues', 'quality-check this skill'. Class-level meta-skill, not a session artifact."
version: 1.2.0
author: Hermes Agent
license: MIT
platforms: [linux]
metadata:
hermes:
tags: [review, quality, audit, skill, documentation, meta]
related_skills: [agent-self-audit, simplify-code, writing-plans, create-plan]
---
# skill-quality-review — Quality Review of Skill Documents
## 1. Overview
This skill reviews a skill's documents (the SKILL.md frontmatter and body, plus any files in `references/`, `templates/`, `scripts/`) for quality. The output is a structured report with:
- **Verdict** — Approved / Approved with fixes / Needs revision
- **Findings** — by severity (Critical / Important / Minor)
- **Verification checklist** — confirming each quality dimension is satisfied
- **Length-trim recommendations** — optional, when file size is a concern
**When to use:** the operator asks for a code review, quality review, or audit of a skill document. Typical phrasings: `review this skill`, `audit the skill`, `code quality review of SKILL.md`, `is this skill well-written`, `check the skill for issues`, `quality-check this skill`, `review the new Section X`.
**When NOT to use:**
- The operator asks for a fix, not a review → just patch the skill directly (use `patch` on the file).
- The operator asks for a self-audit before executing a tool → use `agent-self-audit`.
- The operator asks for code cleanup, not document review → use `simplify-code`.
- The target is not a skill document (e.g., a regular README, a config file) → review ad-hoc.
## 2. The 9 Quality Dimensions
When reviewing a skill, score it against these dimensions. Each is a checklist item in §4.
1. **Self-containment** — Can a fresh agent execute the skill reading only its own files? All commands exact? All paths resolvable from the skill root? Forward references ("see Section 5") that point to a non-existent section are a fail.
2. **Trigger detection** — Are trigger phrases listed (≥3 phrasings: verb + object)? Are they case-insensitive substring matches, or is the match rule specified? Is there a precedence note when the trigger overlaps with another skill's?
3. **Brief / input capture** — If the skill takes operator input (a brief, parameters, etc.), is the schema exact and machine-parseable (a verbatim block, not paraphrased)? Are stopping rules binding?
4. **Dispatch / output commands** — If the skill dispatches other skills or processes, is the exact command line provided? Is flag rationale documented? Are there pre-dispatch checks (e.g., reasoning_effort, profile, model)?
5. **Capture and handoff** — If the skill produces a session id, artifact path, or other resume handle, is the capture rule binding (e.g., "output line 1")? Is the handoff format parseable (delimited block, key-value lines)? Is there an empirical verification step (not just "this should work")?
6. **Cross-references** — Do all `references/`, `templates/`, `scripts/` paths resolve? Are referenced external skills correct? Are paths unambiguous (e.g., `(from subagent-driven-development skill)` if the file lives in another skill)?
7. **Tone** — Is the voice consistent (imperative for agent-facing, operator-friendly for output blocks)? Is peer-to-peer language used where appropriate? Is there voice drift between sections?
8. **Length** — Is the file size justified? Are there trim candidates? Is there duplication between sections (e.g., the same Q-list in two sections)?
9. **Internal consistency** — Do the YAML frontmatter, description, body sections, verification checklist, and See Also list all match? Are phase numbers, trigger phrases, file paths, and command flags consistent across the document?
**Multi-file skills (one SKILL.md + N references + a parent plan):** the 9 dimensions are necessary but not sufficient — they check in-file quality. For cross-file consistency (number agreement across files, section-ID cross-refs, concept drift, peer-skill integration), see `references/multi-file-review-checklist.md`. The 5 cross-file techniques (mental walkthrough, number consistency grep, cross-reference resolution, concept-drift detection, peer-skill integration check) caught real issues in the July 2026 `create-plan` review that the base 9 dimensions miss. Use the multi-file checklist as a parallel add-on, not a replacement for the 9 dimensions.
## 3. Severity Tiers
Findings are sorted into three tiers. Each tier has a clear meaning and a clear action.
- **Critical** — Blocks the skill from working. Empty trigger list, missing dispatch command, wrong path, broken cross-reference, contradiction between frontmatter and body, forward-reference to a non-existent section. **Action: must fix before the skill is buildable.**
- **Important** — Doesn't block, but degrades quality significantly. Section duplication, ambiguous precedence, half-documented flag, citation drift (e.g., "per writing-plans" when the rule lives in ask-hermes), voice drift. **Action: should fix; document the deferral if not fixing now.**
- **Minor** — Cosmetic / readability / future-flag stubs. Column alignment, "TODO: implement later" placeholders that don't ship, redundant "does NOT do" lists when the boundary is already established. **Action: nice to have; trim if file size is a concern.**
## 4. Verification Checklist
A skill passes review when ALL of these are true. Score each ✅ / ⚠️ / ❌ in the output.
- [ ] YAML frontmatter has `name`, `description`, `version`, `author`, `license`, `platforms`, `metadata.hermes.tags`
- [ ] Description's trigger phrase list matches the body
- [ ] Trigger phrases include ≥3 phrasings (verb + object)
- [ ] Trigger phrases are case-insensitive substring matches, or the match rule is specified
- [ ] If the skill has phases, all phase numbers are consistent across frontmatter, body, and pipeline diagram
- [ ] If the skill takes operator input, the input schema is exact (verbatim block, not paraphrased)
- [ ] If the skill dispatches other skills, the exact command line is provided
- [ ] If the skill produces a session id or resume handle, the capture rule is binding
- [ ] All `references/`, `templates/`, `scripts/` paths exist and resolve
- [ ] Cross-references to other skills' files specify which skill (e.g., `(from subagent-driven-development skill)`)
- [ ] Tone is consistent (agent-facing = imperative; operator-facing = friendly)
- [ ] "Does NOT do" or "out of scope" list present when the skill's scope could be confused with another skill's
- [ ] Verification checklist exists in the skill itself (the skill knows how to be tested)
- [ ] No forward references to sections that don't exist
- [ ] No internal contradictions (e.g., "this skill runs Phase X" in §1 vs. "this skill runs Phase Y" in §3)
- [ ] **Multi-file only:** number anchors (phase numbers, turn caps, payload caps, question caps, version strings) agree across all files in the tree
- [ ] **Multi-file only:** section/step cross-references (e.g., `§5.3`, `Step 4.6`) point to real subsections
- [ ] **Multi-file only:** concepts described in multiple files have consistent definitions (no concept drift)
## 5. Length-Justification Heuristic
A skill's length is justified when:
- Each section earns its place (no forward-reference stubs)
- Commands are exact, not summarized
- Tables and code blocks are used for machine-parseable content
- Duplication is removed (if Section A and Section B list the same items, one becomes a pointer)
If a file is over ~30 KB, identify trim candidates with estimated byte savings:
- Duplicated sections (collapse to pointer)
- Future-flag stubs (remove or commit to timeline)
- "Does NOT do" lists (collapse if the boundary is already established elsewhere)
- Empirical-verification paragraphs (deduplicate if mentioned in §6 and pitfall list)
- "Future considerations" / "TODO: future" sections (remove — YAGNI)
**Multi-file heuristic:** if SKILL.md is over ~30 KB AND the file has N `references/` files totaling >50 KB, check whether content in SKILL.md could move to a reference. SKILL.md should be the operator-facing entry point; the references should hold the operational detail. A 33 KB SKILL.md with 5 well-scoped references is healthier than a 60 KB SKILL.md with no references.
## 6. Output Format
The review output should follow this structure exactly. Use the same headings so future agents can parse it:
```markdown
# <SKILL-NAME> QUALITY REVIEW
## Verdict: <Approved | Approved with fixes | Needs revision>
## Critical Issues
<list, or "None.">
## Important Issues
<list>
## Minor Issues
<list>
## Checklist Verifications
| # | Check | Result |
|---|---|---|
| 1 | Self-containment | ✅ / ⚠️ / ❌ + one-line note |
| 2 | Trigger detection | ... |
| 3 | Brief / input capture | ... |
| ... | ... | ... |
## Length-Justification
<paragraph; if trimming, list candidates with estimated byte savings>
## Notes for the Skill Author
<optional: open questions, design alternatives, things the reviewer couldn't decide>
```
The verdict is shorthand for the operator: **Approved** = ship as-is, **Approved with fixes** = ship after the Important and Minor issues are addressed, **Needs revision** = Critical issues must be fixed before the skill is buildable.
**Multi-file reviews:** the operator may also pass a custom section structure for the review (e.g., "use this 11-section layout"). Honor exactly. The base structure above is the default; a user-specified structure overrides it for that review only. Add a final `## What I did` / `## Files reviewed` / `## Files created or modified` block at the end of the report so the operator can see what was actually executed, even when the structure is custom.
## 7. Worked Example
See `references/create-plan-section-5-review-2026-07.md` for a worked example of this methodology applied to a real skill review (the `create-plan` Section 5 review, July 2026). Use it as a template when producing your own review.
For a multi-file review (5+ files), see `references/multi-file-review-checklist.md` §"Worked example pointer."
## 8. Common Pitfalls for the Reviewer
1. **Don't auto-fix the skill.** The user asked for a *review*. Deliver findings; let the operator decide what to apply. Patching a skill without consent is overstepping.
2. **Don't conflate severity with effort.** A Critical issue can be a one-line fix; an Important issue can require a refactor. Severity is about impact, not effort.
3. **Don't invent dimensions.** If a dimension doesn't apply to the skill (e.g., "capture and handoff" for a skill that doesn't produce session ids), skip it. The 9 dimensions are a checklist, not a religion.
4. **Don't reformat the skill in the review.** Comments like "the table would be better as a list" are out of scope. Note them in Minor if you must, but the review is about *correctness*, not style.
5. **Don't propose new features.** A review identifies issues with what exists. If a dimension is missing (e.g., no trigger list), say so under Critical. Don't suggest adding four more triggers.
6. **Don't read into intent.** "The author probably meant X" is speculation. Quote the text and let the operator decide.
## See Also
- `agent-self-audit` — pre-execution audit of an action; a different class of review.
- `simplify-code` — cleanup of recent code changes, not skill documents.
- `writing-plans` — methodology used inside skills like `create-plan`; not a review methodology.
- `references/multi-file-review-checklist.md` — 5 cross-file techniques for reviewing multi-file skills (mental walkthrough, number consistency grep, cross-reference resolution, concept-drift detection, peer-skill integration check).
- `references/create-plan-section-5-review-2026-07.md` — worked example (single-section review).