Files
hermes-skills/deep-web-research/SKILL.md

180 lines
6.8 KiB
Markdown
Raw Normal View History

---
name: deep-web-research
description: Exhaustive deep web research — five-move flow, external findings ledger, mechanical saturation check, disconfirmation, condensation from disk. Opt-in skill for the research profile.
version: 1.0.0
author: Hermes Agent
---
# Deep Web Research
Loaded explicitly via `-s deep-web-research`. Not loaded during normal interactive
use of the research profile. This skill overrides default behavior to enforce
exhaustive research methodology.
## Core Rules (Always in Effect)
These persist across all turns — they are in the skill, not in fading context:
- **Confined to /tmp.** All file writes go to `/tmp/`. Never write outside /tmp.
- **No self-provisioning.** Never install software. No pip, npm, apt, docker, or
any package manager. Use only what's already configured.
- **No repeat searches.** If you catch yourself searching the same thing twice,
stop. That sub-question is saturated.
- **Blacklist after 3 failures.** If a URL returns an error 3 times, blacklist it
and move on. Do not retry indefinitely.
- **Local and free only.** No internet-based paid services, no SaaS APIs with
billing, no metered endpoints. Use any tool already configured that fits this rule.
## Architecture: External Findings Ledger
At 200 turns, early findings scroll out of context. The ledger is the fix.
**Path:** `/tmp/research-<session-id>.md`
**Schema per finding:**
```
## Finding #<N>: <short label>
- Sub-Question: <which sub-question this belongs to>
- Claim: <the factual claim>
- Source: <URL> — <credibility tier: primary|secondary|aggregator|social>
- Confirmed by: <URL> (or "single-sourced")
- Date: <YYYY-MM-DD of the information, not of the search>
- Confidence: confirmed | likely | single-sourced | disputed
- Notes: <contradictions, caveats, context>
```
**Dedup rule:** Before writing a new finding, scan the ledger for an existing
finding with the same claim. If found, add the new source to "Confirmed by"
instead of creating a duplicate. Duplicate findings defeat the saturation check.
**Sub-question tagging:** Every finding must reference which sub-question it
belongs to. This enables the phase gate to verify coverage and the condensation
phase to group findings by topic.
## Five-Move Research Flow
### Move 1: Decompose
Break the question into sub-questions. Write them to the ledger with time
classifications. This makes "multiple angles" principled instead of random.
```
## Sub-Questions
1. <sub-question text> [time: week|month|year|none]
2. <sub-question text> [time: week|month|year|none]
```
The time classification lives on disk, not in fading context. Move 3 reads it
back when applying time filters per sub-question.
**Time classification guide:**
- **Current-state** (releases, prices, news, versions, events): `week` or `month`
- **Established-knowledge** (how something works, architecture, algorithms, history): `none` or `year`
### Move 2: Landscape Pass
Shallow-but-broad sweep. For each sub-question, 1-2 searches, read top 1-2 results.
Extract only: key sources, terms of art, where disagreement lives, major players.
No deep-diving yet. Append landscape notes to the ledger.
### Move 3: Deep-Dive per Sub-Question
For each sub-question:
- Search from 3+ angles with different categories and framings. Use
`mcp_searxng_searxng_web_search`. Apply the time filter from the ledger.
- Read full pages (5-10 per sub-question) via `mcp_searxng_web_url_read`.
Snippets are pointers, not sources.
- Follow citation trails: if a page cites a study/paper/dataset, go read that
source. Depth-first — one trail at a time, to its end, then the next.
- Extract structured data: tables, numbers, dates, versions, names.
- Assign credibility tier at capture time (not post-hoc in output).
- Cross-reference: every claim needs 2+ independent sources.
- Use browser tools (Playwright) when static extraction fails on JS-heavy pages.
- Append every finding to the ledger.
Citation trail stop conditions:
- Reached primary source (original paper, official docs, raw data)
- Dead end (paywall, 404, requires login)
- Circular reference (already read this source)
### Move 4: Disconfirmation Pass
Actively try to falsify each key claim:
- Search for "[claim] wrong", "[claim] criticism", "[claim] outdated"
- Check dates: is a 2024 source being presented as current?
- Look for contradicting evidence
- Append disconfirmation findings to the ledger
This is distinct from cross-referencing. Cross-reference confirms agreement.
Disconfirmation actively hunts for disagreement.
### Move 5: Condensation
Read the full ledger from disk. Read the phase gate file. If any box is
unchecked, do NOT condense — go back and complete that item.
Synthesize:
- Lead with the answer. No process narrative.
- Evidence-backed. Every claim tied to a source from the ledger.
- Concrete. Numbers, dates, names, specifics.
- Uncertainty explicit. "X confirmed by A and B. Y single-sourced from C.
Z unresolved — D and E conflict."
- Sources section: numbered list with URLs and credibility tiers.
## Guardrails
### Saturation Check (Mechanical)
Every 3-4 searches, run this command — do not self-assess:
```
grep -c "^## Finding #" /tmp/research-<sid>.md
```
Compare to the previous count. If zero new findings in the last 3 searches,
that sub-question is saturated. Move to the next. This is a measurable fact,
not a vibe.
### Phase Gate File
The phase gate is a file at `/tmp/research-<sid>-gate.md`. Write it and read it
back — it is not a mental checklist. Before moving to Move 5, read the gate file.
If any box is unchecked, do NOT condense. Go back and complete that item.
```
# Phase Gate
- [ ] All sub-questions have findings in the ledger
- [ ] Disconfirmation pass completed for key claims
- [ ] OR: saturation condition fired (no new findings in last 3 searches)
- [ ] OR: turn ceiling approaching (wrap up what you have)
```
Update this file as you progress.
### Turn Ceiling
200 turns is a safety net, not a target. Hitting the ceiling is a failure to
condense, not the normal path. If the ceiling is hit, deliver partial findings
with a note on what's missing.
## Tool Policy
**One rule: local and free only.** Use any tool already configured on the
research profile that fits this rule. No internet-based paid services, no SaaS
APIs with billing, no metered endpoints. You determine what fits — you are not
given a list of allowed or disallowed tools.
**No self-provisioning.** Never install, pull, or spin up new tools at runtime.
### Tools That Would Have Helped
At the end of the response (after Sources), add a section if any limitations
were hit:
```
Tools that would have improved this research:
- <tool name>: <what it would have enabled>
None: all sources were accessible with available tools.
```
This is informational only. Do your best with what's available and note what
could have been better. Do not stop or block on tool gaps.