diff --git a/deep-research/SKILL.md b/deep-research/SKILL.md index f19cd47..d982aed 100644 --- a/deep-research/SKILL.md +++ b/deep-research/SKILL.md @@ -1,7 +1,7 @@ --- name: deep-research -description: Dispatch exhaustive deep web research to the research profile. Triggered by "deep research", "ask web", or "research this". The research profile runs with the deep-web-research skill loaded — five-move flow, external ledger, mechanical saturation, disconfirmation, condensation from disk. -version: 1.0.1 +description: Dispatch exhaustive deep web research to the research profile. Triggered by "deep research", "ask web", or "research this". The research profile runs with the deep-web-research skill loaded — six-move flow, external ledger, mechanical saturation, disconfirmation, condensation from disk. +version: 2.0.0 author: Hermes Agent license: MIT platforms: [linux] @@ -17,12 +17,14 @@ metadata: Dispatches a research question to the research profile, which runs with the `deep-web-research` skill loaded. The research agent does exhaustive, trail-following research — any tool, any trail, no rush — then condenses everything into a tight, concrete, evidence-backed answer. This agent just relays the result. +See `references/design-rationale.md` for the v2.0 architecture decisions and Claude validation findings. + **Trigger phrases:** "deep research", "ask web", "research this", "deep dive on", "exhaustive research on" ## Command ``` -hermes -p research -s deep-web-research chat -q "" -Q --max-turns 200 --yolo +hermes -p research -s deep-web-research chat -q "" -Q --max-turns 600 --yolo ``` - Output line 1: `session_id: ` — capture this @@ -31,20 +33,21 @@ hermes -p research -s deep-web-research chat -q "" -Q --max-turns 200 **Subsequent asks (resume the session):** ``` -hermes -p research -s deep-web-research chat --resume -q "" -Q --max-turns 200 --yolo +hermes -p research -s deep-web-research chat --resume -q "" -Q --max-turns 600 --yolo ``` ## What the Research Agent Does The `deep-web-research` skill on the research profile enforces: +0. **Analyze & Strategize** — analyze the question, select tools from the arsenal, write strategy to ledger 1. **Decompose** — break the question into sub-questions with time classifications 2. **Landscape Pass** — shallow sweep, identify key sources and terms 3. **Deep-Dive** — 3+ search angles per sub-question, full-page reads, citation trails, depth-first 4. **Disconfirmation** — actively hunt for contradiction and outdated claims 5. **Condensation** — read the external findings ledger from disk, synthesize into a concrete answer -The research agent writes all findings to `/tmp/research-.md` (external ledger) and uses a phase gate file to enforce completion before condensing. Mechanical saturation checks (`grep -c`) prevent endless searching. +The research agent writes all findings to `/tmp/research-.md` (external ledger) and uses a phase gate file to enforce completion before condensing. Mechanical saturation checks (`grep -c`) prevent endless searching. Re-strategize checkpoints after Move 2 and every ~10 findings during Move 3 enable mid-research pivots. ## When to Use @@ -80,33 +83,20 @@ The research agent self-reports are not verified fact. If it claims a file write 1. **Profile flag required.** Always use `-p research`. The sticky default may be general. 2. **Skill flag required.** Always use `-s deep-web-research`. Without it, the research agent runs in normal mode without the methodology. 3. **--yolo is required.** The research agent runs headless. Without it, approval prompts fail closed. -4. **200 turns is the ceiling, not the target.** The research agent should condense well before 200. Hitting the ceiling means it failed to condense. +4. **600 turns is the ceiling, not the target.** The research agent should condense well before 600. Hitting the ceiling means it failed to condense. 5. **Don't re-condense.** The research agent already produced a condensed answer. Relay it, don't summarize it further. 6. **Don't do the research yourself.** If the user triggers deep research, dispatch it. Don't run a few searches and call it done. This is the #1 failure mode: the agent runs 2-3 `mcp_searxng_searxng_web_search` calls, gets empty results, and gives up. That's not deep research — that's a casual lookup. If you catch yourself typing `mcp_searxng_searxng_web_search` for a deep research request, STOP. You're doing it wrong. Dispatch to the research profile. 7. **The methodology skill lives on the research profile.** `deep-web-research` is at `~/.hermes/profiles/research/skills/research/deep-web-research/SKILL.md`. It does NOT exist on the general profile. Don't search for it here — it won't be found. The general profile only has this delegation skill. -8. **`reasoning_effort: xhigh` breaks `glm-5.2:cloud`.** The research profile's default model (`glm-5.2:cloud` via `custom:ollama`) rejects `xhigh` — it only accepts `high`, `medium`, `low`, `max`, or `none`. If the dispatch fails with "HTTP 400: invalid reasoning value: 'xhigh'", check `~/.hermes/profiles/research/config.yaml` → `agent.reasoning_effort`. Set it to `max` (the highest level the model actually supports). Do NOT downgrade to `high` without asking the user — they explicitly want maximum reasoning effort. If the model changes to one that supports `xhigh`, revert to `xhigh`. ## Pre-Dispatch Reasoning Check (MANDATORY) -Before EVERY deep research dispatch, verify the research profile's reasoning_effort is set to the maximum the model supports. This is a hard gate — do not dispatch without checking. +Before EVERY deep research dispatch, set the research profile's reasoning_effort to `max`. This is a hard gate — do not dispatch without it. ``` -# 1. Read the research profile's current model and reasoning_effort -grep -E "default:|reasoning_effort:" ~/.hermes/profiles/research/config.yaml - -# 2. If reasoning_effort is NOT 'max' (or 'xhigh' for models that support it): -# - If model is glm-5.2:cloud → set to 'max' (xhigh rejected) -# - If model is deepseek-v4-pro:cloud → set to 'max' (xhigh rejected) -# - If model is minimax-m3:cloud → set to 'xhigh' (supports it) -# - If unsure what the model supports → set to 'max' (safe default) - -# 3. Apply the fix if needed: hermes -p research config set agent.reasoning_effort max -# OR for models that support xhigh: -hermes -p research config set agent.reasoning_effort xhigh ``` -**Rule:** Always dispatch with the highest reasoning effort the model actually supports. Never silently downgrade. If the model changed and you don't know what it supports, test with `max` first (universally safer than `xhigh`). +`max` is the highest effort setting universally supported by all models. No model-specific table needed. If the model changes to one that supports `xhigh`, the user will set it in config — the pre-dispatch check just ensures it's at `max` minimum. ## Research → Validate → Fix Pipeline diff --git a/deep-web-research/SKILL.md b/deep-web-research/SKILL.md index b9b9729..08cd99c 100644 --- a/deep-web-research/SKILL.md +++ b/deep-web-research/SKILL.md @@ -1,7 +1,7 @@ --- name: deep-web-research -description: Exhaustive deep web research — five-move flow, external findings ledger, mechanical saturation check, disconfirmation, condensation from disk. Opt-in skill for the research profile. -version: 1.0.0 +description: Exhaustive deep web research — six-move flow, external findings ledger, mechanical saturation check, disconfirmation, condensation from disk. Domain-aware tool selection with adaptive re-strategizing. Opt-in skill for the research profile. +version: 2.0.0 author: Hermes Agent --- @@ -25,18 +25,54 @@ These persist across all turns — they are in the skill, not in fading context: - **Local and free only.** No internet-based paid services, no SaaS APIs with billing, no metered endpoints. Use any tool already configured that fits this rule. +## Tool Selection Principles + +These are reasoning heuristics, not rules. Apply judgment. Read once — they are +static reference, not generated per-question. + +1. **Match tool strength to information need.** Don't use SearXNG when arxiv is + purpose-built for paper search. Don't use SearXNG when maigret is purpose-built + for username lookup. Each tool exists because it's better at something than + general search. + +2. **Compose, don't isolate.** Most questions need multiple tool categories. A + person investigation might need OSINT (maigret, holehe) + social (twscrape) + + web (SearXNG) + reference (wikipedia). Compose across categories. + +3. **Start broad, then specialize.** Landscape pass uses broad tools (SearXNG, + wikipedia). Deep-dive uses specialized tools (arxiv, maigret, Firecrawl). Don't + lead with Spiderfoot when a SearXNG search would tell you whether there's even + a domain to scan. + +4. **Follow the trail, not the plan.** If a finding points to a source type you + didn't anticipate, add the tool that handles that source type. The strategy is + a starting point, not a contract. + +5. **Prefer purpose-built over general.** arxiv > SearXNG for papers. maigret > + SearXNG for usernames. holehe > SearXNG for email registration. gnews > SearXNG + for news. Use SearXNG for what doesn't have a purpose-built tool. + +6. **Consult the `search-key` decision matrix.** It maps task types to tools. Use + it as a reference, not a lookup table. If the matrix says `maigret` for username + lookup, that's a strong signal — but verify it makes sense for this specific + question. + +7. **OSINT tools are an escalation ladder, not a default.** Start with SearXNG for + person lookups. Escalate to maigret if you have a username. Escalate to holehe + if you have an email. Escalate to Spiderfoot only if you have a domain and the + scope justifies it. Don't jump to the most aggressive tool first. + ## Architecture: External Findings Ledger -At 200 turns, early findings scroll out of context. The ledger is the fix. +At 600 turns, early findings scroll out of context. The ledger is the fix. **Path:** `/tmp/research-.md` **Schema per finding:** ``` -## Finding #: -- Sub-Question: +### Finding # [SQ]: - Claim: -- Source: — +- Source: — - Confirmed by: (or "single-sourced") - Date: - Confidence: confirmed | likely | single-sourced | disputed @@ -47,11 +83,48 @@ At 200 turns, early findings scroll out of context. The ledger is the fix. finding with the same claim. If found, add the new source to "Confirmed by" instead of creating a duplicate. Duplicate findings defeat the saturation check. -**Sub-question tagging:** Every finding must reference which sub-question it -belongs to. This enables the phase gate to verify coverage and the condensation -phase to group findings by topic. +**Sub-question tagging:** Every finding line includes `[SQ]` — e.g., +`### Finding #3 [SQ2]: ...`. This enables per-sub-question saturation checks +that are immune to header ordering. -## Five-Move Research Flow +**Credibility Tiers:** + +| Tier | Definition | Examples | +|---|---|---| +| `primary` | Original source — the entity that created or owns the data | Official docs, original research paper, company release notes, raw dataset, government database, court filing | +| `secondary` | Reports on or analyzes primary sources with editorial oversight | News article citing a study, analyst report, Wikipedia (well-cited), reputable tech blog | +| `aggregator` | Collects/curates from other sources without original analysis | Hacker News, Reddit, link aggregators, auto-generated comparison sites | +| `social` | Individual opinion without institutional backing | Tweet, personal blog, forum post, YouTube comment | +| `osint` | Machine-generated tool output — marks provenance, not confidence | maigret report, holehe results, Spiderfoot scan, theHarvester output | + +**Important:** `osint` marks *provenance* (where the claim came from), not +*confidence*. A single raw maigret hit is low-confidence osint; the same account +confirmed by a human-authored source is still confirmable. Don't auto-cap +confidence just because the tier is `osint`. + +## Six-Move Research Flow + +### Move 0: Analyze & Strategize + +1. Consult the `search-key` decision matrix — it's your tool reference, not a + lookup you match against. + +2. Call `mcp_searxng_searxng_instance_info` to discover available categories and + engines. Write them to the ledger. + +3. Write this to the ledger, then start Move 1: + + ``` + ## Strategy + - Info needed: + - Where it lives: + - Tools: (purpose-built over general search) + - Pivots: if I find , add + ``` + +4. OSINT gate — only if the plan names maigret / holehe / theHarvester / + Spiderfoot AND the target is a private individual with no public role: log + `## Ethics Note: ` and confirm the scan is proportionate. ### Move 1: Decompose @@ -77,6 +150,21 @@ Shallow-but-broad sweep. For each sub-question, 1-2 searches, read top 1-2 resul Extract only: key sources, terms of art, where disagreement lives, major players. No deep-diving yet. Append landscape notes to the ledger. +**After Move 2: Re-Strategize Checkpoint.** Read the strategy from the ledger. Ask: +- Are the tools I selected actually producing results? +- Have I discovered new angles that need different tools? +- Is the topic different than I initially thought? +- Are there tools I should add or drop? + +If the answer to any is yes, update the strategy section: +``` +### Strategy Revision +- Trigger: +- Added tools: +- Dropped tools: +- New pivot signals: +``` + ### Move 3: Deep-Dive per Sub-Question For each sub-question: @@ -90,13 +178,27 @@ For each sub-question: - Assign credibility tier at capture time (not post-hoc in output). - Cross-reference: every claim needs 2+ independent sources. - Use browser tools (Playwright) when static extraction fails on JS-heavy pages. -- Append every finding to the ledger. +- Append every finding to the ledger with `[SQ]` tag. Citation trail stop conditions: - Reached primary source (original paper, official docs, raw data) - Dead end (paywall, 404, requires login) - Circular reference (already read this source) +**Every ~10 findings during Move 3: Re-Strategize Checkpoint.** Same questions +as after Move 2. Update strategy if needed. + +**Information-void handling:** After 5 searches on a sub-question with zero +relevant results (not zero *new* findings — zero *any* findings), document the +void and move on: +``` +## SQ: — VOID +- Searches attempted: +- Tools tried: +- Reason: no relevant results found +- Note: +``` + ### Move 4: Disconfirmation Pass Actively try to falsify each key claim: @@ -113,13 +215,32 @@ Disconfirmation actively hunts for disagreement. Read the full ledger from disk. Read the phase gate file. If any box is unchecked, do NOT condense — go back and complete that item. -Synthesize: -- Lead with the answer. No process narrative. -- Evidence-backed. Every claim tied to a source from the ledger. -- Concrete. Numbers, dates, names, specifics. -- Uncertainty explicit. "X confirmed by A and B. Y single-sourced from C. - Z unresolved — D and E conflict." -- Sources section: numbered list with URLs and credibility tiers. +Synthesize using this template: + +``` +# + +## Key Findings +- — (, ) +- — (, ) + +## Uncertainty +- : single-sourced from +- : disputed — vs +- : unresolved — no sources found + +## Tools Used +- : +- : + +## Sources +1. — () +2. — () + +## Tools That Would Have Helped +- : +None: all sources were accessible with available tools. +``` ## Guardrails @@ -127,11 +248,12 @@ Synthesize: Every 3-4 searches, run this command — do not self-assess: ``` -grep -c "^## Finding #" /tmp/research-.md +grep -c "^### Finding #.*\[SQ\]" /tmp/research-.md ``` Compare to the previous count. If zero new findings in the last 3 searches, that sub-question is saturated. Move to the next. This is a measurable fact, -not a vibe. +not a vibe. The `[SQ]` tag makes this immune to header ordering — it counts +by tag, not by position. ### Phase Gate File @@ -143,17 +265,34 @@ If any box is unchecked, do NOT condense. Go back and complete that item. # Phase Gate - [ ] All sub-questions have findings in the ledger - [ ] Disconfirmation pass completed for key claims +- [ ] Source diversity: ≥5 unique domains across all findings +- [ ] Source diversity: no single domain >30% of findings +- [ ] Tool diversity: ≥2 different tool categories used (not just SearXNG) +- [ ] Recency spread: findings span appropriate date range for topic - [ ] OR: saturation condition fired (no new findings in last 3 searches) - [ ] OR: turn ceiling approaching (wrap up what you have) ``` Update this file as you progress. +Mechanical domain diversity check: +``` +grep -oP 'Source: \K[^ ]+' /tmp/research-.md | grep -oP 'https?://[^/]+' | sort -u | wc -l +``` + ### Turn Ceiling -200 turns is a safety net, not a target. Hitting the ceiling is a failure to -condense, not the normal path. If the ceiling is hit, deliver partial findings -with a note on what's missing. +600 turns is a safety net, not a target. Hitting the ceiling is a failure to +condense, not the normal path. Ramp: if ≥450 turns used, begin condensation +within 60 turns. If ≥540 turns used, condense immediately with what you have. +If the ceiling is hit, deliver partial findings with a note on what's missing. + +### SearXNG Failure Handling + +If SearXNG returns errors or empty results for 3 consecutive searches (different +queries), wait 10 seconds and retry once. If still failing, note the instance +issue in the ledger and fall back to `web_search` (built-in) or +`duckduckgo-search` (Python library) for the remainder of that sub-question. ## Tool Policy