tools-update-cron: sync 2026-08-30 — 9 skill(s) updated

This commit is contained in:
Hermes Agent
2026-08-30 01:01:13 -05:00
parent 91249bf617
commit 8bbdcde08b
9 changed files with 1177 additions and 1021 deletions
+178
View File
@@ -0,0 +1,178 @@
---
name: ai-vault-kb
description: "Use when writing or searching the ai_vault_kb Qdrant vault."
version: 2.0.0
author: Hermes Agent
license: MIT
platforms: [linux]
metadata:
hermes:
tags: [qdrant, knowledge-base, ai-vault, search, rag, bm25]
related_skills: [research-knowledge-management, deep-web-research, qdrant-collection-management]
---
# AI Vault KB — the fleet's one vault
`ai_vault_kb` is the single Qdrant collection holding all AI/ML knowledge: research,
pipeline state, model configs, prompts, decisions, bugs, hardware facts. One vault,
one collection — never a topic-specific collection.
## THE ONE RULE — writes go through `ai_vault_kb.py`, nothing else
```bash
python3 /home/n8n/bin/ai_vault_kb.py add --type <t> --title "..." --content "..." [flags]
```
That helper is **the only interface that produces a usable, searchable record.**
### Why `mcp__better_qdrant__add_documents` is FORBIDDEN for this collection
It performs a **bare-vector upsert**. For every chunk it writes it produces:
| | `ai_vault_kb.py add` | `mcp__better_qdrant__add_documents` |
|---|---|---|
| dense vector (unnamed slot) | yes | yes |
| **`bm25` sparse vector** | **yes** | **NO** |
| `doc_type` / `title` / `tags` | yes | **NO** |
| `trust` / `host` / `path` / `doc_id` / `pipeline_stage` | yes | **NO** |
A point with no `bm25` slot **cannot be returned by a keyword/BM25 search** — paste an
exact error string, model filename or node name and it will never match. A point with
no `doc_type` **cannot be returned by any typed search** (`--type research`,
`--type issue`, …) or by any faceted list. It survives only in the dense half of a
hybrid query and renders as `[?] (untitled)`.
**It looks ingested and it cannot be found.** This is not a style preference — roughly
4,700 points in this collection were created that way and had to be repaired. Do not
create more. There is no file size, no hurry and no MCP convenience that justifies it.
If you catch yourself reaching for `add_documents` because the helper is awkward for a
big file: the helper takes `--file` and chunks it itself, with no MCP 120 s timeout.
## Writing
```bash
python3 /home/n8n/bin/ai_vault_kb.py add \
--type finding --title "LTX 2.3 audio desync above 121 frames" \
--content "..." \
--stage t2v --tool ltx-video --host 10.0.0.202 \
--trust official --importance 0.7 --tags "ltx-2.3,audio,desync"
```
Long documents: `--file /abs/path/report.md` instead of `--content` (auto-chunked,
one shared `doc_id` across the chunks). `--json` prints `{"doc_id":…, "chunks":N}`.
| Flag | Meaning |
|---|---|
| `--type` **(required)** | `tool` `setting` `workflow` `host` `model` `technique` `issue` `decision` `research` `asset` `prompt` `finding` |
| `--title` **(required)** | what a future search will read as the headline |
| `--content` / `--file` | body text, or a file to chunk |
| `--stage` | `story` `script` `character` `keyframe` `t2v` `i2v` `upscale` `interpolate` `tts` `lipsync` `music` `assembly` `publish` `infra` |
| `--tool` `--host` `--path` `--url` `--version` | provenance; `--host` is the box the fact is about |
| `--status` | `active` `candidate` `deprecated` `broken` `planned` (default `active`) |
| `--trust` | `official` `github` `community` `social` (default `official`) |
| `--tags` | comma-separated; **tags are an exact-match keyword index** — put the slug, the filename, the error code here |
| `--importance` | 0.01.0 |
| `--doc-id` | append more chunks to an existing document |
Unknown vocabulary values warn but are accepted — the schema is faceted, not strict.
A warning is not a failure; do **not** switch to the MCP tool because of one.
### Mandatory before every write: dedup-first
```bash
python3 /home/n8n/bin/ai_vault_kb.py search --query "<the thing you are about to save>"
```
- score **≥ 0.85** — already recorded, skip
- **0.700.84** — add only if meaningfully new
- **< 0.70** — always add
### Host records
`--type host`, one stable `--doc-id` per box. Re-add with the same `--doc-id` to update
or append a dated chunk, rather than creating a second record for the same machine.
## Reading
```bash
# hybrid (dense + BM25, RRF fusion) — the default, use it
python3 /home/n8n/bin/ai_vault_kb.py search --query "ltx 2.3 native audio"
# pure keyword — exact filenames, error strings, node names
python3 /home/n8n/bin/ai_vault_kb.py search --query "ltxv-097-dev-fp8.safetensors" --mode bm25
# typed / faceted
python3 /home/n8n/bin/ai_vault_kb.py list --type issue --tool comfyui --limit 20
python3 /home/n8n/bin/ai_vault_kb.py list --tag 2026-08-09-horizon-scan
python3 /home/n8n/bin/ai_vault_kb.py facet --field tool
python3 /home/n8n/bin/ai_vault_kb.py stats # points, doc_type inventory, health
python3 /home/n8n/bin/ai_vault_kb.py stale --days 30
python3 /home/n8n/bin/ai_vault_kb.py get --id <point-id>
```
`mcp__better_qdrant__search` is **read-only and therefore allowed**, but it is
dense-only — it silently misses anything a keyword query would have found. Prefer the
helper's `search`. Use the MCP one only when you have no shell.
## Deleting
```bash
python3 /home/n8n/bin/ai_vault_kb.py delete --doc-id <uuid> --yes # a whole document
python3 /home/n8n/bin/ai_vault_kb.py delete --id <point-id> --yes # one chunk
```
Per-document delete **exists**. Never
`mcp__better_qdrant__delete_collection(collection="ai_vault_kb")` — that destroys the
fleet's entire brain and there is no undo. Earlier versions of this skill described the
nuke as the only option; that was wrong.
## Verify the write landed
A write is not done until it is retrievable **both ways**. The BM25 leg is the one a
bare upsert cannot pass, so it is the real test:
```bash
D=<doc_id from --json>
python3 /home/n8n/bin/ai_vault_kb.py search --query "<distinctive phrase>" --mode bm25 --doc-id $D
python3 /home/n8n/bin/ai_vault_kb.py list --type <the type you used> --doc-id $D
```
Zero hits on the BM25 leg means something other than `ai_vault_kb.py` wrote it.
## Deep-research reports
Do not ingest them by hand. `publish_report.py` (in the `deep-web-research` skill) is
the only sanctioned path — it publishes, ingests via this helper, verifies hybrid+BM25,
and writes the `.meta.json` receipt in one action.
A `.meta.json` sidecar is a **receipt written after verification, not proof**. Sidecars
written before that rule exists claim `doc_id`s that were never upserted. If you need
to know whether a report is in the vault, ask the vault (`list --tag <slug>`), never the
directory listing.
## Infrastructure
| Setting | Value |
|---|---|
| Qdrant | `http://10.0.0.22:6333`, collection `ai_vault_kb` |
| Embeddings | `snowflake-arctic-embed2` on **mini, `10.0.0.30:11434`** — the vault's embedder (mini also hosts `qwen3-embedding:0.6b` for the Cognee brain; the two are separate) |
| Helper | `/home/n8n/bin/ai_vault_kb.py` (zero deps, urllib only) |
| Vectors | unnamed dense 1024-dim + named sparse `bm25` |
**The CARE rule for mini:** every vault ingest embeds there, and mini also serves the
Cognee brain's embedder (`qwen3-embedding:0.6b`). Batch your writes, keep them bounded,
never hammer it.
## Pitfalls
- **Never hand-roll raw Qdrant HTTP for a write.** A `PUT /collections/ai_vault_kb/points`
with a bare vector reproduces the exact defect the MCP tool causes. Reads
(`GET /collections/...`, `POST .../points/scroll`, `.../points/count`) are fine.
- **Absolute paths** for `--file` / `--path`.
- **Collection name is exact** — `ai_vault_kb`, not `ai-vault-kb`.
- **Don't create topic-specific collections.** No `ltx-research`, no `comfyui-workflows`.
- **`fact_store` and the `memories` collection are not the vault.** Research findings,
model configs, prompt guides and infra facts all go here.
- **A search that returns nothing is a real answer.** Say "not in the vault" and go
research it — do not assume the record exists but is hiding.
+87
View File
@@ -0,0 +1,87 @@
---
name: cognee-brain
description: "Use when user says brain or cognee. Cognee memory brain (Kuzu+LanceDB)."
version: 1.0.0
author: Hermes Agent
license: MIT
platforms: [linux]
metadata:
hermes:
tags: [cognee, brain, memory, knowledge-graph, kuzu, lancedb]
related_skills: [ai-vault-kb, save, memory-ingest]
---
# Cognee Brain — homelab memory layer
The homelab memory layer ("the brain") is **Cognee** on BRAIN-MAIN at `10.0.0.23`.
It replaced the previous memory layer on 2026-08-29. Cognee ingests text, builds a
knowledge graph (entities + relationships) and a vector index side-by-side, and
exposes them through Python, CLI, MCP, and REST.
## When to Use
- User says "brain", "cognee", "what do we know about X", "remember this"
- Before research/infra tasks — query the brain first
- After research with concrete conclusions — write durable facts back
## Endpoints
- API: `http://10.0.0.23:8080` (cognee-backend, cognee/cognee:main)
- MCP: `http://10.0.0.23:8001/mcp` (cognee-mcp, cognee/cognee-mcp:main)
- Dataset: `homelab-stack` (the single shared dataset — use it everywhere)
- LLM: Ollama cloud-passthrough `http://10.0.0.23:11434/v1` (minimax-m3:cloud)
- Embed: mini `http://10.0.0.30:11434/api/embed` (qwen3-embedding:0.6b, 1024 dims)
- Graph: Kuzu (Ladybug) files under `/var/lib/cognee/system/databases`
- Vectors: LanceDB files under `/var/lib/cognee/system/databases/cognee.lancedb`
## MCP tools (auto-discovered in Hermes as `mcp__cognee__*`)
- `remember` — store data. Without `session_id` = permanent memory (add + cognify
pipeline, builds the graph). With `session_id` = session cache only (fast, no
extraction). Pass `data` (text) OR `filename`+`content_base64` (file upload).
`dataset_name` defaults to the client's agent-scoped dataset — ALWAYS pass
`dataset_name="homelab-stack"` explicitly.
- `recall` — search memory. `query` (required), optional `search_type`
(HYBRID_COMPLETION default; CHUNKS/GRAPH_COMPLETION/RAG_COMPLETION/etc.),
`datasets` (comma-separated), `top_k`.
- `forget` — delete. `dataset` (name), `dataset_id`, `data_id`+`dataset`, or
`everything`.
## REST API (for curl / scripts)
- `POST /api/v1/remember` (multipart: `data=@file` + `datasetName=homelab-stack`)
- `POST /api/v1/search` (JSON: `query`, `datasets`, `searchType`)
- `DELETE /api/v1/datasets?dataset_name=...`
- `GET /health``{"status":"ready","health":"healthy","version":"1.5.3-local"}`
**Field-name gotcha:** the search API uses `query` + `searchType` (NOT
`query_text`/`query_type`). Wrong names silently fall back to the default query
"What is in the document?" and return a generic summary.
## Write rules
- **One dataset:** `homelab-stack`. Never create a second dataset for the same
domain — the whole point is a single combined view.
- **`remember` is add+cognify in one call** — no separate cognify step.
- **No group_id, no triplet, no temporal supersession.** Cognee uses datasets. To correct a fact, `remember` the corrected statement;
Cognee's graph merges entities by name.
- **Don't re-ingest the same content twice** — `remember` is not idempotent.
- **Embed dim is 1024** (`EMBEDDING_DIMENSIONS=1024`, qwen3-embedding:0.6b). Never let it
default to 3072.
## Verify a write
- `recall` with a distinctive phrase from the content; a correct answer proves
the graph has it.
- Check `docker logs cognee-backend` for `api.openai.com` / `ProviderConfigMismatch`
— zero hits means the LAN-only provider pairing held.
## Pitfalls
- **`remember` returns `status: completed` synchronously** (unless `background=true`).
A completed status means the graph is built — no async polling needed.
- **Search field names** are `query`/`searchType` (see above).
- **No auth** on API/MCP (`REQUIRE_AUTHENTICATION=false`) — LAN-trust only.
- **Container runs as uid 1000** — `/var/lib/cognee/{system,data}` must stay 1000:1000.
- **MCP allowed_hosts** = 10.0.0.23/42/28/15 (edit `MCP_ALLOWED_HOSTS` in compose to add hosts).
- **Start/stop:** `sudo systemctl start|stop cognee.service` (docker compose up/down).
+76 -1004
View File
File diff suppressed because it is too large Load Diff
+46
View File
@@ -0,0 +1,46 @@
---
name: hermes-capability-lookup
description: "Use when asked if Hermes supports X or how to integrate X."
version: 1.0.0
author: Hermes Agent
metadata:
hermes:
related_skills: [hermes-agent, native-mcp, searxng-smart-search]
---
# Hermes Capability Lookup
Trigger: "does Hermes have X?", "can Hermes use X?", "is there a skill for X?", "how do I integrate X with Hermes?"
## Lookup workflow (in order)
1. **Local profile skills**`search_files` pattern `*<name>*` in `~/.hermes/profiles/<profile>/skills/` plus `skills_list`. Zero hits = not installed locally (may still exist upstream).
2. **Official docs** — hermes-agent.nousresearch.com/docs. Key pages: `/docs/user-guide/features/memory-providers` (bundled memory providers), `/docs/reference/skills-catalog`, `/docs/integrations/`. Read via `mcp_searxng_web_url_read` (one `url` per call).
3. **Upstream repo** — github.com/NousResearch/hermes-agent. SearXNG query: `site:github.com NousResearch hermes-agent <term>`. GitHub issue bodies are extractable via web_url_read (nav chrome dominates but the body is in the result).
4. **If not bundled** — identify integration paths (below), present options with a recommendation (user requires Recommended + Why on every options list).
## Bundled memory providers (9, as of 2026-08)
Honcho, OpenViking, Mem0, Hindsight, Holographic, RetainDB, ByteRover, Supermemory, Memori.
- Only ONE external provider active at a time; built-in MEMORY.md/USER.md always active.
- `hermes memory setup` = interactive picker; `hermes memory status`; `hermes memory off`.
- Full comparison table on the docs memory-providers page.
## Integration paths for external memory systems
1. **MCP registration** (config-only, minutes) — add the external tool's MCP server under `mcp_servers` in config.yaml (see native-mcp skill). Hermes gets the tools immediately. Caveat: capture side may not run — e.g. claude-mem's worker only fills from Claude Code hooks, so Hermes gets read-only search over whatever Claude Code recorded.
2. **Custom memory-provider plugin** — implement Hermes' MemoryProvider contract: `prefetch()`/`queue_prefetch()`, `sync_turn()`, `on_session_end()`, `on_memory_write(action, target, content)`. Docs: `/docs/developer-guide/memory-provider-plugin`. This is how third-party providers wire in (e.g. the local cognee integration, GitHub issue #14368).
3. **HTTP API bridge** — if the external system exposes a REST API (claude-mem worker API), a provider plugin or script can POST observations and GET search results.
## Pitfalls
- `mcp_searxng_web_url_read` takes `url` (singular string), NOT `urls` (list). One URL per call; batch multiple pages as parallel calls.
- "Not bundled" is a dated fact — re-verify against current docs before asserting.
- GitHub issue pages via web_url_read return heavy nav chrome; the issue body is still in the result — search result descriptions often already carry the key sentence.
## References
- `references/hook-system.md` — verified Hermes hook taxonomy: full VALID_HOOKS set, the "no end-of-turn hook" finding, `pre_verify` gate (file-edit + 3-nudge cap), shell-hook allowlist consent, response shapes, SOUL/USER.md paths
- `references/claude-mem-integration.md` — claude-mem: architecture, worker REST API, MCP tools, Hermes integration options
- `references/cognee-status.md` — cognee in Hermes: not bundled, issue #14368 canonical reference, lessons for Lance/Kuzu-backed providers
+175
View File
@@ -0,0 +1,175 @@
---
name: ipmi-bmc-management
description: Use when working with IPMI/BMC out-of-band server mgmt.
version: 1.0.0
author: Hermes Agent
license: MIT
platforms: [linux]
metadata:
hermes:
tags: [ipmi, bmc, redfish, supermicro, out-of-band, hardware, firmware]
related_skills: [proxmox-lxc-deployment, truenas]
---
# IPMI / BMC Out-of-Band Management
Manage servers out-of-band via their BMC (Baseboard Management Controller).
Covers read-only inventory, sensor/thermal readout, firmware version checks,
and the modern Redfish REST API. Credentials live in fact_store, never in this
skill.
## Install
```bash
sudo apt-get install -y ipmitool # CLI (also: freeipmi-tools)
```
## ipmitool — read-only inventory
```bash
ipmitool -I lanplus -H <bmc-ip> -U <user> -P <pass> -C 3 <cmd>
```
Useful read-only commands:
- `mc info` — BMC identity (manufacturer, product ID, firmware rev, IPMI version)
- `fru print` — board serial, mfg date, product info
- `sensor list` / `sdr list` — thermals, voltages, fans, GPU temps
- `chassis status` — power state, fault flags, power-restore policy
- `power status` — on/off
- `lan print` — BMC network config (IP, MAC, gateway, cipher suites)
- `user list` — configured BMC accounts
- `sel info` / `sel list` — system event log (check for full/overflow)
## CRITICAL: Supermicro cipher-suite quirk
Supermicro BMCs (H12SSL-i and similar) FAIL the default RMCP+ cipher suite
(17 / SHA256) with:
```
Error in open session response message : invalid role
Error: Unable to establish IPMI v2 / RMCP+ session
```
**Fix: force cipher suite 3** with `-C 3`. This is the single most common
failure when scripting Supermicro IPMI. `-C 17` and `-L ADMINISTRATOR` do NOT
help — only `-C 3` works. IPMI 1.5 (`-I lan`) also works but is insecure.
## Redfish — the modern API (preferred for firmware inventory)
Supermicro BMCs expose a full Redfish REST API. Use it for firmware version
checks — it's cleaner than raw OEM IPMI commands (which often return
"Request data length invalid").
```bash
# Root (confirms Redfish + vendor)
curl -sk -u 'USER:PASS' https://<bmc-ip>/redfish/v1/
# Firmware inventory — BMC, BIOS, CPLD versions in one shot
curl -sk -u 'USER:PASS' https://<bmc-ip>/redfish/v1/UpdateService/FirmwareInventory
# Per-component version
curl -sk -u 'USER:PASS' https://<bmc-ip>/redfish/v1/UpdateService/FirmwareInventory/BMC
curl -sk -u 'USER:PASS' https://<bmc-ip>/redfish/v1/UpdateService/FirmwareInventory/BIOS
# System info (BiosVersion, Model, ProcessorSummary)
curl -sk -u 'USER:PASS' https://<bmc-ip>/redfish/v1/Systems/1
```
FirmwareInventory members are named `BMC`, `BIOS`, `Motherboard_CPLD_1`,
`GPU<n>` etc. Each returns `Version`, `Updateable`, `ReleaseDate`. GPUs show
`Version: None` (not firmware-managed via BMC).
## Firmware currency check
1. Read current versions via Redfish FirmwareInventory (BMC + BIOS + CPLD).
2. Cross-check against Supermicro download center:
`https://www.supermicro.com/en/support/resources/downloadcenter/firmware/MBD-<board>/BIOS`
(search result snippets list "BIOS Revision: X.Y" and "BMC Firmware Revision: X.Y.Z").
3. Report the gap. Note: Supermicro BMC downgrades are NOT supported after
security updates — upgrades are one-way.
## Firmware UPDATE (flash) — method decision
**The H12/H12SS bundle ships SAA (SuperServer Automation Assistant) UEFI, NOT
SUM.** SUM is the older in-band updater; the H12 firmware zip contains
`SAA.efi` + `flash.nsh`. Two hard rules from Supermicro's own package readme:
1. **"Using AFU tool will end up with the BIOS corruption. It should never be
used!"** — the AMI AFU flasher is forbidden on H12.
2. **BIOS flash on H12SS-and-newer MUST go through the BMC** — DOS/EFI
standalone BIOS flashing is no longer supported. The BMC web UI uploads the
.bin to the BMC flashdisk and the BMC does the flash.
| Method | License? | Notes |
|---|---|---|
| **BMC web UI → Maintenance → Firmware Management** | No | The supported path for BOTH BMC and BIOS on H12. Upload the .bin; BMC does the flash. ~2.5 min BMC, a few min BIOS. |
| SAA UEFI shell (`flash.nsh <rom> <user> <pw>`) | No | In-band alternative; requires booting host into EFI shell. |
| Redfish SimpleUpdate | YES (DCMS) | Returns `SMC.1.0.OemLicenseNotPassed` — "Not licensed... DCMS needed" |
| Redfish OEM SmcUpdateService.Install | No | Available (Targets + InstallOptions) but web UI is the documented path |
| ipmitool hpm upgrade | No | Only for .hpm images; Supermicro ships .bin |
**Probe the license block before assuming Redfish works:**
`curl -sk -u 'U:P' https://<bmc>/redfish/v1/UpdateService/SimpleUpdateActionInfo`
— if it returns `OemLicenseNotPassed`, SimpleUpdate is out.
**Upgrade order:** BMC first (closes BMC CVEs, one-way), then BIOS (closes
CPU microcode/AGESA CVEs). BIOS flash requires host OFF. Apply BMC and BIOS in
one session — mismatched BIOS/BMC version pairs can hang at POST code FF.
**Recovery safety net:** Supermicro BMCs keep quad-image redundancy —
FirmwareInventory lists `BMC/Backup_BMC/Golden_BMC/Staging_BMC` (same for
BIOS). A failed flash can boot from Backup/Golden via the web UI "Recover"
option or Redfish OEM Install with `InstallOptions=["Recover"]`.
**Post-flash checks:** re-test the cipher suite (newer BMC firmware may switch
from cipher 3 to 17/SHA256), re-verify login, and re-read FirmwareInventory to
confirm the new version. A DHCP BMC may also change IP after reset — re-find by
MAC (fixed) via ARP scan if the IP moves.
## Security advisories to check
- Supermicro Security Center: https://www.supermicro.com/en/support/security_center
- Supermicro BMC advisories (e.g. July 2026 CVE-2026-3821, CVSS 8.8, SMASH
arbitrary code execution — fix is a BMC firmware update).
- AMD bulletins (e.g. AMD-SB-7054 / CVE-2025-54502, CVSS 7.1, affects EPYC 7002
"Rome" — fix is RomePI 1.0.0.P AGESA via BIOS update). Cross-check the board's
CPU generation against the bulletin's affected list.
## SEL (system event log) health
`sel info` shows `Percent Used` and `Overflow`. If 100% full with
`Overflow: true`, new events are being DROPPED. Clearing is a WRITE op —
confirm with the operator before `sel clear`. Supermicro OEM events often
decode as `Unknown #0xff` (raw vendor events, not standard IPMI).
## Security notes
- Legacy IPMI (RMCP+ cipher 3) is weak; prefer Redfish over HTTPS where possible.
- Supermicro publishes BMC security advisories (e.g. July 2026, CVE-2026-3821,
up to 8.8 CVSS). Old BMC firmware predates these — flag for update.
- `lan print` shows `Bad Password Threshold` (default 3) and lockout interval —
repeated bad attempts lock the account. Don't brute-force.
## Pitfalls
- **Cipher 17 fails on Supermicro** — always `-C 3` (see above).
- **Raw OEM commands** (`raw 0x30 0x90 ...`) for BIOS version return
"Request data length invalid" on many Supermicro boards — use Redfish instead.
- **BMC up ≠ host up.** The BMC can report "System Power: on" while the host
OS is unreachable (no route to host / ARP INCOMPLETE). Verify host reachability
separately (SSH port probe) before assuming the box is down.
- **Credentials in fact_store, not here.** Look up the BMC IP/user/pass in
fact_store before connecting; never hardcode passwords in a skill.
- **Supermicro download-center PDFs are bot-blocked (403).** Release-notes PDFs
and the firmware download page return 403 to curl/SearXNG/browser. BUT the
firmware itself is freely mirrored and NOT blocked — download the actual
.bin files from `https://ftp.abacus.cz/support/FW/MB/SUPERMICRO/BIOS/<board>/`
(or `https://sm.t3x.net/`), then verify SHA256. Thomas-Krenn also mirrors
tested-stable bundles at `https://www.thomas-krenn.com/en/download?product=<id>`
(their `/redx/tools/mb_download.php/...` links are curl-able). Only the
Supermicro.com release-notes PDFs stay behind the wall — use Thomas-Krenn wiki
changelogs or search-result snippets for those.
## References
- `references/h12ssl-i-firmware-update.md` — H12SSL-i concrete firmware data,
security drivers, license-block evidence, known issues, changelog sources.
+129
View File
@@ -0,0 +1,129 @@
---
name: mem0-memory
description: "Use when configuring Mem0 as Hermes' memory provider."
version: 1.0.0
author: Hermes Agent
license: MIT
metadata:
hermes:
tags: [mem0, memory, qdrant, ollama, hermes, migration]
related_skills: [hermes-agent, holographic-memory, hermes-config-bulk-update]
---
# Mem0 Memory Provider for Hermes Agent
Mem0 is Hermes' server-side LLM fact-extraction memory provider with semantic
search and automatic deduplication. Plugin lives at
`~/.hermes/hermes-agent/plugins/memory/mem0/` (v1.3.0+). It is the sibling of
`holographic-memory` (local SQLite) — see that skill for the provider being
replaced in a migration.
## When to Use
- Setting up mem0 as the memory provider (any of its 3 modes)
- Migrating from holographic (or another provider) to mem0
- Troubleshooting mem0 OSS mode (Qdrant/embedder/LLM wiring)
- Understanding mem0's tools vs holographic's `fact_store`
## Three Connection Modes
| Mode | Trigger | Needs |
|------|---------|-------|
| **Platform** (cloud) | `MEM0_API_KEY` set | API key from app.mem0.ai |
| **Self-hosted server** | `host` set (Docker dashboard URL) | Mem0 server + optional `X-API-Key` |
| **OSS** (in-process) | `mode: oss` | own LLM + embedder + vector store |
Precedence in the plugin: **OSS > host > platform**. Setting `host` routes to
self-hosted HTTP; `mode: oss` overrides and ignores `host`.
## OSS Mode Config (mem0.json)
Config lives in `$HERMES_HOME/mem0.json` (per-profile). Only the secret
`MEM0_API_KEY` belongs in `.env`. Structure:
```json
{
"mode": "oss",
"oss": {
"llm": {"provider": "ollama", "config": {"model": "qwen3:8b", "ollama_base_url": "http://localhost:11434"}},
"embedder": {"provider": "ollama", "config": {"model": "snowflake-arctic-embed2:latest", "ollama_base_url": "http://10.0.0.30:11434", "embedding_dims": 1024}},
"vector_store": {"provider": "qdrant", "config": {"url": "http://10.0.0.161:6333", "collection_name": "mem0_general"}}
}
}
```
Supported OSS providers (from `_oss_providers.py`):
- LLM: `openai`, `ollama`
- Embedder: `openai`, `ollama`
- Vector store: `qdrant` (local `path` or server `url`), `pgvector`
## CRITICAL PITFALL — embedding_dims not auto-set
The plugin's `KNOWN_DIMS` map only lists `nomic-embed-text` (768) and OpenAI
models (`text-embedding-3-small` 1536, `-large` 3072, `ada-002` 1536). It does
**NOT** include `snowflake-arctic-embed2` (1024 dims).
Consequence: `hermes memory setup mem0 --mode oss` will NOT write
`embedding_dims` for snowflake-arctic-embed2, so mem0 creates the Qdrant
collection with wrong/unknown dims and writes fail.
Fix: set `embedding_dims` manually in `mem0.json` (or run setup then patch the
file). Verify the actual dims with:
```bash
curl -s http://<ollama-host>:11434/api/show -d '{"name":"snowflake-arctic-embed2:latest"}' \
| python3 -c "import sys,json; print(json.load(sys.stdin)['model_info']['bert.embedding_length'])"
```
The plugin's `_recreate_collection_if_dims_changed` will delete a stale
collection when dims change, so a wrong first attempt self-heals on the next
correct config — but only if `embedding_dims` is eventually set.
## Per-Profile Isolation on a Shared Qdrant
Holographic was per-profile (own `memory_store.db`). When all profiles point at
ONE shared Qdrant, the default collection name (`mem0`) would pool every
profile's memory together. Preserve isolation with per-profile
`collection_name` (e.g. `mem0_general`, `mem0_finance`, `mem0_base`).
## Tools (vs holographic)
| mem0 | holographic |
|------|-------------|
| `mem0_search` | `fact_store search` |
| `mem0_add` (verbatim, no extraction) | `fact_store add` |
| `mem0_update` | `fact_store update` |
| `mem0_delete` | `fact_store remove` |
No `fact_feedback` equivalent — mem0 has no trust scoring. `mem0_add` stores
verbatim; LLM extraction happens via `sync_turn`, not `mem0_add`.
## Migration Checklist (holographic → mem0)
1. Write per-profile `mem0.json` (OSS mode) with `embedding_dims` set manually.
2. `hermes config set memory.provider mem0` per profile (base + all profiles).
3. Remove holographic: delete plugin dir + all `memory_store.db` files + the
`plugins.hermes-memory-store` block (auto_extract/hrr_dim) from configs.
4. Update tool references: MEMORY.md rules and `save`/`cognee-brain` skills
reference `fact_store`/`fact_feedback` — they orphan on switch.
5. Smoke test ONE profile first: `mem0_add``mem0_search``mem0_delete`,
then confirm the Qdrant collection exists with correct dims before fanning out.
## Pitfalls
- **Network coupling.** mem0 OSS depends on the Qdrant host and embedder host
being reachable. Holographic was fully local (SQLite). If either service is
down, memory writes fail.
- **Circuit breaker.** "Mem0 temporarily unavailable" = 5 consecutive failures
tripped the breaker; resets after 2 minutes.
- **`mem0_add` is verbatim.** No LLM extraction on that path — use `sync_turn`
for extraction.
- **Cloud LLM fact-extraction quality varies.** Tested (2026-08): `kimi-k2.6:cloud`
and `minimax-m3:cloud` preserved full facts including temporal detail;
`deepseek-v4-pro:cloud` dropped "in March"; `glm-5.2:cloud` dropped both
"red" and "in March". For memory extraction, prefer a model that keeps
temporal context.
- **`gemini-3-flash-preview` retired 2026-07-15** — do not select it.
## References
- `references/oss-config-and-dims.md` — full OSS config schema, KNOWN_DIMS map,
and the embedding-dims gotcha with verification commands.
+120
View File
@@ -0,0 +1,120 @@
---
name: model-capability-benchmark
description: "Use when picking a model for a capability. Benchmark."
version: 1.0.0
author: Hermes Agent
license: MIT
platforms: [linux]
metadata:
hermes:
tags: [benchmark, evaluation, model-selection, ollama, extraction]
related_skills: [agent-routing, cognee-brain, save, save-q-memory]
---
# Model Capability Benchmark
Use when the user needs to pick which model to use for a specific capability (memory
extraction, entity/relation extraction, classification, summarization quality, etc.).
The goal is a data-backed model choice, not a vibe check.
## When to Use
Load this skill when the user needs to pick which model to use for a specific
capability (memory extraction, entity/relation extraction, classification,
summarization quality, etc.). The goal is a data-backed model choice, not a vibe check.
Triggers:
- "which model should I use for X"
- "run a test of the models we have"
- "compare models on <capability>"
- Selecting a model for a pipeline (memory extraction, Cognee-style work, etc.)
## Method
0. **Run the SAME test on every candidate — no variations.** When the user says
"test these models" or "do the same test", send the identical prompt, identical
`temperature`, identical system message to every model. Do NOT add toggles
(thinking on/off, different prompts, extra modes) unless the user explicitly asks
for a comparison of those modes. The user corrected this directly: "WHY DIDN'T you
just do the same test??? DO not turn off thinking. Just DO THE SAME TEST." A
benchmark's value is comparability; a variation you introduce silently breaks it.
1. **Enumerate the candidate models.** For Ollama cloud models, `ollama list` shows
them; `curl -s http://localhost:11434/api/tags` reveals `remote_model`/`remote_host`
(cloud models have a `remote_host`; local models have a real byte size). Filter to
the requested scope (e.g. "cloud only" = has `remote_host`). For a non-Ollama
endpoint (e.g. a llama.cpp server on another box), hit its OpenAI-compatible
`/v1/chat/completions` directly with `urllib` — same system prompt, same
`temperature: 0`, same test cases. The harness just needs a `chat(messages)`
function; the scoring is identical regardless of backend.
2. **Write a test-case set.** Each case = input turns + expected extraction + explicit
fail conditions. Cover the hard cases for the capability, not just happy paths. For
memory extraction the canonical hard cases are: identity/stable facts, preference
vs one-off, update/contradiction, relative time, negation, specificity, multi-entity
relation, pronoun resolution (needs prior turn), plan vs completed fact, abstention
(no facts), assistant contamination, quantity/attribute, soft preference + intensity,
two-people-easy-to-merge.
3. **Build a harness** that sends each case to each model with a fixed system prompt
demanding a clean structured list (JSON array), `temperature=0`, and captures raw
output. Save raw outputs to disk (JSON) — never score in the same pass that runs.
4. **Score kept / missed / invented** per case, then total. A good extractor is high
recall (kept) with near-zero inventions. For Cognee-style work also reject answers
that aren't clean entity/relation lists.
5. **Pick the winner** on recall + zero inventions, and report per-case detail for the
interesting failures (not just totals).
## Pitfalls
- **Normalize BOTH sides before substring scoring.** Expected facts and model output
must go through the same `re.sub(r'[^a-z0-9 ]', ' ', s.lower())` — otherwise
hyphenated facts ("sci-fi", "15-gauge", "fine-tune", "gpt-oss-120b") are falsely
marked missed. This bit the first scoring pass.
- **Retired cloud models return HTTP 410.** A model in `ollama list` can still be
retired upstream; every call returns `ERROR: ... was retired at ... (status code: 410)`.
Detect this and exclude the model rather than scoring it as zero-kept.
- **Verify relative-time anchors yourself.** "last Tuesday" relative to a reference
date must be computed with `date -d <ref> +%A` etc. The user's expected answer may
itself be wrong (e.g. "Aug 19" when Aug 19 is a Wednesday) — compute the correct
date and flag the discrepancy rather than silently scoring against a wrong target.
- **Contamination test needs an assistant-role turn.** To test that the model ignores
assistant-suggested facts, the middle turn must be `role: "assistant"`, not user.
- **Substring scoring can give false credit.** A model that *answers* instead of
*extracting* (e.g. recommends vector DBs) may contain the expected keywords in prose
and score as "kept" when it actually failed the format. Inspect raw output for the
cases that matter before trusting the score.
- **Score from raw output, not from a live re-run.** Models are non-deterministic even
at temperature 0; re-running changes results. Persist raw outputs and score the file.
- **Large cloud models are slow — run sequentially in the background.** A 397b/675b/120b
cloud model can take ~20s per test case, so 14 cases ≈ 5 min per model. A single
foreground run of several models will hit the terminal timeout. Run one model per
background process (or a `for` loop over models in ONE background process), and poll
the error log for `DONE <model>` markers rather than blocking on `wait`. Background
processes that get killed mid-run leave a partial JSON — check which models actually
completed before scoring.
- **Strip fences/whitespace in the abstention check.** A model that correctly returns
an empty list may wrap it as ```json\n[]\n``` or `[ ]`. The abstention check must
strip ``` fences and whitespace and accept `[]`, `[ ]`, `""`, `null` — otherwise a
correct abstention is falsely scored as a failure.
- **Negation FORBID substrings false-positive.** A FORBID fact like "allergic to
shellfish" will match inside a *correct* negation ("not allergic to shellfish") and
be falsely scored as invented. For negation cases, either check for the negated form
explicitly (e.g. forbid "allergic to shellfish" only when NOT preceded by "not"), or
inspect the raw output manually before trusting an "invented" flag on a negation test.
- **Non-Ollama endpoints need their own auth.** A llama.cpp server (e.g. e1's
qwen38-27b at `http://10.0.0.26:8099/v1`) may require a Bearer key while `/health`
and `/v1/models` stay public — only a real completion proves auth. Reuse the same
harness by swapping the transport; don't assume the `ollama` python client works for it.
## Support Files
- `references/memory-extraction-results.md` — 2026-08-26 benchmark of 11 Ollama cloud
models on memory extraction, with per-model findings and the winner.
- `references/recommended-sampling-params.md` — official temperature/top_p guidance
per model (DeepSeek V4 Pro = 1.0/1.0, MiniMax M3 = 1.0/0.95, API use-case table),
plus how temperature is actually set for Hermes custom providers
(`extra_body.temperature`, not a top-level key) and how to research user-experience
threads (HN Algolia API). Load when the user asks about optimal temperature for a model.
+121
View File
@@ -0,0 +1,121 @@
---
name: save
description: "Use when user types 'save'. Write to Brain (Cognee) and/or Vault (Qdrant ai_vault_kb)."
version: 1.3.0
author: Hermes Agent
license: MIT
platforms: [linux]
metadata:
hermes:
tags: [memory, brain, vault, cognee, qdrant, save]
related_skills: [cognee-brain, ai-vault-kb]
---
# Save — write to Brain and/or Vault
Triggered when the user types "save". Route the current session's content to the
right store and write it.
## Routing
- **Brain** (Cognee) = durable facts, decisions, preferences, relationships, status changes, corrections. Short atomic statements with time context.
- **Vault** (Qdrant `ai_vault_kb`) = longer context, notes, research, summaries, excerpts — for semantic "find similar" recall.
- **Both** = when something is a clean fact AND rich context: clean version to Brain, fuller version to Vault.
- Be conservative. Only store what's worth remembering. Don't write every message.
---
## BRAIN — Cognee (Kuzu + LanceDB on brain 10.0.0.23)
The brain is Cognee, reachable via the auto-discovered MCP tools `mcp__cognee__*`
(no CLI needed). Endpoints: API `http://10.0.0.23:8080`, MCP `http://10.0.0.23:8001/mcp`.
### Write (permanent memory)
Use the `mcp__cognee__remember` tool. ALWAYS pass `dataset_name="homelab-stack"`.
- `data` = the text to store (atomic facts, one per statement where possible).
- Omit `session_id` — that makes it permanent memory (add + cognify, builds the graph).
- `remember` returns `status: completed` synchronously — the graph is built, no polling.
### Read
Use the `mcp__cognee__recall` tool. `query` (required), `datasets="homelab-stack"`,
optional `search_type` (HYBRID_COMPLETION default; CHUNKS for LLM-free retrieval).
### Write rules
- **One dataset:** `homelab-stack`. Never create a second dataset.
- **No group_id, no triplet, no temporal supersession.** Cognee uses datasets. To correct a fact, `remember` the corrected statement; Cognee
merges entities by name.
- **Don't re-ingest the same content twice** — `remember` is not idempotent.
- **Embed dim is 1024** (qwen3-embedding:0.6b on mini) — never let it default to 3072.
### Verify the write
- `recall` a distinctive phrase from the content; a correct answer proves the graph has it.
- `docker logs cognee-backend` should show zero `api.openai.com` / `ProviderConfigMismatch` hits.
---
## VAULT — Qdrant `ai_vault_kb`
CLI: `python3 /home/n8n/bin/ai_vault_kb.py` (zero deps, urllib only). Qdrant `http://10.0.0.22:6333`, collection `ai_vault_kb`. Embeddings: `snowflake-arctic-embed2` on `10.0.0.30:11434`.
### Write (the ONLY sanctioned path)
```bash
python3 /home/n8n/bin/ai_vault_kb.py add --type <t> --title "..." --content "..." [flags]
```
Long documents: `--file /abs/path/report.md` instead of `--content` (auto-chunked, one shared `doc_id`). `--json` prints `{"doc_id":…, "chunks":N}`.
| Flag | Meaning |
|---|---|
| `--type` **(required)** | `tool` `setting` `workflow` `host` `model` `technique` `issue` `decision` `research` `asset` `prompt` `finding` |
| `--title` **(required)** | headline a future search reads |
| `--content` / `--file` | body text, or a file to chunk |
| `--stage` | `story` `script` `character` `keyframe` `t2v` `i2v` `upscale` `interpolate` `tts` `lipsync` `music` `assembly` `publish` `infra` |
| `--tool` `--host` `--path` `--url` `--version` | provenance; `--host` is the box the fact is about |
| `--status` | `active` `candidate` `deprecated` `broken` `planned` (default `active`) |
| `--trust` | `official` `github` `community` `social` (default `official`) |
| `--tags` | comma-separated; exact-match keyword index — put slug, filename, error code here |
| `--importance` | 0.01.0 |
| `--doc-id` | append more chunks to an existing document |
Unknown vocabulary values warn but are accepted — the schema is faceted, not strict.
### Dedup-first (mandatory before every write)
```bash
python3 /home/n8n/bin/ai_vault_kb.py search --query "<the thing you are about to save>"
```
- score **≥ 0.85** — already recorded, skip
- **0.700.84** — add only if meaningfully new
- **< 0.70** — always add
### Host records
`--type host`, one stable `--doc-id` per box. Re-add with the same `--doc-id` to update/append a dated chunk.
### Verify the write landed (both legs)
```bash
D=<doc_id from --json>
python3 /home/n8n/bin/ai_vault_kb.py search --query "<distinctive phrase>" --mode bm25 --doc-id $D
python3 /home/n8n/bin/ai_vault_kb.py list --type <the type you used> --doc-id $D
```
Zero hits on the BM25 leg means something other than `ai_vault_kb.py` wrote it.
---
## Pitfalls
- **Brain writes are not idempotent** — don't re-ingest the same content twice.
- **Vault writes MUST go through `ai_vault_kb.py`** — never a bare Qdrant upsert or `mcp__better_qdrant__add_documents` (missing bm25/doc_type makes the point unfindable). Never hand-roll raw Qdrant HTTP for a write.
- **Collection name is exact** — `ai_vault_kb`, not `ai-vault-kb`. Don't create topic-specific collections.
- **On correction:** update Brain (remember the corrected fact) and store the correction.
- **`fact_store` and the `memories` collection are not the vault** — research findings, model configs, prompt guides, infra facts go here.
- **A search that returns nothing is a real answer** — say "not in the vault" and go research it.
+245 -17
View File
@@ -1,19 +1,44 @@
---
name: youtube-knowledge-ingestion
description: "Use when user drops a YouTube link. Transcript→brain→NAS."
version: 1.0.0
description: "AUTO-TRIGGER: If the operator's message consists solely of a YouTube URL (youtube.com/watch, youtu.be, youtube.com/shorts — no other words), ALWAYS run the full ingestion flow immediately, without asking any clarifying questions: 1. Verbatim transcript → ai_vault_kb (the vault) per the \"Transcript Ingestion (verbatim)\" section (dedup-first, separate doc if a recipe/summary doc for the video already exists). 2. Distilled summary → Cognee brain (the brain) per the \"Brain Injection (Cognee)\" section. 3. If the video's description or pinned comment links a workflow or downloadable assets, download them to the NAS stock library per the existing procedure. 4. Reply with a minimal 4-line completion status (Brain ingest / Vault ingest / Workflow dl / Workflow(s)); write the full proof report (doc_id, chunk count, excerpt, BM25 outputs, file paths + sizes) to ~/workspace/general/logs/yt_ingestion_proofs.log instead of sending it to the operator. Use when user drops a YouTube link. Transcript→vault+brain→NAS."
version: 1.1.0
platforms: [linux]
---
# YouTube Knowledge Ingestion Pipeline
## AUTO-TRIGGER (read first — highest priority)
AUTO-TRIGGER: If the operator's message consists solely of a YouTube URL
(youtube.com/watch, youtu.be, youtube.com/shorts — no other words), ALWAYS run the full
ingestion flow immediately, without asking any clarifying questions:
1. Verbatim transcript → ai_vault_kb (the vault) per the "Transcript Ingestion (verbatim)"
section (dedup-first, separate doc if a recipe/summary doc for the video already exists).
2. Distilled summary → Cognee brain (the brain) per the "Brain Injection (Cognee)"
section.
3. If the video's description or pinned comment links a workflow or downloadable assets,
download them to the NAS stock library per the existing procedure.
4. Reply with a minimal 4-line completion status (see "Completion output" below); write
the full proof report to ~/workspace/general/logs/yt_ingestion_proofs.log instead of
sending it to the operator.
**The failure mode this trigger exists to prevent (recurring user correction):**
loading the `youtube-content` skill and producing a chat summary. A bare URL is NOT a
request for a summary — it is a request to run THIS pipeline. Do NOT load
`youtube-content` for a bare URL; load THIS skill (`youtube-knowledge-ingestion`).
Do NOT summarize the video in chat under any circumstances. The user has corrected
this multiple times ("WHY DO YOU KEEP DOING THIS — YOU HAVE A YT INGEST SKILL").
The two skills are easy to confuse: `youtube-content` = fetch transcript + format
(summary/thread/blog); `youtube-knowledge-ingestion` = the full transcript→vault +
summary→brain + workflow→NAS pipeline. A bare URL always means the latter.
## When to use
User drops a YouTube link for AI/ML video content (ComfyUI workflows, LTX, MiniMax,
Krea, model tutorials, prompting guides). Run the full three-step pipeline — never skip
Krea, model tutorials, prompting guides). Run the full four-step pipeline — never skip
any step.
## Pipeline (always all three steps)
## Pipeline (always all four steps)
### 1. Transcript
@@ -25,16 +50,16 @@ uv run python3 <SKILL_DIR>/scripts/fetch_transcript.py "URL" --text-only --times
If `youtube-transcript-api` is missing, install with `pip3 install --user youtube-transcript-api`.
### 2. Brain Injection (MANDATORY)
### 2. Vault Injection (ai_vault_kb) (MANDATORY)
Load the `ai-brain-kb` skill. Always dedup-first, then ingest:
Load the `ai-vault-kb` skill. Always dedup-first, then ingest:
```bash
# Dedup check
python3 /home/n8n/bin/ai_brain_kb.py search --query "<key topic>" --limit 5
python3 /home/n8n/bin/ai_vault_kb.py search --query "<key topic>" --limit 5
# Ingest — use --type workflow for tutorials
python3 /home/n8n/bin/ai_brain_kb.py add \
python3 /home/n8n/bin/ai_vault_kb.py add \
--type workflow \
--title "Descriptive Title — Key Topics" \
--stage <t2v|i2v|upscale|...> \
@@ -48,7 +73,7 @@ python3 /home/n8n/bin/ai_brain_kb.py add \
--json
# Verify BM25 leg
python3 /home/n8n/bin/ai_brain_kb.py search --query "<distinctive phrase>" \
python3 /home/n8n/bin/ai_vault_kb.py search --query "<distinctive phrase>" \
--mode bm25 --doc-id <doc_id>
```
@@ -56,7 +81,7 @@ python3 /home/n8n/bin/ai_brain_kb.py search --query "<distinctive phrase>" \
Include specific settings, model names, thresholds, commands, and failure modes.
This is a technical reference, not a blog post.
**MANDATORY: workflow links in the brain content.** Every brain entry must include a
**MANDATORY: workflow links in the vault content.** Every vault entry must include a
`## WORKFLOW LINKS` section listing the DOWNLOADED workflow files on NAS (TrueNAS
10.0.0.117, proxmoxBackup/ai_vid_stock_material/workflows/, exact filenames + the
smbclient get command), the source repo URL, and the video URL. The NAS copies are
@@ -68,7 +93,54 @@ already ingested.
**Tag strategy**: model name, tool name, creator name, key techniques. Tags are
exact-match keyword indexes.
### 3. NAS Workflow Download
### 3. Brain Injection (Cognee) (MANDATORY)
Load the `cognee-brain` skill. Write the distilled summary to the shared Cognee
brain (10.0.0.23) as a narrative finding — LLM extraction pulls out the entities
(models, tools, creators) and facts (claims, settings, comparisons) automatically:
Call the Cognee MCP tool `mcp__cognee__remember` with:
- `data` = the distilled summary (2-4 sentences)
- `dataset_name` = `homelab-stack`
- omit `session_id` (permanent memory — runs add + cognify, builds the graph)
- `remember` is synchronous (`status: completed` means the graph is built). No
polling needed; verify later with `recall` if needed.
- Always use dataset `homelab-stack` (the single shared brain dataset). Never
create a second dataset.
- The brain holds the distilled facts; the vault (ai_vault_kb) holds the verbatim
transcript. Both are written for every video — never skip either.
**MCP timeout + REST fallback (learned 2026-08-29).** The MCP `remember` call can
fail with `TimeoutError: MCP call timed out after 120.0s` because cognify is
variable — the REST path took 77s for a 6-item ingest, while MCP full-size payloads
ran 9.812.2s. The MCP server itself is healthy; it is the client-side 120s
per-tool-call timeout that aborts slow cognify runs. Two-part fix:
1. **Raise the timeout** in `~/.hermes/profiles/general/config.yaml` (cognee MCP
block): `timeout: 120``timeout: 300`. Takes effect on next Hermes restart
(MCP connections are read once at startup). Note: the `patch` tool refuses to
edit Hermes config files — use terminal Python for an exact string replace, then
`python3 -c "import yaml; yaml.safe_load(open('config.yaml'))"` to validate.
2. **REST fallback** when MCP times out (documented in the `cognee-brain` skill):
```bash
# Write (synchronous; returns status:completed + items_processed)
curl -s -m 300 -X POST "http://10.0.0.23:8080/api/v1/remember" \
-F "data=@/tmp/brain_summary.txt" \
-F "datasetName=homelab-stack"
# Verify (field names are query + searchType, NOT query_text/query_type)
curl -s -m 60 -X POST "http://10.0.0.23:8080/api/v1/search" \
-H "Content-Type: application/json" \
-d '{"query":"<distinctive phrase>","datasets":["homelab-stack"],"searchType":"HYBRID_COMPLETION"}'
```
A `status: completed` from the REST write is proof the graph is built — no separate
cognify step. If the MCP call times out, do NOT retry it blindly; use the REST path
and verify with the search call above.
### 4. NAS Workflow Download
If the video description contains workflow links (GitHub repos, direct JSON files),
extract the description with `yt-dlp --print description "URL"`, clone the repos,
@@ -92,15 +164,171 @@ Examples: `amao2001-ltx2.5-video_ltx2_5_t2v1.json`, `vionex-krea2-film-studio-v0
**Target**: `proxmoxBackup/ai_vid_stock_material/workflows/` on TrueNAS (10.0.0.117).
**ComfyUI custom-node repos (no workflow JSONs) → `scripts/`, not `workflows/`.**
When the description links a ComfyUI custom-node pack (e.g. PlagueKind's
`ComfyUI-PlagueKind-Nodes` — the SLA attention node), the repo is Python source with
no `example_workflows/*.json` to copy. Archive it as a tarball under
`ai_vid_stock_material/scripts/` (same pattern as full apps — see
`references/fetch-path-and-walled-links.md`):
```bash
cd /tmp && rm -rf <repo> && git clone --depth 1 <repo-url>
COMMIT=$(cd <repo> && git log -1 --format=%h)
tar czf <creator>-<repo>-<commit>.tar.gz <repo>
smbclient -N //10.0.0.117/proxmoxBackup \
-c 'cd ai_vid_stock_material/scripts; put /tmp/<tarball> <tarball>'
```
`workflows/` stays reserved for ComfyUI JSONs; `scripts/` is the home for node packs
and tools. Record the commit hash in the WORKFLOW LINKS section.
## Transcript Ingestion (verbatim) — when the task says "transcript"
When the task asks for the TRANSCRIPT (not a summary/recipe), the vault doc must contain
the verbatim spoken text. A distilled recipe summary is NOT a transcript.
**Omit useless segments.** "Verbatim" means the informative spoken content word-for-word —
not the filler around it. Cut entirely: sponsor reads / ad segments, song lyrics and
music-only passages, giveaway/merch/Patreon plugs, like-and-subscribe boilerplate, and
unrelated channel promo. Replace each cut with a one-line marker at that spot — [sponsor
segment omitted], [music omitted], [channel promo omitted] — so the cut is visible and
auditable. When in doubt, keep it: anything touching the technical content (settings,
models, node names, reasoning, results) is never filler, even if it sounds chatty. The
word count in the summary header notes omissions, e.g. "2,140 words (3 segments omitted: 2
sponsor, 1 music)."
1. **Fetch the caption track**: `yt-dlp --skip-download --write-auto-subs --write-subs
--sub-langs "en.*" <url>` (fall back to `--sub-langs en` if needed). Clean the VTT/SRT
to plain text: strip WEBVTT headers, cue timestamps, and inline `<c>`/timestamp tags;
dedupe rolling-caption repeated lines. Result must read as continuous spoken sentences.
2. **Body layout**: short distilled summary header (1015 lines max) at top, then the FULL
verbatim transcript. Title pattern: `YT: <video title> (<video-id>) — transcript`.
Use `--type research` (house pattern for video transcripts), `--trust community`,
`--url <video-url>`, tags including the video id.
3. **Helper only**: `python3 /home/n8n/bin/ai_vault_kb.py add ...` — never raw Qdrant HTTP
(bare-vector upserts miss the BM25 sparse slot and are invisible to hybrid search).
4. **Dedup first**: `search --query "<video-id>"`; ≥0.85 skip · 0.700.84 add only if
meaningfully new · <0.70 add. If a summary/recipe doc for the same video already exists,
add the transcript as a SEPARATE doc and mention the other doc_id in the body — do not
overwrite the recipe doc.
5. **Chunk boundaries + ordering**: the helper chunks at 1200 chars with 150 overlap, and
overlap starts land mid-word. To guarantee no chunk starts/ends mid-word, pre-split the
body at sentence boundaries into ≤1200-char pieces and add them sequentially with
`--doc-id` (each piece ≤1200 chars becomes exactly one chunk). Pass `--chunk-index N`
(0-based) on every add-call so reading order is recoverable from metadata: the summary
header piece is `--chunk-index 0`, the next piece `--chunk-index 1`, and so on through
the transcript body; the WORKFLOW LINKS piece is the last (highest) index. Prefer the
bundled `scripts/ingest_transcript.py` — it does the pre-split, the sequential indexed
adds, and the self-QA gate in one run. See `references/transcript-ingestion-detail.md`
for the chunking algorithm, the `--chunk-index` backfill context, and the proof-log
entry format.
6. **Self-QA gate before reporting done (mandatory)**:
- get the doc back; confirm the body contains verbatim spoken sentences (first-person
narration, not bullets);
- confirm no chunk begins or ends mid-word (print first/last 10 chars of every chunk);
- run a search on one distinctive literal phrase copied from the body and confirm the
doc surfaces (proves the BM25 slot).
- **chunk-index ordering**: run `list --doc-id <id> --json` and confirm the
`chunk_index` values are exactly 0 through N1, each appearing once. Then read the
pieces back in index order and confirm they reproduce the narrative sequence
(summary header → transcript body → WORKFLOW LINKS). A doc with duplicate or missing
indices fails the gate — fix it by rebuilding the doc: delete --doc-id <id> --yes,
then re-add ALL pieces in order with correct --chunk-index values (add --doc-id
appends — re-adding a single piece would duplicate it), then re-run this gate.
7. **Completion report → log file, not chat.** Do NOT send the proof report to the
operator. Instead append it to `~/workspace/general/logs/yt_ingestion_proofs.log`
(create the dir if needed) under a dated header per run. The log entry contains the
same content as before: doc_id, chunk count, a 23 line verbatim excerpt, the BM25
search outputs, the chunk-index ordering check output (the `list --doc-id <id> --json`
result showing chunk_index 0..N1 each once), and every downloaded file's full NAS
path + size. Then reply to the operator with only the minimal 4-line status per the
"Completion output" section below.
## Completion output (minimal — replaces the chat proof report)
After a run, reply to the operator with EXACTLY these four lines and nothing more:
```
Brain ingest: done
Vault ingest: done
Workflow dl: done
Workflow(s): <filename(s) as saved on the NAS>
```
Rules:
1. **"done" is earned, not assumed.** A line may say "done" only after its self-QA gate
actually passed — the internal checks (verbatim verify, chunk edges, BM25 probe, NAS
size verify) still run in full; only the reporting shrinks. If a step failed or was
skipped, say so on that line in a few plain words instead of "done":
- Brain ingest: failed — <one-line reason>
- Vault ingest: failed — <one-line reason>
- Workflow dl: none linked
- Workflow(s): <none, or the filenames if some succeeded>
2. **Full proof report goes to the log file**, not chat:
`~/workspace/general/logs/yt_ingestion_proofs.log` (create the dir if needed). Append
one entry per run with a dated header (e.g. `## 2026-08-17 BjZvVx6sDmE — TaoofAI EP.1`)
containing: doc_id, chunk count, verbatim excerpt, BM25 search outputs, and every
downloaded file's full NAS path + size. This log path is the single place any
validator looks for proof.
3. **Keep the doc_id available.** Include it in the log entry. If the operator or a
validator asks "doc id?" after a run, answer with just the doc_id — nothing else.
## Hard rules for vault docs created from videos
- **Every doc created from a video MUST carry the video-ID tag** (e.g. `ZUzeM9OEJ4Y`)
in `--tags`, on every chunk-add call. The video ID also goes in `--url` (source_url).
- **Additionally ensure the video ID appears in each doc's TITLE** (append `(VIDEOID)` if
absent) — tags don't feed BM25, so the tag alone doesn't make the doc rank; title
placement does. The transcript title pattern already complies; older recipe/workflow
docs won't.
- **Trust enum**: `official` | `github` | `community` | `social` — video-derived docs use
`--trust community` unless the creator is the model vendor.
- **Keyword-search caveat**: the helper builds the BM25 sparse vector from title+text
ONLY — tags are payload-only and do not feed keyword search. A doc whose title and
body lack the video ID will rank low in `search --query "<video-id>"` even with the
tag present. For transcript docs the title pattern `YT: <title> (<video-id>) —
transcript` already embeds the ID; for recipe/summary docs, include the video ID in
the body (e.g. the WORKFLOW LINKS section) so at least one chunk matches.
## Hard rules for brain writes created from videos
- **Every video writes to the brain via `mcp__cognee__remember`** (narrative finding)
with `dataset_name="homelab-stack"`. The video URL goes in the data text so the
brain fact is traceable back to the video.
- **The brain write is the distilled summary, not the verbatim transcript.** Keep it to
2-4 sentences of the durable takeaways (what the video demonstrates, key settings,
verdict). The verbatim text lives only in the vault.
- **`remember` is synchronous** — `status: completed` means the graph is built. If a
validator needs proof, `mcp__cognee__recall` a distinctive phrase.
- **`status: completed` is NOT proof the content is retrievable.** A write can return
`completed` yet a later recall answers "does not contain information about <terms>".
The recall is the real gate: after every brain write, `recall` a distinctive phrase
from the summary and confirm the answer actually contains it. If it does not,
re-write the summary and re-verify — do NOT report "Brain ingest: done" on
`status: completed` alone. See `references/brain-write-verification.md`.
## Pitfalls
- **Never skip the brain injection step.** The user's memory directive explicitly
requires both transcript AND brain injection for every YouTube link.
- **Never load `youtube-content` for a bare URL.** The bare-URL trigger means run the
full pipeline via THIS skill. Loading `youtube-content` and summarizing in chat is the
single most common failure — the user has corrected it repeatedly. If you catch
yourself about to produce a chat summary of a video, stop and run the pipeline.
- **MCP `recall` uses `search_type` (snake_case), REST uses `searchType` (camelCase).**
The `mcp__cognee__recall` tool rejects `searchType` with a pydantic validation error
("Unexpected keyword argument searchType") — pass `search_type="HYBRID_COMPLETION"`.
The REST `/api/v1/search` endpoint (curl) uses `searchType` in the JSON body. Don't
mix them up.
- **Never skip the vault injection step.** The user's memory directive explicitly
requires both transcript AND vault injection for every YouTube link.
- **Never skip the brain injection step.** Every video also writes a distilled finding
to the Cognee brain (`mcp__cognee__remember`, dataset `homelab-stack`). Vault =
verbatim transcript; brain = distilled facts. Both are mandatory.
- **Never skip the BM25 verification.** A write that can't be found via BM25 is
effectively invisible — the `ai_brain_kb.py` helper is the only interface that
effectively invisible — the `ai_vault_kb.py` helper is the only interface that
produces searchable points.
- **Don't overwrite existing NAS workflows.** Use creator prefixes to avoid collisions
with workflows already in the directory.
- **The `youtube-content` and `ai-brain-kb` skills are protected** — load them for
reference but don't attempt to patch them. This skill is the integration layer
between them.
- **The `youtube-content`, `ai-vault-kb`, and `cognee-brain` skills are protected** —
load them for reference but don't attempt to patch them. This skill is the integration
layer between them.