178 lines
7.8 KiB
Markdown
178 lines
7.8 KiB
Markdown
---
|
||
name: ai-brain-kb
|
||
description: "Read and write the ai_brain_kb Qdrant collection — the fleet's one brain. Every write goes through ai_brain_kb.py; the better_qdrant MCP write tool is FORBIDDEN because it produces unsearchable points."
|
||
version: 2.0.0
|
||
author: Hermes Agent
|
||
license: MIT
|
||
platforms: [linux]
|
||
metadata:
|
||
hermes:
|
||
tags: [qdrant, knowledge-base, ai-brain, search, rag, bm25]
|
||
related_skills: [research-knowledge-management, deep-web-research, qdrant-collection-management]
|
||
---
|
||
|
||
# AI Brain KB — the fleet's one brain
|
||
|
||
`ai_brain_kb` is the single Qdrant collection holding all AI/ML knowledge: research,
|
||
pipeline state, model configs, prompts, decisions, bugs, hardware facts. One brain,
|
||
one collection — never a topic-specific collection.
|
||
|
||
## THE ONE RULE — writes go through `ai_brain_kb.py`, nothing else
|
||
|
||
```bash
|
||
python3 /home/n8n/bin/ai_brain_kb.py add --type <t> --title "..." --content "..." [flags]
|
||
```
|
||
|
||
That helper is **the only interface that produces a usable, searchable record.**
|
||
|
||
### Why `mcp__better_qdrant__add_documents` is FORBIDDEN for this collection
|
||
|
||
It performs a **bare-vector upsert**. For every chunk it writes it produces:
|
||
|
||
| | `ai_brain_kb.py add` | `mcp__better_qdrant__add_documents` |
|
||
|---|---|---|
|
||
| dense vector (unnamed slot) | yes | yes |
|
||
| **`bm25` sparse vector** | **yes** | **NO** |
|
||
| `doc_type` / `title` / `tags` | yes | **NO** |
|
||
| `trust` / `host` / `path` / `doc_id` / `pipeline_stage` | yes | **NO** |
|
||
|
||
A point with no `bm25` slot **cannot be returned by a keyword/BM25 search** — paste an
|
||
exact error string, model filename or node name and it will never match. A point with
|
||
no `doc_type` **cannot be returned by any typed search** (`--type research`,
|
||
`--type issue`, …) or by any faceted list. It survives only in the dense half of a
|
||
hybrid query and renders as `[?] (untitled)`.
|
||
|
||
**It looks ingested and it cannot be found.** This is not a style preference — roughly
|
||
4,700 points in this collection were created that way and had to be repaired. Do not
|
||
create more. There is no file size, no hurry and no MCP convenience that justifies it.
|
||
|
||
If you catch yourself reaching for `add_documents` because the helper is awkward for a
|
||
big file: the helper takes `--file` and chunks it itself, with no MCP 120 s timeout.
|
||
|
||
## Writing
|
||
|
||
```bash
|
||
python3 /home/n8n/bin/ai_brain_kb.py add \
|
||
--type finding --title "LTX 2.3 audio desync above 121 frames" \
|
||
--content "..." \
|
||
--stage t2v --tool ltx-video --host 10.0.0.202 \
|
||
--trust official --importance 0.7 --tags "ltx-2.3,audio,desync"
|
||
```
|
||
|
||
Long documents: `--file /abs/path/report.md` instead of `--content` (auto-chunked,
|
||
one shared `doc_id` across the chunks). `--json` prints `{"doc_id":…, "chunks":N}`.
|
||
|
||
| Flag | Meaning |
|
||
|---|---|
|
||
| `--type` **(required)** | `tool` `setting` `workflow` `host` `model` `technique` `issue` `decision` `research` `asset` `prompt` `finding` |
|
||
| `--title` **(required)** | what a future search will read as the headline |
|
||
| `--content` / `--file` | body text, or a file to chunk |
|
||
| `--stage` | `story` `script` `character` `keyframe` `t2v` `i2v` `upscale` `interpolate` `tts` `lipsync` `music` `assembly` `publish` `infra` |
|
||
| `--tool` `--host` `--path` `--url` `--version` | provenance; `--host` is the box the fact is about |
|
||
| `--status` | `active` `candidate` `deprecated` `broken` `planned` (default `active`) |
|
||
| `--trust` | `official` `github` `community` `social` (default `official`) |
|
||
| `--tags` | comma-separated; **tags are an exact-match keyword index** — put the slug, the filename, the error code here |
|
||
| `--importance` | 0.0–1.0 |
|
||
| `--doc-id` | append more chunks to an existing document |
|
||
|
||
Unknown vocabulary values warn but are accepted — the schema is faceted, not strict.
|
||
A warning is not a failure; do **not** switch to the MCP tool because of one.
|
||
|
||
### Mandatory before every write: dedup-first
|
||
|
||
```bash
|
||
python3 /home/n8n/bin/ai_brain_kb.py search --query "<the thing you are about to save>"
|
||
```
|
||
|
||
- score **≥ 0.85** — already recorded, skip
|
||
- **0.70–0.84** — add only if meaningfully new
|
||
- **< 0.70** — always add
|
||
|
||
### Host records
|
||
|
||
`--type host`, one stable `--doc-id` per box. Re-add with the same `--doc-id` to update
|
||
or append a dated chunk, rather than creating a second record for the same machine.
|
||
|
||
## Reading
|
||
|
||
```bash
|
||
# hybrid (dense + BM25, RRF fusion) — the default, use it
|
||
python3 /home/n8n/bin/ai_brain_kb.py search --query "ltx 2.3 native audio"
|
||
|
||
# pure keyword — exact filenames, error strings, node names
|
||
python3 /home/n8n/bin/ai_brain_kb.py search --query "ltxv-097-dev-fp8.safetensors" --mode bm25
|
||
|
||
# typed / faceted
|
||
python3 /home/n8n/bin/ai_brain_kb.py list --type issue --tool comfyui --limit 20
|
||
python3 /home/n8n/bin/ai_brain_kb.py list --tag 2026-08-09-horizon-scan
|
||
python3 /home/n8n/bin/ai_brain_kb.py facet --field tool
|
||
python3 /home/n8n/bin/ai_brain_kb.py stats # points, doc_type inventory, health
|
||
python3 /home/n8n/bin/ai_brain_kb.py stale --days 30
|
||
python3 /home/n8n/bin/ai_brain_kb.py get --id <point-id>
|
||
```
|
||
|
||
`mcp__better_qdrant__search` is **read-only and therefore allowed**, but it is
|
||
dense-only — it silently misses anything a keyword query would have found. Prefer the
|
||
helper's `search`. Use the MCP one only when you have no shell.
|
||
|
||
## Deleting
|
||
|
||
```bash
|
||
python3 /home/n8n/bin/ai_brain_kb.py delete --doc-id <uuid> --yes # a whole document
|
||
python3 /home/n8n/bin/ai_brain_kb.py delete --id <point-id> --yes # one chunk
|
||
```
|
||
|
||
Per-document delete **exists**. Never
|
||
`mcp__better_qdrant__delete_collection(collection="ai_brain_kb")` — that destroys the
|
||
fleet's entire brain and there is no undo. Earlier versions of this skill described the
|
||
nuke as the only option; that was wrong.
|
||
|
||
## Verify the write landed
|
||
|
||
A write is not done until it is retrievable **both ways**. The BM25 leg is the one a
|
||
bare upsert cannot pass, so it is the real test:
|
||
|
||
```bash
|
||
D=<doc_id from --json>
|
||
python3 /home/n8n/bin/ai_brain_kb.py search --query "<distinctive phrase>" --mode bm25 --doc-id $D
|
||
python3 /home/n8n/bin/ai_brain_kb.py list --type <the type you used> --doc-id $D
|
||
```
|
||
|
||
Zero hits on the BM25 leg means something other than `ai_brain_kb.py` wrote it.
|
||
|
||
## Deep-research reports
|
||
|
||
Do not ingest them by hand. `publish_report.py` (in the `deep-web-research` skill) is
|
||
the only sanctioned path — it publishes, ingests via this helper, verifies hybrid+BM25,
|
||
and writes the `.meta.json` receipt in one action.
|
||
|
||
A `.meta.json` sidecar is a **receipt written after verification, not proof**. Sidecars
|
||
written before that rule exists claim `doc_id`s that were never upserted. If you need
|
||
to know whether a report is in the brain, ask the brain (`list --tag <slug>`), never the
|
||
directory listing.
|
||
|
||
## Infrastructure
|
||
|
||
| Setting | Value |
|
||
|---|---|
|
||
| Qdrant | `http://10.0.0.22:6333`, collection `ai_brain_kb` |
|
||
| Embeddings | `snowflake-arctic-embed2` on **mini, `10.0.0.30:11434`** — the fleet's only embedder |
|
||
| Helper | `/home/n8n/bin/ai_brain_kb.py` (zero deps, urllib only) |
|
||
| Vectors | unnamed dense 1024-dim + named sparse `bm25` |
|
||
|
||
**The CARE rule for mini:** every ingest embeds there and nothing else in the fleet can.
|
||
Batch your writes, keep them bounded, never hammer it.
|
||
|
||
## Pitfalls
|
||
|
||
- **Never hand-roll raw Qdrant HTTP for a write.** A `PUT /collections/ai_brain_kb/points`
|
||
with a bare vector reproduces the exact defect the MCP tool causes. Reads
|
||
(`GET /collections/...`, `POST .../points/scroll`, `.../points/count`) are fine.
|
||
- **Absolute paths** for `--file` / `--path`.
|
||
- **Collection name is exact** — `ai_brain_kb`, not `ai-brain-kb`.
|
||
- **Don't create topic-specific collections.** No `ltx-research`, no `comfyui-workflows`.
|
||
- **`fact_store` and the `memories` collection are not the brain.** Research findings,
|
||
model configs, prompt guides and infra facts all go here.
|
||
- **A search that returns nothing is a real answer.** Say "not in the brain" and go
|
||
research it — do not assume the record exists but is hiding.
|