tools-update-cron: sync 2026-08-16 — 16 skill(s) updated
This commit is contained in:
+160
-109
@@ -1,126 +1,177 @@
|
||||
---
|
||||
name: ai-brain-kb
|
||||
description: "Manage the ai_brain_kb Qdrant collection — add documents, search, and remove. Central knowledge base for all AI/ML learnings, pipeline details, and research."
|
||||
version: 1.0.0
|
||||
description: "Read and write the ai_brain_kb Qdrant collection — the fleet's one brain. Every write goes through ai_brain_kb.py; the better_qdrant MCP write tool is FORBIDDEN because it produces unsearchable points."
|
||||
version: 2.0.0
|
||||
author: Hermes Agent
|
||||
license: MIT
|
||||
platforms: [linux]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [qdrant, knowledge-base, ai-brain, search, rag]
|
||||
related_skills: [save-q-memory, qdrant-collection-management]
|
||||
tags: [qdrant, knowledge-base, ai-brain, search, rag, bm25]
|
||||
related_skills: [research-knowledge-management, deep-web-research, qdrant-collection-management]
|
||||
---
|
||||
|
||||
# AI Brain KB — Qdrant Knowledge Base Manager
|
||||
# AI Brain KB — the fleet's one brain
|
||||
|
||||
Manage the `ai_brain_kb` Qdrant collection — the central knowledge base for all AI/ML learnings, pipeline details, research, and decisions.
|
||||
`ai_brain_kb` is the single Qdrant collection holding all AI/ML knowledge: research,
|
||||
pipeline state, model configs, prompts, decisions, bugs, hardware facts. One brain,
|
||||
one collection — never a topic-specific collection.
|
||||
|
||||
## Storage Location
|
||||
## THE ONE RULE — writes go through `ai_brain_kb.py`, nothing else
|
||||
|
||||
```bash
|
||||
python3 /home/n8n/bin/ai_brain_kb.py add --type <t> --title "..." --content "..." [flags]
|
||||
```
|
||||
|
||||
That helper is **the only interface that produces a usable, searchable record.**
|
||||
|
||||
### Why `mcp__better_qdrant__add_documents` is FORBIDDEN for this collection
|
||||
|
||||
It performs a **bare-vector upsert**. For every chunk it writes it produces:
|
||||
|
||||
| | `ai_brain_kb.py add` | `mcp__better_qdrant__add_documents` |
|
||||
|---|---|---|
|
||||
| dense vector (unnamed slot) | yes | yes |
|
||||
| **`bm25` sparse vector** | **yes** | **NO** |
|
||||
| `doc_type` / `title` / `tags` | yes | **NO** |
|
||||
| `trust` / `host` / `path` / `doc_id` / `pipeline_stage` | yes | **NO** |
|
||||
|
||||
A point with no `bm25` slot **cannot be returned by a keyword/BM25 search** — paste an
|
||||
exact error string, model filename or node name and it will never match. A point with
|
||||
no `doc_type` **cannot be returned by any typed search** (`--type research`,
|
||||
`--type issue`, …) or by any faceted list. It survives only in the dense half of a
|
||||
hybrid query and renders as `[?] (untitled)`.
|
||||
|
||||
**It looks ingested and it cannot be found.** This is not a style preference — roughly
|
||||
4,700 points in this collection were created that way and had to be repaired. Do not
|
||||
create more. There is no file size, no hurry and no MCP convenience that justifies it.
|
||||
|
||||
If you catch yourself reaching for `add_documents` because the helper is awkward for a
|
||||
big file: the helper takes `--file` and chunks it itself, with no MCP 120 s timeout.
|
||||
|
||||
## Writing
|
||||
|
||||
```bash
|
||||
python3 /home/n8n/bin/ai_brain_kb.py add \
|
||||
--type finding --title "LTX 2.3 audio desync above 121 frames" \
|
||||
--content "..." \
|
||||
--stage t2v --tool ltx-video --host 10.0.0.202 \
|
||||
--trust official --importance 0.7 --tags "ltx-2.3,audio,desync"
|
||||
```
|
||||
|
||||
Long documents: `--file /abs/path/report.md` instead of `--content` (auto-chunked,
|
||||
one shared `doc_id` across the chunks). `--json` prints `{"doc_id":…, "chunks":N}`.
|
||||
|
||||
| Flag | Meaning |
|
||||
|---|---|
|
||||
| `--type` **(required)** | `tool` `setting` `workflow` `host` `model` `technique` `issue` `decision` `research` `asset` `prompt` `finding` |
|
||||
| `--title` **(required)** | what a future search will read as the headline |
|
||||
| `--content` / `--file` | body text, or a file to chunk |
|
||||
| `--stage` | `story` `script` `character` `keyframe` `t2v` `i2v` `upscale` `interpolate` `tts` `lipsync` `music` `assembly` `publish` `infra` |
|
||||
| `--tool` `--host` `--path` `--url` `--version` | provenance; `--host` is the box the fact is about |
|
||||
| `--status` | `active` `candidate` `deprecated` `broken` `planned` (default `active`) |
|
||||
| `--trust` | `official` `github` `community` `social` (default `official`) |
|
||||
| `--tags` | comma-separated; **tags are an exact-match keyword index** — put the slug, the filename, the error code here |
|
||||
| `--importance` | 0.0–1.0 |
|
||||
| `--doc-id` | append more chunks to an existing document |
|
||||
|
||||
Unknown vocabulary values warn but are accepted — the schema is faceted, not strict.
|
||||
A warning is not a failure; do **not** switch to the MCP tool because of one.
|
||||
|
||||
### Mandatory before every write: dedup-first
|
||||
|
||||
```bash
|
||||
python3 /home/n8n/bin/ai_brain_kb.py search --query "<the thing you are about to save>"
|
||||
```
|
||||
|
||||
- score **≥ 0.85** — already recorded, skip
|
||||
- **0.70–0.84** — add only if meaningfully new
|
||||
- **< 0.70** — always add
|
||||
|
||||
### Host records
|
||||
|
||||
`--type host`, one stable `--doc-id` per box. Re-add with the same `--doc-id` to update
|
||||
or append a dated chunk, rather than creating a second record for the same machine.
|
||||
|
||||
## Reading
|
||||
|
||||
```bash
|
||||
# hybrid (dense + BM25, RRF fusion) — the default, use it
|
||||
python3 /home/n8n/bin/ai_brain_kb.py search --query "ltx 2.3 native audio"
|
||||
|
||||
# pure keyword — exact filenames, error strings, node names
|
||||
python3 /home/n8n/bin/ai_brain_kb.py search --query "ltxv-097-dev-fp8.safetensors" --mode bm25
|
||||
|
||||
# typed / faceted
|
||||
python3 /home/n8n/bin/ai_brain_kb.py list --type issue --tool comfyui --limit 20
|
||||
python3 /home/n8n/bin/ai_brain_kb.py list --tag 2026-08-09-horizon-scan
|
||||
python3 /home/n8n/bin/ai_brain_kb.py facet --field tool
|
||||
python3 /home/n8n/bin/ai_brain_kb.py stats # points, doc_type inventory, health
|
||||
python3 /home/n8n/bin/ai_brain_kb.py stale --days 30
|
||||
python3 /home/n8n/bin/ai_brain_kb.py get --id <point-id>
|
||||
```
|
||||
|
||||
`mcp__better_qdrant__search` is **read-only and therefore allowed**, but it is
|
||||
dense-only — it silently misses anything a keyword query would have found. Prefer the
|
||||
helper's `search`. Use the MCP one only when you have no shell.
|
||||
|
||||
## Deleting
|
||||
|
||||
```bash
|
||||
python3 /home/n8n/bin/ai_brain_kb.py delete --doc-id <uuid> --yes # a whole document
|
||||
python3 /home/n8n/bin/ai_brain_kb.py delete --id <point-id> --yes # one chunk
|
||||
```
|
||||
|
||||
Per-document delete **exists**. Never
|
||||
`mcp__better_qdrant__delete_collection(collection="ai_brain_kb")` — that destroys the
|
||||
fleet's entire brain and there is no undo. Earlier versions of this skill described the
|
||||
nuke as the only option; that was wrong.
|
||||
|
||||
## Verify the write landed
|
||||
|
||||
A write is not done until it is retrievable **both ways**. The BM25 leg is the one a
|
||||
bare upsert cannot pass, so it is the real test:
|
||||
|
||||
```bash
|
||||
D=<doc_id from --json>
|
||||
python3 /home/n8n/bin/ai_brain_kb.py search --query "<distinctive phrase>" --mode bm25 --doc-id $D
|
||||
python3 /home/n8n/bin/ai_brain_kb.py list --type <the type you used> --doc-id $D
|
||||
```
|
||||
|
||||
Zero hits on the BM25 leg means something other than `ai_brain_kb.py` wrote it.
|
||||
|
||||
## Deep-research reports
|
||||
|
||||
Do not ingest them by hand. `publish_report.py` (in the `deep-web-research` skill) is
|
||||
the only sanctioned path — it publishes, ingests via this helper, verifies hybrid+BM25,
|
||||
and writes the `.meta.json` receipt in one action.
|
||||
|
||||
A `.meta.json` sidecar is a **receipt written after verification, not proof**. Sidecars
|
||||
written before that rule exists claim `doc_id`s that were never upserted. If you need
|
||||
to know whether a report is in the brain, ask the brain (`list --tag <slug>`), never the
|
||||
directory listing.
|
||||
|
||||
## Infrastructure
|
||||
|
||||
| Setting | Value |
|
||||
|---------|-------|
|
||||
| Qdrant | http://10.0.0.22:6333 |
|
||||
| Collection | `ai_brain_kb` |
|
||||
| Embedding | Ollama (snowflake-arctic-embed2) |
|
||||
| MCP Tool | `mcp__better_qdrant__*` |
|
||||
|---|---|
|
||||
| Qdrant | `http://10.0.0.22:6333`, collection `ai_brain_kb` |
|
||||
| Embeddings | `snowflake-arctic-embed2` on **mini, `10.0.0.30:11434`** — the fleet's only embedder |
|
||||
| Helper | `/home/n8n/bin/ai_brain_kb.py` (zero deps, urllib only) |
|
||||
| Vectors | unnamed dense 1024-dim + named sparse `bm25` |
|
||||
|
||||
## What Goes Here
|
||||
|
||||
Everything AI/ML related that should be searchable across sessions:
|
||||
|
||||
- Pipeline plans, state, and architecture decisions
|
||||
- Story prompts and scene descriptions
|
||||
- Research results (deep research, better-search outputs)
|
||||
- Model configurations, LoRA chains, render settings
|
||||
- Bug diagnoses and fixes
|
||||
- Stock material research
|
||||
- Prompt engineering guides
|
||||
- Hardware/infrastructure details for AI workloads
|
||||
|
||||
## Commands
|
||||
|
||||
### Add Documents
|
||||
|
||||
Add a file (markdown, text, JSON) to the knowledge base. The file is chunked and embedded automatically.
|
||||
|
||||
```
|
||||
mcp__better_qdrant__add_documents(
|
||||
collection="ai_brain_kb",
|
||||
embeddingService="ollama",
|
||||
filePath="/absolute/path/to/file.md"
|
||||
)
|
||||
```
|
||||
|
||||
**Chunking:** Default 500 chars with 50 char overlap. Works for .md, .txt, .json, .py files.
|
||||
|
||||
**After adding:** Confirm chunk count to user.
|
||||
|
||||
### Search
|
||||
|
||||
Semantic search across all knowledge in the collection.
|
||||
|
||||
```
|
||||
mcp__better_qdrant__search(
|
||||
collection="ai_brain_kb",
|
||||
embeddingService="ollama",
|
||||
query="your search query",
|
||||
limit=10
|
||||
)
|
||||
```
|
||||
|
||||
**Tips:**
|
||||
- Use natural language queries — "LTX artifact causes" not "ltx artifact"
|
||||
- Results include score, title, URL (if applicable), summary, and key claims
|
||||
- Higher limit = more context but more tokens
|
||||
|
||||
### Remove Documents
|
||||
|
||||
Delete individual documents by their source path (if tracked) or delete the entire collection and rebuild.
|
||||
|
||||
**Remove entire collection (nuclear option):**
|
||||
```
|
||||
mcp__better_qdrant__delete_collection(collection="ai_brain_kb")
|
||||
```
|
||||
|
||||
**Note:** There is no per-document delete in the current MCP tool. To remove specific content, delete the collection and re-add only the files you want to keep.
|
||||
|
||||
### List All Collections
|
||||
|
||||
See what collections exist on the Qdrant instance:
|
||||
|
||||
```
|
||||
mcp__better_qdrant__list_collections()
|
||||
```
|
||||
|
||||
## Workflow: Save Session Learnings
|
||||
|
||||
After a significant session (new research, bug fix, pipeline change):
|
||||
|
||||
1. **Identify new/changed files** — what markdown docs were created or updated?
|
||||
2. **Add to ai_brain_kb** — use `add_documents` for each file
|
||||
3. **Confirm** — report chunk counts to user
|
||||
4. **Clean up** — if old topic-specific collections exist, merge and delete them
|
||||
|
||||
## Workflow: Research a Topic
|
||||
|
||||
When starting work on an AI/ML topic:
|
||||
|
||||
1. **Search ai_brain_kb first** — what do we already know?
|
||||
2. **If gaps found** — dispatch deep-research or better-search
|
||||
3. **Save results** — add the research output file to ai_brain_kb
|
||||
4. **Proceed** — now you have full context
|
||||
**The CARE rule for mini:** every ingest embeds there and nothing else in the fleet can.
|
||||
Batch your writes, keep them bounded, never hammer it.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **File paths must be absolute** — the MCP tool resolves from the Hermes host filesystem.
|
||||
- **Large files chunk automatically** — 500 char chunks. Very large files (100K+ chars) may produce many chunks; consider summarizing first.
|
||||
- **No per-document delete** — the MCP tool only supports collection-level delete. Plan your adds accordingly.
|
||||
- **Embedding model must be running** — Ollama with `snowflake-arctic-embed2` must be available on the Qdrant host (10.0.0.22:11434).
|
||||
- **Collection name is exact** — `ai_brain_kb`, not `ai-brain-kb` or `ai_brain`.
|
||||
- **Search is semantic, not keyword** — phrase queries naturally. "How to fix LTX artifacts" works better than "ltx artifact fix".
|
||||
- **Don't create topic-specific collections** — everything goes into `ai_brain_kb`. The user's rule: one brain, one collection.
|
||||
|
||||
## Related Skills
|
||||
|
||||
- `save-q-memory` — Manual save to the `memories` collection (personal/behavioral memory, not knowledge base)
|
||||
- `qdrant-collection-management` — Collection-level operations: consolidation, migration, dedup, registry
|
||||
- **Never hand-roll raw Qdrant HTTP for a write.** A `PUT /collections/ai_brain_kb/points`
|
||||
with a bare vector reproduces the exact defect the MCP tool causes. Reads
|
||||
(`GET /collections/...`, `POST .../points/scroll`, `.../points/count`) are fine.
|
||||
- **Absolute paths** for `--file` / `--path`.
|
||||
- **Collection name is exact** — `ai_brain_kb`, not `ai-brain-kb`.
|
||||
- **Don't create topic-specific collections.** No `ltx-research`, no `comfyui-workflows`.
|
||||
- **`fact_store` and the `memories` collection are not the brain.** Research findings,
|
||||
model configs, prompt guides and infra facts all go here.
|
||||
- **A search that returns nothing is a real answer.** Say "not in the brain" and go
|
||||
research it — do not assume the record exists but is hiding.
|
||||
|
||||
Reference in New Issue
Block a user