hermes-skills/local-vector-memory/SKILL.md

name, description, version, category, author, tags

name	description	version	category	author	tags
local-vector-memory	Integrate local vector databases (Qdrant, Chroma, etc.) with custom embedders (Ollama, local models) into Hermes Agent via MCP servers or custom MemoryProvider plugins.	1.0.0	mlops	hermes	qdrant vector-database ollama embeddings mcp memory-provider hermes rag

Local Vector Database Integration for Hermes

When to Use This Skill

Use this when you want Hermes to use a local vector database (Qdrant, ChromaDB, Weaviate, etc.) with a custom embedder (Ollama, local sentence-transformers, etc.) for:

• Persistent cross-session memory with semantic search
• RAG pipelines over local documents
• Deduplication in research workflows
• Agent knowledge bases that survive restarts

Not for: Hosted vector services (Pinecone, Mem0 Platform, Honcho Cloud) — those are managed SaaS, not local. Hindsight was the canonical memory provider through June 2026 but is no longer used. The memories collection in Qdrant (10.0.0.22:6333, 1024-dim, Cosine) is now the standalone primary long-term memory store, maintained by the memories-organizer cron job (id 7dc8ec71118d, 100 pts/run every 30m, model: glm-5.2:cloud, deliver=local). Uses a 10-type classification schema with long-term/short-term memory split, quality scoring, garbage collection (capped at 5/run), and supersession linking. As of 2026-07-04, the cron uses a deterministic Python wrapper (classify_batch.py) that owns all Qdrant mechanics; the LLM only classifies. Error log at /home/n8n/workspace/general/cron_qdrant_errors.md (append-only, only touched on errors). Full design at mlops/qdrant-collection-management/references/memory-schema-gc-proposal.md and wrapper docs at mlops/qdrant-collection-management/references/classify-batch-wrapper.md.

IMPORTANT — Memory system roles: Holographic is the real-time, always-working, automatic memory provider (cross-session recall, fact_store). Qdrant memories is manual-only — the agent only saves to it when explicitly told ("save to memory"). They serve different purposes: Holographic for automatic cross-session recall, Qdrant for explicit long-term knowledge storage.

---

Two Integration Paths

Path A: Qdrant MCP Server (official qdrant/mcp-server-qdrant)

Aspect	Detail
Repo	https://github.com/qdrant/mcp-server-qdrant
Embedders	FastEmbed only (default branch); custom embedder support in custom-embedding-provider branch
Tools exposed	qdrant-store, qdrant-find
Config	QDRANT_URL, COLLECTION_NAME, EMBEDDING_MODEL, EMBEDDING_PROVIDER
Run	uvx mcp-server-qdrant (stdio) or SSE/streamable-http
Add to Hermes	hermes mcp add qdrant --command "uvx mcp-server-qdrant" --env QDRANT_URL=http://10.0.0.22:6333

Custom Embedder with MCP Server

The custom-embedding-provider branch accepts a custom EmbeddingProvider instance implementing:

class EmbeddingProvider(ABC):
    async def embed_documents(self, documents: list[str]) -> list[list[float]]: ...
    async def embed_query(self, query: str) -> list[float]: ...
    def get_vector_name(self) -> str: ...
    def get_vector_size(self) -> int: ...

Example: Ollama embedder (see references/ollama-embedding-provider.py)

class OllamaEmbeddingProvider(EmbeddingProvider):
    def __init__(self, base_url="http://localhost:11434", model="snowflake-arctic-embed2:latest"):
        self.base_url = base_url
        self.model = model
        self.vector_size = 1024
    
    async def embed_documents(self, documents: list[str]) -> list[list[float]]:
        async with httpx.AsyncClient() as client:
            resp = await client.post(f"{self.base_url}/api/embed",
                json={"model": self.model, "input": documents}, timeout=30)
            return resp.json()["embeddings"]
    
    async def embed_query(self, query: str) -> list[float]:
        return (await self.embed_documents([query]))[0]
    
    def get_vector_name(self) -> str:
        return "ollama-arctic-embed2"
    
    def get_vector_size(self) -> int:
        return self.vector_size

Pass to server: embedding_provider=OllamaEmbeddingProvider()

Pitfall: Requires running the MCP server as a separate process. Good for multi-client access; overhead for single-agent use.

---

Path D: better-qdrant-mcp-server (npx-based, recommended for simple RAG)

Repo: https://github.com/wrediam/better-qdrant-mcp-server npm: better-qdrant-mcp-server (runnable via npx)

This is the simplest MCP server for Qdrant + Ollama. Zero install, npx-based (like the existing searxng MCP), supports Ollama with any model via OLLAMA_MODEL env var.

Aspect	Detail
Tools	list_collections, add_documents, search, delete_collection
Embedders (shipped)	Ollama, OpenAI, OpenRouter, FastEmbed
Embedders (working locally)	Ollama only — OpenAI/OpenRouter need API keys (none configured), FastEmbed uses 384/768-dim models that mismatch 1024-dim collections. The npx-cached package has been patched to expose only ollama; see Pitfalls + references/better-qdrant-mcp-patching.md.
Transport	stdio (npx)
Config	QDRANT_URL, DEFAULT_EMBEDDING_SERVICE, OLLAMA_ENDPOINT, OLLAMA_MODEL

Hermes config.yaml entry:

mcp_servers:
  better-qdrant:
    args:
    - -y
    - better-qdrant-mcp-server
    command: npx
    connect_timeout: 30
    env:
      QDRANT_URL: http://10.0.0.22:6333
      DEFAULT_EMBEDDING_SERVICE: ollama
      OLLAMA_ENDPOINT: http://localhost:11434
      OLLAMA_MODEL: snowflake-arctic-embed2:latest
    timeout: 120

Critical pitfall — vector size hardcode: The Ollama embedding service hardcodes vectorSize = 768 (for nomic-embed-text). If add_documents targets a collection that doesn't exist yet, it auto-creates it at 768-dim — then 1024-dim snowflake2 vectors fail to upsert silently.

Fix: Always pre-create collections at 1024-dim via curl or qdrant-client before using add_documents. The search tool is safe for any existing collection regardless of vector size.

Hindsight coexistence: Operates on the memories collection (24K+ pts). Domain collections (RAG, KB) are separate — no conflict.

---

Path B: Custom Hermes MemoryProvider Plugin (recommended for single-agent)

Write a plugin implementing MemoryProvider ABC that:

• Uses your Qdrant client directly (REST or gRPC)
• Embeds via your Ollama endpoint
• Registers as ~/.hermes/plugins/memory/<name>/__init__.py
• Exposes native Hermes memory tools (mem_search, mem_conclude, mem_profile)

Advantages:

• No separate server process
• Native Hermes integration (prefetch, system prompt block, tool schemas)
• Full control over collection schema, filtering, hybrid search
• Uses your existing working patterns (see references/qdrant-crud-patterns.md)

Template: See templates/memory-provider-qdrant.py

---

Your Working Stack (Reference)

Component	Value
Qdrant URL	http://10.0.0.22:6333
Collections	comfyui_kb, comfyui_decisions, private_court_docs, hermes_kb, memories (consolidated, ~24,853 pts), + domain-scoped per project
Vector config	1024-dim, Cosine distance, UUID point IDs
Embedder	Ollama snowflake-arctic-embed2:latest at http://localhost:11434/api/embed
CRUD pattern	PUT /collections/{name}/points?wait=true with {"points": [{"id": UUID, "vector": [...], "payload": {...}}]}
Ingestion	scripts/ingest-pdfs-to-qdrant.py for PDF→Qdrant bulk ingest
Ollama config	OLLAMA_KEEP_ALIVE=-1 (models stay loaded in VRAM indefinitely)
i9 fallback	Ollama still running at localhost:11434 (not used by hindsight, available for rollback)
CRUD pattern	PUT /collections/{name}/points?wait=true with {"points": [{"id": UUID, "vector": [...], "payload": {...}}]}

Proxmox Fleet Overview

4 Proxmox servers + DGX Spark. IPs below are LXC container IPs, NOT host IPs. Each host may run multiple LXCs.

Host	CPU	RAM	GPU	Known LXC IPs	Services on this host
i9 (Hermes-MAIN)	i9-12900KF 24C	94GB	RTX 4090 24GB	localhost, 10.0.0.202	Hermes-MAIN, ComfyUI-MAIN (10.0.0.202:8188). Ollama at localhost:11434.
epyc	EPYC 7F52 32C	252GB	5x RTX 5060 Ti (80GB total)	10.0.0.26	vLLM (10.0.0.26:8000, qwen3.6-35b-a3b-uncensored) — powers Hindsight's LLM.
mini	i9-13900H 20C	94GB	RTX 3050 6GB	10.0.0.30	Ollama 0.30.10, qwen3:8b + snowflake-arctic-embed2, listening on 0.0.0.0:11434. Available for local embedding workloads.
mini2	i9-13900H 20C	47GB	RTX 3050 6GB	unknown	Idle, most underutilized host (load 0.03). Older kernel (6.17.13-2-pve, pve-manager 9.1.6). Future failover candidate.
DGX Spark	GB10 Grace Blackwell	128GB LPDDR5 unified	GB10 integrated	10.0.0.6 (host, powered off)	MSI EdgeXpert AI Mini Desktop (DGX Spark Platform), 4TB NVMe Gen5, WiFi 7, BT 5.3, NVIDIA DGX OS. Currently powered off. Historical vLLM (10.0.0.6:8000) and Ollama (10.0.10:11434) endpoints both offline.

---

Hindsight Plugin (LEGACY — no longer used)

Hindsight was the canonical Hermes memory provider through June 2026. It is no longer in use. The memories Qdrant collection is now the standalone primary memory store, maintained by the memories-organizer cron job. This section is preserved for historical reference only.

How it worked: Hindsight ran an LLM-powered extraction/recall pipeline with local vLLM, hybrid recall (semantic + entity + tag matching), built-in auto-retain, and a single config file (~/.hermes/hindsight/config.json). It stored data in the Qdrant memories collection (24K+ points consolidated from prior providers).

Legacy alternative — Mem0 OSS: The mem0ai Python library was also used historically. As of 2026-06-29, all 20 profiles have migrated away from both mem0_oss and hindsight. The memories Qdrant collection is now standalone.

---

Critical: Hermes Memory Plugin Discovery Path

User plugins MUST be placed at $HERMES_HOME/plugins/<name>/__init__.py — flat, not nested under memory/.

# CORRECT
~/.hermes/profiles/telegram/plugins/<plugin_name>/__init__.py

# WRONG — Hermes will not find this
~/.hermes/profiles/telegram/plugins/<plugin_name>/__init__.py

Discovery heuristic: Hermes checks for MemoryProvider string in the file. The class must subclass MemoryProvider from agent.memory_provider. Verify discovery with:

HERMES_HOME=~/.hermes/profiles/telegram python3 -c "
import sys; sys.path.insert(0, '~/.hermes/hermes-agent')
from plugins.memory import discover_memory_providers
for name, desc, avail in discover_memory_providers(): print(name, avail)
"

---

Path C: Hindsight (LEGACY — no longer used)

Hindsight was the canonical Hermes memory provider through June 2026. It is no longer in use. The memories Qdrant collection is now the standalone primary memory store, maintained by the memories-organizer cron job.

How it differs from Paths A/B: Hindsight runs an LLM-powered extraction/recall pipeline (similar in spirit to mem0's fact extraction), but with:

• Local vLLM as the LLM (no Ollama needed)
• Hybrid recall (semantic + entity + tag matching)
• Built-in auto-retain (no manual memory.add() calls)
• Single config file (~/.hermes/hindsight/config.json)
• Qdrant memories collection (24K+ points consolidated from prior providers)

Config

{
  "mode": "local_embedded",
  "bank_id": "hermes",
  "llm_provider": "openai_compatible",
  "llm_model": "qwen3.6-35b-a3b-uncensored",
  "llm_base_url": "http://10.0.0.26:8000/v1",
  "memory_mode": "hybrid",
  "auto_recall": true,
  "recall_budget": "high",
  "auto_retain": true
}

Enable Hindsight for a profile

Set memory.provider: hindsight in the profile's config.yaml. All 20 profiles now have this setting. See devops/hermes-config-bulk-update/references/memory-provider-switch.md for the bulk-migration playbook.

Daily maintenance

cronjob memories-organizer (id 7dc8ec71118d, 100 pts/run every 30m, model: glm-5.2:cloud, toolsets: terminal+file). Output saved locally (deliver=local). Uses the 10-type classification schema with long-term/short-term memory split, quality scoring, garbage collection (capped at 5/run), and supersession linking. Error log at /home/n8n/workspace/general/cron_qdrant_errors.md (append-only, only touched on errors). See mlops/qdrant-collection-management/references/memory-schema-gc-proposal.md for the full design.

Note: Qdrant memories is manual-only — the agent only saves to it when explicitly told ("save to memory"). Holographic is the real-time automatic memory provider. They serve different purposes.

---

Path C (LEGACY — Mem0 OSS Python Client)

Mem0 OSS is no longer the canonical path. The historical setup used mem0ai library with local Qdrant + Ollama. For reference only — do not use for new setups. See references/mem0-local-config.md.

VRAM Planning (RTX 4090, 24GB) — LEGACY for mem0

Model	VRAM	Headroom for embedder	Verdict
qwen3:4b	~3GB	~20GB	✅ Very safe
qwen3:8b	~6GB	~17GB	✅ Recommended sweet spot
qwen3:14b	~10GB	~13GB	✅ Higher quality
qwen3:30b	~20GB	~3GB	⚠️ Tight — risky
gemma4:12b	~8GB	~15GB	✅ Good alternative

Recommended: qwen3:8b — 30M+ downloads, tools tag, built for structured/JSON output. (Historical: previously used as Mem0's extraction LLM.)

Diagnostic: Check if Hindsight is healthy

Before setting up, verify the Hindsight daemon and Qdrant are reachable:

curl -s http://10.0.0.26:8000/v1/models   # vLLM LLM endpoint
curl -s http://10.0.0.22:6333/collections | jq '.result.collections[].name' | grep -E '(memories|hindsight)'

The memories collection indicates Hindsight is (or was) configured.

Config skeleton (LEGACY mem0 reference — see Hindsight Path C above)

# DEPRECATED as of 2026-06-29. Kept for historical reference only.
from mem0 import Memory
memory = Memory.from_config({...})  # see references/mem0-local-config.md

See references/mem0-local-config.md for the full legacy setup (Qdrant + Ollama + qwen3:8b extraction LLM).

---

Domain-Scoped RAG Workspace (single-collection per project)

When a project needs its own Qdrant collection isolated from the agent's general memory/knowledge collections, set up a scoped workspace:

1. Create the collection

curl -s -X PUT "http://<QDRANT_HOST>:6333/collections/<name>" \
  -H "Content-Type: application/json" \
  -d '{"vectors": {"size": 1024, "distance": "Cosine"}}'

Use 1024-dim to match snowflake-arctic-embed2:latest (the default Ollama embedder in this stack).

2. Create a workspace directory

mkdir -p /workspace/<project>/

3. Write AGENTS.md to enforce scope

AGENTS.md is the convention that works across Hermes, Aider, Codex, Cursor, and most agent tools. It is auto-loaded as project context by Hermes when the working directory is the workspace root. See templates/scoped-workspace-agents.md for a ready-to-fill template.

Critical content: declare only the in-scope Qdrant collection by name, and explicitly name the out-of-scope collections the agent must not touch. Without this, the agent will default to its general-purpose collections.

4. Ingest + query workflow

Automated ingestion script: scripts/ingest-pdfs-to-qdrant.py — extracts, chunks, embeds, and upserts all PDFs in a directory into a target collection. Run it directly:

python3 scripts/ingest-pdfs-to-qdrant.py /path/to/pdfs/ collection_name

Configurable via env vars: QDRANT_URL, OLLAMA_URL, OLLAMA_MODEL, CHUNK_SIZE, OVERLAP, BATCH_SIZE. Requires pymupdf (pip install pymupdf) and Ollama running with snowflake-arctic-embed2:latest.

Manual workflow (when the script doesn't fit):

1. Drop source files into the workspace (or subfolders by type/date).
2. Extract: PDFs via pymupdf / marker-pdf, images via OCR, CSVs as-is.
3. Chunk text.
4. Embed via POST http://localhost:11434/api/embed with {"model": "snowflake-arctic-embed2:latest", "input": [...]}
5. Upsert: PUT /collections/<name>/points?wait=true with {"points": [{"id": UUID, "vector": [...], "payload": {"source": "file.pdf", "chunk": 0, ...}}]}
6. Query: embed the question with the same model, POST /collections/<name>/points/search with the vector, return top-k chunks with payload.

Pitfalls

• Dimension mismatch: collection dim must match embedder output. snowflake-arctic-embed2 = 1024, nomic-embed-text = 768, mxbai-embed-large = 1024. Check both before creating the collection — recreating requires re-embedding.

• better-qdrant MCP add_documents timeout: The MCP tool has a 120s timeout and fails on files over ~500KB. For bulk ingestion, use a direct Python script (see references/direct-embed-script.py) that calls Ollama and Qdrant REST APIs directly — no timeout ceiling, batch control, and progress reporting.

• better-qdrant MCP auto-creates at 768-dim: If the target collection doesn't exist, add_documents creates it at 768-dim (hardcoded for nomic-embed-text). 1024-dim vectors then fail silently. Always pre-create collections at the correct dimension via curl before using the MCP tool.

• better-qdrant MCP dead embedders stripped from npx cache: The shipped package exposes openai, openrouter, fastembed, ollama in its tool schemas, but only ollama works in a local-only stack (no API keys; FastEmbed dims mismatch 1024-dim collections). The npx-cached build/ has been patched to expose only ollama so the agent never selects a broken service. The patch persists until npx upgrades/reinstalls the package — re-apply after any version bump. See references/better-qdrant-mcp-patching.md for the exact files and procedure. The running MCP process keeps the old (four-option) schema until the next Hermes restart; this is cosmetic since the removed services threw at runtime anyway.

• No AGENTS.md = no scope: without it, the agent will read from its general memory collections by default and ignore the project collection entirely.

• Cross-collection bleed: if multiple projects share one collection, payload filtering by project field is the only way to separate them. Cleaner to give each project its own collection.

• Collection name reuse: Qdrant returns true on PUT even if the collection already exists with different config. Check existing config with GET /collections/<name> before creating.

• NEVER assume model for cron jobs: When creating agent-driven cron jobs (like the weekly cleanup), always ask the user which model/provider to use. Do not default to the session model or pick one yourself. Script-only cron jobs (no_agent=true) don't need a model.

• Ollama /api/show does not return model digest: Use GET /api/ps to get the digest field for embedding model hash verification. /api/show returns model_info and details but no digest. The hash check catches silent model version changes that would degrade search quality on old embeddings.

• Ollama idle model eviction: If using Ollama for any local embedding workload, models unload from VRAM after 5 minutes by default. For production, set OLLAMA_KEEP_ALIVE=-1 via systemd override to keep models loaded permanently. Without this, the first call after idle pays a 20+ second cold-start penalty. Override file: /etc/systemd/system/ollama.service.d/override.conf with Environment=OLLAMA_KEEP_ALIVE=-1.

• HNSW indexing threshold: Qdrant only builds the HNSW index at 10K+ vectors. Below that, indexed_vectors_count shows 0 — this is normal, search still works via flat scan. Do NOT use indexed_vectors_count as a proxy for collection activity or health. A 2,000-point collection with 0 indexed vectors is perfectly healthy — it just hasn't hit the indexing threshold. The only reliable activity signals are: point-level timestamps in payloads, access logs (if enabled), or knowledge of which projects/scripts reference the collection.

• MCP-first for Qdrant operations: Use the mcp_better_qdrant_* tools for what they cover (list_collections, search, add_documents, delete_collection). Fall back to curl only for operations the MCP doesn't expose: collection info/config inspection (GET /collections/{name}), point scrolling with vectors (POST /collections/{name}/points/scroll), point counting with filters, and batch upserts with custom vectors. The MCP add_documents tool auto-embeds from files — for point-level migration with pre-existing vectors, use curl PUT /collections/{name}/points directly.

• Qdrant point count may lag during bulk ingestion: After batch upserts, points_count reflects stored points but indexed_vectors_count may trail behind while the optimizer processes segments. The indexing_threshold (default 10,000) controls when HNSW indexing kicks in. Filtered counts via POST /collections/{name}/points/count with payload filters are accurate immediately — they scan all segments, not just indexed ones.

• Cron delivery cross-profile: deliver=telegram only resolves if the profile running the cron job has Telegram configured in its platforms.telegram block. If Telegram lives in a dedicated profile (common pattern), cron jobs under other profiles must use deliver=all or accept local-only delivery. The script still runs and syncs — only the notification is affected.

See references/scoped-workspace-pattern.md for a worked example.

Decision Guide

Need	Use
Multi-agent / multi-client access to same Qdrant	MCP Server (Path A)
Single Hermes agent, native memory tools, full control	Custom MemoryProvider (Path B)
Production agent memory with semantic + entity recall	Qdrant memories collection (standalone, maintained by memories-organizer cron)
Rapid prototyping, already have MCP client	MCP Server
Complex payload filtering, hybrid search, custom scoring	Custom MemoryProvider
Want to use hermes memory CLI / /memory slash commands	Custom MemoryProvider
Domain-specific RAG isolated from general agent memory	Domain-Scoped Workspace (above)

---

References

• references/qdrant-batch-upsert-shapes.md — Authoritative. Tested batch upsert shapes for Qdrant 1.17.0: {"points": [...]} works with UUIDs/integers, {"batch": {"ids": [...], "vectors": [...], "payloads": [...]}} also works, qdrant-client recommended. Point IDs must be UUIDs or unsigned integers — string IDs like "test-1" fail.

• references/qdrant-collection-consolidation.md — Collection consolidation pattern. Migrate multiple Qdrant collections into one unified collection: scroll with vectors, transform payloads (add source/migrated_at, normalize text fields), re-embed dimension mismatches, batch upsert via temp files, validate with filtered counts, delete sources.

• references/mem0-local-config.md — LEGACY (deprecated 2026-06-29). Mem0 OSS local setup: Qdrant + Ollama + qwen3:8b extraction LLM. Kept for historical reference; all 20 profiles now use Hindsight.

• references/qdrant-crud-patterns.md — Historical ComfyUI patterns (May 2026). Superseded by qdrant-batch-upsert-shapes.md for batch shape details; still useful for payload schema examples.

• references/ollama-embedding-provider.py — Ready-to-use Ollama EmbeddingProvider for MCP server

• templates/memory-provider-qdrant.py — Scaffold for custom Hermes MemoryProvider plugin

• references/qdrant-mcp-server-details.md — Full MCP server config, env vars, deployment options

• references/better-qdrant-mcp-patching.md — How to strip the dead embedding services (openai/openrouter/fastembed) from the npx-cached better-qdrant-mcp-server package so only ollama is exposed. Files to edit, verification, npx-upgrade caveats.

• references/mem0-host-migration.md — LEGACY (superseded by Hindsight). Historical record of moving mem0's LLM+embedder Ollama endpoint off the main host (i9 → mini LXC at 10.0.0.30:11434). Migration completed June 2026; all 16 mem0.json files were pointed at mini. As of 2026-06-29, mem0.json files have been removed and all 20 profiles use Hindsight instead.

• references/mem0-validation-checklist.md — LEGACY 10-phase validation checklist for mem0 OSS. Superseded by Hindsight validation in devops/hindsight-memory-setup/SKILL.md.

---

Related Skills

• ecosystem-surveillance — Uses Qdrant for research deduplication
• comfyui — Contains working Qdrant + Ollama patterns in references/qdrant-crud-notes.md
• dspy — Builds RAG with pluggable retrievers (can use Qdrant via custom retriever)
• hermes-agent — Hermes configuration, MCP server management, memory provider setup
• social-media-scraping — Free, local X/Twitter scraping via twscrape (cookie auth). Uses same Ollama + Qdrant stack pattern for potential social media data ingestion into vector DB.