163 lines
6.4 KiB
Markdown
163 lines
6.4 KiB
Markdown
|
|
---
|
||
|
|
name: local-deep-research
|
||
|
|
description: "Set up, configure, and operate local-deep-research (LDR) — the LearningCircuit deep research agent. Covers user creation, Ollama/SearXNG config, API usage, and troubleshooting."
|
||
|
|
version: 1.0.0
|
||
|
|
author: Hermes Agent
|
||
|
|
license: MIT
|
||
|
|
platforms: [linux]
|
||
|
|
metadata:
|
||
|
|
hermes:
|
||
|
|
tags: [research, deep-research, ollama, searxng, ldr]
|
||
|
|
related_skills: [ask-claude]
|
||
|
|
---
|
||
|
|
|
||
|
|
# Local Deep Research — Setup & Operation
|
||
|
|
|
||
|
|
local-deep-research (LDR) is a multi-step research agent by LearningCircuit. It plans sub-queries, searches via configurable engines, scrapes results, and synthesizes findings. Built for Ollama + SearXNG. Web UI on port 5000, Python API client available.
|
||
|
|
|
||
|
|
## Quick Reference
|
||
|
|
|
||
|
|
| Task | Approach |
|
||
|
|
|------|----------|
|
||
|
|
| Create user | Direct DB insert + `DatabaseManager.create_user_database()` |
|
||
|
|
| Configure model | Settings API: `llm.provider`, `llm.model`, `llm.ollama.url` |
|
||
|
|
| Configure search | Settings API: `search.tool`, `search.engine.web.searxng.default_params.instance_url` |
|
||
|
|
| Test query | `LDRClient.quick_research()` or `quick_query()` convenience function |
|
||
|
|
| Start server | `python3 -c "from local_deep_research.web.app import main; main()"` with env vars |
|
||
|
|
|
||
|
|
## Auth Architecture
|
||
|
|
|
||
|
|
LDR does NOT store password hashes. Authentication works by attempting to decrypt the user's per-user SQLCipher database with the supplied password. If decryption succeeds, the password is correct.
|
||
|
|
|
||
|
|
- **Auth DB**: SQLite at `get_data_directory()/ldr_auth.db` — `users` table (id, username, created_at, last_login, database_version)
|
||
|
|
- **Per-user DB**: Encrypted SQLCipher file, created by `DatabaseManager.create_user_database(username, password)`
|
||
|
|
- **Registration flow**: Insert user row → create encrypted DB. Both must succeed.
|
||
|
|
|
||
|
|
## Creating a User (Programmatic)
|
||
|
|
|
||
|
|
The web UI registration hits rate limits. Use direct DB access instead:
|
||
|
|
|
||
|
|
```python
|
||
|
|
from local_deep_research.database.encrypted_db import DatabaseManager
|
||
|
|
from local_deep_research.database.models import User
|
||
|
|
from local_deep_research.database.auth_db import auth_db_session
|
||
|
|
|
||
|
|
username = 'hermes'
|
||
|
|
password = 'research123'
|
||
|
|
|
||
|
|
with auth_db_session() as session:
|
||
|
|
new_user = User(username=username)
|
||
|
|
session.add(new_user)
|
||
|
|
session.commit()
|
||
|
|
|
||
|
|
db_manager = DatabaseManager()
|
||
|
|
db_manager.create_user_database(username, password)
|
||
|
|
```
|
||
|
|
|
||
|
|
This bypasses rate limits and CSRF entirely. Works even when the server is not running.
|
||
|
|
|
||
|
|
## Configuration
|
||
|
|
|
||
|
|
### Via Settings API (preferred — persists across restarts)
|
||
|
|
|
||
|
|
```python
|
||
|
|
from local_deep_research.api.client import LDRClient
|
||
|
|
|
||
|
|
client = LDRClient(base_url="http://localhost:5000")
|
||
|
|
client.login("hermes", "research123")
|
||
|
|
|
||
|
|
# LLM config
|
||
|
|
client.update_setting("llm.provider", "ollama")
|
||
|
|
client.update_setting("llm.model", "deepseek-v4-pro:cloud")
|
||
|
|
client.update_setting("llm.ollama.url", "http://localhost:11434")
|
||
|
|
|
||
|
|
# Search config
|
||
|
|
client.update_setting("search.tool", "searxng")
|
||
|
|
client.update_setting("search.engine.web.searxng.default_params.instance_url", "http://10.0.0.8:8888")
|
||
|
|
|
||
|
|
client.logout()
|
||
|
|
```
|
||
|
|
|
||
|
|
### Via Environment Variables (override at server start)
|
||
|
|
|
||
|
|
```bash
|
||
|
|
LDR_LLM_PROVIDER=ollama
|
||
|
|
LDR_LLM_MODEL=deepseek-v4-pro:cloud
|
||
|
|
LDR_LLM_OLLAMA_URL=http://localhost:11434
|
||
|
|
LDR_SEARCH_ENGINE_WEB_SEARXNG_DEFAULT_PARAMS_INSTANCE_URL=http://10.0.0.8:8888
|
||
|
|
```
|
||
|
|
|
||
|
|
Env vars override web UI settings and lock them (read-only in UI). Format: `LDR_` + setting key with dots replaced by underscores, UPPERCASED.
|
||
|
|
|
||
|
|
### Settings API Structure
|
||
|
|
|
||
|
|
Settings are returned as nested dicts with metadata. The actual value is in the `value` key:
|
||
|
|
|
||
|
|
```python
|
||
|
|
r = client.session.get("http://localhost:5000/settings/api")
|
||
|
|
data = r.json()
|
||
|
|
settings = data.get("settings", {})
|
||
|
|
# settings["llm.model"]["value"] → "deepseek-v4-pro:cloud"
|
||
|
|
```
|
||
|
|
|
||
|
|
Key settings to verify:
|
||
|
|
- `llm.provider` → `ollama`
|
||
|
|
- `llm.model` → model name (any model in `ollama list`)
|
||
|
|
- `llm.ollama.url` → `http://localhost:11434`
|
||
|
|
- `search.tool` → `searxng`
|
||
|
|
- `search.engine.web.searxng.default_params.instance_url` → SearXNG URL
|
||
|
|
|
||
|
|
## Running a Research Query
|
||
|
|
|
||
|
|
```python
|
||
|
|
from local_deep_research.api.client import LDRClient
|
||
|
|
|
||
|
|
client = LDRClient(base_url="http://localhost:5000")
|
||
|
|
client.login("hermes", "research123")
|
||
|
|
|
||
|
|
result = client.quick_research(
|
||
|
|
"Your research question here",
|
||
|
|
model="deepseek-v4-pro:cloud",
|
||
|
|
search_engines=["searxng"],
|
||
|
|
iterations=2,
|
||
|
|
timeout=600
|
||
|
|
)
|
||
|
|
|
||
|
|
# result is a dict with "summary" and "sources" keys
|
||
|
|
print(result.get("summary", "No summary"))
|
||
|
|
```
|
||
|
|
|
||
|
|
Or the one-liner:
|
||
|
|
```python
|
||
|
|
from local_deep_research.api.client import quick_query
|
||
|
|
summary = quick_query("hermes", "research123", "Your question", base_url="http://localhost:5000")
|
||
|
|
```
|
||
|
|
|
||
|
|
## Starting the Server
|
||
|
|
|
||
|
|
```bash
|
||
|
|
# With env vars
|
||
|
|
LDR_LLM_MODEL=deepseek-v4-pro:cloud \
|
||
|
|
LDR_SEARCH_ENGINE_WEB_SEARXNG_DEFAULT_PARAMS_INSTANCE_URL=http://10.0.0.8:8888 \
|
||
|
|
python3 -c "from local_deep_research.web.app import main; main()"
|
||
|
|
```
|
||
|
|
|
||
|
|
Server listens on 0.0.0.0:5000 by default. First request returns 302 redirect to /auth/login.
|
||
|
|
|
||
|
|
## Model Selection
|
||
|
|
|
||
|
|
LDR uses `ChatOllama` from langchain — any model in `ollama list` works, including cloud-routed aliases. LDR doesn't know or care whether the model is local or remote.
|
||
|
|
|
||
|
|
For deep research, model quality matters significantly. The model needs strong multi-step reasoning and long context. Small local models (qwen3:8b, granite4.1:3b) produce poor research. Use the strongest available model.
|
||
|
|
|
||
|
|
## Pitfalls
|
||
|
|
|
||
|
|
- **Rate limiting on registration**: The web UI rate-limits registration attempts. Always create the first user via direct DB insert (see "Creating a User" above).
|
||
|
|
- **Settings API uses dot notation in URL path**: `PUT /settings/api/llm.model` with JSON body `{"value": "model-name"}`. The key in the URL path matches the setting key.
|
||
|
|
- **Settings are nested dicts, not flat values**: `settings["llm"]["model"]` won't work. Use `settings["llm.model"]["value"]` or the `update_setting()` helper.
|
||
|
|
- **SearXNG URL must be set explicitly**: LDR defaults to `localhost:8080`. If your SearXNG is elsewhere, set `search.engine.web.searxng.default_params.instance_url`.
|
||
|
|
- **Ollama URL defaults to localhost:11434**: Usually correct, but verify with `ollama list` that it's reachable there.
|
||
|
|
- **Server must be running for API client**: The `LDRClient` connects to the running Flask server. Start it first.
|
||
|
|
- **Env vars lock settings**: Settings set via env var become read-only in the web UI. Use the API for mutable config.
|
||
|
|
- **Context window for local providers**: Default is 20480 tokens. For deep research with large context, increase `llm.local_context_window_size`.
|