a1cc5b5477
Documented 6 areas needing improvement: - Semantic Deduplication (High) - Search Result Deduplication (Medium) - Gem Quality Scoring (Medium) - Temporal Decay (Low) - Gem Merging/Updating (Low) - Importance Calibration (Low)
301 lines
11 KiB
Markdown
301 lines
11 KiB
Markdown
# TrueRecall v2 - Master Audit Checklist (LOCAL)
|
|
|
|
**For:** `.local_projects/true-recall-v2/` (Working/Development Directory)
|
|
**Version:** 2.2
|
|
**Last Updated:** 2026-02-25 10:07 CST
|
|
|
|
---
|
|
|
|
## Overview
|
|
|
|
This checklist validates the **local working directory** with real IPs, paths, and credentials. Use this for development, debugging, and local testing.
|
|
|
|
---
|
|
|
|
## Recent Fixes (2026-02-25)
|
|
|
|
| Issue | Status | Fix |
|
|
|-------|--------|-----|
|
|
| Embedding model mismatch | ✅ Fixed | Changed curator to `snowflake-arctic-embed2` |
|
|
| Gems had no vectors | ✅ Fixed | Updated `store_gem()` to use `text` field |
|
|
| JSON parsing errors | ✅ Fixed | Simplified extraction prompt |
|
|
| Watcher stuck on old session | ✅ **Fixed 12:22** | Restarted watcher service |
|
|
| Plugin capture 0 exchanges | ✅ **Fixed 12:34** | Added `extractMessageText()` for array content |
|
|
| Plugin exchanges working | ✅ **Verified 12:41** | 9 exchanges extracted per session |
|
|
|
|
### Needed Improvements
|
|
|
|
| Issue | Description | Priority |
|
|
|-------|-------------|----------|
|
|
| **Semantic Deduplication** | No dedup between similar gems. Same fact phrased differently creates multiple gems. | High |
|
|
| **Search Result Deduplication** | Similar gems both injected, causing redundancy. | Medium |
|
|
| **Gem Quality Scoring** | No quality metric for gems. | Medium |
|
|
| **Temporal Decay** | All gems treated equally regardless of age. | Low |
|
|
| **Gem Merging/Updating** | Old gems not updated when preferences change. | Low |
|
|
| **Importance Calibration** | All curator gems marked "medium" importance. | Low |
|
|
|
|
---
|
|
|
|
## SECTION 1: System Requirements
|
|
|
|
### 1.1 Python Environment
|
|
|
|
| # | Check | Command | Expected | Status |
|
|
|---|-------|---------|----------|--------|
|
|
| 1.1.1 | Python version | `python3 --version` | 3.8+ | ☐ |
|
|
| 1.1.2 | pip available | `pip3 --version` | Working | ☐ |
|
|
| 1.1.3 | curl available | `curl --version` | Working | ☐ |
|
|
| 1.1.4 | jq available | `jq --version` | Working | ☐ |
|
|
|
|
### 1.2 Network Services
|
|
|
|
| # | Check | Command | Expected | Status |
|
|
|---|-------|---------|----------|--------|
|
|
| 1.2.1 | Qdrant reachable | `curl -s http://<QDRANT_IP>:6333` | Returns version | ☐ |
|
|
| 1.2.2 | Ollama reachable | `curl -s http://<OLLAMA_IP>:11434/api/tags` | Returns models | ☐ |
|
|
| 1.2.3 | Redis reachable | `redis-cli -h <REDIS_IP> ping` | PONG | ☐ |
|
|
| 1.2.4 | Kokoro reachable | `curl -s http://<KOKORO_IP>:8880` | 200 OK | ☐ |
|
|
|
|
### 1.3 OpenClaw
|
|
|
|
| # | Check | Command | Expected | Status |
|
|
|---|-------|---------|----------|--------|
|
|
| 1.3.1 | Gateway status | `openclaw gateway status` | Active | ☐ |
|
|
| 1.3.2 | Config valid | `openclaw doctor` | No errors | ☐ |
|
|
| 1.3.3 | Plugin loaded | `openclaw status | grep memory-qdrant` | Enabled | ☐ |
|
|
|
|
---
|
|
|
|
## SECTION 2: Project Files (Local)
|
|
|
|
### 2.1 Core Files Exist
|
|
|
|
| # | File | Path | Status |
|
|
|---|------|------|--------|
|
|
| 2.1.1 | README.md | `.local_projects/true-recall-v2/README.md` | ☐ |
|
|
| 2.1.2 | session.md | `.local_projects/true-recall-v2/session.md` | ☐ |
|
|
| 2.1.3 | checklist.md | `.local_projects/true-recall-v2/checklist.md` | ☐ |
|
|
| 2.1.4 | curator-prompt.md | `.local_projects/true-recall-v2/curator-prompt.md` | ☐ |
|
|
|
|
### 2.2 Scripts Exist
|
|
|
|
| # | File | Path | Status |
|
|
|---|------|------|--------|
|
|
| 2.2.1 | curator_timer.py | `.local_projects/true-recall-v2/tr-continuous/curator_timer.py` | ☐ |
|
|
| 2.2.2 | curator_config.json | `.local_projects/true-recall-v2/tr-continuous/curator_config.json` | ☐ |
|
|
| 2.2.3 | install.py | `.local_projects/true-recall-v2/install.py` | ☐ |
|
|
|
|
### 2.3 Watcher Files
|
|
|
|
| # | File | Path | Status |
|
|
|---|------|------|--------|
|
|
| 2.3.1 | realtime_qdrant_watcher.py | `skills/qdrant-memory/scripts/realtime_qdrant_watcher.py` | ☐ |
|
|
| 2.3.2 | mem-qdrant-watcher.service | `/etc/systemd/system/mem-qdrant-watcher.service` | ☐ |
|
|
|
|
---
|
|
|
|
## SECTION 3: Configuration Validation
|
|
|
|
### 3.1 curator_config.json
|
|
|
|
| # | Setting | Key | Expected | Status |
|
|
|---|---------|-----|----------|--------|
|
|
| 3.1.1 | Timer minutes | `timer_minutes` | 5 | ☐ |
|
|
| 3.1.2 | Batch size | `max_batch_size` | 100 | ☐ |
|
|
| 3.1.3 | User ID | `user_id` | rob | ☐ |
|
|
| 3.1.4 | Source collection | `source_collection` | memories_tr | ☐ |
|
|
| 3.1.5 | Target collection | `target_collection` | gems_tr | ☐ |
|
|
|
|
### 3.2 openclaw.json Plugin Config
|
|
|
|
| # | Setting | Key | Expected | Status |
|
|
|---|---------|-----|----------|--------|
|
|
| 3.2.1 | Qdrant URL | `qdrantUrl` | http://<QDRANT_IP>:6333 | ☐ |
|
|
| 3.2.2 | Ollama URL | `ollamaUrl` | http://<OLLAMA_IP>:11434 | ☐ |
|
|
| 3.2.3 | Embedding model | `embeddingModel` | snowflake-arctic-embed2 | ☐ |
|
|
| 3.2.4 | Capture collection | `captureCollection` | memories_tr | ☐ |
|
|
| 3.2.5 | Recall collection | `collectionName` | gems_tr | ☐ |
|
|
| 3.2.6 | Auto capture | `autoCapture` | true | ☐ |
|
|
| 3.2.7 | Auto recall | `autoRecall` | true | ☐ |
|
|
|
|
---
|
|
|
|
## SECTION 4: Qdrant Collections
|
|
|
|
### 4.1 Collection Status
|
|
|
|
| # | Check | Command | Expected | Status |
|
|
|---|-------|---------|----------|--------|
|
|
| 4.1.1 | memories_tr exists | `curl -s http://<QDRANT_IP>:6333/collections/memories_tr | jq .result.status` | green | ☐ |
|
|
| 4.1.2 | gems_tr exists | `curl -s http://<QDRANT_IP>:6333/collections/gems_tr | jq .result.status` | green | ☐ |
|
|
| 4.1.3 | memories_tr points | `curl -s http://<QDRANT_IP>:6333/collections/memories_tr | jq .result.points_count` | 12000+ | ☐ |
|
|
| 4.1.4 | gems_tr points | `curl -s http://<QDRANT_IP>:6333/collections/gems_tr | jq .result.points_count` | 70+ | ☐ |
|
|
|
|
### 4.2 Data Integrity
|
|
|
|
| # | Check | Command | Expected | Status |
|
|
|---|-------|---------|----------|--------|
|
|
| 4.2.1 | Uncurated count | Count `curated: false` | 1500+ | ☐ |
|
|
| 4.2.2 | Curated count | Count `curated: true` | 11000+ | ☐ |
|
|
| 4.2.3 | Can write points | Test insert | Success | ☐ |
|
|
| 4.2.4 | Can read points | Test query | Success | ☐ |
|
|
|
|
---
|
|
|
|
## SECTION 5: Services
|
|
|
|
### 5.1 Watcher Service
|
|
|
|
| # | Check | Command | Expected | Status |
|
|
|---|-------|---------|----------|--------|
|
|
| 5.1.1 | Service loaded | `systemctl status mem-qdrant-watcher | grep Loaded` | loaded | ☐ |
|
|
| 5.1.2 | Service active | `systemctl is-active mem-qdrant-watcher` | active | ☐ |
|
|
| 5.1.3 | Service enabled | `systemctl is-enabled mem-qdrant-watcher` | enabled | ☐ |
|
|
| 5.1.4 | Process running | `pgrep -f realtime_qdrant_watcher` | PID exists | ☐ |
|
|
| 5.1.5 | Logs available | `journalctl -u mem-qdrant-watcher -n 5` | Recent entries | ☐ |
|
|
|
|
### 5.2 Timer Curator
|
|
|
|
| # | Check | Command | Expected | Status |
|
|
|---|-------|---------|----------|--------|
|
|
| 5.2.1 | Cron job exists | `crontab -l | grep true-recall` | Entry present | ☐ |
|
|
| 5.2.2 | Cron interval | Visual check | */5 * * * * | ☐ |
|
|
| 5.2.3 | Log file exists | `ls -la /var/log/true-recall-timer.log` | File exists | ☐ |
|
|
| 5.2.4 | Recent activity | `tail -5 /var/log/true-recall-timer.log` | Recent timestamp | ☐ |
|
|
| 5.2.5 | Script executable | `test -x curator_timer.py` | Yes | ☐ |
|
|
|
|
---
|
|
|
|
## SECTION 6: Function Tests
|
|
|
|
### 6.1 Capture Test
|
|
|
|
| # | Step | Expected | Status |
|
|
|---|------|----------|--------|
|
|
| 6.1.1 | Send test message | Message captured | ☐ |
|
|
| 6.1.2 | Wait 10 seconds | Processing time | ☐ |
|
|
| 6.1.3 | Check memories_tr count | Increased by 2 | ☐ |
|
|
| 6.1.4 | Verify content | Content matches | ☐ |
|
|
|
|
### 6.2 Curation Test
|
|
|
|
| # | Step | Expected | Status |
|
|
|---|------|----------|--------|
|
|
| 6.2.1 | Note uncurated count | Baseline | ☐ |
|
|
| 6.2.2 | Run curator manually | Completes | ☐ |
|
|
| 6.2.3 | Check gems_tr | New gems added | ☐ |
|
|
| 6.2.4 | Verify curated flag | Marked true | ☐ |
|
|
|
|
### 6.3 Recall Test
|
|
|
|
| # | Step | Expected | Status |
|
|
|---|------|----------|--------|
|
|
| 6.3.1 | Start new conversation | Context loaded | ☐ |
|
|
| 6.3.2 | Send relevant query | Gems injected | ☐ |
|
|
| 6.3.3 | Verify injection | Context visible | ☐ |
|
|
|
|
---
|
|
|
|
## SECTION 7: Error Checks
|
|
|
|
### 7.1 Common Errors
|
|
|
|
| # | Error | Check | Fix | Status |
|
|
|---|-------|-------|-----|--------|
|
|
| 7.1.1 | Qdrant unreachable | `curl http://<QDRANT_IP>:6333` | Start Qdrant | ☐ |
|
|
| 7.1.2 | Ollama unreachable | `curl http://<OLLAMA_IP>:11434` | Start Ollama | ☐ |
|
|
| 7.1.3 | Watcher not running | `systemctl status mem-qdrant-watcher` | Restart service | ☐ |
|
|
| 7.1.4 | Curator not running | `tail /var/log/true-recall-timer.log` | Check cron | ☐ |
|
|
| 7.1.5 | No gems extracted | Check config.json | Verify model | ☐ |
|
|
|
|
### 7.2 Log Analysis
|
|
|
|
| # | Log | Location | Check For | Status |
|
|
|---|-----|----------|-----------|--------|
|
|
| 7.2.1 | Watcher log | `journalctl -u mem-qdrant-watcher` | Errors, crashes | ☐ |
|
|
| 7.2.2 | Curator log | `/var/log/true-recall-timer.log` | Failures, 0 gems | ☐ |
|
|
| 7.2.3 | OpenClaw log | `/tmp/openclaw/openclaw-*.log` | Plugin errors | ☐ |
|
|
| 7.2.4 | System log | `journalctl -n 50` | Service failures | ☐ |
|
|
|
|
---
|
|
|
|
## SECTION 8: Security (Local - Expected)
|
|
|
|
### 8.1 Private Info (Acceptable in Local)
|
|
|
|
| # | Item | Location | Expected | Status |
|
|
|---|------|----------|----------|--------|
|
|
| 8.1.1 | Private IPs | Scripts | 10.0.0.x | ✅ OK |
|
|
| 8.1.2 | Absolute paths | Scripts | /root/... | ✅ OK |
|
|
| 8.1.3 | Usernames | Config | rob | ✅ OK |
|
|
| 8.1.4 | Internal URLs | Config | http://10.0.0.x | ✅ OK |
|
|
|
|
### 8.2 Credentials (Should NOT Be in Code)
|
|
|
|
| # | Check | Command | Expected | Status |
|
|
|---|-------|---------|----------|--------|
|
|
| 8.2.1 | No tokens in .py | `grep -r "token" *.py` | Only env vars | ☐ |
|
|
| 8.2.2 | No passwords | `grep -r "password" *.py` | None found | ☐ |
|
|
| 8.2.3 | No API keys | `grep -rE "[a-zA-Z0-9]{32,}" *.py` | None found | ☐ |
|
|
| 8.2.4 | .git/config clean | `cat .git/config | grep url` | No tokens | ☐ |
|
|
|
|
---
|
|
|
|
## SECTION 9: Sync Check (Local vs Git)
|
|
|
|
### 9.1 Compare Directories
|
|
|
|
| # | Check | Command | Expected | Status |
|
|
|---|-------|---------|----------|--------|
|
|
| 9.1.1 | File count match | Compare `.local_projects/` vs `.git_projects/` | Similar | ☐ |
|
|
| 9.1.2 | Key files exist | README, session, checklist in both | Yes | ☐ |
|
|
| 9.1.3 | Scripts in git | curator_timer.py in git | Yes | ☐ |
|
|
| 9.1.4 | Config in git | curator_config.json in git | Yes | ☐ |
|
|
|
|
### 9.2 Sanitization Verification
|
|
|
|
| # | Check | Local | Git | Status |
|
|
|---|-------|-------|-----|--------|
|
|
| 9.2.1 | IPs in local | 10.0.0.x | ✅ Expected | - |
|
|
| 9.2.2 | IPs in git | Placeholders | ✅ Expected | - |
|
|
| 9.2.3 | Paths in local | /root/... | ✅ Expected | - |
|
|
| 9.2.4 | Paths in git | ~/... | ✅ Expected | - |
|
|
|
|
---
|
|
|
|
## Sign-Off
|
|
|
|
| Role | Name | Date | Signature |
|
|
|------|------|------|-----------|
|
|
| Auditor | | | |
|
|
| Developer | | | |
|
|
| Reviewer | | | |
|
|
|
|
---
|
|
|
|
## Quick Commands Reference
|
|
|
|
```bash
|
|
# Check all services
|
|
systemctl status mem-qdrant-watcher
|
|
curl -s http://<QDRANT_IP>:6333/collections/memories_tr | jq .result.points_count
|
|
curl -s http://<QDRANT_IP>:6333/collections/gems_tr | jq .result.points_count
|
|
tail -20 /var/log/true-recall-timer.log
|
|
|
|
# Check Qdrant for curated status
|
|
curl -s -X POST http://<QDRANT_IP>:6333/collections/memories_tr/points/count \
|
|
-d '{"filter":{"must":[{"key":"curated","match":{"value":false}}]}}'
|
|
|
|
# Manual curator run
|
|
cd ~/.openclaw/workspace/.local_projects/true-recall-v2/tr-continuous
|
|
python3 curator_timer.py --dry-run
|
|
|
|
# Restart services
|
|
sudo systemctl restart mem-qdrant-watcher
|
|
```
|
|
|
|
---
|
|
|
|
*This checklist is for LOCAL working directory validation only.*
|
|
*For git/public checks, see `audit_checklist.md` in `.git_projects/true-recall-v2/`*
|