true-recall-gems/function_check.md

# TrueRecall v2 - Function Check (LOCAL)

**Quick validation checklist for OUR TrueRecall v2 setup**

**User:** rob  
**Qdrant:** http://<QDRANT_IP>:6333  
**Ollama:** http://<OLLAMA_IP>:11434  
**Timer:** 5 minutes  
**Working Dir:** ~/.openclaw/workspace/.local_projects/true-recall-gems

---

## Quick Status Check

```bash
cd ~/.openclaw/workspace/.local_projects/true-recall-gems
```

---

## 1. Directory Structure

| Check | Command | Expected |
|-------|---------|----------|
| Local project exists | `ls ~/.openclaw/workspace/.local_projects/true-recall-gems` | Files listed |
| Git project exists | `ls ~/.openclaw/workspace/.git_projects/true-recall-gems` | Files listed |
| Watcher script | `ls ~/.openclaw/workspace/skills/qdrant-memory/scripts/realtime_qdrant_watcher.py` | File exists |

**Our Paths:**
- Local: `~/.openclaw/workspace/.local_projects/true-recall-gems/`
- Git: `~/.openclaw/workspace/.git_projects/true-recall-gems/`
- Watcher: `~/.openclaw/workspace/skills/qdrant-memory/scripts/realtime_qdrant_watcher.py`
- Systemd: `/etc/systemd/system/mem-qdrant-watcher.service`

---

## 2. Services

| Check | Command | Expected |
|-------|---------|----------|
| Watcher running | `systemctl is-active mem-qdrant-watcher` | `active` |
| Watcher enabled | `systemctl is-enabled mem-qdrant-watcher` | `enabled` |
| Cron job set | `crontab -l \| grep true-recall` | `*/5 * * * *` entry |

**Our Service:**
- Service: `mem-qdrant-watcher.service`
- Status: `systemctl status mem-qdrant-watcher --no-pager`
- Logs: `journalctl -u mem-qdrant-watcher -n 20`
- Cron: `*/5 * * * *` (every 5 minutes)

---

## 3. Qdrant Collections

| Check | Command | Expected |
|-------|---------|----------|
| memories_tr status | `curl -s http://<QDRANT_IP>:6333/collections/memories_tr \| jq .result.status` | `green` |
| gems_tr status | `curl -s http://<QDRANT_IP>:6333/collections/gems_tr \| jq .result.status` | `green` |
| memories_tr count | `curl -s http://<QDRANT_IP>:6333/collections/memories_tr \| jq .result.points_count` | `12000+` |
| gems_tr count | `curl -s http://<QDRANT_IP>:6333/collections/gems_tr \| jq .result.points_count` | `70+` |

**Our Qdrant:**
- URL: http://<QDRANT_IP>:6333
- Collections: memories_tr, gems_tr
- Embedding Model: snowflake-arctic-embed2

---

## 4. Curation Status

| Check | Command | Expected |
|-------|---------|----------|
| Uncurated count | See Section 7 | `1490` |
| Curated count | See Section 7 | `11239` |
| Curator config | `cat tr-continuous/curator_config.json` | `timer_minutes: 5` |

**Our Config:**
- Timer: 5 minutes (`*/5 * * * *`)
- Batch Size: 100
- User ID: rob
- Source: memories_tr
- Target: gems_tr
- Curator Log: `/var/log/true-recall-timer.log`

---

## 5. Capture Test

| Step | Action | Check |
|------|--------|-------|
| 1 | Send a test message to Kimi | Message received |
| 2 | Wait 10 seconds | Allow processing |
| 3 | Check memories count increased | `curl -s http://<QDRANT_IP>:6333/collections/memories_tr \| jq .result.points_count` |
| 4 | Verify memory has user_id | `user_id: "rob"` in payload |
| 5 | Verify memory has curated=false | `curated: false` in payload |

**Our Watcher:**
- Script: `~/.openclaw/workspace/skills/qdrant-memory/scripts/realtime_qdrant_watcher.py`
- User ID: `rob`
- Collection: `memories_tr`
- Embeddings: `snowflake-arctic-embed2`

---

## 6. Curation Test

| Step | Action | Check |
|------|--------|-------|
| 1 | Note current gems count | Baseline |
| 2 | Run curator manually | `cd tr-continuous && python3 curator_timer.py` |
| 3 | Check gems count increased | New gems added |
| 4 | Check memories marked curated | `curated: true` |
| 5 | Check curator log | `tail /var/log/true-recall-timer.log` |

---

## 7. Recall Test ✅ **WORKING**

| Step | Action | Check |
|------|--------|-------|
| 1 | Start new conversation | Context loads |
| 2 | Ask about previous topic | Gems injected |
| 3 | Verify context visible | ✅ **Score 0.587** - Working! |

**Verified 2026-02-25:** Context injection successfully returns relevant gems with similarity scores above threshold (0.5+).

---

## 8. Path Validation

| Path | Check | Status |
|------|-------|--------|
| Watcher script | `skills/qdrant-memory/scripts/realtime_qdrant_watcher.py` | ☐ |
| Curator script | `.local_projects/true-recall-gems/tr-continuous/curator_timer.py` | ☐ |
| Config file | `.local_projects/true-recall-gems/tr-continuous/curator_config.json` | ☐ |
| Log file | `/var/log/true-recall-timer.log` | ☐ |

---

## 9. Quick Commands Reference

```bash
# Check all services
systemctl status mem-qdrant-watcher --no-pager
tail -20 /var/log/true-recall-timer.log

# Check Qdrant collections (Our Qdrant: <QDRANT_IP>:6333)
curl -s http://<QDRANT_IP>:6333/collections/memories_tr | jq '{status: .result.status, points: .result.points_count}'
curl -s http://<QDRANT_IP>:6333/collections/gems_tr | jq '{status: .result.status, points: .result.points_count}'

# Check uncurated memories (Our user_id: rob)
curl -s -X POST http://<QDRANT_IP>:6333/collections/memories_tr/points/count \
  -d '{"filter":{"must":[{"key":"user_id","match":{"value":"rob"}},{"key":"curated","match":{"value":false}}]}}' | jq .result.count

# Run curator manually (Our path: .local_projects)
cd ~/.openclaw/workspace/.local_projects/true-recall-gems/tr-continuous
python3 curator_timer.py

# Check OpenClaw plugin
openclaw status | grep memory-qdrant

# Restart watcher (if needed)
sudo systemctl restart mem-qdrant-watcher

# View watcher logs
journalctl -u mem-qdrant-watcher -n 50 --no-pager
```

---

## Recent Fixes (2026-02-25)

| Issue | Status | Fix |
|-------|--------|-----|
| Embedding model mismatch | ✅ Fixed | Changed curator from `mxbai-embed-large` to `snowflake-arctic-embed2` |
| Gems had no vectors | ✅ Fixed | Updated `store_gem()` to use `text` field |
| JSON parsing errors | ✅ Fixed | Simplified extraction prompt |
| Field mismatch | ✅ Fixed | Curator now supports both `text` and `content` fields |
| Context injection | ✅ **WORKING** | Verified with score 0.587 on test query |
| **Watcher session bug** | ✅ **Fixed 12:22** | Watcher was stuck on old session, restarted and now follows current session |
| **Plugin capture** | ✅ **Fixed 12:34** | Added `extractMessageText()` to handle OpenAI-style content arrays |
| **Plugin exchanges** | ✅ **Verified 12:41** | Now extracting exchanges: parsed 17 user, 116 assistant, 9 exchanges |
| **Gem ID collision** | ✅ **Fixed 12:50** | Hash now uses `embedding_text_for_hash[:100]` instead of empty fields |
| **Meta-gem filtering** | ✅ **Fixed 12:52** | Curator skips patterns: "gems extracted", "curator", "✅", "🔍", debug messages, system messages |
| **gems_tr cleaned** | ✅ **Done 12:53** | Removed 5 meta-gems, kept 1 real gem |
| **Gem format (1st person)** | ✅ **Fixed 13:15** | Changed from "User decided..." to "I decided..." for better query matching |

### Needed Improvements

| Issue | Description | Priority |
|-------|-------------|----------|
| **Semantic Deduplication** | No dedup between similar gems. Same fact phrased differently creates multiple gems. | High |
| **Search Result Deduplication** | Similar gems both injected, causing redundancy. | Medium |
| **Gem Quality Scoring** | No quality metric for gems. | Medium |
| **Temporal Decay** | All gems treated equally regardless of age. | Low |
| **Gem Merging/Updating** | Old gems not updated when preferences change. | Low |
| **Importance Calibration** | All curator gems marked "medium" importance. | Low |

**Result:** Context injection now functional. Gems are embedded and searchable. Both watcher and plugin capture working.

| Check | Date | Status |
|-------|------|--------|
| All services running | 2026-02-25 | ✅ |
| Collections healthy | 2026-02-25 | ✅ |
| Capture working | 2026-02-25 | ✅ |
| Curation working | 2026-02-25 | ✅ |
| Recall working | 2026-02-25 | ✅ **Context injection verified** |

---

*Last updated: 2026-02-25 12:04 CST*  
*User: rob*  
*Qdrant: <QDRANT_IP>:6333*  
*Timer: 5 minutes*  
*Collections: memories_tr (12,729), gems_tr (14+)*  
*Status: ✅ Context injection WORKING*