Add all custom Hermes skills from general profile (33 skills)
agent-workflows: workspace-context-organization autonomous-ai-agents: hermes-agent computer-use devops: hermes-config-bulk-update, hermes-profile-management, holographic-memory, telegram-integration, webhook-subscriptions email: himalaya mcp: native-mcp, searxng-smart-search media: voice-systems, youtube-content mlops: local-vector-memory, qdrant-collection-management productivity: maps, notion, ocr-and-documents project-knowledge-base research: arxiv, blogwatcher, ecosystem-surveillance save-agents-md social-media: social-media-scraping, xurl software-development: agent-self-audit, simplify-code, spike, subagent-driven-development, systematic-debugging, test-driven-development, writing-plans user-response-style
This commit is contained in:
Hermes Agent
140
ecosystem-surveillance/SKILL.md
Normal file
140
ecosystem-surveillance/SKILL.md
Normal file
@@ -0,0 +1,140 @@
|
||||
---
|
||||
name: ecosystem-surveillance
|
||||
description: Multi-platform ecosystem research and surveillance using searxng as primary search, supplemented by GitHub API, site-specific APIs, and direct page fetches. Deduplicates against Qdrant vectors and produces structured markdown research artifacts.
|
||||
version: 1.0.0
|
||||
category: research
|
||||
author: hermes
|
||||
---
|
||||
|
||||
# Ecosystem Surveillance Research
|
||||
|
||||
Trigger: user issues "fringe research" or requests systematic, multi-platform ecosystem surveillance across Reddit, GitHub, Twitter/X, DuckDuckGo, and arXiv. They expect comprehensive coverage, deduplication against existing Qdrant vectors, and a structured markdown research artifact as output.
|
||||
|
||||
## Core Search Strategy
|
||||
|
||||
**SearXNG is the ONLY web search source. NEVER use the web_search tool (API provider).** This is a hard rule, not a preference. If SearXNG fails, notify the user immediately — do not fall back to the web_search tool. Only use direct DuckDuckGo/Brave/Google as a last resort if the entire SearXNG ecosystem is down.
|
||||
|
||||
**Always validate search paths before modifying or claiming knowledge.** Confirm with a live search that the information is current — don't rely solely on internal knowledge or the long-term memory store.
|
||||
|
||||
**All global rules are GLOBAL.** When the user says "make global" or "prominent," the rule applies across all profiles and sessions, not just this one.
|
||||
|
||||
### Known searxng instances (try in order)
|
||||
|
||||
| Instance | URL | Notes |
|
||||
|----------|-----|-------|
|
||||
| search.sapti.me | `https://search.sapti.me` | Rate-limits after a few queries; add delays |
|
||||
| search.nousresearch.com | `https://search.nousresearch.com` | Internal — may be down |
|
||||
| search.nadeau.lu | `https://search.nadeau.lu` | Luxembourg instance |
|
||||
| search.mascot.xyz | `https://search.mascot.xyz` | Switzerland |
|
||||
| searx.be | `https://searx.be` | Belgium; often overloaded |
|
||||
| searx.fmac.xyz | `https://searx.fmac.xyz` | Community instance |
|
||||
| searxng.site | `https://searxng.site` | Public |
|
||||
|
||||
### Query format
|
||||
|
||||
```bash
|
||||
curl -s --max-time 15 \
|
||||
"https://<INSTANCE>/search?q=<URL_ENCODED_QUERY>&format=json" \
|
||||
| jq -r '.results[:10] | .[].url'
|
||||
```
|
||||
|
||||
Example:
|
||||
```bash
|
||||
curl -s --max-time 15 \
|
||||
"https://search.sapti.me/search?q=AI+assisted+GPU+mining+RTX+4090&format=json" \
|
||||
| jq -r '.results[:5] | .[].url'
|
||||
```
|
||||
|
||||
### Fallback: direct search engine
|
||||
|
||||
When searxng is entirely unreachable, use DuckDuckGo direct:
|
||||
```bash
|
||||
curl -s "https://lite.duckduckgo.com/lite/?q=<QUERY>" \
|
||||
-A "Mozilla/5.0" \
|
||||
| grep -oE 'https?://[^"<>]+' \
|
||||
| grep -v duckduckgo
|
||||
```
|
||||
|
||||
Or fetch known authoritative pages directly (whattomine.com, GitHub, etc.) without searching.
|
||||
|
||||
## Platform Coverage Pattern
|
||||
|
||||
For "fringe research" class queries, cover in this order:
|
||||
|
||||
1. **Searxng** — broad web search for recent articles, forum posts, blog announcements
|
||||
2. **GitHub** — code repos, open-source projects in the space (`github.com/search?q=<query>&type=repositories`)
|
||||
3. **Twitter/X** — via `xurl` CLI or searxng with `site:twitter.com` or `site:x.com`
|
||||
4. **Reddit** — via searxng with `site:reddit.com`
|
||||
5. **arXiv** — academic papers on the topic (see `arxiv` skill)
|
||||
6. **Direct authoritative sources** — project docs, official calculators, API endpoints
|
||||
|
||||
## Deduplication
|
||||
|
||||
Before adding results to the final artifact:
|
||||
- Check against existing Qdrant vectors in the `ecosystem_research` collection
|
||||
- Normalize URLs (drop tracking params, `?utm_*`, `#fragments`)
|
||||
- Group by source domain; flag if same story appears across multiple platforms
|
||||
- De-duplicate by content hash if fetching full pages
|
||||
|
||||
### Qdrant Configuration
|
||||
|
||||
- **Instance**: `http://10.0.0.22:6333`
|
||||
- **Collection**: `ecosystem_research` (1024-dim Cosine, created for this skill)
|
||||
- **Embedding model**: `snowflake-arctic-embed2:latest` via Ollama at `http://localhost:11434`
|
||||
- **qdrant-client**: already installed (v1.18.0)
|
||||
|
||||
Store research findings as vectors for future deduplication:
|
||||
```python
|
||||
from qdrant_client import QdrantClient
|
||||
client = QdrantClient(url="http://10.0.0.22:6333")
|
||||
# Upsert with snowflake-arctic-embed2 embeddings (1024-dim)
|
||||
```
|
||||
|
||||
## Research Artifact Format
|
||||
|
||||
Produce structured markdown:
|
||||
|
||||
```markdown
|
||||
# Research: <Topic>
|
||||
Date: <ISO date>
|
||||
Sources: <count>
|
||||
|
||||
## Executive Summary
|
||||
<2-3 sentences>
|
||||
|
||||
## Key Findings
|
||||
### <Finding 1>
|
||||
- Source: <URL>
|
||||
- Date: <date>
|
||||
- Summary: <brief>
|
||||
|
||||
### <Finding 2>
|
||||
...
|
||||
|
||||
## Code / Projects
|
||||
| Repo | Stars | Lang | Description |
|
||||
|------|-------|------|-------------|
|
||||
| ... | ... | ... | ... |
|
||||
|
||||
## Data Points
|
||||
| Metric | Value | Source |
|
||||
|--------|-------|--------|
|
||||
| ... | ... | ... |
|
||||
|
||||
## Gaps / Uncertainties
|
||||
- <what we don't know yet>
|
||||
```
|
||||
|
||||
## Pitfalls
|
||||
|
||||
1. **Searxng rate-limiting**: Most public instances limit to ~1 req/sec. Add `sleep 2` between queries or you get 429/Too Many Requests.
|
||||
2. **Don't search for things the user already told you**: If the user says "we have an RTX 4090," don't search "does RTX 4090 exist?" Search for the thing they need *about* it (profitability, tools, config).
|
||||
3. **Instance cycling**: If one searxng instance is down, try another immediately — do not give up after the first failure.
|
||||
4. **Narrow queries beat broad ones**: `"RTX 4090 mining profitability 2025"` returns better results than `"GPU crypto"`.
|
||||
5. **Browser tools are slow**: For plain-text endpoints (`.md`, `.txt`, `.json`, raw GitHub), prefer `curl` in terminal. Use `browser_navigate` only for interactive JS-heavy pages.
|
||||
6. **Verify from multiple sources**: Mining profitability numbers, prices, and difficulty change constantly. Cross-reference whattomine.com, 2cryptocalc.com, and nicehash.com before presenting a figure as current.
|
||||
|
||||
## References
|
||||
|
||||
- `references/search-rules.md` — Complete SearXNG web search rules: hard constraints, instance details, validation pattern, and internal-knowledge vs web-search separation.
|
||||
- `references/search-instance-health.md` — Known searxng instances and their uptime/status from recent sessions
|
||||
Reference in New Issue
Block a user