hermes-skills/ltx-video-pipeline/SKILL.md

---
name: ltx-video-pipeline
description: "LTX Video pipeline on 10.0.0.202 — model chain wiring, render settings, concat, stock management, and Transition LoRA multi-scene workflows."
version: 2.0.0
author: Hermes Agent
license: MIT
platforms: [linux]
metadata:
  hermes:
    tags: [ltx, video, comfyui, pipeline, ai-video, transition-lora]
    related_skills: [ai-vid-stock, truenas, better-search]
---

# LTX Video Pipeline

End-to-end LTX Video pipeline on the ComfyUI LXC at 10.0.0.202.

## Environment

| Setting | Value |
|---------|-------|
| Target | 10.0.0.202 (Proxmox LXC 9009) |
| SSH | `sshpass -p 'passw0rd' ssh [email protected]` |
| ComfyUI | port 8188, venv ~/comfy-env, install ~/comfy-ui/ |
| GPU | RTX 4090 24GB VRAM |
| ffmpeg | /usr/bin/ffmpeg (7.1.5) |
| State file | `~/workspace/general/ltx-pipeline-state.md` |
| Continue file | `~/workspace/general/ltx-video-pipeline-continue.md` |
| Plan file | `~/workspace/general/ltx-pipeline-plan.md` |

## Model Chain

### DEFAULT: Single-Stage I2V (fp8 + TenStrip cond-safe or Distilled, NO ID LoRA) — USE THIS FOR ALL SCENES

**The 6 fixes from deep research (2026-07-22, 15 sources, v2 correction) are the baseline.** The 2-clip test confirmed: single-stage 18 steps, guide_strength 1.0, no ID LoRA, 768×512, simplified prompts — clean 241-frame output, no Director wiring bug.

**⚠️ Fix #5 (Distilled LoRA 1.0) was CORRECTED by follow-up deep research (2026-07-22, 15 sources).** Community consensus: Distilled LoRA at 1.0 causes quality degradation for I2V. The correct range is 0.5-0.7. The confusion was conflating two different parameters: I2V conditioning strength (guide_strength, should be 1.0) vs Distilled LoRA strength (should be 0.5-0.7). Sources: aistudynow.com ("Do not set the strength to 1.0"), official ComfyUI guide (uses 0.5), TenStrip experiments ("The official rank 384 LoRA can actively work against conditioned inputs").

**PREFERRED: TenStrip cond-safe LoRA (rank-72, 662 MB) at strength 1.0.** This is purpose-built for I2V — zeroes out cross-attention bridges, adaln/scale-shift tables, gate logits, and prompt scale-shift that fight I2V conditioning. "This is technically what an official I2V distilled lora should have had." Download from huggingface.co/TenStrip/LTX2.3_Distilled_Lora_1.1_Experiments — file: `ltx-2.3-22b-distilled-lora-1.1_fro90_ceil72_condsafe.safetensors`. Safe at 1.0 on first pass I2V. Upscale pass at 0.4.

**FALLBACK: Official Distilled LoRA (rank-384, 7.1 GB) at strength 0.5-0.7.** Use only if TenStrip cond-safe is not available. The official LoRA's cross-attention bridges actively fight I2V conditioning — do NOT use at 1.0.

```
UNETLoader (fp8 distilled transformer)
  → LTX2LoraLoaderAdvanced (TenStrip cond-safe rank-72, strength 1.0)  ← PREFERRED
  → LTX2LoraLoaderAdvanced (distilled-lora-384-1.1, strength 0.5-0.7) ← FALLBACK
    → LTXDirector (single-stage, 16-20 steps)
```

**Why this chain:**
- **TenStrip cond-safe at 1.0** — purpose-built for I2V, zeroes out conditioning-fighting layers. Safe at full strength.
- **NO ID LoRA** — TalkVid-3K is trained for talking-head footage, not action scenes or camera pans. It adds frontal-face bias with no upside. Drop it for ALL scenes unless doing a dedicated talking-head video.
- **Single-stage (16-20 steps)** — the two-stage refiner (4 steps at denoise 0.42) may be smearing, not refining. Single-stage at 18 steps produced clean output in the 2-clip test.
- **guide_strength 1.0** — I2V conditioning at 1.0 anchors the start frame properly. The old 0.5 was too weak, causing identity drift. The old "DO NOT use 1.0" rule was based on v4 failures where ID LoRA was ALSO at 1.0 — the duplication was from stacked LoRA interference, not from guide_strength alone. With ID LoRA dropped, guide_strength 1.0 is clean.

**ID LoRA drop rule (when keeping it for talking-head only):** Rewire node 131's `model` input from `["201", 0]` to `["200", 0]` and delete node 201.

**Full two-stage → single-stage transformation recipe:** See `references/workflow-transformation-two-to-single-stage.md` — complete node deletion list (9 nodes), rewiring table (6 connections), settings changes, timeline data updates, and verification checklist. 31 nodes → 22 nodes. First applied 2026-07-22 on the 2-clip test.

### DEPRECATED: Two-Stage I2V (fp8 + Distilled + ID) — v4/v5 era
```
UNETLoader (fp8 distilled transformer)
  → LTX2LoraLoaderAdvanced (distilled-lora-384-1.1, strength 0.7)
    → LTX2LoraLoaderAdvanced (id-lora-talkvid-3k, strength 0.6)
      → LTXDirector (two-stage: 8+4 steps)
```
**Do not use this chain for new work.** It produced artifacts in v4 and v5. Kept for reference only.

### Transition/Morph Scenes ONLY (fp8 + Transition + Distilled + ID)
```
UNETLoader (fp8 distilled transformer)
  → LoraLoaderModelOnly (ltx2.3-transition.safetensors, strength 1.0)
    → LTX2LoraLoaderAdvanced (distilled-lora-384-1.1, strength 0.7)
      → LTX2LoraLoaderAdvanced (id-lora-talkvid-3k, strength 0.6)
        → LTXDirector
```

**CRITICAL: Do NOT use the Transition LoRA chain for standard I2V scenes.** The Transition LoRA forces transformation/morphing behavior even without the `zhuanchang` trigger word. On standard I2V, it causes subject duplication and wrong-scene hallucination (confirmed: man getting out of bed split into two; kitchen scene became man+soup). Only use it for actual scene-to-scene morphing, identity transformations, or style changes.

**Transition LoRA uses standard `LoraLoaderModelOnly`, NOT `LTX2LoraLoaderAdvanced`.** It must be the FIRST LoRA in the chain (closest to UNETLoader).

## CRITICAL: Director Wiring Bug (Node 132 Latent Input)

**This bug caused 3 failed render batches (2026-07-21).** Subject duplication persisted across all settings changes because the wiring was wrong.

### The Bug

Node 132 (LTXDirectorGuide, the refiner pass) takes its `latent` input from the **uncropped** stage-1 output (`["34", 0]` — LTXVSeparateAVLatent) instead of the **cropped** output (`["55", 2]` — LTXDirectorCropGuides).

LTXDirectorGuide appends guide frames as extra latent frames, then records how many to remove. LTXDirectorCropGuides trims them. But if the refiner takes the uncropped latent, the guide frames leak through as actual video — a frozen duplicate of the subject.

### Verification

**Correct output: 240-241 frames for 10s@24fps.** The Director may produce 241 due to a 1-frame rounding quirk — this is normal. If ffprobe shows 249 frames, the bug is present (guide frames leaked). All 3 failed batches produced 249 frames; v4 success produced 241.

### The Fix

In every scene JSON, ensure:
```json
"132": {
  "inputs": {
    "latent": ["55", 2]   // NOT ["34", 0]
  }
}
```

### How It Was Found

Claude Opus SSH'd into 10.0.0.202, read the workflow JSON, read the LTXDirectorGuide source code (`ltx_director_guide.py`), checked ffprobe frame counts, and compared against the shipped reference workflow. Full diagnosis in `references/claude-wiring-diagnosis-2026-07-21.md`.

| Setting | Value | Notes |
|---------|-------|-------|
| Resolution | 768×512 | LTX trained for widescreen; 512×512 is suboptimal |
| FPS | 24 | Standard cinematic |
| Duration | 5s (121 frames) for testing, 10s (241 frames) for final | Test at 5s first — temporal coherence degrades after 5-6s |
| Sampler | euler | |
| Scheduler | **simple** | NOT linear_quadratic — distilled model is fragile with non-standard schedules |
| Steps | 16-20 single-stage | No refiner. Two-stage (8+4) may smear, not refine |
| CFG | 1.0 | Distilled model — do NOT raise for standard scenes |
| guide_strength | **1.0** | Confirmed clean in 2-clip test (2026-07-22). The old "DO NOT use 1.0" rule was from v4 where ID LoRA was also at 1.0 — the duplication was stacked LoRA interference, not guide_strength alone. With ID LoRA dropped, 1.0 is clean. |
| Distilled LoRA | **TenStrip cond-safe 1.0** (preferred) or **official 0.5-0.7** (fallback) | Official LoRA at 1.0 degrades I2V quality — use TenStrip cond-safe instead |
| ID LoRA | **Dropped** | TalkVid-3K is talking-head only; drop for all non-talking scenes |
| Peak VRAM | ~23.5GB / 24GB | |

### guide_strength — RESOLVED (2026-07-22)

**Use 1.0.** The 2-clip test confirmed: guide_strength 1.0 is clean with ID LoRA dropped. The old "DO NOT use 1.0" rule was based on v4 failures where ID LoRA was ALSO at 1.0 — the duplication was from stacked LoRA interference, not from guide_strength alone. With ID LoRA dropped, 1.0 is the correct value for proper I2V start-frame anchoring.

From the LTX Director GitHub issue #258: "Hard pin = velocity discontinuity. When a pin lands where motion is active, the static frame overrides motion mid-stream → snap." This applies when guide_strength is combined with other strong conditioning (ID LoRA, Transition LoRA). With a clean single-LoRA chain, 1.0 is safe.

**Verification:** After rendering, check the actual guide_strength used by extracting metadata:
```bash
ffprobe -v quiet -show_entries format_tags=prompt output.mp4 | grep -oP 'guide_strength.*?(\d+\.?\d*)'
```

### CFG Per Scene Type

| Scene Type | CFG | Notes |
|------------|-----|-------|
| Standard I2V (distilled) | 1.0 | Do NOT raise |
| Transition LoRA scenes | 4.0 | Required for zhuanchang to activate properly |

## Prompting Rules (CRITICAL — session 2026-07-21)

LTX-2.3 needs detailed, structured prompts. Short 1-2 sentence prompts produce bad output: subject duplication, wrong scenes entirely, static camera, identity drift, unnatural motion, flickering. The user confirmed: "I watched the video, it worked but it's all messed up. I think the issue is the prompts."

**Two confirmed failure modes from our renders:**

1. **Subject duplication** — "The man getting out of bed split into two men. One got out of bed and he was still in bed at the same time." Root cause: `guide_strength: 1.0` (hard pin) + no camera direction + no motion detail.

2. **Wrong scene entirely** — "Scene two was a man and a woman, he was scooping soup." Root cause: `zhuanchang` on a non-transition scene + prompt too short (15 words). The model filled gaps with random kitchen training data.

### Required Elements in Every Prompt

1. **Shot description** — close-up, medium shot, wide shot, low angle, tracking shot, overhead, POV
2. **Camera movement** — dolly in/out, pan left/right, tilt up/down, zoom, tracking, static, handheld, crane, Steadicam
3. **Subject action** — detailed motion: gait, speed, gestures, facial expression changes, body language. Present tense. Sequential.
4. **Environment detail** — lighting (golden hour, overcast, neon, soft studio, morning sunlight, dusk), textures, atmosphere, weather
5. **Temporal flow** — what happens first, then what changes, how the scene evolves over the 10 seconds
6. **Audio** — ambient sounds, specific SFX, music, dialogue in quotes

### Prompt Order

```
Shot + Camera → Subject + Action → Lighting + Environment → Audio + Mood
```

### Transition LoRA (`zhuanchang`)

- **Trigger word:** `zhuanchang` — append to END of prompt
- **When to use:** ONLY on scenes that involve a transformation, morph, or scene transition (present→memory, reality→vision, character morph, style change, environment transition)
- **When NOT to use:** Standard single-scene I2V shots with no transformation. Using it on every scene forces the model to try to morph when it shouldn't — this causes flickering, identity drift, and hallucinated content (confirmed: man+soup scene).
- **Strength:** 1.0
- **CFG:** 4.0 (NOT 1.0 — Transition LoRA needs higher CFG)

### Prompt Template (Standard I2V — no zhuanchang)

```
[Shot type and camera language]. [Subject and scene description]. 
[Describe the action in sequence over the full duration — what happens first, then what changes]. 
[Lighting, texture, atmosphere, composition cues]. [Audio description].
```

### Prompt Template (Transition Scene — with zhuanchang)

```
[Shot type and camera language]. [Subject and scene description]. 
[Describe the motion, transformation, or transition process in detail — how one state morphs into another]. 
[Lighting, texture, atmosphere, and composition cues]. zhuanchang
```

### Example: Bad vs Good Prompt

**Bad (what we used — produced messed up output):**
> A man wakes up in a small cabin bedroom, morning sunlight streaming through the window. He sits up slowly, rubbing his eyes., zhuanchang

**Problems:** No camera direction, no shot framing, no motion detail, no temporal flow, no audio, zhuanchang on a non-transition scene, only 25 words.

**Good (with camera, shot, motion, temporal flow, audio):**
> Medium shot, static camera. A man in his 30s lies in a rustic cabin bed, morning sunlight streaming through a window casting warm golden light across rumpled sheets. Over 10 seconds: he stirs, eyes slowly opening, then pushes himself up to sitting, rubbing his eyes with both hands. He blinks, adjusting to the light, then looks toward the window with a calm expression. Warm golden hour light, dust motes floating in sunbeams, wooden cabin interior with exposed log walls. Birds chirping outside, soft rustle of bedsheets. Photorealistic, 35mm film, shallow depth of field.

**What changed:** Added shot framing, camera type, sequential action over time, lighting detail, environment texture, audio, removed zhuanchang. 80 words vs 25.

## Artifact Prevention (DEEP RESEARCH — 2026-07-22, 18 sources)

The 7 fixes below supersede the old 6-agent review (2026-07-21). These are the new defaults.

### The 6 Fixes (ranked by impact, v2 corrected 2026-07-22)

| # | Fix | From | To | Why |
|---|-----|------|----|-----|
| 1 | ID LoRA | TalkVid-3K (talking-head) | Drop entirely | Trained for static faces, fights against action motion |
| 2 | I2V conditioning | 0.5 | 1.0 | Too weak to anchor start frame — causes identity drift |
| 3 | Stages | Two-stage (8+4 refiner) | Single-stage (16-20 steps) | Refiner may be smearing, not refining |
| 4 | Resolution | 512×512 | 768×512 | LTX trained for widescreen, square is suboptimal |
| 5 | Distilled LoRA | Official 384 @ 1.0 | **TenStrip cond-safe @ 1.0** (preferred) or official @ 0.5-0.7 (fallback) | Official LoRA fights I2V conditioning at high strength. TenStrip cond-safe zeroes out those layers. |
| 6 | Prompts | 150-200 words | 2-3 actions, 50-80 words | LTX can only execute 2-3 simultaneous actions |

**⚠️ Fix #5 was CORRECTED by follow-up deep research (2026-07-22, 15 sources).** The original claim (Distilled LoRA 0.7→1.0) was wrong. Community consensus: official Distilled LoRA at 1.0 causes quality degradation for I2V. The correct range is 0.5-0.7. TenStrip cond-safe is the solution — purpose-built for I2V at 1.0.

### Deep Research Dispatch for Artifact Investigation

When artifacts persist after applying the ranked fixes above, dispatch a focused deep-research pass with the exact setup and prompts. The pattern:

1. **Write the full research question to a file** — include: exact model chain, all render settings, all scene prompts verbatim, what's been tried (v4, v5), and specific research questions (e.g., "does ID LoRA cause artifacts on full-body action scenes?", "is fp8_scaled the worst variant?")
2. **Dispatch to research profile:** `hermes -p research -s deep-web-research chat -q "Read the full research question from <path>..." -Q --max-turns 600 --yolo`
3. **Expected output:** `/home/n8n/workspace/research/results/<YYYY-MM-DD>-ltx-2.3-artifacts-deep.md`
4. **Do NOT poll** — the `deep-research` skill's post-dispatch rule applies. Wait for the `notify_on_complete` notification.
5. **If the result file is missing after completion:** check the process log for the session_id, then check the ledger at `/tmp/research-<date>-<slug>.md`. If the ledger has only the strategy section (Move 0), the research didn't progress — re-dispatch with a fresh session. See `research-dispatch-pitfalls` Pitfall 12 for the full diagnosis workflow. Real failure (2026-07-22): deep research on LTX artifacts completed but result file never created — agent exited after Move 0.

See `references/artifact-deep-research-template.md` for the question template used in the 2026-07-22 dispatch (10 specific research questions, full setup, all 6 scene prompts).

### Fix Application Order (test after each)

1. Drop ID LoRA — zero-cost, biggest single impact
2. Switch to single-stage 16-20 steps — eliminates refiner smear risk
3. Download TenStrip cond-safe LoRA — purpose-built for I2V at 1.0
4. Set guide_strength to 1.0 — proper I2V anchoring (safe with ID LoRA dropped)
5. Switch to 768×512 — widescreen training distribution
6. Simplify prompts to 50-80 words — 2-3 actions max
7. Git pull deps (with caution — see Pitfalls: may land incompatible native workflows)

### Negative Prompt (updated)

```
no extra limbs, no face warp, no object duplication
no text artifacts, no floating logos, no watermark
no extreme motion blur, no rolling shutter wobble
no flicker, no frame-to-frame texture shift
no Dutch angle, no rapid handheld, keep horizon level
text, watermark, subtitle, logo, readable letters, garbled text
```

Added the text/watermark line — LTX-2.3's upscaler v1.0 was trained on data contaminated with endscreen logos (GitHub #148, HF discussion #13).

### Common Prompt Mistakes

| Mistake | Symptom | Fix |
|---------|---------|-----|
| guide_strength 1.0 | Subject splits into two, ghosting | Drop to 0.7 (0.5 for high motion) |
| zhuanchang on every scene | Wrong scene entirely, hallucinated content | Only use on actual transition/morph scenes |
| No camera direction | Static or random movement | Add dolly/pan/tracking/static |
| No shot framing | Inconsistent framing, zoom jumps | Add close-up/medium/wide/angle |
| Prompts too short (15-25 words) | Model fills gaps with random training data | 50-80 words minimum |
| No motion detail | Character freezes or glides unnaturally | Describe sequential action in present tense |
| No per-scene lighting | Lighting doesn't match scene context | Describe light source, quality, color temp |
| Same global prompt for all scenes | No scene-specific atmosphere | Vary lighting/audio/environment per scene |
| No temporal structure | Model doesn't know what to animate when | "Over 10 seconds: first X, then Y, finally Z" |
| Emotional labels without physical cues | Abstract expressions, no visible emotion | "Shoulders slumped, eyes downcast" not "sad" |
| Conflicting descriptions | Model averages competing signals | One speed, one camera, one lighting logic |
| Text/logos in prompts | Garbled text output | LTX cannot generate readable text |

### Prompting Workflow

1. Write the full prompt with all required elements (shot, camera, action, lighting, environment)
2. Only append `zhuanchang` if the scene involves a transformation/morph
3. Use the same character description across all scenes (from global prompt or character sheet)
4. Vary lighting and atmosphere per scene to match the setting
5. Test one scene first before rendering all 6
6. **MANDATORY: Validate every prompt against the 10-point checklist in `references/prompt-validation-checklist.md` BEFORE submitting to render queue.** This is a FIRM quality gate — do not skip. The user's standing rule: "ensure prompts match ltx standard. ALWAYS. This should be FIRM in memory and validate before EVERY run." If any check fails, fix the prompt and re-validate. Do not submit a workflow with a failing prompt.
7. Verify guide_strength is 1.0 in every scene JSON before submitting

## Story Structure (CRITICAL — session 2026-07-21)

The user's feedback on v4: "the videos had all kinds of inconsistencies" despite fixed wiring and good individual prompts. The root cause: the story didn't flow between scenes. Each scene was a standalone vignette (wake up → kitchen → porch → forest → stream → overlook) with no causal chain. LTX needs a narrative thread where each scene follows from the previous one.

### Story Design Rules

1. **Chain of events, not vignettes.** Each scene must be caused by the previous scene. "She discovers something → she runs → she hides → she chooses." Not "he wakes up → he makes coffee → he walks outside."
2. **One character, one journey.** Multi-character stories cause identity drift. The ID LoRA helps but isn't perfect across different faces.
3. **No dialogue, no plot twists.** LTX can't do dialogue well. Visual storytelling: action, reaction, environment, choice.
4. **6 scenes × 10s = 60s.** This is the sweet spot. Each scene is one beat in the story.
5. **Clear visual variety per scene.** Each scene should have a distinct setting, lighting, and camera language. This prevents the model from blending scenes together.
6. **Match LTX prompt style for EVERY scene.** See Prompting Rules above. Every scene prompt must have: shot type, camera movement, subject action (present tense, sequential), lighting, audio. No exceptions.

### Story Template

```
Scene 1: INCITING INCIDENT — character discovers/encounters something
Scene 2: ESCALATION — the situation intensifies, stakes rise
Scene 3: REACTION — character responds, makes a decision
Scene 4: CHASE/STRUGGLE — physical action, pursuit, or confrontation
Scene 5: LOW POINT — character is isolated, vulnerable, reflects
Scene 6: CHOICE/RESOLUTION — character makes the final decision, walks toward outcome
```

### User Preference: Detail Over Simplicity

The user initially asked for simpler stories, then reversed: "go back to your original." The detailed 4-story cyberpunk templates (Ghost in the Wire, Chrome Angels, The Last Human Job, Neon Baptism) are the preferred level of detail. Each story has: logline, chain of events, per-scene setting/action/camera, and start frame requirements. See `references/story-structure-guide.md` for the full 4-story templates.

## Multi-Story Workflow (CRITICAL — session 2026-07-21)

**Run one story at a time, end to end.** Do NOT batch all 4 stories. Complete each story fully before starting the next:

1. Generate start frames → wait for queue drain
2. Build scene JSONs → submit render → wait for queue drain
3. Verify frame counts (241 per scene, not 249)
4. Concat → upload to TrueNAS
5. Save all artifacts (prompts, JSONs, stock) to TrueNAS
6. Update state file
7. Only then start the next story

**Before scaling to a full 6-scene story, test with 2 clips first.** A 2-clip test (shared background, camera pan between subjects) validates the model chain, prompt quality, and transition smoothness at minimal cost before committing to a full render. See `references/claude-live-ssh-inspection.md` for the pattern of having Claude SSH into .202 to inspect live state and build the test plan.

### 2-Clip Test Pattern (Panorama + Frame-B Transition)

Proven pattern for testing continuous camera motion across clips (2026-07-22, two iterations):

1. **Generate one wide Flux panorama** (1536×512) with both subjects in a shared scene — guarantees identical background/lighting
2. **Crop 3 windows** (768×512 each): A (subject 1), B (center transition — EMPTY mid-room), C (subject 2)
3. **Clip 1:** Plain I2V from frame A, prompt drives a slow steady pan. Drop ID LoRA.
4. **Clip 2:** Plain I2V from **frame B** (the pre-rendered empty mid-room crop). Prompt continues the pan, reveals subject 2.

**CRITICAL: Do NOT use true-extend (extract last frame from Clip 1 → use as Clip 2 start).** True-extend carries latent memory of subject 1 into Clip 2. LTX hallucinates a second figure at subject 1's location even though the start frame shows empty room. Confirmed 2026-07-22: Clip 2 generated a second woman stepping up from the couch where the man was. Fix: use the pre-rendered empty frame B as Clip 2's start frame — clean start, no latent memory. Small seam risk at boundary but eliminates hallucination.

**Why frame-B transition, not FLF:** FLF (First-Last-Frame) decelerates toward its target keyframe — clip 1 slows down approaching B, clip 2 speeds up leaving it. This creates a velocity "hitch" at the seam. Frame-B transition uses plain I2V for both clips with the pre-rendered empty frame as Clip 2's anchor — constant velocity, no deceleration, no latent memory.

Full plan at `~/workspace/general/plans/2026-07-22-2clip-test-plan.md`. See `references/2clip-test-pattern.md`.

### Claude Live-SSH Inspection Pattern

When you need a plan built from live system state (not from memory or docs), have Claude SSH into .202 and inspect before planning:

1. **Write the question** to a local temp file, scp to 10.0.0.28
2. **Include SSH credentials in the prompt** — Claude needs `sshpass -p 'passw0rd' ssh [email protected]` to reach .202
3. **Tell Claude what to inspect** — model files, workflow JSONs, queue status, disk space, existing outputs
4. **Claude inspects live, then builds the plan** — it reads real workflow JSONs to understand node structure, checks which LoRAs are actually on disk, verifies queue is empty before submitting
5. **Claude writes the plan back** — it can scp files to 10.0.0.42 (the Hermes host) or save to .202 and have you retrieve them

This pattern produced the 2-clip FLF panorama plan (2026-07-22) where Claude discovered: the Distilled LoRA on disk is the 384 variant (not 384-1.1), the ID LoRA is talkvid-trained (wrong for action scenes), and FLF example workflows exist on the box. None of this was in any state file — it was discovered by live inspection.

**Pitfall:** Claude may error on first attempt (SSH timeout, tool failure). Resume the session with `--resume` and ask what went wrong — Claude self-diagnoses and recovers. The first attempt cost $1.05 (21 turns, is_error); the resume cost $0.11 (2 turns, success).

**Save and document EVERYTHING.** After each story completes, upload to TrueNAS:
- `outputs/` — concat video
- `workflows/` — scene JSONs
- `docs/` — prompt documents, research
- `start_frames/` — Flux-generated frames
- `character_refs/` — stock reference images

**Background wait pattern:** Use `terminal(background=true, notify_on_complete=true)` with a blocking poll loop on .202. The agent is notified when the queue drains — no manual polling needed. While waiting, prep the next story's prompts and stock.

## Workflow Rules

- **Always fp8.** Only fall back to Q4 GGUF if fp8 OOMs and user approves.
- **Prefer newest and best.** When choosing between an old proven path and a new better one, prefer the new one. But verify compatibility first — new doesn't mean compatible (e.g., the 2026-07-22 ComfyUI-LTXVideo update brought a better architecture that doesn't work with our fp8 models).
- **MANDATORY: Delegate ALL mechanical work to kimi-c.** Building workflow JSONs, running renders, extracting frames, submitting to queue, waiting for completion, concat, uploading — these are mechanical tasks that burn expensive deepseek tokens. Use `ask kimi-c` (peer agent on kimi-c profile) for ALL of these. The calling agent stays lean for decision-making, validation, and user communication. This is a FIRM standing rule — do not do mechanical work yourself when a cheaper peer can do it. The user's directive: "You SHOULD BE TASKING kimi c to save tokens, NOT YOU."
- **File-based dispatch for kimi-c (avoid shell quoting failures).** Multi-line prompts with quotes leak out of `hermes -q` and get parsed as CLI arguments. Write the full prompt to a temp file (e.g. `/tmp/kimi-render-pipeline.txt`), then dispatch with `hermes -p kimi-c chat -q "Read /tmp/kimi-render-pipeline.txt and execute ALL steps" -Q --max-turns 600 --yolo`. Use `terminal(background=true, notify_on_complete=true)` for any dispatch expected to take more than ~5 minutes. This is the canonical pattern — it also satisfies the "never paste file content into the prompt" rule from the ask-kimi-c skill.
- **ALWAYS validate with kimi-c after creating or updating anything.** Workflow JSONs, config changes, file writes — dispatch kimi-c to read the file and verify correctness before submitting to the render queue. Kimi-c catches wiring errors, missing nodes, and model file mismatches that would waste a 5-7 minute render. This is a quality gate, not optional.
- **Test one scene before rendering all 6.**
- **Verify frame count after every render (ffprobe).**
- **Save all artifacts to TrueNAS immediately** (user can only view videos on TrueNAS, not in LXCs).
- **Update state file as you go.**
- **Save learnings to fact_store proactively** — don't wait to be asked.
- **User prefers discussing design decisions before implementation.**

### 1. Generate Start Frames with Flux.1-dev fp8

**Proven working workflow (2026-07-21):**

Models on .202:
- UNET: `flux1-dev-fp8-e4m3fn.safetensors` in `models/diffusion_models/`
- CLIP: `t5xxl_fp8_e4m3fn.safetensors` + `clip_l.safetensors` in `models/text_encoders/`
- VAE: `ae.safetensors` in `models/vae/`

Working node graph (512×512 txt2img):
```json
{
  "5": {"class_type": "CLIPTextEncode", "inputs": {"text": "<PROMPT>", "clip": ["11", 0]}},
  "6": {"class_type": "CLIPTextEncode", "inputs": {"text": "blurry, low quality, distorted face, bad anatomy, watermark, text, logo", "clip": ["11", 0]}},
  "7": {"class_type": "FluxGuidance", "inputs": {"conditioning": ["5", 0], "guidance": 3.5}},
  "8": {"class_type": "VAEDecode", "inputs": {"samples": ["3", 0], "vae": ["10", 0]}},
  "9": {"class_type": "SaveImage", "inputs": {"filename_prefix": "ltx_start_frame_XX", "images": ["8", 0]}},
  "10": {"class_type": "VAELoader", "inputs": {"vae_name": "ae.safetensors"}},
  "11": {"class_type": "DualCLIPLoader", "inputs": {"clip_name1": "t5xxl_fp8_e4m3fn.safetensors", "clip_name2": "clip_l.safetensors", "type": "flux"}},
  "16": {"class_type": "UNETLoader", "inputs": {"unet_name": "flux1-dev-fp8-e4m3fn.safetensors", "weight_dtype": "fp8_e4m3fn"}},
  "3": {"class_type": "KSampler", "inputs": {"seed": 42, "steps": 20, "cfg": 1.0, "sampler_name": "euler", "scheduler": "simple", "denoise": 1.0, "model": ["16", 0], "positive": ["7", 0], "negative": ["6", 0], "latent_image": ["27", 0]}},
  "27": {"class_type": "EmptySD3LatentImage", "inputs": {"width": 512, "height": 512, "batch_size": 1}}
}
```

**Submission:** MUST wrap in `{"prompt": <workflow>}` envelope. The ComfyUI API rejects bare workflow JSON with `"no_prompt"` error.

```bash
# Build JSON, then submit:
curl -s -X POST http://localhost:8188/prompt -H "Content-Type: application/json" -d @/tmp/flux_scene_01.json
```

**Output:** Files land in `~/comfy-ui/output/` as `<filename_prefix>_00001_.png`. Move to `~/comfy-ui/input/` for LTX Director to find them.

- Use Flux.1-dev fp8 or Qwen-Image-2512 on .202
- 512×512, photorealistic
- Save to `~/comfy-ui/input/`

### 2. Generate TTS Audio
- Edge TTS (free) or Piper
- ~2-4s per scene
- Save to `/tmp/ltx_audio/`

### 3. Build Scene JSONs
- Template: use an existing scene JSON from a prior render
- Modify: start_frame path, audio path, segment prompt
- For Transition LoRA: add `LoraLoaderModelOnly` node, rewire chain, append `zhuanchang` to prompts
- Save to `/tmp/api_scene_XX.json`

### 4. Submit to Queue
```bash
curl -s -X POST http://localhost:8188/prompt -H "Content-Type: application/json" -d @/tmp/api_scene_XX.json
```
- Submit all scenes at once
- Queue processes sequentially (one GPU)

### 5. Wait for Completion
- Poll queue: `curl -s http://localhost:8188/queue | python3 -c "import json,sys; d=json.load(sys.stdin); print(len(d['queue_running']), len(d['queue_pending']))"`
- GPU: `nvidia-smi --query-gpu=utilization.gpu,memory.used --format=csv,noheader`
- ~5-7 min per scene at 8 steps
- Blocking wait loop (from Hermes host):
```bash
sshpass -p 'passw0rd' ssh [email protected] "while true; do q=\$(curl -s http://localhost:8188/queue | python3 -c \"import json,sys; d=json.load(sys.stdin); print(len(d['queue_running'])+len(d['queue_pending']))\"); if [ \"\$q\" -eq 0 ]; then echo 'QUEUE_EMPTY'; break; fi; echo \"queue: \$q\"; sleep 30; done"
```

### 6. Identify Output Files
```bash
python3 << 'PYEOF'
import subprocess, os, re
outdir = os.path.expanduser("~/comfy-ui/output/video")
for f in sorted(os.listdir(outdir)):
    if not f.startswith("LTX_Director_"): continue
    path = os.path.join(outdir, f)
    r = subprocess.run(["ffprobe", "-v", "quiet", "-show_entries", "format_tags=prompt", "-of", "csv=p=0", path], capture_output=True, text=True)
    img = re.search(r"imageFile.*?ltx_start_frame_(\d+)", r.stdout)
    scene = f"Scene_{img.group(1)}" if img else "?"
    has_trans = "LoraLoaderModelOnly" in r.stdout
    print(f"{f} | {scene} | Transition={has_trans}")
PYEOF
```

### 7. Concat with ffmpeg
```bash
# Create concat list
cat > /tmp/concat.txt << EOF
file '/home/n8n/comfy-ui/output/video/LTX_Director_XXXXX_.mp4'
file '/home/n8n/comfy-ui/output/video/LTX_Director_YYYYY_.mp4'
...
EOF

# Stream copy (no re-encode)
ffmpeg -f concat -safe 0 -i /tmp/concat.txt -c copy ~/comfy-ui/output/video/LTX_Director_60s.mp4 -y
```

### 8. Save to Stock
- **smbclient is NOT on .202** — scp files to Hermes host first, then upload to TrueNAS from there
- Upload outputs: `smbclient -N //10.0.0.117/proxmoxBackup -c 'cd ai_vid_stock_material\\outputs; put <local_path> <filename>'`
- Upload workflow JSONs: `smbclient -N //10.0.0.117/proxmoxBackup -c 'cd ai_vid_stock_material\\workflows; put <local_path> <filename>'`
- Update README.md inventory
- See `ai-vid-stock` skill for full commands

## Stock Materials

All stock materials (start frames, audio, character refs, workflows, outputs) are stored on TrueNAS at `//10.0.0.117/proxmoxBackup/ai_vid_stock_material/`. Use the `ai-vid-stock` skill for add/search/remove operations.

**Known gap (2026-07-21):** `better-search` finds sources but doesn't download them to stock. The research agent reports URLs; the dispatcher agent must manually download and upload. Future: extend `better-search-research` methodology to accept a `--target` parameter and auto-download top results to the specified stock directory.

**Character reference sources (from better-search research):**
- freeaivideohub.com/character-sheets — 12 free photorealistic multi-view character sheets
- Pexels API — 700K+ free portrait photos, commercial use
- FFHQ — 70K faces at 1024×1024 (NVIDIA, non-commercial)
- CivitAI — community LoRAs and LTX workflows

## Pitfalls

- **CRITICAL: True-extend (extract last frame → use as next clip start) carries latent memory.** LTX remembers subjects from prior clips even when the start frame shows empty space. Confirmed 2026-07-22: Clip 2 hallucinated a second woman stepping up from the couch where the man was in Clip 1. Fix: use the pre-rendered empty mid-room frame B as Clip 2's start frame instead of the extracted last frame. Clean start, no latent memory. See §2-Clip Test Pattern.
- **ffmpeg last-frame extraction:** `ffmpeg -sseof -3 -i input.mp4 -vsync vfr -q:v 2 output.png` may fail. Working alternative: `ffmpeg -sseof -3 -i input.mp4 -update 1 -frames:v 1 -q:v 2 output.png`. The `-update 1` flag is needed for single-frame PNG output from video input.
- **CRITICAL: git pull on ComfyUI-LTXVideo may land incompatible native workflows.** The 2026-07-22 update (30K lines) brought a new native single-stage architecture (CheckpointLoaderSimple, LTXVScheduler, SamplerCustomAdvanced, GemmaAPITextEncode) that uses full checkpoints (46 GB), not our fp8 UNET-only transformer (23 GB). The new workflow drops LTXDirector entirely. Our fp8 chain (UNETLoader → LTX2LoraLoaderAdvanced → LTXDirector) is incompatible with the new architecture. **Before git pulling, snapshot the working state.** If the pull lands new example workflows, do NOT assume they work with our fp8 models — verify compatibility first. The new architecture requires a full checkpoint that won't fit 24 GB VRAM.
- **CRITICAL: Upload to TrueNAS after EVERY render.** The user can only view videos on TrueNAS (10.0.0.117), not in LXC containers. scp from .202 to Hermes host, then smbclient to TrueNAS. Do NOT skip this — the user has no other way to review output. (Learned 2026-07-22: user said "I can only view videos on truenas, not in lxc's.")
- **CRITICAL: Do NOT interrupt a running Claude session.** If Claude is mid-build (background process running), do not scp new question files or send follow-ups unless the user explicitly says "tell Claude now." Wait for the session to complete. Mid-build interruptions can cause the session to error out. (Learned 2026-07-22: sent a fact_store instruction mid-build; user corrected: "I said when claude is complete.")
- **CRITICAL: Claude's intermediate errors are normal self-correction.** Claude will make mistakes, hit errors, then try a different approach. `is_error: true` on intermediate turns does NOT mean the session failed. Wait for the final turn output. Only diagnose after the background process completes. Jumping to conclusions mid-session wastes turns and money. (Learned 2026-07-22: killed a working session twice, $1.78 wasted. User: "you just need to wait for the final turn output. Not jump to conclusions.")
- **Transition LoRA uses LoraLoaderModelOnly, not LTX2LoraLoaderAdvanced.** Wiring it wrong silently fails — the workflow runs but produces no transition effect.
- **`zhuanchang` trigger word is required** for Transition LoRA to activate. Append to end of segment prompts.
- **CFG must be 1.0** for distilled model. Higher values cause artifacts.
- **Queue is sequential** — submitting 6 scenes means ~30-40 min total. Use the blocking wait loop from step 5.
- **Output files accumulate** — the Director workflow produces 2 outputs per scene (guide pass + main pass). Identify the final output by checking for `LoraLoaderModelOnly` in metadata (Transition batch) or by resolution (512×512 = fp8 batch).
- **cifs kernel module not available** on Hermes host — use smbclient for TrueNAS, not mount.cifs.
- **smbclient is NOT installed on .202** — scp files to Hermes host first, then upload to TrueNAS from there. Do not try to install smbclient on .202 (no root access).
- **State file is the crash-recovery artifact** — update `~/workspace/general/ltx-pipeline-state.md` as you go. A new session reads it to resume.
- **ComfyUI API requires `{"prompt": <workflow>}` envelope.** Submitting bare workflow JSON returns `"no_prompt"` error. Always wrap in the prompt key.
- **241 frames is normal for 10s@24fps.** The Director may produce 241 due to a 1-frame rounding quirk. 249 frames means the wiring bug is present (guide frames leaked). Verify with ffprobe.
- **Duration control: set `segment[\"length\"]`, NOT `duration_seconds` or `segment[\"end\"]`.** The LTXDirector ignores node-level `duration_seconds` and segment `end` — it uses `segment[\"length\"]` as the frame count. For 5s@24fps: `td[\"segments\"][0][\"length\"] = 120`. Setting only `duration_seconds` or `end` silently produces the full 10s output. Confirmed 2026-07-21: two failed attempts before finding the correct field.
- **Story must have a causal chain.** Standalone vignettes (wake up → kitchen → porch → forest) produce visual inconsistencies because the model has no narrative thread. Each scene must follow from the previous one. See `references/story-structure-guide.md` for the full 4-story templates and design rules.
- **User prefers detailed stories over simplified ones.** When the user said "keep it simple" and then "go back to your original," the original detailed templates were preferred. Don't over-simplify — the 4-story cyberpunk templates with loglines, scene tables, and character descriptions are the right level of detail.