--- name: ltx-video-pipeline description: "LTX Video pipeline on 10.0.0.202 — model chain wiring, render settings, concat, stock management, and Transition LoRA multi-scene workflows." version: 2.2.0 author: Hermes Agent license: MIT platforms: [linux] metadata: hermes: tags: [ltx, video, comfyui, pipeline, ai-video, transition-lora] related_skills: [ai-vid-stock, truenas, better-search] --- # LTX Video Pipeline End-to-end LTX Video pipeline on the ComfyUI LXC at 10.0.0.202. ## Naming Convention (FIRM — user directive 2026-07-22) All output files uploaded to TrueNAS MUST use this format: ``` LTX_YYYY-MM-DD___.mp4 ``` Examples: - `LTX_2026-07-22_cyberpunk-boss-pan_euler_10s.mp4` - `LTX_2026-07-22_cyberpunk-boss-pan_euler-ancestral_5s.mp4` - `LTX_2026-07-22_boss-stock-test_euler-ancestral_10s.mp4` Components: - `YYYY-MM-DD`: render date - ``: short kebab-case description of content (e.g., `cyberpunk-boss-pan`, `boss-stock-test`, `woman-window-reveal`) - ``: `euler` or `euler-ancestral` - ``: `5s` or `10s` Do NOT use generic names like `LTX_cyberpunk_C1_00005_.mp4` or `LTX_2clip_test_C1_00002_.mp4` — these are ComfyUI auto-generated filenames that convey zero information about what's in the video. Rename during the concat/upload step. ## Character Descriptions See `references/character-descriptions.md` for the canonical character descriptions used in prompts and Flux panorama generation. These are the ground truth — do not improvise or use generic descriptions. ## Environment | Setting | Value | |---------|-------| | Target | 10.0.0.202 (Proxmox LXC 9009) | | SSH | `sshpass -p 'passw0rd' ssh n8n@10.0.0.202` | | ComfyUI | port 8188, venv ~/comfy-env, install ~/comfy-ui/ | | GPU | RTX 4090 24GB VRAM | | ffmpeg | /usr/bin/ffmpeg (7.1.5) | | State file | `~/workspace/general/ltx-pipeline-state.md` | | Continue file | `~/workspace/general/ltx-pipeline-continue.md` | | Plan file | `~/workspace/general/ltx-pipeline-plan.md` | ## Resolution Limits ### .202 (RTX 4090 24GB, fp8 UNET, spatial x2 upscaler) | Resolution | Latent | VRAM Est. | Feasible | |---|---|---|---| | 1280×720 (720p) | 640×360 | ~20GB | Yes — same ballpark as 1280×736 | | 1408×768 | 704×384 | ~21-22GB | Probably — tight but fp8 helps | | 1536×864 | 768×432 | ~22-23GB | Maybe — right at the edge | | 1920×1080 (1080p) | 960×540 | OOM | No — 24GB not enough | Higher res = longer renders (roughly linear with pixel count). 1408×768 is ~15% more pixels than 1280×736, adds ~3-4 min per scene. The spatial upscaler x2 means the model works at half res — that's the real bottleneck, not output pixels. ### DGX (10.0.0.6, 128GB unified, bf16 UNET 42GB) Can push to 1920×1080 or higher. bf16 UNET + tile 1408 already in the DGX max quality workflow. See `references/dgx-spark-comfyui.md`. ## DGX Workflow Porting (.202 → DGX) When porting workflow modifications from .202 to DGX, the changes are parameter-level — nothing structural depends on fp8 vs bf16 or single vs multi-GPU: **What ports (same changes, different workflow file):** - Variable durations (EmptyLTXVLatentVideo length values) - Rewritten prompts (text node strings) - Per-scene save_output=true (VHS VideoCombine) - Brand names stripped, static wardrobe removed **What stays DGX-specific (don't touch):** - bf16 UNET + bf16 Gemma model loaders - LTXVSequenceParallelMultiGPUPatcher - tile 1408 resolution - Different VAE/CLIP loader nodes **Process:** Node types are the same (LTXVConditioning, VHS_VideoCombine, EmptyLTXVLatentVideo) — just wired to different model files. The DGX workflow has different node ID numbering, so changes must be mapped to the correct node IDs. Load the DGX workflow in ComfyUI UI, identify the right nodes, apply the same edits. **⚠️ CORRECTED 2026-07-30: DGX and .202 workflows have IDENTICAL structure.** Both use the same single-render pattern: 1 EmptyLatent, 1 LTXVConditioning, 8 LoadImage, 9 VHS nodes. The initial assumption of a structural mismatch was wrong — confirmed by byte-level node inventory diff. Porting changes from .202 to DGX is parameter-only: PROMPT text, LENGTH values, VHS save_output/prefix. No restructuring needed. The 3 DGX quality differences (bf16 UNET, bf16 Gemma CLIP, MultiGPU patcher) are the ONLY structural differences. See `plans/2026-07-30-dgx-fullhqhours.md` for the corrected plan. ## Related Skills & References - **Official ComfyUI skill (v5.1.0)** — enabled in general profile at `~/.hermes/profiles/general/skills/creative/comfyui/`. 36 files: 10 scripts, 7 workflows, 4 references, 7 tests. The execution scripts (`run_workflow.py`, `run_batch.py`, `extract_schema.py`, `check_deps.py`, `ws_monitor.py`) are directly useful for LTX pipeline work — they handle param injection, submission, monitoring, and output download. Use `--host http://10.0.0.202:8188` for remote execution (scripts default to 127.0.0.1:8188). **`fetch_logs.py` is essential for debugging failed renders** — retrieves full Python tracebacks from the ComfyUI history API. See §Debugging Failed Renders. The `comfy-cli` lifecycle commands (install, launch, stop) are NOT needed — we use systemd. Hub install blocked by security scanner (25 findings, community source) — manually copied from bundled location instead. - **Consolidated ComfyUI reference** — `~/workspace/general/comfy_skill.md` (11 sections, 51 chunks in ai_brain_kb). Covers all ComfyUI knowledge: environment, update procedure, API format, Open WebUI integration, LTX pipeline, WanVideo, SageAttention, fact_store, AI brain, and the official skill. Search ai_brain_kb with natural language queries for any ComfyUI topic. - **`ai-brain-kb` skill** — manage the `ai_brain_kb` Qdrant collection. After every deep research run or significant session, add result files to the brain for cross-session semantic search. - **`holographic-memory` skill** — fact_store for real-time working memory. Save pipeline discoveries, bug fixes, and configuration details proactively during sessions. - **LTX Director 2.0 full reference** — `references/ltx-director-2.0-how-it-works.md` — timeline_data schema, keyframe loading from source code, end frames feature (native FLF2V), two-stage pipeline architecture, model list, example workflows, storybook pipeline integration pattern. From README + source code + hotfix workflow JSON (2026-07-29). - **LTX Director best practices (July 2026 research)** — `references/ltx-director-best-practices-2026-07-29.md` — 18-source deep research: prompting rules, keyframe best practices, timeline standards, render settings, current model recommendations, known bugs. Supersedes older prompting and settings assumptions. - **Qwen Image prompting (July 2026 research)** — `references/qwen-image-prompting-2026-07-29.md` — 18-source deep research: txt2img structured prompt format, img2img natural language instructions, character consistency workflow, Next-Scene LoRA usage, Lightning 4-step trade-offs, common pitfalls. For generating photorealistic keyframes for LTX video stories. - **DGX Spark ComfyUI** — `references/dgx-spark-comfyui.md` — full infrastructure reference for the DGX Spark (10.0.0.6): access, Docker stack, Manager versioning pitfall (V3.41 trap), models, custom nodes, torch update constraints, and fleet context. Check DGX before pulling models from HuggingFace — LAN copies are faster. ## Model Chain ### DEFAULT: Single-Stage I2V (dev-fp8 + TenStrip cond-safe, NO ID LoRA) — CONFIRMED CLEAN 2026-07-22 **✅ USER-CONFIRMED BASELINE (Fix 1b, 2026-07-22):** dev-fp8 + TenStrip cond-safe @ 1.0, LTXDirector chain, guide_strength 0.80, euler/simple 18 steps, CFG 1.0, 768×512, 121 frames, no NAG (ConditioningZeroOut). Single-clip test produced clean output. 2-clip test (boss + woman, frame-B transition) also passed — both clips clean, 242-frame concat, user-approved. This is the production baseline. **⚠️ CRITICAL: The TenStrip cond-safe LoRA is a DISTILLATION LoRA — it MUST be paired with the DEV base, NOT the distilled base.** Running cond-safe on the already-distilled transformer causes double-distillation: structure loss, degraded conditioning fidelity, and increased hallucination (closet doors, second characters, frozen objects). From the model page: "Applying them on top of an already-distilled model double-distills and degrades quality." See `references/double-distillation-artifacts.md` for the full root cause analysis (2026-07-22, Grok diagnosis). **HARD RULES (from failures — do not violate):** 1. **LTXDirector chain ONLY** — SamplerCustomAdvanced/Dev_FullSteps incompatible with fp8 UNET (produces blur of colors, no video) 2. **NAG (LTX2_NAG) crashes with MultimodalGuider** (GitHub #535, tensor 1024 vs 6144) — use ConditioningZeroOut for negative prompts 3. **euler_ancestral_cfg_pp FAILS with fp8** — use plain euler 4. **dev-fp8 + cond-safe is the correct pairing** (NOT distilled + cond-safe) 5. **Always debug failed renders with ComfyUI history API:** `curl HOST:8188/history/` — returns full Python traceback with node ID, exception type, and error message. See §Debugging Failed Renders. **PREFERRED: TenStrip cond-safe LoRA (rank-72, 662 MB) at strength 1.0, paired with dev-fp8 UNET.** This is purpose-built for I2V — zeroes out cross-attention bridges, adaln/scale-shift tables, gate logits, and prompt scale-shift that fight I2V conditioning. "This is technically what an official I2V distilled lora should have had." Download from huggingface.co/TenStrip/LTX2.3_Distilled_Lora_1.1_Experiments — file: `ltx-2.3-22b-distilled-lora-1.1_fro90_ceil72_condsafe.safetensors`. Safe at 1.0 on first pass I2V. Upscale pass at 0.4. **FALLBACK: Official Distilled LoRA (rank-384, 7.1 GB) at strength 0.5-0.7, paired with distilled fp8 UNET.** Use only if TenStrip cond-safe is not available. The official LoRA's cross-attention bridges actively fight I2V conditioning — do NOT use at 1.0. **ONLY Architecture: LTXDirector (our proven chain)** ``` UNETLoader (dev-fp8 transformer) ← MUST be dev, not distilled → LTX2LoraLoaderAdvanced (TenStrip cond-safe rank-72, strength 1.0) ← PREFERRED → LTXDirector (single-stage, 16-20 steps) ``` Uses `guide_strength` parameter (0.80 confirmed). Has timeline_data segments, Director Guide, Director Crop Guides. Produces 121 frames (5.04s) or 241 frames (10s). Frame count verification: 249 = wiring bug. **⚠️ Dev_FullSteps native workflow (SamplerCustomAdvanced) is INCOMPATIBLE with fp8 UNET-only.** Confirmed 2026-07-22: Fix 1 test used Dev_FullSteps (SamplerCustomAdvanced, BasicScheduler, KSamplerSelect, CFGGuider, LTX2_NAG) with dev-fp8 UNET — output was a blur of colors, no actual video. The native workflow requires a full checkpoint (46GB). Our fp8 UNET-only setup (23GB) MUST use the LTXDirector chain. Do NOT use SamplerCustomAdvanced, BasicScheduler, or KSamplerSelect with fp8 UNET-only. The NAG+MultimodalGuider incompatibility (GitHub #535) was a secondary issue — even with CFGGuider workaround, the native workflow produces garbage with fp8 UNET-only. **ALTERNATE (no cond-safe available):** ``` UNETLoader (distilled fp8 transformer) → LTX2LoraLoaderAdvanced (distilled-lora-384-1.1, strength 0.5-0.7) ← FALLBACK → LTXDirector (single-stage, 16-20 steps) ``` **Why this chain:** - **dev-fp8 + TenStrip cond-safe at 1.0** — correct pairing. Cond-safe is a distillation LoRA designed for the dev base. Together they produce the cleanest I2V conditioning. - **NO ID LoRA** — TalkVid-3K is trained for talking-head footage, not action scenes or camera pans. It adds frontal-face bias with no upside. Drop it for ALL scenes unless doing a dedicated talking-head video. - **Single-stage (16-20 steps)** — the two-stage refiner (4 steps at denoise 0.42) may be smearing, not refining. Single-stage at 18 steps produced clean output in the 2-clip test. - **guide_strength 0.75-0.85** — community-recommended range for multi-clip pans. The old 1.0 value was tested on the broken double-distillation setup; the correct value for dev-fp8 + cond-safe is being validated (2026-07-22 test in progress at 0.80). **ID LoRA drop rule (when keeping it for talking-head only):** Rewire node 131's `model` input from `["201", 0]` to `["200", 0]` and delete node 201. **Full two-stage → single-stage transformation recipe:** See `references/workflow-transformation-two-to-single-stage.md` — complete node deletion list (9 nodes), rewiring table (6 connections), settings changes, timeline data updates, and verification checklist. 31 nodes → 22 nodes. First applied 2026-07-22 on the 2-clip test. ### DEPRECATED: Two-Stage I2V (fp8 + Distilled + ID) — v4/v5 era ``` UNETLoader (fp8 distilled transformer) → LTX2LoraLoaderAdvanced (distilled-lora-384-1.1, strength 0.7) → LTX2LoraLoaderAdvanced (id-lora-talkvid-3k, strength 0.6) → LTXDirector (two-stage: 8+4 steps) ``` **Do not use this chain for new work.** It produced artifacts in v4 and v5. Kept for reference only. ### Transition/Morph Scenes ONLY (fp8 + Transition + Distilled + ID) ``` UNETLoader (fp8 distilled transformer) → LoraLoaderModelOnly (ltx2.3-transition.safetensors, strength 1.0) → LTX2LoraLoaderAdvanced (distilled-lora-384-1.1, strength 0.7) → LTX2LoraLoaderAdvanced (id-lora-talkvid-3k, strength 0.6) → LTXDirector ``` **CRITICAL: Do NOT use the Transition LoRA chain for standard I2V scenes.** The Transition LoRA forces transformation/morphing behavior even without the `zhuanchang` trigger word. On standard I2V, it causes subject duplication and wrong-scene hallucination (confirmed: man getting out of bed split into two; kitchen scene became man+soup). Only use it for actual scene-to-scene morphing, identity transformations, or style changes. **Transition LoRA uses standard `LoraLoaderModelOnly`, NOT `LTX2LoraLoaderAdvanced`.** It must be the FIRST LoRA in the chain (closest to UNETLoader). ## CRITICAL: Director Wiring Bug (Node 132 Latent Input) **This bug caused 3 failed render batches (2026-07-21).** Subject duplication persisted across all settings changes because the wiring was wrong. ### The Bug Node 132 (LTXDirectorGuide, the refiner pass) takes its `latent` input from the **uncropped** stage-1 output (`["34", 0]` — LTXVSeparateAVLatent) instead of the **cropped** output (`["55", 2]` — LTXDirectorCropGuides). LTXDirectorGuide appends guide frames as extra latent frames, then records how many to remove. LTXDirectorCropGuides trims them. But if the refiner takes the uncropped latent, the guide frames leak through as actual video — a frozen duplicate of the subject. ### Verification **Correct output: 240-241 frames for 10s@24fps.** The Director may produce 241 due to a 1-frame rounding quirk — this is normal. If ffprobe shows 249 frames, the bug is present (guide frames leaked). All 3 failed batches produced 249 frames; v4 success produced 241. ### The Fix In every scene JSON, ensure: ```json "132": { "inputs": { "latent": ["55", 2] // NOT ["34", 0] } } ``` ### How It Was Found Claude Opus SSH'd into 10.0.0.202, read the workflow JSON, read the LTXDirectorGuide source code (`ltx_director_guide.py`), checked ffprobe frame counts, and compared against the shipped reference workflow. Full diagnosis in `references/claude-wiring-diagnosis-2026-07-21.md`. ### Single-Clip I2V (OLD BASELINE — dev-fp8 + cond-safe, single-stage) These settings are for standalone single-clip I2V renders using the dev-fp8 + TenStrip cond-safe chain. For multi-scene Director timelines, see §Config Split below. | Setting | Value | Notes | |---------|-------|-------| | Resolution | 768×512 | LTX trained for widescreen; 512×512 is suboptimal | | FPS | 24 | Standard cinematic | | Duration | 5s (121 frames) for testing, 10s (241 frames) for final | Test at 5s first — temporal coherence degrades after 5-6s | | Sampler | **euler** (ONLY — mandatory) | euler_ancestral_cfg_pp FAILS catastrophically with fp8 distilled model (hallucinated subjects, backwards limbs). Plain euler only. See §Sampler below. | | Scheduler | **simple** | NOT linear_quadratic — distilled model is fragile with non-standard schedules | | Steps | 16-20 single-stage | No refiner. Two-stage (8+4) may smear, not refine | | CFG | 1.0 | Distilled model — do NOT raise for standard scenes | | guide_strength | **0.80** (confirmed clean) | 2-clip cyberpunk test 2026-07-22: 0.80 with dev-fp8 + cond-safe produced clean output, no artifacts, smooth pan continuity. Range 0.75-0.85 is safe. The old 1.0 value was tested on the BROKEN double-distillation setup (distilled + cond-safe) — the artifacts were attributed to guide_strength but the root cause was the model/LoRA mismatch. | | Distilled LoRA | **TenStrip cond-safe 1.0** (preferred) or **official 0.5-0.7** (fallback) | Official LoRA at 1.0 degrades I2V quality — use TenStrip cond-safe instead | | ID LoRA | **Dropped** | TalkVid-3K is talking-head only; drop for all non-talking scenes | | Peak VRAM | ~23.5GB / 24GB | | ### Multi-Scene Director (PRODUCTION — distilled-1.1 fp8, two-stage hotfix workflow) These are the settings from the WhatDreamsCost hotfix workflow and the MShahzadAbdulmajeed 11-segment community example. Use these for storybook/multi-scene Director timelines. | Setting | Value | Notes | |---------|-------|-------| | Resolution | 768×512 | Must be divisible by 32 | | FPS | 24 | Standard cinematic | | Segment duration | ~72 frames / 3s | Community standard from 11-segment example | | Stage 1 steps | **10** | euler, linear_quadratic, denoise 1.0 | | Stage 2 steps | **10** | euler, linear_quadratic, denoise 0.42, spatial x2 upscale | | CFG | 1.0 | Distilled model | | guide_strength | **1.00** | Used across all 11 segments in the community-proven example. Lower = more freedom, higher = tighter anchor. | | UNET | **distilled-1.1 int8_convrot** (NEW Jul 2026) | Faster, less VRAM than fp8_scaled. Fallback: distilled-1.1 fp8_scaled | | Distilled LoRA | Kijai compressed rank-111 (2.74GB) or official 384-1.1 (7.61GB) | | | Spatial Upscaler | **x2-1.1** (MANDATORY) | v1.0 has logo corruption and flickering | | Peak VRAM | ~23.5GB / 24GB | | ### Sampler — `euler` for distilled, `euler_ancestral` for dev (UPDATED 2026-07-22) **`euler_ancestral_cfg_pp` FAILS with fp8 distilled model.** Confirmed 2026-07-22: 2-clip test produced catastrophic artifacts — hallucinated wrong man, woman falls face-first, legs turn backwards. Community consensus (RuneXX) was correct: `euler_ancestral_cfg_pp` is for the full dev model (non-distilled, CFG>1), not our fp8 distilled pipeline. **With dev-fp8 + cond-safe (correct pairing):** `euler_ancestral` may work. The Dev_FullSteps workflow uses `euler_ancestral` for pass 1 and `euler` for pass 2. Test `euler_ancestral` on the dev-fp8 + cond-safe setup — if it produces clean output, prefer it for better temporal coherence. If artifacts appear, fall back to plain `euler`. **For distilled fp8 (fallback path):** Use plain `euler` ONLY. Scheduler: `simple`. Steps: 8. CFG: 1.0. ### Duration — 5s (121 frames) Sweet Spot (VALIDATED 2026-07-22) Community sweet spot is 5-6s (121f requested). 10s risks slow-motion. Frame count must be 8n+1 — 121 works (8×15+1). Change `duration_frames: 240` → `121`, `end_frame: 240` → `121`, `duration_seconds: 10.0` → `5.04`. Keep steps=18, scheduler=simple, denoise=1. **Actual output is 121 frames at 5.04s (24fps).** With 121 requested frames and the Fix 1b single-stage LTXDirector chain (dev-fp8 + cond-safe, euler/simple 18 steps), ffprobe confirms exactly 121 frames, 5.04s. The 8n+1 rounding quirk (producing 129 frames) was observed on the older two-stage workflow — the single-stage chain produces the exact requested frame count. For 10s renders (241 requested), output is 241 frames. Confirmed in 2-clip cyberpunk test 2026-07-22: 121 frames per clip, 242 total after concat, 10.08s, clean output with TenStrip cond-safe + euler. ## Debugging Failed Renders When a render produces garbage output (blur of colors, no video, wrong content), use the ComfyUI history API to get the full traceback. The bundled `comfyui` skill (v5.1.0) provides `fetch_logs.py` for this. ### Quick diagnosis (from Hermes host) ```bash # 1. Get recent history entries sshpass -p 'passw0rd' ssh n8n@10.0.0.202 "curl -s http://localhost:8188/history | python3 -c \" import json, sys d = json.load(sys.stdin) for pid, entry in sorted(d.items(), key=lambda x: x[1].get('timestamp',0) if isinstance(x[1],dict) else 0, reverse=True)[:5]: if not isinstance(entry, dict): continue status = entry.get('status', {}) print(f'{pid}: status={status.get(\\\"status_str\\\",\\\"?\\\")} completed={status.get(\\\"completed\\\",\\\"?\\\")}') \"" # 2. Get full traceback for a failed prompt_id sshpass -p 'passw0rd' ssh n8n@10.0.0.202 "curl -s http://localhost:8188/history/ | python3 -c \" import json, sys d = json.load(sys.stdin) entry = d.get('', {}) status = entry.get('status', {}) for msg in status.get('messages', []): if isinstance(msg, list) and msg[0] == 'execution_error': err = msg[1] print(f\\\"Node: {err.get('node_id')} ({err.get('node_type')})\\\") print(f\\\"Error: {err.get('exception_message')}\\\") print(f\\\"Type: {err.get('exception_type')}\\\") \"" ``` ### Using the bundled comfyui skill's fetch_logs.py The official comfyui skill at `~/.hermes/profiles/general/skills/creative/comfyui/scripts/fetch_logs.py` provides a cleaner interface: ```bash sshpass -p 'passw0rd' ssh n8n@10.0.0.202 "cd ~/comfy-ui && python3 /path/to/fetch_logs.py " ``` ### Common failure signatures | Symptom | Likely cause | Check | |---------|-------------|-------| | Blur of colors, no video | SamplerCustomAdvanced with fp8 UNET-only | Use LTXDirector chain instead | | `tensor a (1024) must match tensor b (6144)` | NAG + MultimodalGuider incompatibility (#535) | Use CFGGuider with cfg=1.0 | | 249 frames instead of 241 | Director wiring bug (guide frames leaked) | Node 132 latent must be ["55", 2] | | Hallucinated second character | Double-distillation (cond-safe on distilled) | Switch UNET to dev-fp8 | | Frozen objects, static props | guide_strength too high | Drop to 0.75-0.85 | Director 2.0 is already on .202 (commit 0dfa657). We use the LTXDirector node — 2.0 is the same node with new features. Key additions relevant to our pipeline: ### NAG (Negative-Aware Guidance) — INCOMPATIBLE, DO NOT USE **NAG is incompatible with our fp8 setup.** Confirmed 2026-07-22: LTX2_NAG crashes with MultimodalGuider (GitHub #535, tensor size mismatch 1024 vs 6144). Even with CFGGuider workaround, the Dev_FullSteps native workflow that NAG requires is itself incompatible with fp8 UNET-only (produces blur of colors). **Use ConditioningZeroOut for negative prompts instead.** This is the confirmed production approach — simple, stable, no compatibility issues. NAG may be re-evaluated if we ever switch to a full checkpoint (46GB) that supports the native workflow, but for fp8 UNET-only it's a dead end. ### IC-LoRA Ingredients (Character Consistency) — CONFIRMED WORKING 2026-07-22 LTX-2.3-22b-IC-LoRA-Ingredients model for character consistency via reference style sheet. Downloaded on .202 (1.31GB). **CONFIRMED WORKING with fp8 UNET-only + LTXDirector.** The `LTXICLoRALoaderModelOnly` node accepts standard MODEL input from UNETLoader — no full checkpoint needed. Grok confirmed compatibility; single-clip boss test rendered successfully 2026-07-22. **Model chain with IC-LoRA:** ``` UNETLoader (dev-fp8) → LTX2LoraLoaderAdvanced (TenStrip cond-safe rank-72, strength 1.0) → LTXICLoRALoaderModelOnly (Ingredients, strength 1.0) → LTXDirector ``` **IC-LoRA settings (hybrid, from Grok):** | Setting | Value | Notes | |---------|-------|-------| | Steps | 24 | Up from baseline 18 | | CFG | 2.5 | Up from baseline 1.0 | | IC-LoRA strength | 1.0 | Start here, raise to 1.2-1.3 if identity weak | | cond-safe strength | 1.0 | Unchanged | | guide_strength | 0.80 | Unchanged | | Resolution | 768×448 | IC-LoRA trained bucket (768×512 works but expect mild drift) | | Sampler | euler, simple scheduler | Unchanged | **Reference sheet requirements:** - Multi-panel composite on black background: face close-up + body turnaround + key props - Resize to 768×448, loop to 121-frame static video at 24fps - Save to `~/comfy-ui/input/` on .202 - Boss sheet: `SHEETS2_00005_Boss.png` (3328×2432) on TrueNAS → resize → `ic_ref_boss_768x448_121f.mp4` **IC-LoRA prompt format:** ``` Reference sheet: . . Generated video: ``` **CRITICAL: IC-LoRA adds ~28 reference frames to the latent.** Output is always 249 frames (not 121). Trim frames 28-148 post-render: ```bash ffmpeg -y -i input.mp4 -vf 'select=between(n\,28\,148),setpts=N/FRAME_RATE/TB' -r 24 -c:v libx264 output.mp4 ``` **Working wiring (confirmed 2026-07-22):** See `references/ic-lora-wiring.md` for the exact node connections, workflow JSON template, and submission pattern. Key connections: - CFGGuider(17): positive/negative from IC-LoRA Guide (302,0/302,1) - CropGuides(55): positive/negative from IC-LoRA Guide (302,0/302,1) - Concat(18): video_latent from DirectorGuide (133,2) — NOT IC-LoRA Guide - IC-LoRA Guide(302): positive/negative from DirectorGuide (133,0/133,1), latent from DirectorGuide (133,2) **IC-LoRA v5 workflow bug (session 2026-07-22):** The shipped `/tmp/api_ic_lora_v5.json` had node 18 video_latent pointing to 302,2 (IC-LoRA Guide latent) instead of 133,2. This leaked the reference sheet into the output (char sheet visible at end of video). Fix with one-time python edit before every IC-LoRA submit: ```bash sshpass -p 'passw0rd' ssh n8n@10.0.0.202 " python3 -c ' import json with open(\"/tmp/api_ic_lora_v5.json\") as f: w = json.load(f) w[\"18\"][\"inputs\"][\"video_latent\"] = [\"133\", 2] with open(\"/tmp/api_ic_lora_v5.json\", \"w\") as f: json.dump(w, f, indent=2) print(\"Fixed node 18 to 133,2\") ' " ``` Always verify the connection after edit. The background render monitor (terminal background=true + notify_on_complete) is the recommended pattern for IC-LoRA renders (24 steps, longer runtime). **Stock integration:** The character stock on TrueNAS (`SHEETS2_00005_Boss.png`, `cyberpunk_woman_neon_*.jpg`) is the input format IC-LoRA Ingredients expects. Convert stock images to reference sheet videos for IC-LoRA. For standard I2V without IC-LoRA, use Flux-generated panoramas. ### IC-LoRA Control (Canny/Depth/Pose) Motion transfer from reference video. Not relevant for our I2V pipeline — we control motion via prompts and start frames. ### guide_strength — CORRECTED (2026-07-22) **Use 0.80 (CONFIRMED CLEAN).** The 2-clip cyberpunk test (2026-07-22) with dev-fp8 + cond-safe at guide_strength 0.80 produced clean output on both clips — no artifacts, smooth pan continuity, user-approved. Range 0.75-0.85 is safe; 0.80 is the confirmed production value. From the LTX Director GitHub issue #258: "Hard pin = velocity discontinuity. When a pin lands where motion is active, the static frame overrides motion mid-stream → snap." This applies when guide_strength is combined with other strong conditioning (ID LoRA, Transition LoRA). With a clean single-LoRA chain, 0.80 is the recommended starting point. **Verification:** After rendering, check the actual guide_strength used by extracting metadata: ```bash ffprobe -v quiet -show_entries format_tags=prompt output.mp4 | grep -oP 'guide_strength.*?(\d+\.?\d*)' ``` ### CFG Per Scene Type | Scene Type | CFG | Notes | |------------|-----|-------| | Standard I2V (distilled) | 1.0 | Do NOT raise | | Transition LoRA scenes | 4.0 | Required for zhuanchang to activate properly | ## Prompting Rules (CRITICAL — session 2026-07-21) LTX-2.3 needs detailed, structured prompts. Short 1-2 sentence prompts produce bad output: subject duplication, wrong scenes entirely, static camera, identity drift, unnatural motion, flickering. The user confirmed: "I watched the video, it worked but it's all messed up. I think the issue is the prompts." ### Director-Specific Prompting (from research, July 2026) When using LTX Director for multi-segment timelines, the Prompt Relay system (which Director is built on) has specific rules: - **Segment 1: Static description only.** No motion. No action. Nothing not present in the input image. Describe only the visible state. - **Segments 2+: Motion changes only.** Describe only what changes or moves during that period. Do NOT re-establish what segment 1 already described. - **Global prompt:** Camera behavior, style, lighting — applied across ALL segments. Example: "static tripod, no pan/zoom/handheld motion, consistent daylight" - **Per-segment prompts:** 1-3 sentences each. Overly long prompts dilute attention. - **Never repeat global-prompt content in per-segment prompts** — causes the model to over-weight static descriptions, reducing motion. - **Prompt Relay syntax:** Pick inline (pipe-separated) OR block (newline headers). Never mix both in one prompt. - **Word count:** No strict limit, but community pattern is concise. The MShahzadAbdulmajeed 11-segment example uses short action-focused prompts per segment. Sources: Kijai's PromptRelay README, MShahzadAbdulmajeed 11-segment workflow (July 2026). ### Standalone I2V Prompting (original rules, still valid for single-clip renders) **Two confirmed failure modes from our renders:** 1. **Subject duplication** — "The man getting out of bed split into two men. One got out of bed and he was still in bed at the same time." Root cause: `guide_strength: 1.0` (hard pin) + no camera direction + no motion detail. 2. **Wrong scene entirely** — "Scene two was a man and a woman, he was scooping soup." Root cause: `zhuanchang` on a non-transition scene + prompt too short (15 words). The model filled gaps with random kitchen training data. ### Required Elements in Every Prompt 1. **Shot description** — close-up, medium shot, wide shot, low angle, tracking shot, overhead, POV 2. **Camera movement** — dolly in/out, pan left/right, tilt up/down, zoom, tracking, static, handheld, crane, Steadicam 3. **Subject action** — detailed motion: gait, speed, gestures, facial expression changes, body language. Present tense. Sequential. 4. **Environment detail** — lighting (golden hour, overcast, neon, soft studio, morning sunlight, dusk), textures, atmosphere, weather 5. **Temporal flow** — what happens first, then what changes, how the scene evolves over the 10 seconds 6. **Audio** — ambient sounds, specific SFX, music, dialogue in quotes ### Prompt Order ``` Shot + Camera → Subject + Action → Lighting + Environment → Audio + Mood ``` ### Transition LoRA (`zhuanchang`) - **Trigger word:** `zhuanchang` — append to END of prompt - **When to use:** ONLY on scenes that involve a transformation, morph, or scene transition (present→memory, reality→vision, character morph, style change, environment transition) - **When NOT to use:** Standard single-scene I2V shots with no transformation. Using it on every scene forces the model to try to morph when it shouldn't — this causes flickering, identity drift, and hallucinated content (confirmed: man+soup scene). - **Strength:** 1.0 - **CFG:** 4.0 (NOT 1.0 — Transition LoRA needs higher CFG) ### Prompt Template (Standard I2V — no zhuanchang) ``` [Shot type and camera language]. [Subject and scene description]. [Describe the action in sequence over the full duration — what happens first, then what changes]. [Lighting, texture, atmosphere, composition cues]. [Audio description]. ``` ### Prompt Template (Transition Scene — with zhuanchang) ``` [Shot type and camera language]. [Subject and scene description]. [Describe the motion, transformation, or transition process in detail — how one state morphs into another]. [Lighting, texture, atmosphere, and composition cues]. zhuanchang ``` ### Example: Bad vs Good Prompt **Bad (what we used — produced messed up output):** > A man wakes up in a small cabin bedroom, morning sunlight streaming through the window. He sits up slowly, rubbing his eyes., zhuanchang **Problems:** No camera direction, no shot framing, no motion detail, no temporal flow, no audio, zhuanchang on a non-transition scene, only 25 words. **Good (with camera, shot, motion, temporal flow, audio):** > Medium shot, static camera. A man in his 30s lies in a rustic cabin bed, morning sunlight streaming through a window casting warm golden light across rumpled sheets. Over 10 seconds: he stirs, eyes slowly opening, then pushes himself up to sitting, rubbing his eyes with both hands. He blinks, adjusting to the light, then looks toward the window with a calm expression. Warm golden hour light, dust motes floating in sunbeams, wooden cabin interior with exposed log walls. Birds chirping outside, soft rustle of bedsheets. Photorealistic, 35mm film, shallow depth of field. **What changed:** Added shot framing, camera type, sequential action over time, lighting detail, environment texture, audio, removed zhuanchang. 80 words vs 25. ## Artifact Prevention (DEEP RESEARCH — 2026-07-22, 18 sources) The 7 fixes below supersede the old 6-agent review (2026-07-21). These are the new defaults. ### The 7 Fixes (ranked by impact, v3 corrected 2026-07-22) | # | Fix | From | To | Why | |---|-----|------|----|-----| | 1 | Model/LoRA pairing | distilled + cond-safe (DOUBLE-DISTILLATION) | **dev-fp8 + cond-safe** (correct pairing) | Cond-safe is a distillation LoRA — must pair with dev base. Double-distillation causes hallucinated environments, characters, and frozen objects. See `references/double-distillation-artifacts.md`. | | 2 | ID LoRA | TalkVid-3K (talking-head) | Drop entirely | Trained for static faces, fights against action motion | | 3 | guide_strength | 1.0 (tested on broken setup) | **0.80 (CONFIRMED)** | 2-clip test passed clean at 0.80 with dev-fp8 + cond-safe. Range 0.75-0.85 is safe. | | 4 | Stages | Two-stage (8+4 refiner) | Single-stage (16-20 steps) | Refiner may be smearing, not refining | | 5 | Resolution | 512×512 | 768×512 | LTX trained for widescreen, square is suboptimal | | 6 | Distilled LoRA | Official 384 @ 1.0 | **TenStrip cond-safe @ 1.0** (preferred, with dev-fp8) or official @ 0.5-0.7 (fallback, with distilled) | Official LoRA fights I2V conditioning at high strength. TenStrip cond-safe zeroes out those layers — but MUST be paired with dev-fp8. | | 7 | Prompts | 150-200 words | 2-3 actions, 50-80 words | LTX can only execute 2-3 simultaneous actions | **⚠️ Fix #5 was CORRECTED by follow-up deep research (2026-07-22, 15 sources).** The original claim (Distilled LoRA 0.7→1.0) was wrong. Community consensus: official Distilled LoRA at 1.0 causes quality degradation for I2V. The correct range is 0.5-0.7. TenStrip cond-safe is the solution — purpose-built for I2V at 1.0. ### Deep Research Dispatch for Artifact Investigation When artifacts persist after applying the ranked fixes above, dispatch a focused deep-research pass with the exact setup and prompts. The pattern: 1. **Write the full research question to a file** — include: exact model chain, all render settings, all scene prompts verbatim, what's been tried (v4, v5), and specific research questions (e.g., "does ID LoRA cause artifacts on full-body action scenes?", "is fp8_scaled the worst variant?") 2. **Dispatch to research profile:** `hermes -p research -s deep-web-research chat -q "Read the full research question from ..." -Q --max-turns 600 --yolo` 3. **Expected output:** `/home/n8n/workspace/research/results/-ltx-2.3-artifacts-deep.md` 4. **Do NOT poll** — the `deep-research` skill's post-dispatch rule applies. Wait for the `notify_on_complete` notification. 5. **If the result file is missing after completion:** check the process log for the session_id, then check the ledger at `/tmp/research--.md`. If the ledger has only the strategy section (Move 0), the research didn't progress — re-dispatch with a fresh session. See `research-dispatch-pitfalls` Pitfall 12 for the full diagnosis workflow. Real failure (2026-07-22): deep research on LTX artifacts completed but result file never created — agent exited after Move 0. See `references/artifact-deep-research-template.md` for the question template used in the 2026-07-22 dispatch (10 specific research questions, full setup, all 6 scene prompts). ### Fix Application Order (test after each) 1. Fix model/LoRA pairing — switch UNET to dev-fp8 + keep cond-safe @ 1.0. HIGHEST impact, fixes double-distillation root cause. ✅ DONE. 2. Lower guide_strength to 0.80 — confirmed clean in 2-clip test. ✅ DONE. 3. ~~Enable NAG with strong negatives~~ — NAG is INCOMPATIBLE with fp8 UNET-only. Use ConditioningZeroOut instead. ✅ DONE. 4. Drop ID LoRA — zero-cost, eliminates talking-head bias on action scenes. ✅ DONE. 5. Switch to single-stage 16-20 steps — eliminates refiner smear risk. ✅ DONE. 6. Switch to 768×512 — widescreen training distribution. ✅ DONE. 7. Simplify prompts to 50-80 words — 2-3 actions max. 8. Git pull deps (with caution — see Pitfalls: may land incompatible native workflows) ### Negative Prompt (updated) ``` no extra limbs, no face warp, no object duplication no text artifacts, no floating logos, no watermark no extreme motion blur, no rolling shutter wobble no flicker, no frame-to-frame texture shift no Dutch angle, no rapid handheld, keep horizon level text, watermark, subtitle, logo, readable letters, garbled text ``` Added the text/watermark line — LTX-2.3's upscaler v1.0 was trained on data contaminated with endscreen logos (GitHub #148, HF discussion #13). ### Common Prompt Mistakes | Mistake | Symptom | Fix | |---------|---------|-----| | guide_strength 1.0 | Subject splits into two, ghosting | Drop to 0.7 (0.5 for high motion) | | zhuanchang on every scene | Wrong scene entirely, hallucinated content | Only use on actual transition/morph scenes | | No camera direction | Static or random movement | Add dolly/pan/tracking/static | | No shot framing | Inconsistent framing, zoom jumps | Add close-up/medium/wide/angle | | Prompts too short (15-25 words) | Model fills gaps with random training data | 50-80 words minimum | | No motion detail | Character freezes or glides unnaturally | Describe sequential action in present tense | | No per-scene lighting | Lighting doesn't match scene context | Describe light source, quality, color temp | | Same global prompt for all scenes | No scene-specific atmosphere | Vary lighting/audio/environment per scene | | No temporal structure | Model doesn't know what to animate when | "Over 10 seconds: first X, then Y, finally Z" | | Emotional labels without physical cues | Abstract expressions, no visible emotion | "Shoulders slumped, eyes downcast" not "sad" | | Conflicting descriptions | Model averages competing signals | One speed, one camera, one lighting logic | | Text/logos in prompts | Garbled text output | LTX cannot generate readable text | ### Prompting Workflow 1. Write the full prompt with all required elements (shot, camera, action, lighting, environment) 2. **Use canonical character descriptions from `references/character-descriptions.md`** — do NOT improvise or use generic descriptions. The user provided exact descriptions for each character; use them verbatim in every prompt. 3. Only append `zhuanchang` if the scene involves a transformation/morph 4. Use the same character description across all scenes (from `references/character-descriptions.md` or character sheet) 5. Vary lighting and atmosphere per scene to match the setting 6. Test one scene first before rendering all 6 7. **MANDATORY: Validate every prompt against the 10-point checklist in `references/prompt-validation-checklist.md` BEFORE submitting to render queue.** This is a FIRM quality gate — do not skip. The user's standing rule: "ensure prompts match ltx standard. ALWAYS. This should be FIRM in memory and validate before EVERY run." If any check fails, fix the prompt and re-validate. Do not submit a workflow with a failing prompt. 8. Verify guide_strength is 0.80 in every scene JSON before submitting ## Storybook Pipeline (NEW — 2026-07-29, peer-reviewed by Grok + Claude) Repeatable template system for story-driven multi-scene video using LTX Director. Full plan: `~/workspace/general/ltx23_plan.md` (also in ltx23_kb). ### How Director Is Actually Used (User's Workflow) Director is a **visual timeline editor** inside ComfyUI — not a programmatic pipeline. The user loads the Director workflow, drops images on tracks, pastes prompts, and hits render. Director handles everything internally: joining scenes, audio generation, upscaling. **Incremental build pattern** (the correct workflow, not monolithic timeline): 1. Generate first clip (up to 20s) with a start frame + prompt 2. Load that clip back into Director, extend duration (e.g., 20s → 40s) 3. Provide a prompt for the NEW section only 4. Director generates the extension and joins it seamlessly to the existing clip 5. Repeat: 40s → 60s, 60s → 80s, etc. **Keyframe placement**: Images can go at start, middle, OR end of any segment. This means you can anchor the beginning of a scene, the midpoint, or the destination. **Why this matters**: Each segment is rendered and reviewed before extending. If segment 3 fails, you only re-render that extension — not the whole timeline. This is the correct pattern; do NOT design for monolithic all-at-once rendering. **What the agent produces:** 1. Global prompt (camera/style/lighting for all scenes) 2. Per-scene table: keyframe prompt (for Qwen Image) + Director prompt 3. Build plan: incremental order, keyframe placement, durations 4. **Generated keyframe images** — agent submits Qwen T2I/I2I jobs via ComfyUI API on .202, downloads outputs, uploads to TrueNAS keyframes/ directory 5. User loads keyframes into Director, pastes prompts, renders manually **Pre-generation validation:** Before generating any keyframes, cross-reference the story against the ltx23_kb Qdrant collection for prompting rule compliance. Search for "LTX Director prompting rules", "prompt format", "common mistakes", and "keyframe best practices". Flag any violations (e.g., Segment 1 containing motion, global-prompt content repeated in per-segment, missing FLF2V where appropriate). ### Core Design (from peer review) 1. **Don't generate workflow JSON — patch frozen golden graphs.** Workflow topology bugs (wiring, crop guides, IC-LoRA) are the #1 failure mode. Keep 2-3 golden graph files, patch only whitelisted fields: `timeline_data` string, image filenames, seed, dims, frame count. Both Grok and Claude independently flagged this. 2. **Per-segment rendering, not monolithic timeline.** A 6-scene timeline that dies at scene 5 costs the whole run. Render one segment per job, validate each (`frame count == duration*fps+1`), keep it, move on. Enables cheap single-shot re-rolls. 3. **Character consistency is won at the keyframe stage.** Prompt-only drifts by scene 3. Lock a character reference sheet and inject it into every keyframe generation prompt. IC-LoRA helps within LTX but can't fix drift Flux already introduced. 4. **Silent audio for v1.** LTX 2.3 does joint AV generation natively. Per-scene TTS inside ComfyUI fights that. Render silent or with LTX native ambience, mix narration in the ffmpeg assembly step. 5. **Start simple — one opinionated path.** Flux|Qwen × start-only|FLF2V × IC-LoRA|prompt-only × TTS|silent = 16 paths. Ship one end-to-end first (Flux keyframes, start-only, prompt-only, silent), then parameterize. ### Pipeline Stages ``` story.json → keyframe_prompts.json → [validate against ltx23_kb] → [agent generates keyframes via Qwen on .202] → keyframes on TrueNAS → [user loads into Director] → [user renders on .202] → assembly (ffmpeg concat + TrueNAS upload) ``` **Pre-generation validation (MANDATORY):** Before generating any keyframes, cross-reference the story against the ltx23_kb Qdrant collection. Search for "LTX Director prompting rules", "prompt format", "common mistakes", and "keyframe best practices". Flag violations: Segment 1 containing motion, global-prompt content repeated in per-segment, missing FLF2V where appropriate, keyframe density below community standard (~3s per keyframe). Fix violations before generating images. ### Config Split (Clarified 2026-07-29) The skill previously mixed two different chains. They are separate: | Use Case | Chain | Steps | Scheduler | Model | |----------|-------|-------|-----------|-------| | Single-clip I2V (old baseline) | dev-fp8 + cond-safe, single-stage LTXDirector | 18 | simple | dev-fp8 | | Multi-scene Director (hotfix workflow) | distilled-1.1 fp8, two-stage (8+4) | 8+4 | linear_quadratic | distilled-1.1 fp8 | The Director hotfix workflow from WhatDreamsCost uses distilled-1.1 fp8 with the two-stage pipeline (Stage 1: 8 steps denoise 1.0, Stage 2: 4 steps denoise 0.42, spatial x2 upscale). This is the **production Director chain**. The old single-stage dev-fp8 + cond-safe chain is for standalone I2V clips, not multi-scene Director timelines. ### Storybook Pitfalls (from "Burn the Helix" session, 2026-07-29) 1. **Qwen I2I official template has placeholder images.** The Comfy-Org `qwen_image_edit_2511.json` template ships with `LoadImage` nodes pointing to `leather_sofa.png` and `texture_fur.png` — these don't exist on disk. Before submitting, change both LoadImage nodes: node 41 → character portrait, node 83 → empty environment T2I output. Otherwise you get 5 node errors on submit. 2. **FLF2V is a pixel morph, not an animation.** LTX Director's FLF2V interpolates pixels between start and end keyframes — it does NOT animate the prompt's action. Result: empty alley fades to a still image of Kael kneeling, with no walk-in, no kneel, no motion. **For scenes where action matters more than destination** (walk-ins, climbs, runs, reveals): use start-only I2V (one keyframe at start, no end keyframe). **For scenes where destination matters more** (specific pose, chip falling, exact framing): use FLF2V. This is the single most important keyframe strategy decision per scene. 3. **Start+mid keyframes cause early character bleed.** When using start (empty) + mid (character) keyframes, Director blends toward the mid keyframe — the character appears in frame well before the midpoint. Avoid start+mid entirely. Use either start-only I2V (action scenes) or FLF2V (destination scenes). 4. **Gaze direction drifts downward.** LTX over-interprets "walking" as looking at the ground. End keyframes with "gaze forward hard" still produce downward gaze. Fix: add "eyes locked on distant [target], chin up" to end keyframe prompts. The explicit upward cue counteracts the walking-downward association. 5. **One scene at a time.** Do not present all scenes' prompts at once. The user works incrementally — generate images for one scene, render, review, then move to the next. Only show the current scene's Director prompt, total time, and keyframe placement. Before giving Director prompts for a scene, confirm the user has generated the keyframe images for that scene — don't assume they exist. 6. **Compounding timeline is what the user enters.** Director's "extend timeline" field takes the TOTAL time, not the new scene's duration. Show it as: "Scene 2 — Dead Drop Alley (12s) — Total: 22s". Keep a running table: 10s → 22s → 32s → 44s → 58s → 68s → 80s. 7. **Keyframe naming convention.** sN-1 = scene N start frame (empty environment, T2I). sN-2 = scene N end frame (character in scene, I2I with portrait as image1). Plus one base character portrait (T2I, used as image1 reference for all I2I generations). 8. **Qwen I2I official template has model subdirectory mismatch.** The Comfy-Org template expects models in the root of `models/loras/` and `models/diffusion_models/`, but they may be in subdirectories (e.g., `Qwen-Image/Edit/`). ComfyUI only scans the root of each model folder. Fix: create symlinks from the root to the actual files. Example: `ln -sf Qwen-Image/Edit/Qwen-Image-Edit-2511-Lightning-4steps-V1.0-bf16.safetensors ~/comfy-ui/models/loras/Qwen-Image-Edit-2511-Lightning-4steps-V1.0-bf16.safetensors`. Same pattern for diffusion_models. Refresh ComfyUI (F5) after creating symlinks — it rescans on reload. ### LTX Director Reference Full reference: `references/ltx-director-2.0-how-it-works.md` — timeline_data schema, keyframe loading from source code, end frames feature (native FLF2V), two-stage pipeline architecture, model list, example workflows. From README + source code + hotfix workflow JSON (2026-07-29). ## Story Structure (CRITICAL — session 2026-07-21) The user's feedback on v4: "the videos had all kinds of inconsistencies" despite fixed wiring and good individual prompts. The root cause: the story didn't flow between scenes. Each scene was a standalone vignette (wake up → kitchen → porch → forest → stream → overlook) with no causal chain. LTX needs a narrative thread where each scene follows from the previous one. ### Story Design Rules 1. **Chain of events, not vignettes.** Each scene must be caused by the previous scene. "She discovers something → she runs → she hides → she chooses." Not "he wakes up → he makes coffee → he walks outside." 2. **One character, one journey.** Multi-character stories cause identity drift. The ID LoRA helps but isn't perfect across different faces. 3. **No dialogue, no plot twists.** LTX can't do dialogue well. Visual storytelling: action, reaction, environment, choice. 4. **6 scenes × 10s = 60s.** This is the sweet spot. Each scene is one beat in the story. 5. **Clear visual variety per scene.** Each scene should have a distinct setting, lighting, and camera language. This prevents the model from blending scenes together. 6. **Match LTX prompt style for EVERY scene.** See Prompting Rules above. Every scene prompt must have: shot type, camera movement, subject action (present tense, sequential), lighting, audio. No exceptions. ### Story Template ``` Scene 1: INCITING INCIDENT — character discovers/encounters something Scene 2: ESCALATION — the situation intensifies, stakes rise Scene 3: REACTION — character responds, makes a decision Scene 4: CHASE/STRUGGLE — physical action, pursuit, or confrontation Scene 5: LOW POINT — character is isolated, vulnerable, reflects Scene 6: CHOICE/RESOLUTION — character makes the final decision, walks toward outcome ``` ### User Preference: Detail Over Simplicity The user initially asked for simpler stories, then reversed: "go back to your original." The detailed 4-story cyberpunk templates (Ghost in the Wire, Chrome Angels, The Last Human Job, Neon Baptism) are the preferred level of detail. Each story has: logline, chain of events, per-scene setting/action/camera, and start frame requirements. See `references/story-structure-guide.md` for the full 4-story templates. ## Multi-Story Workflow (CRITICAL — session 2026-07-21) **Run one story at a time, end to end.** Do NOT batch all 4 stories. Complete each story fully before starting the next: 1. Generate start frames → wait for queue drain 2. Build scene JSONs → submit render → wait for queue drain 3. Verify frame counts (241 per scene, not 249) 4. Concat → upload to TrueNAS 5. Save all artifacts (prompts, JSONs, stock) to TrueNAS 6. Update state file 7. Only then start the next story **CRITICAL: 2-clip tests ONLY until everything is 100%.** Do NOT scale to 6 scenes until: transition is smooth, character is consistent, no artifacts. Fix one thing at a time on 2 clips, validate with user, then next fix. User directive 2026-07-22: "Stick with 2 scenes until everything is 100%." A 2-clip test (shared background, camera pan between subjects) validates the model chain, prompt quality, and transition smoothness at minimal cost before committing to a full render. See `references/claude-live-ssh-inspection.md` for the pattern of having Claude SSH into .202 to inspect live state and build the test plan. ### 2-Clip Test Pattern (Panorama + Frame-B Transition) Proven pattern for testing continuous camera motion across clips (2026-07-22, two iterations): 1. **Generate one wide Flux panorama** (1536×512) with both subjects in a shared scene — guarantees identical background/lighting 2. **Crop 3 windows** (768×512 each): A (subject 1), B (center transition — EMPTY mid-room), C (subject 2) 3. **Clip 1:** Plain I2V from frame A, prompt drives a slow steady pan. Drop ID LoRA. 4. **Clip 2:** Plain I2V from **frame B** (the pre-rendered empty mid-room crop). Prompt continues the pan, reveals subject 2. **CRITICAL: Do NOT use true-extend (extract last frame from Clip 1 → use as Clip 2 start).** True-extend carries latent memory of subject 1 into Clip 2. LTX hallucinates a second figure at subject 1's location even though the start frame shows empty room. Confirmed 2026-07-22: Clip 2 generated a second woman stepping up from the couch where the man was. Fix: use the pre-rendered empty frame B as Clip 2's start frame — clean start, no latent memory. Small seam risk at boundary but eliminates hallucination. **Why frame-B transition, not FLF:** FLF (First-Last-Frame) decelerates toward its target keyframe — clip 1 slows down approaching B, clip 2 speeds up leaving it. This creates a velocity "hitch" at the seam. Frame-B transition uses plain I2V for both clips with the pre-rendered empty frame as Clip 2's anchor — constant velocity, no deceleration, no latent memory. Full plan at `~/workspace/general/plans/2026-07-22-2clip-test-plan.md`. See `references/2clip-test-pattern.md`. ### Claude Live-SSH Inspection Pattern When you need a plan built from live system state (not from memory or docs), have Claude SSH into .202 and inspect before planning: 1. **Write the question** to a local temp file, scp to 10.0.0.28 2. **Include SSH credentials in the prompt** — Claude needs `sshpass -p 'passw0rd' ssh n8n@10.0.0.202` to reach .202 3. **Tell Claude what to inspect** — model files, workflow JSONs, queue status, disk space, existing outputs 4. **Claude inspects live, then builds the plan** — it reads real workflow JSONs to understand node structure, checks which LoRAs are actually on disk, verifies queue is empty before submitting 5. **Claude writes the plan back** — it can scp files to 10.0.0.42 (the Hermes host) or save to .202 and have you retrieve them This pattern produced the 2-clip FLF panorama plan (2026-07-22) where Claude discovered: the Distilled LoRA on disk is the 384 variant (not 384-1.1), the ID LoRA is talkvid-trained (wrong for action scenes), and FLF example workflows exist on the box. None of this was in any state file — it was discovered by live inspection. **Pitfall:** Claude may error on first attempt (SSH timeout, tool failure). Resume the session with `--resume` and ask what went wrong — Claude self-diagnoses and recovers. The first attempt cost $1.05 (21 turns, is_error); the resume cost $0.11 (2 turns, success). **Pitfall (user intent / scope validation — 2026-07-22 session):** When user says "proceed" or "it looks okay" after a single-clip review/fix, do NOT assume it means "complete more single-clip work". User frequently intends immediate move to the 2-clip test (the actual goal). Always restate: "Single clip clean — proceeding to 2-clip IC-LoRA test?" and get explicit confirmation before building/submitting 2-clip workflows. "Stop" commands require immediate halt — no autonomous continuation. This overrides the pending list when user intent differs. **Save and document EVERYTHING.** After each story completes, upload to TrueNAS: - `outputs/` — concat video - `workflows/` — scene JSONs - `docs/` — prompt documents, research - `start_frames/` — Flux-generated frames - `character_refs/` — stock reference images **Background wait pattern:** Use `terminal(background=true, notify_on_complete=true)` with a blocking poll loop on .202. The agent is notified when the queue drains — no manual polling needed. While waiting, prep the next story's prompts and stock. ## Workflow Rules - **Always fp8.** Only fall back to Q4 GGUF if fp8 OOMs and user approves. - **Prefer newest and best.** When choosing between an old proven path and a new better one, prefer the new one. But verify compatibility first — new doesn't mean compatible (e.g., the 2026-07-22 ComfyUI-LTXVideo update brought a better architecture that doesn't work with our fp8 models). - **MANDATORY: Delegate ALL mechanical work to Claude (ask-claude) FIRST, glm as FALLBACK.** Building workflow JSONs, running renders, extracting frames, submitting to queue, waiting for completion, concat, uploading — these are mechanical tasks that burn expensive deepseek tokens. Use `delegate_task` for ALL of these. Claude is PRIMARY (free — Anthropic Pro subscription, infrastructure cost) and can SSH to .202. glm is FALLBACK when Claude unavailable or tokens exhausted. The calling agent stays lean for decision-making, validation, and user communication. User directive 2026-07-22: "I would prefer that you start with claude and glm is fallback. Claude can also ssh." - **MANDATORY: Validate non-visual artifacts before submission.** Workflow JSONs, config files, scripts — dispatch a validation sub-agent to verify correctness before submitting to the render queue. Catches wiring errors, missing nodes, and model file mismatches that would waste a 5-7 minute render. User validates video output by watching — do NOT dispatch validation sub-agents for renders. User directive 2026-07-22: "do not validate when complete. I will validate by watching videos." - **CRITICAL: glm profile needs the ComfyUI skill.** The glm profile does NOT have the ComfyUI skill by default — it only lives in the general profile. Before dispatching LTX work to glm, verify the skill is available: `hermes -p glm skills list | grep comfyui`. If missing, copy it: `cp -r ~/.hermes/profiles/general/skills/creative/comfyui ~/.hermes/profiles/glm/skills/creative/comfyui`. Then include "FIRST: Load the ComfyUI skill (skill_view(name='comfyui'))" in the glm prompt. The skill provides `run_workflow.py`, `ws_monitor.py`, and `extract_schema.py` — use `--host http://10.0.0.202:8188` for remote execution. Confirmed 2026-07-22: first glm dispatch was missing the skill and had to be killed + re-dispatched. - **CRITICAL: Dispatch glm in background mode for LTX renders.** LTX renders take 5-7 minutes per clip plus workflow building and concat — easily 15+ minutes total. The foreground 600s timeout will kill the dispatch mid-render. Use `terminal(background=true, notify_on_complete=true)` for all glm LTX dispatches. Confirmed 2026-07-22: 2-clip cyberpunk render completed successfully in background mode. - **File-based API submission.** Do NOT inline complex JSON in curl commands — Hermes' command parser blocks them. Write the workflow to a file (e.g., `/tmp/submit_ltx_render.sh`), then execute with `bash /tmp/submit_ltx_render.sh`. See `references/api-submission-pattern.md` for the complete pattern including polling, verification, and TrueNAS upload. - **ALWAYS validate with glm after creating or updating anything.** Workflow JSONs, config changes, file writes — dispatch glm via `delegate_task` to read the file and verify correctness before submitting to the render queue. Glm catches wiring errors, missing nodes, and model file mismatches that would waste a 5-7 minute render. This is a quality gate, not optional. - **Test one scene before rendering all 6.** - **Verify frame count after every render (ffprobe).** - **Save all artifacts to TrueNAS immediately** (user can only view videos on TrueNAS, not in LXCs). - **Update state file as you go.** - **Save learnings to fact_store proactively** — don't wait to be asked. - **User prefers discussing design decisions before implementation.** ### 1. Generate Start Frames with Flux.1-dev fp8 **Proven working workflow (2026-07-21):** Models on .202: - UNET: `flux1-dev-fp8-e4m3fn.safetensors` in `models/diffusion_models/` - CLIP: `t5xxl_fp8_e4m3fn.safetensors` + `clip_l.safetensors` in `models/text_encoders/` - VAE: `ae.safetensors` in `models/vae/` Working node graph (768×512 txt2img — preferred resolution for LTX widescreen): ```json { "5": {"class_type": "CLIPTextEncode", "inputs": {"text": "", "clip": ["11", 0]}}, "6": {"class_type": "CLIPTextEncode", "inputs": {"text": "blurry, low quality, distorted face, bad anatomy, watermark, text, logo", "clip": ["11", 0]}}, "7": {"class_type": "FluxGuidance", "inputs": {"conditioning": ["5", 0], "guidance": 3.5}}, "8": {"class_type": "VAEDecode", "inputs": {"samples": ["3", 0], "vae": ["10", 0]}}, "9": {"class_type": "SaveImage", "inputs": {"filename_prefix": "ltx_start_frame_XX", "images": ["8", 0]}}, "10": {"class_type": "VAELoader", "inputs": {"vae_name": "ae.safetensors"}}, "11": {"class_type": "DualCLIPLoader", "inputs": {"clip_name1": "t5xxl_fp8_e4m3fn.safetensors", "clip_name2": "clip_l.safetensors", "type": "flux"}}, "16": {"class_type": "UNETLoader", "inputs": {"unet_name": "flux1-dev-fp8-e4m3fn.safetensors", "weight_dtype": "fp8_e4m3fn"}}, "3": {"class_type": "KSampler", "inputs": {"seed": 42, "steps": 20, "cfg": 1.0, "sampler_name": "euler", "scheduler": "simple", "denoise": 1.0, "model": ["16", 0], "positive": ["7", 0], "negative": ["6", 0], "latent_image": ["27", 0]}}, "27": {"class_type": "EmptySD3LatentImage", "inputs": {"width": 768, "height": 512, "batch_size": 1}} } ``` Full template at `templates/flux-768x512-txt2img.json`. The old 512×512 template (`templates/flux-512-txt2img.json`) is kept for reference but 768×512 is the production default. **Submission:** MUST wrap in `{"prompt": }` envelope. The ComfyUI API rejects bare workflow JSON with `"no_prompt"` error. ```bash # Build JSON, then submit: curl -s -X POST http://localhost:8188/prompt -H "Content-Type: application/json" -d @/tmp/flux_scene_01.json ``` **Output:** Files land in `~/comfy-ui/output/` as `_00001_.png`. Move to `~/comfy-ui/input/` for LTX Director to find them. - Use Flux.1-dev fp8 or Qwen-Image-2512 on .202 - 512×512, photorealistic - Save to `~/comfy-ui/input/` ### 2. Generate TTS Audio - Edge TTS (free) or Piper - ~2-4s per scene - Save to `/tmp/ltx_audio/` ### 3. Build Scene JSONs - Template: use an existing scene JSON from a prior render - Modify: start_frame path, audio path, segment prompt - For Transition LoRA: add `LoraLoaderModelOnly` node, rewire chain, append `zhuanchang` to prompts - Save to `/tmp/api_scene_XX.json` - **To modify and re-render existing workflows** (change sampler, duration, etc. without rebuilding): see `references/modify-and-re-render-recipe.md` — proven pattern with Python in-place editing, verification, and submission. - **To programmatically build multiple clip workflows from templates** (common modifications across C1/C2 with per-clip overrides): see `references/programmatic-workflow-modification.md` — scp a Python script to .202, avoids shell heredoc quoting issues. - **To recover a lost workflow from a successful render's output file:** see `references/workflow-extraction-from-output.md` — extract the full workflow JSON from the output MP4's metadata (ffprobe → prompt tag). Use this when the original `/tmp/` JSON was deleted or you need to audit what actually ran. ### 4. Submit to Queue ```bash curl -s -X POST http://localhost:8188/prompt -H "Content-Type: application/json" -d @/tmp/api_scene_XX.json ``` - Submit all scenes at once - Queue processes sequentially (one GPU) ### 5. Wait for Completion - Poll queue: `curl -s http://localhost:8188/queue | python3 -c "import json,sys; d=json.load(sys.stdin); print(len(d['queue_running']), len(d['queue_pending']))"` - GPU: `nvidia-smi --query-gpu=utilization.gpu,memory.used --format=csv,noheader` - ~5-7 min per scene at 8 steps - Blocking wait loop (from Hermes host): ```bash sshpass -p 'passw0rd' ssh n8n@10.0.0.202 "while true; do q=\$(curl -s http://localhost:8188/queue | python3 -c \"import json,sys; d=json.load(sys.stdin); print(len(d['queue_running'])+len(d['queue_pending']))\"); if [ \"\$q\" -eq 0 ]; then echo 'QUEUE_EMPTY'; break; fi; echo \"queue: \$q\"; sleep 30; done" ``` ### 6. Identify Output Files **Preferred: Use ComfyUI history API (reliable, works regardless of output directory).** After queue drains, query the history for each prompt_id to get exact filenames: ```bash # Get output filename from prompt_id curl -s http://localhost:8188/history/ | python3 -c " import json,sys d=json.load(sys.stdin) entry=d.get('',{}) outputs=entry.get('outputs',{}) node37=outputs.get('37',{}) images=node37.get('images',[]) for img in images: print(img.get('filename','?')) " ``` **Fallback: Search both output directories.** Outputs land in `~/comfy-ui/output/` or `~/comfy-ui/output/video/` depending on the workflow's `filename_prefix`. Use `find` with a reference file timestamp: ```bash # Find all mp4s newer than a known reference file find ~/comfy-ui/output -name "*.mp4" -newer /tmp/api_2clip_C1_v2.json -type f # Or list by modification time ls -lt ~/comfy-ui/output/*.mp4 ~/comfy-ui/output/video/*.mp4 2>/dev/null | head -10 ``` For metadata-based identification (when outputs have predictable prefixes): ```bash python3 << 'PYEOF' import subprocess, os, re outdir = os.path.expanduser("~/comfy-ui/output/video") for f in sorted(os.listdir(outdir)): if not f.startswith("LTX_Director_"): continue path = os.path.join(outdir, f) r = subprocess.run(["ffprobe", "-v", "quiet", "-show_entries", "format_tags=prompt", "-of", "csv=p=0", path], capture_output=True, text=True) img = re.search(r"imageFile.*?ltx_start_frame_(\d+)", r.stdout) scene = f"Scene_{img.group(1)}" if img else "?" has_trans = "LoraLoaderModelOnly" in r.stdout print(f"{f} | {scene} | Transition={has_trans}") PYEOF ``` ### 7. Concat with ffmpeg ```bash # Create concat list cat > /tmp/concat.txt << EOF file '/home/n8n/comfy-ui/output/video/LTX_Director_XXXXX_.mp4' file '/home/n8n/comfy-ui/output/video/LTX_Director_YYYYY_.mp4' ... EOF # Stream copy (no re-encode) ffmpeg -f concat -safe 0 -i /tmp/concat.txt -c copy ~/comfy-ui/output/video/LTX_Director_60s.mp4 -y ``` ### 8. Save to Stock - **smbclient is NOT on .202** — scp files to Hermes host first, then upload to TrueNAS from there - Upload outputs: `smbclient -N //10.0.0.117/proxmoxBackup -c 'cd ai_vid_stock_material\\outputs; put '` - Upload workflow JSONs: `smbclient -N //10.0.0.117/proxmoxBackup -c 'cd ai_vid_stock_material\\workflows; put '` - Update README.md inventory - See `ai-vid-stock` skill for full commands ## Stock Materials All stock materials (start frames, audio, character refs, workflows, outputs) are stored on TrueNAS at `//10.0.0.117/proxmoxBackup/ai_vid_stock_material/`. Use the `ai-vid-stock` skill for add/search/remove operations. **Known gap (2026-07-21):** `better-search` finds sources but doesn't download them to stock. The research agent reports URLs; the dispatcher agent must manually download and upload. Future: extend `better-search-research` methodology to accept a `--target` parameter and auto-download top results to the specified stock directory. **Character reference sources (from better-search research):** - freeaivideohub.com/character-sheets — 12 free photorealistic multi-view character sheets - Pexels API — 700K+ free portrait photos, commercial use - FFHQ — 70K faces at 1024×1024 (NVIDIA, non-commercial) - CivitAI — community LoRAs and LTX workflows **Using stock as I2V start frames:** See `references/stock-as-i2v-start-frame.md` — pattern for using character stock images as direct I2V start frames without IC-LoRA Ingredients. Simpler, works with existing 6-fix baseline. For multi-clip character consistency, use IC-LoRA Ingredients instead. ## Freshness Protocol — Version-Aware Research (FIRM — user directive 2026-07-30) **The AI tools space moves weekly.** APIs break, nodes get renamed, workflows from last month don't run today. A 4-month-old ComfyUI tutorial is archaeology. The agent MUST NOT find an old blog post and confidently apply it without verification. ### Mandatory Before Any Fix or Recommendation 1. **Check installed version first.** Before searching "how to do X", determine what version is actually running on the target system. The fix for ComfyUI nightly is different from the fix for 3.12.0 stable. ```bash # ComfyUI version ssh n8n@10.0.0.202 "cd ~/comfy-ui && git log -1 --format='%h %s %ai'" # Custom nodes ssh n8n@10.0.0.202 "cd ~/comfy-ui/custom_nodes/ComfyUI-LTXDirector && git log -1 --format='%h %s %ai'" ``` 2. **Check release notes before random fixes.** When something breaks, check the project's GitHub releases/changelog FIRST — the breakage is often documented as an intentional change. Do NOT jump to searching error messages. 3. **Date-gate all searches.** For fast-moving projects, use time filters: `after:2026-06-01` or "past month". A blog post from February titled "How to use ComfyUI MultiGPU" is almost certainly wrong today. 4. **Prefer official docs over tutorials.** The project's own README, docs site, or GitHub releases page is more likely current than a third-party tutorial. 5. **Test one atomic step before building the whole plan.** Verify step 1 actually works before writing step 2. A 10-step workflow where step 7 fails because step 1's assumptions were wrong wastes hours. 6. **Cross-reference with ltx23_kb first.** Search the knowledge base for what we already know about the current version before searching the web. ### Search Query Template for Fast-Moving Projects ``` site:github.com/{owner}/{repo} (release OR changelog OR breaking) after:{3_months_ago} site:github.com/{owner}/{repo}/issues (error OR fix OR workaround) updated:>={last_week} "{project} {current_version}" (fix OR error OR workaround OR update) ``` ### Red Flags (abort and re-verify) - Tutorial/blog post older than 3 months with no version mentioned - GitHub issue marked "closed" but the fix references a different version - Reddit thread where the top comment says "this is outdated, use X instead" - Any source that says "just do Y" without mentioning a specific version ## Pitfalls - **CRITICAL: Double-distillation — TenStrip cond-safe LoRA MUST be paired with dev-fp8 UNET, NOT distilled.** The cond-safe LoRA is a distillation LoRA. Running it on the already-distilled transformer damages conditioning paths, causing hallucinated environments (closet doors), hallucinated characters (second girl), and object persistence artifacts (frozen cup on tricep). From the model page: "Applying them on top of an already-distilled model double-distills and degrades quality." Fix: switch UNET to dev-fp8 (already on disk at `~/comfy-ui/models/diffusion_models/ltx-2.3-22b-dev_transformer_only_fp8_scaled.safetensors`, 22GB) OR drop cond-safe and use pure distilled. See `references/double-distillation-artifacts.md` for full root cause analysis with Grok diagnosis, secondary causes, and fix priority order. Confirmed 2026-07-22: 2-clip cyberpunk test produced major artifacts from this exact misconfiguration. - **CRITICAL: True-extend (extract last frame → use as next clip start) carries latent memory.** LTX remembers subjects from prior clips even when the start frame shows empty space. Confirmed 2026-07-22: Clip 2 hallucinated a second woman stepping up from the couch where the man was in Clip 1. Fix: use the pre-rendered empty mid-room frame B as Clip 2's start frame instead of the extracted last frame. Clean start, no latent memory. See §2-Clip Test Pattern. - **ffmpeg last-frame extraction:** `ffmpeg -sseof -3 -i input.mp4 -vsync vfr -q:v 2 output.png` may fail. Working alternative: `ffmpeg -sseof -3 -i input.mp4 -update 1 -frames:v 1 -q:v 2 output.png`. The `-update 1` flag is needed for single-frame PNG output from video input. - **CRITICAL: git pull on ComfyUI-LTXVideo may land incompatible native workflows.** The 2026-07-22 update (30K lines) brought a new native single-stage architecture (CheckpointLoaderSimple, LTXVScheduler, SamplerCustomAdvanced, GemmaAPITextEncode) that uses full checkpoints (46 GB), not our fp8 UNET-only transformer (23 GB). The new workflow drops LTXDirector entirely. Our fp8 chain (UNETLoader → LTX2LoraLoaderAdvanced → LTXDirector) is incompatible with the new architecture. **Before git pulling, snapshot the working state.** If the pull lands new example workflows, do NOT assume they work with our fp8 models — verify compatibility first. The new architecture requires a full checkpoint that won't fit 24 GB VRAM. - **CRITICAL: Start frame resolution determines output resolution.** LTXDirector doubles the start frame dimensions. 256×256 start frame → 512×512 output. 768×512 start frame → 768×512 output. The `custom_width`/`custom_height` inputs on LTXDirector do NOT override — resolution is derived from the start frame. Generate Flux start frames at the TARGET output resolution. Confirmed 2026-07-22: Story 1 v1 rendered at 512×512 because start frames were 256×256. - **CRITICAL: Sub-agent validation mandate.** After any glm subagent completes work (renders, file ops, JSON builds), re-dispatch a separate validation subagent to independently verify every output — ffprobe resolution/frame counts, file existence, concat integrity. Never trust subagent output without independent verification. User directive 2026-07-22: "After glm does the work, re-dispatch to validate. Every time." - **Subagent timeout: was 600s, now 1200s (20 min).** `delegation.child_timeout_seconds` and `terminal.timeout` both set to 1200 across general, grok45, and grok43 profiles (2026-07-22). This covers most single-clip renders (~3-4 min) plus workflow building. For multi-scene renders exceeding 20 min, still use `terminal(background=true, notify_on_complete=true)`. Confirmed 2026-07-22: Story 1 delegation timed out at 600s with 57 API calls — renders completed but subagent couldn't report. IC-LoRA delegation also timed out at 600s mid-render. - **CRITICAL: Dev_FullSteps native workflow (SamplerCustomAdvanced) produces garbage with fp8 UNET-only.** Confirmed 2026-07-22: Fix 1 test used Dev_FullSteps (SamplerCustomAdvanced, BasicScheduler, KSamplerSelect, CFGGuider) with dev-fp8 UNET — output was a blur of colors, no actual video. The native workflow requires a full checkpoint (46GB). Our fp8 UNET-only setup (23GB) MUST use the LTXDirector chain (UNETLoader → LTX2LoraLoaderAdvanced → LTXDirector). Do NOT use SamplerCustomAdvanced, BasicScheduler, or KSamplerSelect with fp8 UNET-only. - **CRITICAL: Upload to TrueNAS after EVERY render.** The user can only view videos on TrueNAS (10.0.0.117), not in LXC containers. scp from .202 to Hermes host, then smbclient to TrueNAS. Do NOT skip this — the user has no other way to review output. (Learned 2026-07-22: user said "I can only view videos on truenas, not in lxc's.") - **CRITICAL: Do NOT interrupt a running Claude session.** If Claude is mid-build (background process running), do not scp new question files or send follow-ups unless the user explicitly says "tell Claude now." Wait for the session to complete. Mid-build interruptions can cause the session to error out. (Learned 2026-07-22: sent a fact_store instruction mid-build; user corrected: "I said when claude is complete.") - **CRITICAL: Claude's intermediate errors are normal self-correction.** Claude will make mistakes, hit errors, then try a different approach. `is_error: true` on intermediate turns does NOT mean the session failed. Wait for the final turn output. Only diagnose after the background process completes. Jumping to conclusions mid-session wastes turns and money. (Learned 2026-07-22: killed a working session twice, $1.78 wasted. User: "you just need to wait for the final turn output. Not jump to conclusions.") - **Transition LoRA uses LoraLoaderModelOnly, not LTX2LoraLoaderAdvanced.** Wiring it wrong silently fails — the workflow runs but produces no transition effect. - **`zhuanchang` trigger word is required** for Transition LoRA to activate. Append to end of segment prompts. - **CFG must be 1.0** for distilled model. Higher values cause artifacts. - **Queue is sequential** — submitting 6 scenes means ~30-40 min total. Use the blocking wait loop from step 5. - **`process(action='wait')` timeout is clamped to 60s regardless of what you request.** The configured limit on the Hermes host caps wait timeouts at 60 seconds. For renders that take 5-7 minutes each, a single `process(action='wait', timeout=900)` will time out after 60s with the process still running. Pattern: use `process(action='wait', timeout=60)` in a loop, checking `status` after each call. When status is still `"running"`, call wait again. When status is `"exited"`, the queue has drained. Confirmed 2026-07-22: 2-clip render took 3 wait cycles (~3 min total) despite requesting 900s. - **Output files accumulate** — the Director workflow produces 2 outputs per scene (guide pass + main pass). Identify the final output by checking for `LoraLoaderModelOnly` in metadata (Transition batch) or by resolution (512×512 = fp8 batch). - **Output directory varies by workflow prefix.** The `SaveImage`/`SaveVideo` node's `filename_prefix` determines where files land. Workflows with prefix `LTX_Director_` go to `~/comfy-ui/output/video/`. Workflows with custom prefixes (e.g., `LTX_2clip_test_C1`) go to `~/comfy-ui/output/` directly. When identifying new outputs, search both directories: `find ~/comfy-ui/output -name "*.mp4" -newer -type f`. Do NOT assume all outputs are in `video/`.\n- **CRITICAL: Template file naming — verify you're using the right base files before re-rendering.** The .202 `/tmp/` directory accumulates multiple generations of workflow JSONs with confusingly similar names: `api_2clip_C1.json` (old living room test), `api_cyberpunk_C1.json` (cyberpunk test), `api_2clip_01.json` (earlier living room variant). When modifying and re-rendering, ALWAYS inspect the timeline_data segment's `imageFile` and `prompt` fields to confirm you're working with the correct template. A one-line Python check: `python3 -c "import json; d=json.load(open('/tmp/api_XXX.json')); td=json.loads(d['prompt']['131']['inputs']['timeline_data']); print(td['segments'][0]['imageFile'], td['segments'][0]['prompt'][:80])"`. Confirmed 2026-07-22: pointed delegation at wrong template, produced living room output instead of cyberpunk. - **cifs kernel module not available** on Hermes host — use smbclient for TrueNAS, not mount.cifs. - **smbclient is NOT installed on .202** — scp files to Hermes host first, then upload to TrueNAS from there. Do not try to install smbclient on .202 (no root access). - **State file is the crash-recovery artifact** — update `~/workspace/general/ltx-pipeline-state.md` as you go. A new session reads it to resume. - **ComfyUI API requires `{"prompt": }` envelope.** Submitting bare workflow JSON returns `"no_prompt"` error. Always wrap in the prompt key. - **241 frames is normal for 10s@24fps.** The Director may produce 241 due to a 1-frame rounding quirk. 249 frames means the wiring bug is present (guide frames leaked). Verify with ffprobe. - **Duration control: set `segment[\"length\"]`, NOT `duration_seconds` or `segment[\"end\"]`.** The LTXDirector ignores node-level `duration_seconds` and segment `end` — it uses `segment[\"length\"]` as the frame count. For 5s@24fps: `td[\"segments\"][0][\"length\"] = 120`. Setting only `duration_seconds` or `end` silently produces the full 10s output. Confirmed 2026-07-21: two failed attempts before finding the correct field. - **Story must have a causal chain.** Standalone vignettes (wake up → kitchen → porch → forest) produce visual inconsistencies because the model has no narrative thread. Each scene must follow from the previous one. See `references/story-structure-guide.md` for the full 4-story templates and design rules. - **User prefers detailed stories over simplified ones.** When the user said "keep it simple" and then "go back to your original," the original detailed templates were preferred. Don't over-simplify — the 4-story cyberpunk templates with loglines, scene tables, and character descriptions are the right level of detail. - **CRITICAL: Do NOT copy ComfyUI built-in blueprints as workflow files.** The blueprints in `~/comfy-ui/blueprints/` use subgraph wrappers (single node of type `fd6ee5f8-...`) that are NOT complete workflows. They only work when loaded through ComfyUI's template browser. For direct workflow files, download the official expanded templates from Comfy-Org GitHub: `https://raw.githubusercontent.com/Comfy-Org/workflow_templates/refs/heads/main/templates/image_qwen_Image_2512.json` (T2I) and `https://raw.githubusercontent.com/Comfy-Org/workflow_templates/refs/heads/main/templates/image_qwen_image_edit_2511.json` (I2I). These have full node graphs (4-6 nodes with links and groups). (Learned 2026-07-29: copied blueprints as workflows; user said "YOU DID NOT SET THE QWEN WORKFLOWS UP CORRECTLy" — blueprints are incomplete subgraph wrappers.) - **CRITICAL: Prompt wardrobe/appearance audit before EVERY render.** Mismatched wardrobe terms between prompts and reference frames cost a 35-minute re-render (RapVidFinal, 2026-07-30). Before queueing any multi-scene workflow, grep all PROMPT nodes for wardrobe/appearance terms (colors, clothing items, accessories) and verify they match the reference frame spec. A 30-second grep saves a full re-render. Example: `grep -i 'tracksuit\|jacket\|shirt\|pants\|chains\|color' workflow.json | sort | uniq -c`. If any prompt says "black" and frames are "red", fix before queueing. This is a mandatory pre-queue step. - **I2V Prompt Engineering (RapVidFinal lessons, 2026-07-30):** See `references/ltx23-i2v-prompt-engineering.md` — proven patterns from 3 rounds of Claude+Grok dual-agent validation: brand name stripping, I2V static wardrobe rule, action thinning (2-3 max), physical aggression cues, lightning through-line, single camera/lighting logic per scene. All 8 RapVidFinalHours prompts passed final validation at 60-93 words each. - **CRITICAL: Do NOT overcomplicate. Do exactly what the user asks — nothing more.** One command that succeeds = task complete. STOP. No follow-up research, no validation, no alternative downloads, no format checks, no "let me also..." additions. The user will tell you if something is wrong. This is the #1 user frustration pattern — agent completes the task then keeps going with unnecessary extras. (Learned 2026-07-29: user said "STOP", "Why do you keep making things so complicated???", "stop. Stop doing overkill. JUST DO simply what I ask.") The pattern `ssh ... python3 -c "import json; open('/tmp/file.json','w').write(escaped_json)"` fails because shell escaping corrupts the JSON (exit code 2, no file created). The working pattern: write the JSON locally → `sshpass -p 'passw0rd' scp local.json n8n@10.0.0.202:/tmp/remote.json` → then curl from .202. Confirmed 2026-07-22: 6 LTX scene JSONs failed to write via inline Python; scp succeeded on all 6.plates from Comfy-Org GitHub: `https://raw.githubusercontent.com/Comfy-Org/workflow_templates/refs/heads/main/templates/image_qwen_Image_2512.json` (T2I) and `https://raw.githubusercontent.com/Comfy-Org/workflow_templates/refs/heads/main/templates/image_qwen_image_edit_2511.json` (I2I). These have full node graphs (4-6 nodes with links and groups). (Learned 2026-07-29: copied blueprints as workflows; user said "YOU DID NOT SET THE QWEN WORKFLOWS UP CORRECTLy" — blueprints are incomplete subgraph wrappers.) - **CRITICAL: Prompt wardrobe/appearance audit before EVERY render.** Mismatched wardrobe terms between prompts and reference frames cost a 35-minute re-render (RapVidFinal, 2026-07-30). Before queueing any multi-scene workflow, grep all PROMPT nodes for wardrobe/appearance terms (colors, clothing items, accessories) and verify they match the reference frame spec. A 30-second grep saves a full re-render. Example: `grep -i 'tracksuit\|jacket\|shirt\|pants\|chains\|color' workflow.json | sort | uniq -c`. If any prompt says "black" and frames are "red", fix before queueing. This is a mandatory pre-queue step. - **I2V Prompt Engineering (RapVidFinal lessons, 2026-07-30):** See `references/ltx23-i2v-prompt-engineering.md` — proven patterns from 3 rounds of Claude+Grok dual-agent validation: brand name stripping, I2V static wardrobe rule, action thinning (2-3 max), physical aggression cues, lightning through-line, single camera/lighting logic per scene. All 8 RapVidFinalHours prompts passed final validation at 60-93 words each. - **CRITICAL: Do NOT overcomplicate. Do exactly what the user asks — nothing more.** One command that succeeds = task complete. STOP. No follow-up research, no validation, no alternative downloads, no format checks, no "let me also..." additions. The user will tell you if something is wrong. This is the #1 user frustration pattern — agent completes the task then keeps going with unnecessary extras. (Learned 2026-07-29: user said "STOP", "Why do you keep making things so complicated???", "stop. Stop doing overkill. JUST DO simply what I ask.") The pattern `ssh ... python3 -c "import json; open('/tmp/file.json','w').write(escaped_json)"` fails because shell escaping corrupts the JSON (exit code 2, no file created). The working pattern: write the JSON locally → `sshpass -p 'passw0rd' scp local.json n8n@10.0.0.202:/tmp/remote.json` → then curl from .202. Confirmed 2026-07-22: 6 LTX scene JSONs failed to write via inline Python; scp succeeded on all 6. - **CRITICAL: IC-LoRA adds ~28 reference frames to the latent — always trim post-render.** Output is 249 frames regardless of requested duration. Trim frames 28-148 with ffmpeg: `ffmpeg -y -i input.mp4 -vf 'select=between(n\,28\,148),setpts=N/FRAME_RATE/TB' -r 24 -c:v libx264 output.mp4`. Do NOT try to fix this in the workflow — the extra frames are inherent to how LTXAddVideoICLoRAGuide injects reference conditioning. Confirmed 2026-07-22: 4 renders, all produced 249 frames. - **CRITICAL: Subagent delegation timeout (600s) is too short for IC-LoRA renders.** IC-LoRA renders take longer (24 steps vs 18, higher CFG) plus workflow building and reference sheet conversion. The subagent timed out at 600s mid-render. Use `delegate_task` with the understanding that the subagent may time out — check the live transcript to see how far it got, then finish the work yourself. The subagent did the heavy lifting (reference sheet conversion, workflow building, first render) — the orchestrator just needed to fix wiring and resubmit. Confirmed 2026-07-22: Claude subagent timed out at 600s with 25 API calls, but the reference video and workflow JSON were correct. - **CRITICAL: Do NOT copy ComfyUI built-in blueprints as workflow files.** The blueprints in `~/comfy-ui/blueprints/` use subgraph wrappers (single node of type `fd6ee5f8-...`) that are NOT complete workflows. They only work when loaded through ComfyUI's template browser. For direct workflow files, download the official expanded templates from Comfy-Org GitHub: `https://raw.githubusercontent.com/Comfy-Org/workflow_templates/refs/heads/main/templates/image_qwen_Image_2512.json` (T2I) and `https://raw.githubusercontent.com/Comfy-Org/workflow_templates/refs/heads/main/templates/image_qwen_image_edit_2511.json` (I2I). These have full node graphs (4-6 nodes with links and groups). (Learned 2026-07-29: copied blueprints as workflows; user said "YOU DID NOT SET THE QWEN WORKFLOWS UP CORRECTLy" — blueprints are incomplete subgraph wrappers.) - **CRITICAL: Prompt wardrobe/appearance audit before EVERY render.** Mismatched wardrobe terms between prompts and reference frames cost a 35-minute re-render (RapVidFinal, 2026-07-30). Before queueing any multi-scene workflow, grep all PROMPT nodes for wardrobe/appearance terms (colors, clothing items, accessories) and verify they match the reference frame spec. A 30-second grep saves a full re-render. Example: `grep -i 'tracksuit\|jacket\|shirt\|pants\|chains\|color' workflow.json | sort | uniq -c`. If any prompt says "black" and frames are "red", fix before queueing. This is a mandatory pre-queue step. - **I2V Prompt Engineering (RapVidFinal lessons, 2026-07-30):** See `references/ltx23-i2v-prompt-engineering.md` — proven patterns from 3 rounds of Claude+Grok dual-agent validation: brand name stripping, I2V static wardrobe rule, action thinning (2-3 max), physical aggression cues, lightning through-line, single camera/lighting logic per scene. All 8 RapVidFinalHours prompts passed final validation at 60-93 words each. - **CRITICAL: Do NOT overcomplicate. Do exactly what the user asks — nothing more.** One command that succeeds = task complete. STOP. No follow-up research, no validation, no alternative downloads, no format checks, no "let me also..." additions. The user will tell you if something is wrong. This is the #1 user frustration pattern — agent completes the task then keeps going with unnecessary extras. (Learned 2026-07-29: user said "STOP", "Why do you keep making things so complicated???", "stop. Stop doing overkill. JUST DO simply what I ask.") The pattern `ssh ... python3 -c "import json; open('/tmp/file.json','w').write(escaped_json)"` fails because shell escaping corrupts the JSON (exit code 2, no file created). The working pattern: write the JSON locally → `sshpass -p 'passw0rd' scp local.json n8n@10.0.0.202:/tmp/remote.json` → then curl from .202. Confirmed 2026-07-22: 6 LTX scene JSONs failed to write via inline Python; scp succeeded on all 6.