Add creative/comfyui v5.1.0 (official bundled) and media/ltx-video-pipeline v2.0.0 (custom LTX pipeline)

2026-07-22 16:17:52 -05:00
parent 7e81fdac31
commit d94132f4ca
60 changed files with 10421 additions and 0 deletions
@@ -0,0 +1,71 @@
+# 2-Clip Test Pattern — Panorama + True-Extend
+
+Proven 2026-07-22. Validates model chain, prompt quality, and transition smoothness at minimal cost before committing to a full 6-scene story.
+
+## Steps
+
+1. **Generate one wide Flux panorama** (1536×512) with both subjects in a shared scene. Use Flux.1-dev fp8 on .202. Prompt: describe the full room with subject 1 on left, subject 2 on right, open space between them. This guarantees identical background/lighting/perspective.
+
+2. **Crop 3 windows** (768×512 each) with ffmpeg:
+   - A = left (subject 1 in frame, subject 2 off-screen right)
+   - B = center (mid-room transition point)
+   - C = right (subject 2 in frame, subject 1 off-screen left)
+   Upload to `~/comfy-ui/input/`.
+
+3. **Clip 1 workflow:** Copy v5 fix template (`/tmp/api_s1_fix_01.json`). Drop ID LoRA (rewire 131.model from ["201",0] to ["200",0], delete node 201). Set imageFile to A, prompt drives slow steady rightward pan, 10s/240 frames. Submit.
+
+4. **Extract last frame:** `ffmpeg -sseof -3 -i clip1.mp4 -vsync vfr -q:v 2 ~/comfy-ui/input/last_frame.png`
+
+5. **Clip 2 workflow:** Same template. imageFile = extracted last frame. Prompt continues the pan, reveals subject 2. Frame C is compositional target, not hard landing point. Submit.
+
+6. **Concat:** `ffmpeg -f concat -safe 0 -i list.txt -c copy output_20s.mp4`
+
+## Why True-Extend, Not FLF
+
+FLF decelerates toward its target keyframe — clip 1 slows down approaching B, clip 2 speeds up leaving it. Creates a velocity "hitch" at the seam. True-extend avoids this: constant velocity, no deceleration. LTX's Extend mode is purpose-built for seamless continuation.
+
+Sources: WaveSpeedAI (2026), LTX blog "How to Extend AI Videos" (2026).
+
+## Model Config
+
+- UNET: fp8 distilled transformer
+- LoRA: Distilled only @ 0.7 (NO ID LoRA — talking-head domain, wrong for camera moves)
+- Two-stage: 8 steps denoise 1.0 + 4 steps denoise 0.42
+- Sampler: euler, Scheduler: simple, CFG: 1.0, guide_strength: 0.5
+- Resolution: 768×512, 24fps, 10s per clip
+- Audio: disabled
+
+## Validation
+
+- Seam: no flash/pop/jump at boundary; pan speed continuous
+- Pan: steady rightward throughout, no reversal/wobble/stall
+- Background: same room/lighting across both clips
+- Subjects: coherent, no duplication or morphing
+- Artifacts: no limb melting, flicker, ghosting, warping
+- Timing: ~10s each at 24fps (~240 frames)
+
+## Execution Confirmation (2026-07-22)
+
+Successfully executed end-to-end on .202:
+- Flux panorama: 1536×512, 948KB, rendered in ~30s
+- Crops: A (842K), B (858K), C (845K) via ffmpeg crop filter
+- Clip 1: 241 frames, 10.04s, 1.0MB — status success
+- Last frame extracted: 611K PNG
+- Clip 2: 241 frames, 10.04s, 1.6MB — status success
+- Concat: 482 frames, 20.08s, 2.5MB — minor non-monotonic DTS warning (harmless)
+- Both clips used Distilled-only (no ID LoRA), two-stage, 768×512, 24fps
+
+## TrueNAS Upload (MANDATORY)
+
+**User can only view videos on TrueNAS, not in LXC containers.** After every render, upload outputs to TrueNAS immediately:
+```bash
+# scp from .202 to Hermes host first (smbclient not on .202)
+scp [email protected]:~/comfy-ui/output/<file>.mp4 /tmp/
+# Then upload to TrueNAS
+smbclient -N //10.0.0.117/proxmoxBackup -c 'cd ai_vid_stock_material\\outputs; put /tmp/<file>.mp4 <file>.mp4'
+```
+Do NOT skip this step — the user cannot review videos any other way.
+
+## Fallback
+
+If true-extend produces visible jump: use FLF A→B / B→C with pre-rendered keyframe B. Pixel-exact seam but may have subtle velocity change.