Files
hermes-skills/ltx-video-pipeline/references/2clip-test-pattern.md
T

3.4 KiB
Raw Blame History

2-Clip Test Pattern — Panorama + True-Extend

Proven 2026-07-22. Validates model chain, prompt quality, and transition smoothness at minimal cost before committing to a full 6-scene story.

Steps

  1. Generate one wide Flux panorama (1536×512) with both subjects in a shared scene. Use Flux.1-dev fp8 on .202. Prompt: describe the full room with subject 1 on left, subject 2 on right, open space between them. This guarantees identical background/lighting/perspective.

  2. Crop 3 windows (768×512 each) with ffmpeg:

    • A = left (subject 1 in frame, subject 2 off-screen right)
    • B = center (mid-room transition point)
    • C = right (subject 2 in frame, subject 1 off-screen left) Upload to ~/comfy-ui/input/.
  3. Clip 1 workflow: Copy v5 fix template (/tmp/api_s1_fix_01.json). Drop ID LoRA (rewire 131.model from ["201",0] to ["200",0], delete node 201). Set imageFile to A, prompt drives slow steady rightward pan, 10s/240 frames. Submit.

  4. Extract last frame: ffmpeg -sseof -3 -i clip1.mp4 -vsync vfr -q:v 2 ~/comfy-ui/input/last_frame.png

  5. Clip 2 workflow: Same template. imageFile = extracted last frame. Prompt continues the pan, reveals subject 2. Frame C is compositional target, not hard landing point. Submit.

  6. Concat: ffmpeg -f concat -safe 0 -i list.txt -c copy output_20s.mp4

Why True-Extend, Not FLF

FLF decelerates toward its target keyframe — clip 1 slows down approaching B, clip 2 speeds up leaving it. Creates a velocity "hitch" at the seam. True-extend avoids this: constant velocity, no deceleration. LTX's Extend mode is purpose-built for seamless continuation.

Sources: WaveSpeedAI (2026), LTX blog "How to Extend AI Videos" (2026).

Model Config

  • UNET: fp8 distilled transformer
  • LoRA: Distilled only @ 0.7 (NO ID LoRA — talking-head domain, wrong for camera moves)
  • Two-stage: 8 steps denoise 1.0 + 4 steps denoise 0.42
  • Sampler: euler, Scheduler: simple, CFG: 1.0, guide_strength: 0.5
  • Resolution: 768×512, 24fps, 10s per clip
  • Audio: disabled

Validation

  • Seam: no flash/pop/jump at boundary; pan speed continuous
  • Pan: steady rightward throughout, no reversal/wobble/stall
  • Background: same room/lighting across both clips
  • Subjects: coherent, no duplication or morphing
  • Artifacts: no limb melting, flicker, ghosting, warping
  • Timing: ~10s each at 24fps (~240 frames)

Execution Confirmation (2026-07-22)

Successfully executed end-to-end on .202:

  • Flux panorama: 1536×512, 948KB, rendered in ~30s
  • Crops: A (842K), B (858K), C (845K) via ffmpeg crop filter
  • Clip 1: 241 frames, 10.04s, 1.0MB — status success
  • Last frame extracted: 611K PNG
  • Clip 2: 241 frames, 10.04s, 1.6MB — status success
  • Concat: 482 frames, 20.08s, 2.5MB — minor non-monotonic DTS warning (harmless)
  • Both clips used Distilled-only (no ID LoRA), two-stage, 768×512, 24fps

TrueNAS Upload (MANDATORY)

User can only view videos on TrueNAS, not in LXC containers. After every render, upload outputs to TrueNAS immediately:

# scp from .202 to Hermes host first (smbclient not on .202)
scp [email protected]:~/comfy-ui/output/<file>.mp4 /tmp/
# Then upload to TrueNAS
smbclient -N //10.0.0.117/proxmoxBackup -c 'cd ai_vid_stock_material\\outputs; put /tmp/<file>.mp4 <file>.mp4'

Do NOT skip this step — the user cannot review videos any other way.

Fallback

If true-extend produces visible jump: use FLF A→B / B→C with pre-rendered keyframe B. Pixel-exact seam but may have subtle velocity change.