72 lines
3.4 KiB
Markdown
72 lines
3.4 KiB
Markdown
|
# 2-Clip Test Pattern — Panorama + True-Extend
|
|||
|
|
|
|||
|
|
Proven 2026-07-22. Validates model chain, prompt quality, and transition smoothness at minimal cost before committing to a full 6-scene story.
|
|||
|
|
|
|||
|
|
## Steps
|
|||
|
|
|
|||
|
|
1. **Generate one wide Flux panorama** (1536×512) with both subjects in a shared scene. Use Flux.1-dev fp8 on .202. Prompt: describe the full room with subject 1 on left, subject 2 on right, open space between them. This guarantees identical background/lighting/perspective.
|
|||
|
|
|
|||
|
|
2. **Crop 3 windows** (768×512 each) with ffmpeg:
|
|||
|
|
- A = left (subject 1 in frame, subject 2 off-screen right)
|
|||
|
|
- B = center (mid-room transition point)
|
|||
|
|
- C = right (subject 2 in frame, subject 1 off-screen left)
|
|||
|
|
Upload to `~/comfy-ui/input/`.
|
|||
|
|
|
|||
|
|
3. **Clip 1 workflow:** Copy v5 fix template (`/tmp/api_s1_fix_01.json`). Drop ID LoRA (rewire 131.model from ["201",0] to ["200",0], delete node 201). Set imageFile to A, prompt drives slow steady rightward pan, 10s/240 frames. Submit.
|
|||
|
|
|
|||
|
|
4. **Extract last frame:** `ffmpeg -sseof -3 -i clip1.mp4 -vsync vfr -q:v 2 ~/comfy-ui/input/last_frame.png`
|
|||
|
|
|
|||
|
|
5. **Clip 2 workflow:** Same template. imageFile = extracted last frame. Prompt continues the pan, reveals subject 2. Frame C is compositional target, not hard landing point. Submit.
|
|||
|
|
|
|||
|
|
6. **Concat:** `ffmpeg -f concat -safe 0 -i list.txt -c copy output_20s.mp4`
|
|||
|
|
|
|||
|
|
## Why True-Extend, Not FLF
|
|||
|
|
|
|||
|
|
FLF decelerates toward its target keyframe — clip 1 slows down approaching B, clip 2 speeds up leaving it. Creates a velocity "hitch" at the seam. True-extend avoids this: constant velocity, no deceleration. LTX's Extend mode is purpose-built for seamless continuation.
|
|||
|
|
|
|||
|
|
Sources: WaveSpeedAI (2026), LTX blog "How to Extend AI Videos" (2026).
|
|||
|
|
|
|||
|
|
## Model Config
|
|||
|
|
|
|||
|
|
- UNET: fp8 distilled transformer
|
|||
|
|
- LoRA: Distilled only @ 0.7 (NO ID LoRA — talking-head domain, wrong for camera moves)
|
|||
|
|
- Two-stage: 8 steps denoise 1.0 + 4 steps denoise 0.42
|
|||
|
|
- Sampler: euler, Scheduler: simple, CFG: 1.0, guide_strength: 0.5
|
|||
|
|
- Resolution: 768×512, 24fps, 10s per clip
|
|||
|
|
- Audio: disabled
|
|||
|
|
|
|||
|
|
## Validation
|
|||
|
|
|
|||
|
|
- Seam: no flash/pop/jump at boundary; pan speed continuous
|
|||
|
|
- Pan: steady rightward throughout, no reversal/wobble/stall
|
|||
|
|
- Background: same room/lighting across both clips
|
|||
|
|
- Subjects: coherent, no duplication or morphing
|
|||
|
|
- Artifacts: no limb melting, flicker, ghosting, warping
|
|||
|
|
- Timing: ~10s each at 24fps (~240 frames)
|
|||
|
|
|
|||
|
|
## Execution Confirmation (2026-07-22)
|
|||
|
|
|
|||
|
|
Successfully executed end-to-end on .202:
|
|||
|
|
- Flux panorama: 1536×512, 948KB, rendered in ~30s
|
|||
|
|
- Crops: A (842K), B (858K), C (845K) via ffmpeg crop filter
|
|||
|
|
- Clip 1: 241 frames, 10.04s, 1.0MB — status success
|
|||
|
|
- Last frame extracted: 611K PNG
|
|||
|
|
- Clip 2: 241 frames, 10.04s, 1.6MB — status success
|
|||
|
|
- Concat: 482 frames, 20.08s, 2.5MB — minor non-monotonic DTS warning (harmless)
|
|||
|
|
- Both clips used Distilled-only (no ID LoRA), two-stage, 768×512, 24fps
|
|||
|
|
|
|||
|
|
## TrueNAS Upload (MANDATORY)
|
|||
|
|
|
|||
|
|
**User can only view videos on TrueNAS, not in LXC containers.** After every render, upload outputs to TrueNAS immediately:
|
|||
|
|
```bash
|
|||
|
|
# scp from .202 to Hermes host first (smbclient not on .202)
|
|||
|
|
scp [email protected]:~/comfy-ui/output/<file>.mp4 /tmp/
|
|||
|
|
# Then upload to TrueNAS
|
|||
|
|
smbclient -N //10.0.0.117/proxmoxBackup -c 'cd ai_vid_stock_material\\outputs; put /tmp/<file>.mp4 <file>.mp4'
|
|||
|
|
```
|
|||
|
|
Do NOT skip this step — the user cannot review videos any other way.
|
|||
|
|
|
|||
|
|
## Fallback
|
|||
|
|
|
|||
|
|
If true-extend produces visible jump: use FLF A→B / B→C with pre-rendered keyframe B. Pixel-exact seam but may have subtle velocity change.
|