Fix: flatten comfyui and ltx-video-pipeline to Gitea convention (flat skill dirs). comfyui v5.1.0, ltx-video-pipeline v2.0.0.
This commit is contained in:
Hermes Agent
@@ -0,0 +1,71 @@
|
||||
# 2-Clip Test Pattern — Panorama + True-Extend
|
||||
|
||||
Proven 2026-07-22. Validates model chain, prompt quality, and transition smoothness at minimal cost before committing to a full 6-scene story.
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Generate one wide Flux panorama** (1536×512) with both subjects in a shared scene. Use Flux.1-dev fp8 on .202. Prompt: describe the full room with subject 1 on left, subject 2 on right, open space between them. This guarantees identical background/lighting/perspective.
|
||||
|
||||
2. **Crop 3 windows** (768×512 each) with ffmpeg:
|
||||
- A = left (subject 1 in frame, subject 2 off-screen right)
|
||||
- B = center (mid-room transition point)
|
||||
- C = right (subject 2 in frame, subject 1 off-screen left)
|
||||
Upload to `~/comfy-ui/input/`.
|
||||
|
||||
3. **Clip 1 workflow:** Copy v5 fix template (`/tmp/api_s1_fix_01.json`). Drop ID LoRA (rewire 131.model from ["201",0] to ["200",0], delete node 201). Set imageFile to A, prompt drives slow steady rightward pan, 10s/240 frames. Submit.
|
||||
|
||||
4. **Extract last frame:** `ffmpeg -sseof -3 -i clip1.mp4 -vsync vfr -q:v 2 ~/comfy-ui/input/last_frame.png`
|
||||
|
||||
5. **Clip 2 workflow:** Same template. imageFile = extracted last frame. Prompt continues the pan, reveals subject 2. Frame C is compositional target, not hard landing point. Submit.
|
||||
|
||||
6. **Concat:** `ffmpeg -f concat -safe 0 -i list.txt -c copy output_20s.mp4`
|
||||
|
||||
## Why True-Extend, Not FLF
|
||||
|
||||
FLF decelerates toward its target keyframe — clip 1 slows down approaching B, clip 2 speeds up leaving it. Creates a velocity "hitch" at the seam. True-extend avoids this: constant velocity, no deceleration. LTX's Extend mode is purpose-built for seamless continuation.
|
||||
|
||||
Sources: WaveSpeedAI (2026), LTX blog "How to Extend AI Videos" (2026).
|
||||
|
||||
## Model Config
|
||||
|
||||
- UNET: fp8 distilled transformer
|
||||
- LoRA: Distilled only @ 0.7 (NO ID LoRA — talking-head domain, wrong for camera moves)
|
||||
- Two-stage: 8 steps denoise 1.0 + 4 steps denoise 0.42
|
||||
- Sampler: euler, Scheduler: simple, CFG: 1.0, guide_strength: 0.5
|
||||
- Resolution: 768×512, 24fps, 10s per clip
|
||||
- Audio: disabled
|
||||
|
||||
## Validation
|
||||
|
||||
- Seam: no flash/pop/jump at boundary; pan speed continuous
|
||||
- Pan: steady rightward throughout, no reversal/wobble/stall
|
||||
- Background: same room/lighting across both clips
|
||||
- Subjects: coherent, no duplication or morphing
|
||||
- Artifacts: no limb melting, flicker, ghosting, warping
|
||||
- Timing: ~10s each at 24fps (~240 frames)
|
||||
|
||||
## Execution Confirmation (2026-07-22)
|
||||
|
||||
Successfully executed end-to-end on .202:
|
||||
- Flux panorama: 1536×512, 948KB, rendered in ~30s
|
||||
- Crops: A (842K), B (858K), C (845K) via ffmpeg crop filter
|
||||
- Clip 1: 241 frames, 10.04s, 1.0MB — status success
|
||||
- Last frame extracted: 611K PNG
|
||||
- Clip 2: 241 frames, 10.04s, 1.6MB — status success
|
||||
- Concat: 482 frames, 20.08s, 2.5MB — minor non-monotonic DTS warning (harmless)
|
||||
- Both clips used Distilled-only (no ID LoRA), two-stage, 768×512, 24fps
|
||||
|
||||
## TrueNAS Upload (MANDATORY)
|
||||
|
||||
**User can only view videos on TrueNAS, not in LXC containers.** After every render, upload outputs to TrueNAS immediately:
|
||||
```bash
|
||||
# scp from .202 to Hermes host first (smbclient not on .202)
|
||||
scp [email protected]:~/comfy-ui/output/<file>.mp4 /tmp/
|
||||
# Then upload to TrueNAS
|
||||
smbclient -N //10.0.0.117/proxmoxBackup -c 'cd ai_vid_stock_material\\outputs; put /tmp/<file>.mp4 <file>.mp4'
|
||||
```
|
||||
Do NOT skip this step — the user cannot review videos any other way.
|
||||
|
||||
## Fallback
|
||||
|
||||
If true-extend produces visible jump: use FLF A→B / B→C with pre-rendered keyframe B. Pixel-exact seam but may have subtle velocity change.
|
||||
@@ -0,0 +1,126 @@
|
||||
# ComfyUI API Submission & TrueNAS Upload Pattern
|
||||
|
||||
## Problem
|
||||
Submitting complex workflow JSONs via shell heredocs often fails due to quote escaping issues. Additionally, smbclient is NOT installed on the ComfyUI LXC (10.0.0.202), so TrueNAS uploads must be done from the Hermes host.
|
||||
|
||||
## Solution: File-Based API Submission
|
||||
|
||||
### Step 1: Write Workflow to File
|
||||
```bash
|
||||
cat > /tmp/submit_ltx_render.sh << 'EOF'
|
||||
#!/bin/bash
|
||||
curl -s -X POST http://10.0.0.202:8188/prompt \
|
||||
-H "Content-Type: application/json" \
|
||||
-d '{
|
||||
"prompt": {
|
||||
"35": {"class_type": "UNETLoader", "inputs": {"unet_name": "ltx-2.3-22b-distilled_transformer_only_fp8_scaled.safetensors", "weight_dtype": "default"}},
|
||||
"131": {"class_type": "LTXDirector", "inputs": {
|
||||
"model": ["200", 0],
|
||||
"clip": ["12", 0],
|
||||
"audio_vae": ["8", 0],
|
||||
"start_second": 0,
|
||||
"end_second": 5.0,
|
||||
"duration_seconds": 5.0,
|
||||
"start_frame": 0,
|
||||
"end_frame": 120,
|
||||
"duration_frames": 120,
|
||||
"timeline_data": "{\"mainTrackEnabled\": true, \"audioTrackEnabled\": false, \"motionTrackEnabled\": false, \"global_prompt\": \"<PROMPT>\", \"segments\": [{\"type\": \"image\", \"start\": 0.0, \"length\": 120, \"prompt\": \"<DETAILED_PROMPT>\", \"imageFile\": \"<START_FRAME>\", \"guide_strength\": 1.0, \"audioFile\": \"\", \"end\": 5.0}]}",
|
||||
"guide_strength": 1.0
|
||||
}},
|
||||
"200": {"class_type": "LTX2LoraLoaderAdvanced", "inputs": {"lora_name": "ltx-2.3-22b-distilled-lora-1.1_fro90_ceil72_condsafe.safetensors", "model": ["35", 0], "strength_model": 1.0, "video": 1.0, "video_to_audio": 0.0, "audio": 0.0, "audio_to_video": 0.0, "other": 0.0}}
|
||||
// ... rest of workflow nodes
|
||||
}
|
||||
}'
|
||||
EOF
|
||||
chmod +x /tmp/submit_ltx_render.sh
|
||||
```
|
||||
|
||||
### Step 2: Submit and Capture Prompt ID
|
||||
```bash
|
||||
RESULT=$(bash /tmp/submit_ltx_render.sh)
|
||||
PROMPT_ID=$(echo "$RESULT" | python3 -c "import json,sys; print(json.load(sys.stdin)['prompt_id'])")
|
||||
echo "Submitted: $PROMPT_ID"
|
||||
```
|
||||
|
||||
### Step 3: Poll for Completion
|
||||
```bash
|
||||
while true; do
|
||||
STATUS=$(curl -s "http://10.0.0.202:8188/history/$PROMPT_ID")
|
||||
echo "$STATUS" | grep -q '"status_str":"success"' && echo "COMPLETE" && break
|
||||
echo "$STATUS" | grep -q '"status_str":"error"' && echo "ERROR" && echo "$STATUS" && break
|
||||
echo "Running..."
|
||||
sleep 10
|
||||
done
|
||||
```
|
||||
|
||||
### Step 4: Verify Output
|
||||
```bash
|
||||
# Check duration
|
||||
ffprobe -v error -show_entries format=duration -of default=noprint_wrappers=1:nokey=1 ~/comfy-ui/output/<filename>.mp4
|
||||
|
||||
# Check frames
|
||||
ffprobe -v error -select_streams v:0 -show_entries stream=nb_frames -of default=noprint_wrappers=1:nokey=1 ~/comfy-ui/output/<filename>.mp4
|
||||
|
||||
# Check resolution
|
||||
ffprobe -v error -select_streams v:0 -show_entries stream=width,height -of default=noprint_wrappers=1:nokey=1 ~/comfy-ui/output/<filename>.mp4
|
||||
```
|
||||
|
||||
## TrueNAS Upload (From Hermes Host)
|
||||
|
||||
### Prerequisites
|
||||
- smbclient is NOT on .202 (LXC)
|
||||
- Copy files to Hermes host first (10.0.0.42)
|
||||
- Upload from Hermes host using smbclient
|
||||
|
||||
### Upload Command
|
||||
```bash
|
||||
# Copy from LXC to Hermes host
|
||||
sshpass -p 'passw0rd' scp [email protected]:~/comfy-ui/output/<filename>.mp4 ~/comfy-ui/output/
|
||||
|
||||
# Upload to TrueNAS
|
||||
smbclient //10.0.0.117/proxmoxBackup -U n8n -c "cd ai_vid_stock_material\\\\outputs; put <local_filename>.mp4"
|
||||
```
|
||||
|
||||
### Alternative: Mount and Copy
|
||||
```bash
|
||||
# Mount TrueNAS share (if cifs kernel module available)
|
||||
sudo mount -t cifs //10.0.0.117/proxmoxBackup/ai_vid_stock_material/outputs /mnt/truenas_outputs -o user=n8n,vers=3.0
|
||||
|
||||
# Copy file
|
||||
cp <local_filename>.mp4 /mnt/truenas_outputs/
|
||||
|
||||
# Unmount
|
||||
sudo umount /mnt/truenas_outputs
|
||||
```
|
||||
|
||||
## Verification Checklist
|
||||
- [ ] Prompt ID captured from API response
|
||||
- [ ] Queue status shows `queue_running: []` and `queue_pending: []`
|
||||
- [ ] History shows `"status_str":"success"`
|
||||
- [ ] ffprobe confirms expected duration (5s = 121 frames at 24fps, 10s = 241 frames)
|
||||
- [ ] ffprobe confirms resolution (768×512 for 6-fix baseline)
|
||||
- [ ] File uploaded to TrueNAS at `//10.0.0.117/proxmoxBackup/ai_vid_stock_material/outputs/`
|
||||
- [ ] Filename follows naming convention: `LTX_YYYY-MM-DD_<what>_<sampler>_<duration>.mp4`
|
||||
|
||||
## Common Errors
|
||||
|
||||
### "no_prompt" Error
|
||||
- **Cause:** Submitted bare workflow JSON instead of `{\"prompt\": <workflow>}` envelope
|
||||
- **Fix:** Wrap workflow in `{"prompt": ...}` before sending to `/prompt` endpoint
|
||||
|
||||
### "tree connect failed: NT_STATUS_BAD_NETWORK_NAME"
|
||||
- **Cause:** Wrong SMB path or missing credentials
|
||||
- **Fix:** Use `//10.0.0.117/proxmoxBackup` (not the full path), then `cd` into subdirectory
|
||||
|
||||
### Quote Escaping Failures
|
||||
- **Cause:** Inline JSON with quotes breaks shell parsing
|
||||
- **Fix:** Write JSON to file first, then read with `-d @/path/to/file.json`
|
||||
|
||||
### VRAM Stuck at Idle
|
||||
- **Cause:** Job never started (error during parsing)
|
||||
- **Fix:** Check `curl -s http://localhost:8188/queue` for empty `queue_running`, then check `curl -s http://localhost:8188/history/<prompt_id>` for error messages
|
||||
|
||||
## Related Files
|
||||
- `/tmp/submit_ltx_render.sh` — Example submission script
|
||||
- `~/comfy-ui/output/` — ComfyUI output directory on .202
|
||||
- `//10.0.0.117/proxmoxBackup/ai_vid_stock_material/outputs/` — TrueNAS destination
|
||||
@@ -0,0 +1,34 @@
|
||||
# Artifact Deep Research — Question Template
|
||||
|
||||
Template for dispatching a focused deep-research pass when LTX-2.3 artifacts persist after applying the ranked fixes. Used 2026-07-22 for Story 1 "Ghost in the Wire."
|
||||
|
||||
## Structure
|
||||
|
||||
1. **OUR SETUP** — exact model chain, render settings, hardware
|
||||
2. **STORY PROMPTS** — all scene prompts verbatim
|
||||
3. **WHAT WE'VE ALREADY TRIED** — v4 and v5 settings + results
|
||||
4. **EXISTING RESEARCH** — path to prior artifact research
|
||||
5. **WHAT WE NEED NOW** — 10 specific research questions targeting our exact setup
|
||||
|
||||
## Dispatch Command
|
||||
|
||||
```bash
|
||||
hermes -p research -s deep-web-research chat -q "Read the full research question from /home/n8n/workspace/research/<date>-ltx-artifacts-question.md and execute it exhaustively..." -Q --max-turns 600 --yolo
|
||||
```
|
||||
|
||||
## Key Research Questions (from 2026-07-22 dispatch)
|
||||
|
||||
1. Given our EXACT prompts, what specific artifacts would LTX-2.3 produce with these scene descriptions?
|
||||
2. Does ID LoRA (talkvid-3k, trained for talking-head) cause artifacts on full-body action scenes?
|
||||
3. Is the LTX Director refiner stage (4 steps, denoise 0.42) introducing artifacts?
|
||||
4. Is fp8_scaled the worst fp8 variant? Compare vs fp8_e4m3fn vs v1.1 for I2V with LoRAs.
|
||||
5. Does 512×512 square aspect produce more artifacts than widescreen?
|
||||
6. Is Distilled LoRA at 0.7 still too high when stacked with ID LoRA?
|
||||
7. Are there NEW fixes or LoRAs released in the last 30 days?
|
||||
8. Could Flux.1-dev start frame inconsistencies cause artifacts?
|
||||
9. Are there known LTX Director node bugs beyond the wiring bug we already fixed?
|
||||
10. Does LTX-2.3 struggle with cyberpunk/neon-heavy scenes specifically?
|
||||
|
||||
## Expected Output
|
||||
|
||||
`/home/n8n/workspace/research/results/<YYYY-MM-DD>-ltx-2.3-artifacts-deep.md` with YAML frontmatter and findings ranked by likelihood of causing OUR specific artifacts.
|
||||
@@ -0,0 +1,46 @@
|
||||
# LTX-2.3 Artifact Root Causes — 6-Agent Consensus (2026-07-21)
|
||||
|
||||
Full research result from 6 independent agents diagnosing Story 1 "Ghost in the Wire" artifacts.
|
||||
|
||||
## Methodology
|
||||
|
||||
6 agents reviewed the exact technical setup independently:
|
||||
- Claude Opus 4.8 (SSH print mode, 9 turns)
|
||||
- Kimi K2.7 Code (kimi-c profile, 30 turns)
|
||||
- Kimi K2.6 (kimi profile, 30 turns)
|
||||
- MiniMax M3 (minimax profile, 30 turns)
|
||||
- GLM-5.2 (glm profile, 30 turns)
|
||||
- Deep research (research profile, 600 turns, 18 sources including Reddit, GitHub, HuggingFace)
|
||||
|
||||
## Consensus Findings
|
||||
|
||||
### CRITICAL (6/6 agents agree)
|
||||
1. **LoRA stacking at 1.0+1.0** — Distilled + ID LoRA both at full strength cause interference on faces/hands. Fix: Distilled → 0.7, ID → 0.6 (or drop ID if no audio).
|
||||
|
||||
### HIGH (4-5/6 agents agree)
|
||||
2. **10s exceeds temporal coherence** — LTX-2.3 degrades after 5-6s. Fix: test at 5s (121 frames).
|
||||
3. **linear_quadratic scheduler** — Distilled model is fragile with non-standard schedules. Fix: switch to `simple`.
|
||||
4. **Inconsistent guide_strength** — 0.5 on scene 1, 0.7 on others breaks flow. Fix: 0.5 uniform.
|
||||
|
||||
### MEDIUM (2-3/6 agents agree)
|
||||
5. **Audio track enabled with no audio** — AudioVAE NaN risk. Fix: disable audioTrackEnabled.
|
||||
6. **Text in prompts** — LTX can't render readable text. Fix: replace with abstract descriptions.
|
||||
7. **512×512 square aspect** — LTX prefers widescreen. Fix: 768×512 (requires new frames).
|
||||
8. **fp8_scaled v1.0 quality** — Kijai's v1.1 is better. Fix: model upgrade.
|
||||
|
||||
## Deep Research Sources (18 total)
|
||||
|
||||
Key community sources:
|
||||
- GitHub #148: End-of-video logo/watermark artifacts (upscaler v1.0 contamination)
|
||||
- HF Discussion #13: Upscaler sigma fix table by frame count
|
||||
- Reddit r/StableDiffusion: Skin compression fix (Nearest Exact interpolation)
|
||||
- Reddit r/StableDiffusion: Deformed bodies / identity drift in I2V
|
||||
- Reddit r/comfyui: Official workflow vs ComfyUI built-in (prompt ignoring bug)
|
||||
- Reddit r/StableDiffusion: RL LoRA for coherence (OmniNFT)
|
||||
- HuggingFace RuneXX: Dev model sampler comparison
|
||||
- HuggingFace LiconStudio: MSR V2 LoRA for identity preservation
|
||||
- GitHub #255: Image conditioning does not preserve identity
|
||||
- GitHub #244: Noise in distilled 1.1 model
|
||||
- LTX.io blog: Official artifact reduction guide
|
||||
|
||||
Full research: `~/workspace/research/results/2026-07-21-ltx-2.3-artifacts.md`
|
||||
@@ -0,0 +1,24 @@
|
||||
# Canonical Character Descriptions
|
||||
|
||||
Ground truth for all LTX prompts and Flux panorama generation. Do NOT improvise or use generic descriptions — these are the exact descriptions the user provided.
|
||||
|
||||
## Boss
|
||||
|
||||
Black suit, muscular build, shaved head, goatee. Stern expression, chrome temple implant. Stock reference: `SHEETS2_00005_Boss.png` on TrueNAS.
|
||||
|
||||
**Prompt fragment:** "A muscular boss with shaved head and goatee wearing a black suit sits at a terminal in a neon-lit cyberpunk room. His chrome temple implant glints in the neon glow. Stern expression."
|
||||
|
||||
## Woman
|
||||
|
||||
Mid-20s, short dark hair, chrome temple implants, cybernetic fingers, black synth-leather jacket. Stock references: `cyberpunk_woman_neon_01.jpg`, `cyberpunk_woman_neon_02.jpg` on TrueNAS.
|
||||
|
||||
**Prompt fragment:** "A woman in her mid-20s with short dark hair, chrome temple implants, and cybernetic fingers wears a black synth-leather jacket. She stands by a window gazing out at the neon cityscape."
|
||||
|
||||
## Pitfall: Generic Descriptions
|
||||
|
||||
Using generic descriptions like "stern boss" or "woman by window" instead of the canonical descriptions causes:
|
||||
- Face distortion (model doesn't know what "stern boss" looks like)
|
||||
- Character morphing (model blends between prompt and start frame)
|
||||
- Wrong character entirely (model substitutes its own training data)
|
||||
|
||||
Always use the exact canonical descriptions above in every prompt that references these characters.
|
||||
@@ -0,0 +1,55 @@
|
||||
# Claude Live SSH Inspection — Plan-Building Pattern
|
||||
|
||||
When Claude needs to build a plan for a target machine it can SSH into, have it inspect the live state FIRST before writing the plan. This eliminates guesswork about what's installed, what models are on disk, what workflows exist, and what the queue looks like.
|
||||
|
||||
## Pattern
|
||||
|
||||
1. **Write the question** — include the goal, what to inspect, and the deliverable path
|
||||
2. **scp to 10.0.0.28** — standard ask-claude flow
|
||||
3. **Claude SSHs into the target** — inspects files, checks queue, reads workflow JSONs, verifies model paths
|
||||
4. **Claude writes the plan** — saves to its own host (10.0.0.28); scp back to Hermes host after
|
||||
|
||||
## When to Use
|
||||
|
||||
- Building a plan that depends on specific files/models/workflows on a target machine
|
||||
- The target's state is complex enough that describing it in the prompt would be error-prone
|
||||
- You want Claude to verify claims against reality (e.g., "is this LoRA actually on disk?")
|
||||
|
||||
## Example (2026-07-22)
|
||||
|
||||
Goal: 2-clip LTX-2.3 test plan. Target: 10.0.0.202 (ComfyUI).
|
||||
|
||||
Claude was told to SSH into .202 and inspect:
|
||||
- LoRAs on disk: `ls ~/comfy-ui/models/loras/`
|
||||
- Workflow JSONs: `ls /tmp/api_s1_*.json`
|
||||
- Diffusion models: `ls ~/comfy-ui/models/diffusion_models/`
|
||||
- Start frames: `ls ~/comfy-ui/input/ltx_start_frame*`
|
||||
- Queue status: `curl -s http://localhost:8188/queue`
|
||||
|
||||
Key discoveries from live inspection (none were in any state file):
|
||||
- Distilled LoRA on disk is the 384 variant, not 384-1.1
|
||||
- ID LoRA (talkvid-3k) is trained for talking-head, wrong for action scenes
|
||||
- FLF example workflows exist on the box
|
||||
- Existing start frames are all cyberpunk single-subject — can't be reused
|
||||
- Queue empty, clear to run
|
||||
|
||||
Plan produced: FLF panorama approach — one wide Flux panorama, 3 cropped windows (A/B/C), Clip 1 FLF A→B, Clip 2 FLF B→C, shared frame B = seamless pan.
|
||||
|
||||
## Error Recovery
|
||||
|
||||
Claude may error on first attempt (SSH timeout, tool failure, wrong command format). **Resume, don't start fresh:**
|
||||
|
||||
| Attempt | Turns | Cost | Result |
|
||||
|---------|-------|------|--------|
|
||||
| Fresh session | 21 | $1.05 | is_error: true, empty result |
|
||||
| Resume (--resume) | 2 | $0.11 | Success — self-diagnosed, recovered, produced plan |
|
||||
|
||||
**Pattern:** When Claude returns `is_error: true` with empty result, write a short follow-up asking what went wrong, scp it, and resume with `--resume <session_id>`. Claude self-diagnoses and recovers. Do NOT start a fresh session — you lose the context and pay the full cost again.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- Claude cannot write files to the Hermes host directly. It writes to 10.0.0.28 and you scp back, or it outputs inline for you to save.
|
||||
- SSH credentials must be in the question (Claude doesn't have them in its environment). Use the exact command: `sshpass -p 'passw0rd' ssh [email protected]`
|
||||
- Keep the inspection list focused — 5-6 specific checks, not a full system audit.
|
||||
- **Live-SSH inspection can fail silently.** Claude may error out with `is_error: true` and empty result after 20+ turns. Resume with `--resume` — Claude self-diagnoses. Do not start fresh.
|
||||
- **Don't send follow-up instructions mid-build.** If the user says "tell Claude X once he completes," wait for the current turn to finish. Sending mid-execution wastes the instruction (Claude is busy) and frustrates the user.
|
||||
@@ -0,0 +1,52 @@
|
||||
# Claude Wiring Diagnosis — 2026-07-21
|
||||
|
||||
Claude Opus 4.8 SSH'd into 10.0.0.202 and diagnosed the subject duplication bug.
|
||||
|
||||
## Root Cause
|
||||
|
||||
Node 132 (LTXDirectorGuide, the refiner pass) had its `latent` input wired to `["34", 0]` (uncropped LTXVSeparateAVLatent output) instead of `["55", 2]` (LTXDirectorCropGuides cropped output).
|
||||
|
||||
## Mechanism (from source code review)
|
||||
|
||||
`LTXDirectorGuide` doesn't blend the I2V image in — it **appends** it as extra latent frames on the end of the sequence, then records how many to remove later in a conditioning key (`ltx_director_guide.py:596`). `LTXDirectorCropGuides` trims exactly that many afterward.
|
||||
|
||||
The critical line is `ltx_director_guide.py:339`:
|
||||
```python
|
||||
initial_latent_length = int(latent_length) # captured from whatever arrives
|
||||
```
|
||||
|
||||
The graph runs 133 first (8 steps, denoise 1.0), then 132 (4 steps, 0.42). Node 55 correctly crops 133's guide frames off... but `55:2` (the cropped latent) is never consumed. Node 132 takes `34:0` instead — the raw, uncropped sampler output. Only 55's *conditioning* outputs are wired, which is why the break is invisible in the UI.
|
||||
|
||||
So 132 counts stage-1's leftover guide frames as **real video content**, appends a second guide on top, and the final crop removes only one set. The orphaned guide frames decode as actual footage: a second copy of the man, frozen in the start-frame pose, while the real generation gets out of bed.
|
||||
|
||||
## Empirical Confirmation
|
||||
|
||||
- **ffprobe: 249 frames / 10.375s. Requested: 240 / 10.0s.** The overshoot is leftover guide latent, decoded as video.
|
||||
- **Batches 1, 2, and 3 are all 249 frames** (`00007`, `00013`, `00019`, `00025`–`00030`). Byte-identical symptom across every "fix" — the wiring never changed.
|
||||
- **Log shows two `[LTXDirectorGuide] execute started` per render, both "Using Appended Keyframe Guidance"** — double append, confirmed.
|
||||
|
||||
## Reference Workflow Comparison
|
||||
|
||||
`example_workflows/LTX_Director_2_Workflow_Hotfix.json` wires it as:
|
||||
```
|
||||
stage1 → 34 Separate → 55 CropGuides → 14 LTXVLatentUpsampler → 132 Guide
|
||||
(55:2, cropped)
|
||||
```
|
||||
|
||||
Our v3 deleted node 14 (`LTXVLatentUpsampler`) and reconnected 132 straight to `34:0`. Removing the upsampler is what orphaned the crop.
|
||||
|
||||
## Fix
|
||||
|
||||
Point `132.inputs.latent` at `["55", 2]` instead of `["34", 0]`. Restoring the upsampler between them is the fully-correct form.
|
||||
|
||||
## Verification
|
||||
|
||||
Output should be exactly **240 frames** for 10s@24fps. 249 frames = bug present.
|
||||
|
||||
## Other Findings
|
||||
|
||||
- **ID LoRA / distilled LoRA at 1.0** — not the cause. Log confirms `is_lora_active: False, ic_lora_name: None`.
|
||||
- **Scene 2 "soup" problem** — separate issue. The start frame `ltx_start_frame_02_00001_.png` shows a man over a tall steaming stockpot, which reads as soup. Bad start frame, not a prompt bug.
|
||||
- **Seed is 0 and shared** — bump to random. ComfyUI caches identical graphs.
|
||||
- **Session ID:** 5381500e-a071-42f5-8d6b-d780520a981b
|
||||
- **Cost:** $1.34 USD, 31 turns
|
||||
@@ -0,0 +1,108 @@
|
||||
# IC-LoRA Ingredients — Reference Sheet Control for LTX 2.3
|
||||
|
||||
**Model:** Lightricks/LTX-2.3-22b-IC-LoRA-Ingredients
|
||||
**File:** `ltx-2.3-22b-ic-lora-ingredients-0.9.safetensors` (1.31 GB)
|
||||
**HF URL:** https://huggingface.co/Lightricks/LTX-2.3-22b-IC-LoRA-Ingredients
|
||||
**Status:** Downloaded on .202 (2026-07-22, 1.31 GB) — gated, requires "Agree and Access" on HF website + valid token
|
||||
**Token:** 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 (user: MDKRUSH, 2026-07-22)
|
||||
**Trained bucket:** 768×448, 121 frames, 24 fps
|
||||
|
||||
## fp8 Compatibility Gap (UNRESOLVED — deep research in progress 2026-07-22)
|
||||
|
||||
The official IC-LoRA workflow (`LTX-2.3_ICLoRA_Ingredients_Single_Stage_Distilled.json`) uses a completely different architecture from our pipeline:
|
||||
|
||||
| Component | Official IC-LoRA Workflow | Our Pipeline |
|
||||
|-----------|--------------------------|--------------|
|
||||
| Model loader | CheckpointLoaderSimple (full 46GB checkpoint) | UNETLoader (fp8 distilled, 23GB) |
|
||||
| Sampler | SamplerCustomAdvanced + ManualSigmas + CFGGuider | LTXDirector (single-stage) |
|
||||
| Text encoder | GemmaAPITextEncode | LTXConditioning |
|
||||
| Total nodes | 38 | 22 |
|
||||
|
||||
**Key question:** Can `LTXICLoRALoaderModelOnly` accept a model from UNETLoader (fp8 distilled transformer) instead of CheckpointLoaderSimple? The node takes a "model" input — compatibility with fp8 UNET-only model is unverified. Deep research dispatched 2026-07-22 to answer this and 11 other compatibility questions. Results pending at `/home/n8n/workspace/research/results/2026-07-22-ic-lora-fp8-compatibility.md`.
|
||||
|
||||
## What It Does
|
||||
|
||||
Conditions video generation on a **reference sheet** — a single composite image inventorying characters, props, and location. The model reads the reference latents in-context and renders a new clip whose characters, props, and setting match the sheet. This is the solution for using stock character sheets (like `SHEETS2_00005_Boss.png` on TrueNAS) as character consistency anchors.
|
||||
|
||||
## Recommended Settings (from official README)
|
||||
|
||||
| Setting | Value | Notes |
|
||||
|---------|-------|-------|
|
||||
| LoRA strength | **1.4** | NOT 1.0 — official recommendation |
|
||||
| Inference steps | **30** | Higher than our standard 18 |
|
||||
| Guidance scale | **4.0** | NOT 1.0 — video-to-video mode |
|
||||
| Resolution | 768×448 | Trained bucket — best results here |
|
||||
| Frames | 121 | 24 fps |
|
||||
| Negative prompt | `worst quality, inconsistent motion, blurry, jittery, distorted` | |
|
||||
| STG | mode `stg_v`, block 29, scale 1.0 | Helps motion stability |
|
||||
|
||||
## Prompt Format
|
||||
|
||||
Two-part structure (matching training):
|
||||
|
||||
```
|
||||
Reference sheet: <description of the panels in the sheet — characters, props, location>
|
||||
Generated video: <description of the action / shot you want generated>
|
||||
```
|
||||
|
||||
The `Reference sheet:` text describes what's in the panels. The `Generated video:` text drives the action. The model reads the reference latents for "what things look like" and the prompt for "what happens."
|
||||
|
||||
## Control Signal Requirements
|
||||
|
||||
- Reference sheet: single composite image with one clean panel per visual element
|
||||
- Each character: face close-up + body turnaround
|
||||
- Each prop: product-style render
|
||||
- One clean location panel
|
||||
- Laid out on black background with NO text
|
||||
- **Bigger panels carry over better** — give important elements larger panels
|
||||
- Reference must be looped into a static video ≥ 121 frames at output resolution
|
||||
|
||||
## Node Chain (ComfyUI)
|
||||
|
||||
```
|
||||
CheckpointLoaderSimple (full 46GB checkpoint — adapt to UNETLoader fp8 for 24GB)
|
||||
→ LTXICLoRALoaderModelOnly (ingredients-0.9.safetensors, strength 1.4)
|
||||
→ LTXAddVideoICLoRAGuide (reference sheet as static video input)
|
||||
→ LTXVCropGuides → SamplerCustomAdvanced
|
||||
```
|
||||
|
||||
## Nodes on .202
|
||||
|
||||
- `LTXICLoRALoaderModelOnly` — in `iclora.py`, loads IC-LoRA and extracts `latent_downscale_factor`
|
||||
- `LTXAddVideoICLoRAGuide` — in `iclora.py`, applies reference video conditioning
|
||||
- `LTXAddVideoICLoRAGuideAdvanced` — extended version with `attention_strength` and `attention_mask`
|
||||
|
||||
## Example Workflow
|
||||
|
||||
On .202: `~/comfy-ui/custom_nodes/ComfyUI-LTXVideo/example_workflows/2.3/LTX-2.3_ICLoRA_Ingredients_Single_Stage_Distilled.json`
|
||||
|
||||
Uses 39 nodes including: CheckpointLoaderSimple, LTXICLoRALoaderModelOnly, LTXAddVideoICLoRAGuide, GemmaAPITextEncode, LTXVCropGuides, SamplerCustomAdvanced, ManualSigmas.
|
||||
|
||||
## Download Process
|
||||
|
||||
1. Visit https://huggingface.co/Lightricks/LTX-2.3-22b-IC-LoRA-Ingredients
|
||||
2. Click "Agree and Access" (gated model)
|
||||
3. Download with HF token:
|
||||
```bash
|
||||
HF_TOKEN='hf_...' huggingface-cli download Lightricks/LTX-2.3-22b-IC-LoRA-Ingredients \
|
||||
ltx-2.3-22b-ic-lora-ingredients-0.9.safetensors \
|
||||
--local-dir ~/comfy-ui/models/loras/
|
||||
```
|
||||
4. Place in `~/comfy-ui/models/loras/` on .202
|
||||
|
||||
## Stock Integration
|
||||
|
||||
The boss stock on TrueNAS (`ai_vid_stock_material/character_refs/SHEETS2_00005_Boss.png`) is a character sheet — exactly the input format IC-LoRA Ingredients expects. To use it:
|
||||
|
||||
1. Download the character sheet from TrueNAS
|
||||
2. Create a reference sheet image with the boss character panel(s)
|
||||
3. Loop into a 121-frame static video at 768×448
|
||||
4. Feed as the reference/control input to LTXAddVideoICLoRAGuide
|
||||
|
||||
## Tips (from README)
|
||||
|
||||
- **Bigger panels carry over better** — give important characters/props larger panels
|
||||
- **Identity drift fix:** ensure clean front-facing close-up + full turnaround for each character
|
||||
- **Element not appearing:** add a dedicated panel for any prop/character that needs to persist
|
||||
- **Reference too short:** static video must be ≥ 121 frames
|
||||
- **Element-driven reference-sheet generator** exists for authoring sheets (referenced in README)
|
||||
@@ -0,0 +1,118 @@
|
||||
# Modify-and-Re-Render Recipe
|
||||
|
||||
Proven pattern for changing settings on existing workflow JSONs and re-rendering without rebuilding from scratch. Used 2026-07-22 for the sampler+duration 2-clip test.
|
||||
|
||||
## When to Use
|
||||
|
||||
- Changing sampler, steps, scheduler, duration, or other render settings
|
||||
- Testing a new LoRA or model chain variant on existing scenes
|
||||
- Re-running a failed batch with corrected settings
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Copy existing workflow JSONs** to `_v2` variants (preserve originals):
|
||||
```bash
|
||||
cp /tmp/api_2clip_C1.json /tmp/api_2clip_C1_v2.json
|
||||
cp /tmp/api_2clip_C2.json /tmp/api_2clip_C2_v2.json
|
||||
```
|
||||
|
||||
2. **Modify in-place with Python via scp** (SSH heredoc quoting is fragile — scp the script, then run it):
|
||||
|
||||
Write the modification script locally, scp it to .202, then execute:
|
||||
```bash
|
||||
# Write script locally
|
||||
cat > /tmp/modify_v3.py << 'PYEOF'
|
||||
import json
|
||||
for fname in ["/tmp/api_boss_C1_v3.json", "/tmp/api_boss_C2_v3.json"]:
|
||||
with open(fname) as f:
|
||||
d = json.load(f)
|
||||
p = d["prompt"]
|
||||
|
||||
# Sampler change (node 20 is KSamplerSelect, not KSampler)
|
||||
p["20"]["inputs"]["sampler_name"] = "euler"
|
||||
|
||||
# Duration change (node-level)
|
||||
p["131"]["inputs"]["duration_frames"] = 121
|
||||
p["131"]["inputs"]["end_frame"] = 121
|
||||
p["131"]["inputs"]["duration_seconds"] = 5.04
|
||||
|
||||
# Duration change (timeline_data JSON — CRITICAL, don't skip)
|
||||
td = json.loads(p["131"]["inputs"]["timeline_data"])
|
||||
td["normalDurationFrames"] = 121
|
||||
for seg in td["segments"]:
|
||||
seg["length"] = 121
|
||||
seg["end"] = 5.04
|
||||
p["131"]["inputs"]["timeline_data"] = json.dumps(td)
|
||||
|
||||
with open(fname, "w") as f:
|
||||
json.dump(d, f, indent=2)
|
||||
print(f"Updated {fname}: sampler={p['20']['inputs']['sampler_name']}")
|
||||
PYEOF
|
||||
|
||||
# scp to .202 and run
|
||||
sshpass -p 'passw0rd' scp /tmp/modify_v3.py [email protected]:/tmp/
|
||||
sshpass -p 'passw0rd' ssh [email protected] python3 /tmp/modify_v3.py
|
||||
```
|
||||
|
||||
**Why scp, not heredoc:** SSH heredoc quoting strips quotes from Python string-keyed dicts, causing `KeyError` on nodes like `"20"`. The scp approach preserves the script exactly as written. Confirmed 2026-07-22: heredoc failed twice (KeyError on node 20); scp worked first try.
|
||||
|
||||
3. **Verify all changes** before submitting:
|
||||
```python
|
||||
for fname in files:
|
||||
d = json.load(open(fname))
|
||||
p = d["prompt"]
|
||||
assert p["20"]["inputs"]["sampler_name"] == "euler_ancestral_cfg_pp"
|
||||
assert p["131"]["inputs"]["duration_frames"] == 121
|
||||
td = json.loads(p["131"]["inputs"]["timeline_data"])
|
||||
assert td["segments"][0]["length"] == 121
|
||||
```
|
||||
|
||||
4. **Submit to queue:**
|
||||
```bash
|
||||
curl -s -X POST http://localhost:8188/prompt -H "Content-Type: application/json" -d @/tmp/api_2clip_C1_v2.json
|
||||
curl -s -X POST http://localhost:8188/prompt -H "Content-Type: application/json" -d @/tmp/api_2clip_C2_v2.json
|
||||
```
|
||||
|
||||
5. **Wait for queue drain** (blocking poll from Hermes host):
|
||||
```bash
|
||||
sshpass -p 'passw0rd' ssh [email protected] "while true; do q=\$(curl -s http://localhost:8188/queue | python3 -c \"import json,sys; d=json.load(sys.stdin); print(len(d['queue_running'])+len(d['queue_pending']))\"); if [ \"\$q\" -eq 0 ]; then echo 'QUEUE_EMPTY'; break; fi; echo \"queue: \$q\"; sleep 30; done"
|
||||
```
|
||||
|
||||
6. **Identify new outputs** (search both output directories):
|
||||
```bash
|
||||
find ~/comfy-ui/output -name "*.mp4" -newer /tmp/api_2clip_C1_v2.json -type f
|
||||
```
|
||||
|
||||
7. **Verify frame counts:**
|
||||
```bash
|
||||
ffprobe -v quiet -show_entries stream=nb_frames,codec_name,width,height,duration -of csv=p=0 <file>
|
||||
```
|
||||
|
||||
8. **Concat, scp, upload to TrueNAS** (standard pipeline steps 7-8).
|
||||
|
||||
## Key Pitfalls
|
||||
|
||||
- **sshpass is NOT on .202 — pull files from Hermes host instead of pushing from .202.** When transferring output files from .202 to the Hermes host, `sshpass` is not installed on .202. Use `sshpass -p 'passw0rd' scp n8n@10.0.0.202:<remote_path> <local_path>` from the Hermes host (pull), not `ssh ... 'sshpass scp ...'` from .202 (push). Confirmed 2026-07-22: push attempt failed with `sshpass: command not found`.
|
||||
- **CRITICAL: Verify you have the RIGHT template before modifying.** `/tmp/` accumulates multiple generations with confusingly similar names: `api_2clip_C1.json` (old living room), `api_cyberpunk_C1.json` (cyberpunk), `api_2clip_01.json` (earlier variant). ALWAYS inspect `imageFile` and `prompt` before modifying:
|
||||
```bash
|
||||
python3 -c "import json; d=json.load(open('/tmp/api_XXX.json')); td=json.loads(d['prompt']['131']['inputs']['timeline_data']); print(td['segments'][0]['imageFile'], td['segments'][0]['prompt'][:80])"
|
||||
```
|
||||
Confirmed 2026-07-22: pointed delegation at wrong template, produced living room output instead of cyberpunk.
|
||||
- **Must update BOTH node-level AND timeline_data JSON fields.** The LTXDirector reads `segment["length"]` from timeline_data for actual frame count. Node-level `duration_frames` alone won't change output duration.
|
||||
- **121 frames = 8×15+1** satisfies the 8n+1 requirement for LTX.
|
||||
- **5.04 seconds** is the correct `duration_seconds` for 121 frames at 24fps (121/24 ≈ 5.0417).
|
||||
- **Verify with ffprobe, not by file size.** File size varies with content complexity; frame count is the ground truth.
|
||||
- **Use ComfyUI history API to identify outputs reliably.** After queue drains, query each prompt_id:
|
||||
```bash
|
||||
curl -s http://localhost:8188/history/<prompt_id> | python3 -c "
|
||||
import json,sys
|
||||
d=json.load(sys.stdin)
|
||||
entry=d.get('<prompt_id>',{})
|
||||
outputs=entry.get('outputs',{})
|
||||
node37=outputs.get('37',{})
|
||||
images=node37.get('images',[])
|
||||
for img in images:
|
||||
print(img.get('filename','?'))
|
||||
"
|
||||
```
|
||||
This works regardless of which output directory the workflow uses. Fallback: `find ~/comfy-ui/output -name "*.mp4" -newer <reference_file> -type f`.
|
||||
@@ -0,0 +1,44 @@
|
||||
# Multi-Agent Parallel Review Pattern (2026-07-21)
|
||||
|
||||
Proven pattern for diagnosing complex technical issues: dispatch 5+ independent agents simultaneously with the same diagnostic brief, then consolidate findings into a ranked consensus.
|
||||
|
||||
## When to Use
|
||||
|
||||
- Complex artifact/quality issues where no single agent has the full answer
|
||||
- User reports "a LOT of artifacts" or "all kinds of inconsistencies" but can't pinpoint the cause
|
||||
- Need both technical review (model settings, workflow) AND community research (GitHub issues, Reddit, social media)
|
||||
|
||||
## Pattern
|
||||
|
||||
### 1. Write a single diagnostic brief
|
||||
Save to `/tmp/<topic>-review.txt`. Include:
|
||||
- Exact technical setup (model, settings, prompts, frame counts)
|
||||
- What the user is seeing (symptoms)
|
||||
- What to investigate (specific questions)
|
||||
- Web search mandate: "Use mcp_searxng_searxng_web_search for every claim and cite the source URL"
|
||||
|
||||
### 2. Dispatch all peers in parallel
|
||||
Use `terminal(background=true, notify_on_complete=true)` for each:
|
||||
- Claude Opus (ask.sh on 10.0.0.28) — best for technical reasoning
|
||||
- Kimi K2.7 Code (kimi-c profile) — good for code-level analysis
|
||||
- Kimi K2.6 (kimi profile) — broad knowledge
|
||||
- MiniMax M3 (minimax profile) — alternative perspective
|
||||
- GLM-5.2 (glm profile) — alternative perspective
|
||||
- Deep research (research profile, 600 turns) — GitHub, Reddit, HuggingFace, social media
|
||||
|
||||
### 3. Consolidate findings
|
||||
When all complete, build a ranked table:
|
||||
- CRITICAL: all agents agree
|
||||
- HIGH: 4-5 agents agree
|
||||
- MEDIUM: 2-3 agents agree
|
||||
- UNIQUE: single agent, high-impact
|
||||
|
||||
### 4. Apply fixes in order
|
||||
Test after each fix. Start with zero-cost config changes before model downloads or frame regeneration.
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **kimi-c may hit tool name issues** — the SearXNG MCP tool uses double underscores (`mcp__searxng__`) but the prompt says single. kimi-c may burn turns trying to resolve this. Accept partial results.
|
||||
- **MiniMax may produce truncated output** — the process log may only show reasoning, not the final answer. Check the full log with `process(action='log')`.
|
||||
- **Deep research can't access X/Twitter or Discord** — SearXNG returns empty for social media queries. Document as uncertainty, not failure.
|
||||
- **Don't wait for all before acting** — apply consensus fixes as soon as 3+ agents agree. The remaining peers add detail but shouldn't block progress.
|
||||
@@ -0,0 +1,72 @@
|
||||
# Official LTX Sample Prompts
|
||||
|
||||
Extracted from ltx.io/blog/ltx-2-3-prompt-guide and ltx.io/blog/how-to-write-a-prompt (June 2026). These are the official examples — use as templates for structure and style.
|
||||
|
||||
## Example 1: News Broadcast (T2V, dialogue + camera)
|
||||
|
||||
EXT. SMALL TOWN STREET – MORNING – LIVE NEWS BROADCAST
|
||||
|
||||
The shot opens on a news reporter standing in front of a row of cordoned-off cars, yellow caution tape fluttering behind him. The light is warm, early sun reflecting off the camera lens. The faint hum of chatter and distant drilling fills the air. The reporter, composed but visibly excited, looks directly into the camera, microphone in hand.
|
||||
|
||||
Reporter (live): "Thank you, Sylvia. And yes — this is a sentence I never thought I'd say on live television — but this morning, here in the quiet town of New Castle, Vermont… black gold has been found!"
|
||||
|
||||
He gestures slightly toward the field behind him.
|
||||
|
||||
Reporter (grinning): "If my cameraman can pan over, you'll see what all the excitement's about."
|
||||
|
||||
The camera pans right, slowly revealing a construction site surrounded by workers in hard hats. A beat of silence — then, with a sudden roar, a geyser of oil erupts from the ground, blasting upward in a violent plume. Workers cheer and scramble, the black stream glistening in the morning light. The camera shakes slightly, trying to stay focused through the chaos.
|
||||
|
||||
Reporter (off-screen, shouting over the noise): "There it is, folks — the moment New Castle will never forget!"
|
||||
|
||||
The camera catches the sunlight gleaming off the oil mist before pulling back, revealing the entire scene — the small-town skyline silhouetted against the wild fountain of oil.
|
||||
|
||||
## Example 2: Frog Yoga (T2V, comedy, dialogue + timing)
|
||||
|
||||
The camera opens in a calm, sunlit frog yoga studio. Warm morning light washes over the wooden floor as incense smoke drifts lazily in the air. The senior frog instructor sits cross-legged at the center, eyes closed, voice deep and calm.
|
||||
|
||||
"We are one with the pond."
|
||||
All the frogs answer softly: "Ommm..."
|
||||
|
||||
"We are one with the mud."
|
||||
"Ommm..."
|
||||
|
||||
He smiles faintly. "We are one with the flies."
|
||||
|
||||
A pause. The camera pans to the side towards one frog who twitches, eyes darting. Suddenly its tongue snaps out, catching a fly mid-air and pulling it into its mouth.
|
||||
|
||||
The master exhales slowly, still serene. "But we do not chase the flies..." Beat. "not during class."
|
||||
|
||||
The guilty frog lowers its head in shame, folding its hands back into a meditative pose. The other frogs resume their chant: "Ommm..."
|
||||
|
||||
Camera holds for a moment on the embarrassed frog, eyes closed too tightly, pretending nothing happened.
|
||||
|
||||
## Example 3: Dialogue with Acting Directions (from prompt guide)
|
||||
|
||||
A middle-aged man with greying hair speaks in a sad, slow-paced voice, "I remember after you kids came along..." He pauses and looks to the side, then continues, "your mom..." His eyes widen momentarily. He finishes with a cracking voice, "said something to me I never quite understood." The camera slowly zooms into his face. The audio is crisp with faint room tone.
|
||||
|
||||
## Example 4: Structured Prompt (from "How to Write a Prompt")
|
||||
|
||||
**Subject:** The sun and a high-tech cityscape with glowing skyscrapers
|
||||
**Action:** The sun rising above the horizon, a second black sun appearing in the distance
|
||||
**Framing:** Wide shot capturing the full skyline
|
||||
**Lighting and Style:** Warm golden light, sci-fi cinematic register
|
||||
**Camera Motion:** Slow upward pan revealing more of the sky
|
||||
|
||||
**Final Prompt:** "Wide shot of the sun rising over a high-tech futuristic city with glowing skyscrapers. Warm, golden lighting. Camera slowly pans upward to reveal another sun — black and ominous in the distance."
|
||||
|
||||
## Example 5: Vague vs Structured (from "How to Write a Prompt")
|
||||
|
||||
| Vague | Structured |
|
||||
|-------|------------|
|
||||
| "A woman walking in a city at night." | "Medium shot, eye level, tracking from behind. A woman in a navy trench coat walks through a rain-soaked Tokyo side street at night, neon reflections in the puddles, shallow depth of field." |
|
||||
| "Show a sunrise in a city" | "Wide shot of the sun rising over a high-tech futuristic city with glowing skyscrapers. Warm golden lighting. Camera slowly pans upward to reveal a second black sun in the distance." |
|
||||
| "Make a video of someone walking" | "Medium shot tracking a woman in a red coat walking through a crowded Tokyo street at night, neon signs reflecting in puddles, handheld camera following from behind." |
|
||||
|
||||
## Key Patterns
|
||||
|
||||
- **Dialogue:** Break into short phrases with acting directions between each line
|
||||
- **Structure:** framing → subject → action → style → motion (shot-size-first) OR subject → action → framing → style → motion (subject-first)
|
||||
- **I2V:** Focus on motion and action — start frame already defines the visual
|
||||
- **Length:** Longer prompts consistently outperform short ones on 2.3
|
||||
- **Emotion:** Physical cues, never abstract labels
|
||||
- **Camera:** State "static" explicitly if no movement; state exact motion otherwise
|
||||
@@ -0,0 +1,65 @@
|
||||
# LTX Pipeline State Template
|
||||
|
||||
Copy this template at the start of any LTX pipeline session. Update as you go.
|
||||
|
||||
```markdown
|
||||
# LTX Video Pipeline — Session State
|
||||
|
||||
**Date:** YYYY-MM-DD | **Target:** 10.0.0.202 (RTX 4090 24GB)
|
||||
|
||||
## Current Model Chain
|
||||
|
||||
```
|
||||
UNETLoader (fp8 distilled)
|
||||
→ [LoraLoaderModelOnly (Transition) — only if using transitions]
|
||||
→ LTX2LoraLoaderAdvanced (Distilled, strength 1.0)
|
||||
→ LTX2LoraLoaderAdvanced (ID, strength 1.0)
|
||||
→ LTXDirector
|
||||
```
|
||||
|
||||
## Render Settings
|
||||
|
||||
| Setting | Value |
|
||||
|---------|-------|
|
||||
| Resolution | 512×512 |
|
||||
| FPS | 24 |
|
||||
| Duration | 10s per scene |
|
||||
| guide_strength | 0.7 |
|
||||
| CFG | 1.0 (4.0 for Transition scenes) |
|
||||
| Steps | 8 main / 4 guide |
|
||||
|
||||
## Scene Inventory
|
||||
|
||||
| # | Prompt Summary | Start Frame | Audio | Output File | Status |
|
||||
|---|---------------|-------------|-------|-------------|--------|
|
||||
| 1 | | | | | |
|
||||
| 2 | | | | | |
|
||||
| 3 | | | | | |
|
||||
| 4 | | | | | |
|
||||
| 5 | | | | | |
|
||||
| 6 | | | | | |
|
||||
|
||||
## Queue Status
|
||||
|
||||
- Running:
|
||||
- Pending:
|
||||
- GPU:
|
||||
|
||||
## Outputs
|
||||
|
||||
| File | Scene | Duration | Size |
|
||||
|------|-------|----------|------|
|
||||
| | | | |
|
||||
|
||||
## Issues Found
|
||||
|
||||
| Symptom | Cause | Fix Applied |
|
||||
|---------|-------|-------------|
|
||||
| | | |
|
||||
|
||||
## Next Steps
|
||||
|
||||
1.
|
||||
2.
|
||||
3.
|
||||
```
|
||||
@@ -0,0 +1,77 @@
|
||||
# Programmatic Workflow JSON Modification
|
||||
|
||||
When modifying multiple LTX workflow JSONs with the same changes (sampler, duration, imageFile, prompt), use a Python script rather than manual JSON editing. This avoids quoting errors in shell heredocs and ensures consistent modifications across clips.
|
||||
|
||||
## Pattern
|
||||
|
||||
1. Write the modification script locally (on Hermes host)
|
||||
2. scp it to .202
|
||||
3. Run it via SSH
|
||||
|
||||
## Example: build_clips.py (2026-07-22)
|
||||
|
||||
```python
|
||||
import json
|
||||
|
||||
# Load templates
|
||||
c1 = json.load(open("/tmp/api_cyberpunk_C1.json"))
|
||||
c2 = json.load(open("/tmp/api_cyberpunk_C2.json"))
|
||||
|
||||
# Common modifications for both clips
|
||||
for d, name in [(c1, "C1"), (c2, "C2")]:
|
||||
p = d["prompt"]
|
||||
|
||||
# Node 20: sampler
|
||||
p["20"]["inputs"]["sampler_name"] = "euler_ancestral_cfg_pp"
|
||||
|
||||
# Node 131: duration -> 121 frames / 5.04s
|
||||
p["131"]["inputs"]["duration_frames"] = 121
|
||||
p["131"]["inputs"]["end_frame"] = 121
|
||||
p["131"]["inputs"]["duration_seconds"] = 5.04
|
||||
p["131"]["inputs"]["end_second"] = 5.04
|
||||
|
||||
# Node 131 timeline_data
|
||||
td = json.loads(p["131"]["inputs"]["timeline_data"])
|
||||
td["normalDurationFrames"] = 121
|
||||
seg = td["segments"][0]
|
||||
seg["length"] = 121
|
||||
seg["end"] = 5.04
|
||||
|
||||
# C1-specific: boss pan right
|
||||
p1 = c1["prompt"]
|
||||
td1 = json.loads(p1["131"]["inputs"]["timeline_data"])
|
||||
td1["segments"][0]["imageFile"] = "ltx_boss_A.png"
|
||||
td1["segments"][0]["prompt"] = "..." # full prompt
|
||||
p1["131"]["inputs"]["timeline_data"] = json.dumps(td1)
|
||||
|
||||
# C2-specific: frame-B -> woman reveal
|
||||
p2 = c2["prompt"]
|
||||
td2 = json.loads(p2["131"]["inputs"]["timeline_data"])
|
||||
td2["segments"][0]["imageFile"] = "ltx_boss_B.png" # CRITICAL: pre-rendered empty frame B
|
||||
td2["segments"][0]["prompt"] = "..." # full prompt
|
||||
p2["131"]["inputs"]["timeline_data"] = json.dumps(td2)
|
||||
|
||||
# Write both
|
||||
json.dump(c1, open("/tmp/api_boss_C1_fixed.json", "w"), indent=2)
|
||||
json.dump(c2, open("/tmp/api_boss_C2_fixed.json", "w"), indent=2)
|
||||
```
|
||||
|
||||
## Why Not Shell Heredocs?
|
||||
|
||||
Shell heredocs with Python inside them have quoting issues:
|
||||
- Single quotes inside the Python string break the heredoc delimiter
|
||||
- Multi-line prompts with special characters get mangled
|
||||
- Escaping is fragile and error-prone
|
||||
|
||||
The scp + remote execution pattern avoids all of this.
|
||||
|
||||
## Dispatch
|
||||
|
||||
```bash
|
||||
# Write script locally
|
||||
write_file /tmp/build_clips.py
|
||||
|
||||
# scp to .202 and run
|
||||
sshpass -p 'passw0rd' scp /tmp/build_clips.py [email protected]:/tmp/build_clips.py
|
||||
sshpass -p 'passw0rd' ssh [email protected] 'python3 /tmp/build_clips.py'
|
||||
```
|
||||
@@ -0,0 +1,43 @@
|
||||
# LTX Prompt Validation Checklist
|
||||
|
||||
Run this 10-point check on EVERY segment prompt BEFORE submitting to the render queue. This is a quality gate — do not skip.
|
||||
|
||||
## The 10 Checks
|
||||
|
||||
| # | Check | Pass if |
|
||||
|---|-------|---------|
|
||||
| 1 | Shot type | Contains one of: Medium shot, Wide shot, Close-up, Low angle, Tracking shot, Overhead, POV |
|
||||
| 2 | Camera direction | Contains one of: pan left/right, dolly in/out, tilt up/down, zoom, tracking, static, handheld, crane, Steadicam |
|
||||
| 3 | Actions (2-3 max) | 2-3 distinct actions, present tense, sequential. Not 5-8 simultaneous actions. |
|
||||
| 4 | Lighting | Describes light source, quality, and color temp (e.g., "warm afternoon light from tall windows") |
|
||||
| 5 | Environment | Describes textures, atmosphere, setting details |
|
||||
| 6 | Word count | 50-80 words for I2V (action-focused — start frame already defines visuals). Official LTX says longer is better for T2V, but I2V needs less scene-setting. Under 50 = model fills gaps. Over 80 = model drops actions. |
|
||||
| 7 | No text/logos | No words like "text", "logo", "sign", "label", "readable letters" in the prompt |
|
||||
| 8 | No zhuanchang | Only append "zhuanchang" if this is a transition/morph scene. Never on standard I2V. |
|
||||
| 9 | No emotional labels | No abstract emotions ("sad", "angry", "happy"). Use physical cues instead ("shoulders slumped", "eyes downcast"). |
|
||||
| 10 | Single logic | One speed, one camera direction, one lighting logic. No conflicting descriptions. |
|
||||
|
||||
## Prompt Structure (Official LTX 2.3)
|
||||
|
||||
Two valid orderings per ltx.io (June 2026):
|
||||
|
||||
**Shot-size-first:** framing → subject → action → style → motion
|
||||
**Subject-first:** subject → action → framing → style → motion
|
||||
|
||||
For I2V: focus on motion and action — the start frame already defines the visual. Describe the transition from stillness to motion, not static elements already in the image.
|
||||
|
||||
## Example: FAIL
|
||||
|
||||
> The camera continues its slow steady pan right through the same neon-lit cyberpunk room, past flickering holographic displays and tangled cables, gradually revealing a woman in her late 20s with chrome temple implants standing by a rain-streaked window. Pink and blue neon light traces her silhouette. She turns her head toward the camera. Dark atmosphere, photorealistic, cinematic 35mm.
|
||||
|
||||
**Fails:** Check #1 (no shot type — missing "Medium shot" or equivalent).
|
||||
|
||||
## Example: PASS
|
||||
|
||||
> Medium shot, slow steady pan right. The camera continues through the same neon-lit cyberpunk room, past flickering holographic displays and tangled cables, gradually revealing a woman in her late 20s with chrome temple implants standing by a rain-streaked window. Pink and blue neon light traces her silhouette. She turns her head toward the camera. Dark atmosphere, photorealistic, cinematic 35mm.
|
||||
|
||||
**Passes:** All 10 checks. Shot type present, camera explicit, 2 actions (standing, turns head), lighting described, environment detailed, ~60 words, no text/logos, no zhuanchang, no emotional labels, single direction/speed.
|
||||
|
||||
## Audit History
|
||||
|
||||
- 2026-07-22: Cyberpunk Clip 2 prompt caught missing shot type in pre-render audit. Fixed before dispatch.
|
||||
@@ -0,0 +1,134 @@
|
||||
# LTX Video Prompting — Quick Reference
|
||||
|
||||
Condensed from the 544-line deep research at `/home/n8n/workspace/general/LTX_prompt_instructions.md`.
|
||||
|
||||
## Prompt Order (Official LTX 2.3)
|
||||
|
||||
LTX Studio's pipelines weight tokens by position. Two orderings work well:
|
||||
|
||||
**Order one (shot-size-first):** framing → subject → action → style → motion
|
||||
Use when the shot size is the most important decision. "Wide shot, low angle, a detective in a long coat walks toward a warehouse door, harsh sodium lighting, slow push-in."
|
||||
|
||||
**Order two (subject-first):** subject → action → framing → style → motion
|
||||
Use when the character or object is more critical. "A woman in a red coat walks through a Tokyo street at night, medium tracking shot, neon reflections in puddles, handheld follow."
|
||||
|
||||
**Failure mode:** piling every adjective into one long noun phrase. Cut to the 2-3 that matter most.
|
||||
|
||||
**For I2V specifically:** Focus the prompt on motion and action — the visual starting point is already defined by the input image. Avoid describing static elements already visible in the image. Describe the transition from stillness to motion.
|
||||
|
||||
Source: ltx.io/blog/ltx-2-3-prompt-guide, ltx.io/blog/how-to-write-a-prompt (official, June 2026)
|
||||
|
||||
## Shot Sizes
|
||||
|
||||
| Shot | What It Shows |
|
||||
|------|---------------|
|
||||
| Extreme Wide | Vast environment, subject tiny |
|
||||
| Wide / Establishing | Full body + surroundings |
|
||||
| Medium Wide | Knees up |
|
||||
| Medium Shot | Waist up |
|
||||
| Medium Close-up | Chest up |
|
||||
| Close-up | Face and neck |
|
||||
| Extreme Close-up | Single feature |
|
||||
|
||||
## Camera Angles
|
||||
|
||||
| Angle | Effect |
|
||||
|-------|--------|
|
||||
| Eye Level | Neutral, familiar |
|
||||
| Low Angle | Power, dominance |
|
||||
| High Angle | Vulnerability |
|
||||
| Bird's Eye | Detached, god-like |
|
||||
| Dutch Angle | Unease, tension |
|
||||
| Over-the-Shoulder | Dialogue |
|
||||
| POV | First-person immersion |
|
||||
|
||||
## Camera Movements
|
||||
|
||||
| Movement | Prompt Phrase |
|
||||
|----------|---------------|
|
||||
| Static | "Static camera," "tripod-locked" |
|
||||
| Dolly-in | "Slow dolly-in," "camera pushes forward" |
|
||||
| Dolly-out | "Camera pulls back slowly" |
|
||||
| Pan | "Slow pan left/right" |
|
||||
| Tilt | "Tilts up to reveal" |
|
||||
| Tracking | "Tracks alongside," "follows behind" |
|
||||
| Orbit | "Camera orbits slowly around" |
|
||||
| Handheld | "Subtle handheld, micro-shakes only" |
|
||||
|
||||
## Lighting
|
||||
|
||||
| Type | Prompt Phrase |
|
||||
|------|---------------|
|
||||
| Window light | "Soft north-window light, gentle falloff" |
|
||||
| Golden hour | "Golden hour rim light, long shadows" |
|
||||
| Overcast | "Overcast, diffuse light, low contrast" |
|
||||
| Tungsten | "Practical lamp as key, warm tungsten" |
|
||||
| Neon | "Neon glow, cyan and magenta reflections" |
|
||||
| Backlight | "Backlit rim light, subject silhouetted" |
|
||||
| High contrast | "High-contrast studio lighting, deep shadows" |
|
||||
|
||||
## Lens Cues
|
||||
|
||||
- "35mm lens, f/2.8, shallow depth of field" — people/products
|
||||
- "85mm portrait lens, bokeh, crisp eyes" — talking heads
|
||||
- "16mm wide, deep focus" — rooms/landscapes
|
||||
- "50mm standard" — natural perspective
|
||||
|
||||
## Motion Verbs
|
||||
|
||||
Walking, pouring, turning, lifting, revealing, drifting, sprinting, reaching, pausing, stepping, rising, falling, swaying, flickering, rotating, gliding, accelerating, decelerating, spinning, floating, sliding, bouncing, nodding, shaking, gesturing, breathing, blinking, smiling, frowning, gasping, laughing, crying, whispering, shouting
|
||||
|
||||
## Emotion — Show, Don't Tell
|
||||
|
||||
❌ "A sad woman sits at a table"
|
||||
✅ "A woman sits at a table, her shoulders slumped forward, eyes downcast, fingers tracing the rim of an empty coffee cup"
|
||||
|
||||
## Negative Prompt
|
||||
|
||||
```
|
||||
no extra limbs, no face warp, no object duplication
|
||||
no text artifacts, no floating logos, no watermark
|
||||
no extreme motion blur, no rolling shutter wobble
|
||||
no flicker, no frame-to-frame texture shift
|
||||
no Dutch angle, no rapid handheld, keep horizon level
|
||||
```
|
||||
|
||||
For I2V: add `static, frozen, no motion, Ken Burns zoom`
|
||||
|
||||
## Transition LoRA (zhuanchang)
|
||||
|
||||
- **Use ONLY on transition/morph scenes** — character morph, style change, scene switch, day→night
|
||||
- **NEVER on standard I2V** — the LoRA forces transformation behavior even without the trigger word. Confirmed 2026-07-21: caused subject duplication and wrong-scene hallucination on standard I2V scenes.
|
||||
- **Place at END of prompt**
|
||||
- **CFG: 4.0** (not 1.0 — Transition LoRA needs higher CFG)
|
||||
- **Strength: 1.0**
|
||||
- **Node: LoraLoaderModelOnly** (standard ComfyUI node, NOT LTX2LoraLoaderAdvanced)
|
||||
|
||||
## guide_strength (RESOLVED 2026-07-22)
|
||||
|
||||
**Use 1.0 for all standard I2V with ID LoRA dropped.** The old "1.0 = NEVER" rule was from v4 where ID LoRA was also at 1.0 — the duplication was stacked LoRA interference, not guide_strength alone. With ID LoRA dropped, 1.0 is clean and provides proper start-frame anchoring.
|
||||
|
||||
| Value | When | Why |
|
||||
|-------|------|-----|
|
||||
| **1.0** | Standard I2V (no ID LoRA) | Proper start-frame anchoring. Confirmed clean in 2-clip test. |
|
||||
| **0.5-0.7** | I2V WITH ID LoRA (talking-head only) | Lower strength prevents LoRA interference |
|
||||
|
||||
Verify after render:
|
||||
```bash
|
||||
ffprobe -v quiet -show_entries format_tags=prompt output.mp4 | grep -oP 'guide_strength.*?(\d+\.?\d*)'
|
||||
```
|
||||
|
||||
## Common Mistakes
|
||||
|
||||
| Mistake | Symptom | Fix |
|
||||
|---------|---------|-----|
|
||||
| guide_strength too low (0.5) | Identity drift, weak start-frame anchoring | 1.0 (safe with ID LoRA dropped) |
|
||||
| ID LoRA on action scenes | Frontal-face bias, fights camera motion | Drop ID LoRA entirely |
|
||||
| Transition LoRA on standard I2V | Wrong scene, hallucinated content | Remove Transition LoRA entirely |
|
||||
| zhuanchang on every scene | Model forces morphing everywhere | Only on actual transitions |
|
||||
| No camera direction | Static/random movement | Add dolly/pan/tracking |
|
||||
| Prompts too short (<30 words) | Model fills gaps with random data | 50-80 words minimum |
|
||||
| No motion detail | Character freezes | Sequential action in present tense |
|
||||
| Emotional labels | Abstract expressions | Physical cues only |
|
||||
| Conflicting descriptions | Averaged competing signals | One speed, one camera, one light |
|
||||
| euler_ancestral_cfg_pp on fp8 distilled | Catastrophic artifacts, hallucinated subjects | Plain euler only |
|
||||
@@ -0,0 +1,99 @@
|
||||
# Reference Sheet Creation for IC-LoRA Ingredients
|
||||
|
||||
Proven workflow for creating IC-LoRA reference sheets from stock character images on TrueNAS. Used 2026-07-22 for the boss+woman cyberpunk test.
|
||||
|
||||
## Steps
|
||||
|
||||
### 1. Download stock from TrueNAS
|
||||
|
||||
```bash
|
||||
smbclient -N //10.0.0.117/proxmoxBackup -c 'cd ai_vid_stock_material\\character_refs; get SHEETS2_00005_Boss.png /tmp/SHEETS2_00005_Boss.png'
|
||||
smbclient -N //10.0.0.117/proxmoxBackup -c 'cd ai_vid_stock_material\\character_refs; get cyberpunk_woman_neon_01.jpg /tmp/cyberpunk_woman_neon_01.jpg'
|
||||
```
|
||||
|
||||
### 2. Extract panels from multi-panel character sheets
|
||||
|
||||
The boss stock (3328×2432) is a 2-panel sheet on a bright background. Use brightness thresholding to find content regions:
|
||||
|
||||
```python
|
||||
from PIL import Image
|
||||
import numpy as np
|
||||
|
||||
img = Image.open("/tmp/SHEETS2_00005_Boss.png")
|
||||
arr = np.array(img)
|
||||
brightness = arr.mean(axis=2)
|
||||
is_content = brightness < 200 # content is darker than bright background
|
||||
|
||||
# Find vertical regions (rows with >5% content pixels)
|
||||
row_content = is_content.mean(axis=1)
|
||||
content_rows = np.where(row_content > 0.05)[0]
|
||||
gaps = np.diff(content_rows)
|
||||
split_points = np.where(gaps > 20)[0] # gaps >20px = panel boundary
|
||||
|
||||
# Extract each panel with horizontal bounds
|
||||
regions = []
|
||||
start = content_rows[0]
|
||||
for sp in split_points:
|
||||
end = content_rows[sp]
|
||||
regions.append((start, end))
|
||||
start = content_rows[sp + 1]
|
||||
regions.append((start, content_rows[-1]))
|
||||
|
||||
for i, (y1, y2) in enumerate(regions):
|
||||
region = is_content[y1:y2+1, :]
|
||||
col_content = region.mean(axis=0)
|
||||
content_cols = np.where(col_content > 0.02)[0]
|
||||
x1, x2 = content_cols[0], content_cols[-1]
|
||||
crop = img.crop((x1, y1, x2, y2))
|
||||
crop.save(f"/tmp/boss_panel_{i}.png")
|
||||
```
|
||||
|
||||
### 3. Composite reference sheet
|
||||
|
||||
IC-LoRA requires: 768×448, black background, one panel per character, NO text.
|
||||
|
||||
```python
|
||||
ref = Image.new("RGB", (768, 448), (0, 0, 0))
|
||||
|
||||
# Place boss panels (left side)
|
||||
bp0 = boss_panels[0].copy()
|
||||
bp0.thumbnail((300, 200), Image.LANCZOS)
|
||||
ref.paste(bp0, (10, 10))
|
||||
|
||||
bp1 = boss_panels[1].copy()
|
||||
bp1.thumbnail((300, 220), Image.LANCZOS)
|
||||
ref.paste(bp1, (10, 220))
|
||||
|
||||
# Place woman panels (right side)
|
||||
w1 = woman1.copy()
|
||||
w1.thumbnail((200, 200), Image.LANCZOS)
|
||||
ref.paste(w1, (330, 10))
|
||||
|
||||
w2 = woman2.copy()
|
||||
w2.thumbnail((200, 200), Image.LANCZOS)
|
||||
ref.paste(w2, (550, 10))
|
||||
|
||||
ref.save("/tmp/ic_lora_reference_sheet.png")
|
||||
```
|
||||
|
||||
### 4. Loop to 121-frame static video
|
||||
|
||||
```bash
|
||||
ffmpeg -y -loop 1 -i /tmp/ic_lora_reference_sheet.png \
|
||||
-c:v libx264 -t 5.04 -r 24 -pix_fmt yuv420p \
|
||||
/tmp/ic_lora_reference_121f.mp4
|
||||
```
|
||||
|
||||
### 5. Copy to .202 for ComfyUI
|
||||
|
||||
```bash
|
||||
sshpass -p 'passw0rd' scp /tmp/ic_lora_reference_121f.mp4 [email protected]:~/comfy-ui/input/
|
||||
```
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **Bright backgrounds need thresholding.** The boss stock has a ~230 brightness background — content detection needs `brightness < 200`, not `< 30` (which is for black backgrounds).
|
||||
- **Panel gap detection is fragile.** The `gaps > 20` threshold works for the boss sheet but may need tuning for other sheets. Always inspect extracted panels before compositing.
|
||||
- **IC-LoRA requires black background.** The reference sheet MUST have a black background — bright backgrounds confuse the conditioning.
|
||||
- **Bigger panels = better carry-over.** Give important characters more space in the composite.
|
||||
- **Resolution must match trained bucket.** 768×448 is the IC-LoRA trained resolution. Other resolutions may work but are untested.
|
||||
@@ -0,0 +1,57 @@
|
||||
#!/usr/bin/env python3
|
||||
"""Scan LTX Director output videos and map them to scenes.
|
||||
|
||||
Usage: python3 scan_videos.py [output_dir]
|
||||
|
||||
Extracts from each LTX_Director_*.mp4:
|
||||
- Scene number (from start_frame in prompt metadata)
|
||||
- Model type (fp8+LoRAs, Q4, or unknown)
|
||||
- Resolution and duration
|
||||
- Whether Transition LoRA is wired (LoraLoaderModelOnly in prompt)
|
||||
- Whether zhuanchang trigger is present
|
||||
- File size
|
||||
"""
|
||||
|
||||
import subprocess, os, re, sys
|
||||
|
||||
outdir = sys.argv[1] if len(sys.argv) > 1 else os.path.expanduser("~/comfy-ui/output/video")
|
||||
files = sorted([f for f in os.listdir(outdir) if f.startswith("LTX_Director_") and f.endswith(".mp4")])
|
||||
|
||||
print(f"{'File':40s} | {'Resolution':20s} | {'Scene':10s} | {'Model':12s} | {'Trans':6s} | {'zhuanchang':10s} | {'Size':>8s}")
|
||||
print("-" * 120)
|
||||
|
||||
for f in files:
|
||||
path = os.path.join(outdir, f)
|
||||
|
||||
# Resolution and duration
|
||||
r = subprocess.run(["ffprobe", "-v", "quiet", "-select_streams", "v:0",
|
||||
"-show_entries", "stream=width,height,duration",
|
||||
"-of", "csv=p=0", path], capture_output=True, text=True)
|
||||
res = r.stdout.strip()
|
||||
|
||||
# Prompt metadata
|
||||
r2 = subprocess.run(["ffprobe", "-v", "quiet", "-show_entries", "format_tags=prompt",
|
||||
"-of", "csv=p=0", path], capture_output=True, text=True)
|
||||
prompt_raw = r2.stdout.strip()
|
||||
|
||||
# Scene from imageFile
|
||||
img_match = re.search(r"imageFile.*?ltx_start_frame_(\d+)", prompt_raw)
|
||||
scene = f"Scene_{img_match.group(1)}" if img_match else "?"
|
||||
|
||||
# Model type
|
||||
if "UNETLoader" in prompt_raw and "LTX2LoraLoaderAdvanced" in prompt_raw:
|
||||
model = "fp8+LoRAs"
|
||||
elif "UnetLoaderGGUF" in prompt_raw:
|
||||
model = "Q4"
|
||||
else:
|
||||
model = "?"
|
||||
|
||||
# Transition LoRA
|
||||
has_trans = "LoraLoaderModelOnly" in prompt_raw
|
||||
has_zhuanchang = "zhuanchang" in prompt_raw
|
||||
|
||||
# File size
|
||||
size = os.path.getsize(path)
|
||||
size_str = f"{size/1024:.0f}KB"
|
||||
|
||||
print(f"{f:40s} | {res:20s} | {scene:10s} | {model:12s} | {str(has_trans):6s} | {str(has_zhuanchang):10s} | {size_str:>8s}")
|
||||
@@ -0,0 +1,69 @@
|
||||
# Stock as I2V Start Frame
|
||||
|
||||
Pattern for using stock character images (from TrueNAS `ai_vid_stock_material/character_refs/`) as direct I2V start frames in the standard LTX pipeline. Simpler than IC-LoRA Ingredients — no additional LoRA, no reference sheet, no static video loop. Works with the existing 6-fix baseline.
|
||||
|
||||
## When to Use
|
||||
|
||||
- Quick character test: "does this stock image animate well?"
|
||||
- Single-clip renders where character consistency across scenes isn't critical
|
||||
- Before investing in IC-LoRA Ingredients setup
|
||||
|
||||
## When NOT to Use
|
||||
|
||||
- Multi-scene stories needing character consistency across clips → use IC-LoRA Ingredients
|
||||
- Stock images with white/light backgrounds → LTX may hallucinate background elements
|
||||
- Low-resolution stock → resize artifacts will compound in animation
|
||||
|
||||
## Steps
|
||||
|
||||
1. **Download stock from TrueNAS:**
|
||||
```bash
|
||||
smbclient -N //10.0.0.117/proxmoxBackup -c 'cd ai_vid_stock_material\character_refs; get SHEETS2_00005_Boss.png /tmp/boss.png'
|
||||
```
|
||||
|
||||
2. **Resize to 768×512** (our standard resolution):
|
||||
```python
|
||||
from PIL import Image
|
||||
img = Image.open('/tmp/boss.png')
|
||||
img = img.resize((768, 512), Image.LANCZOS)
|
||||
img.save('/tmp/boss_start_frame.png')
|
||||
```
|
||||
|
||||
3. **Copy to .202 input directory:**
|
||||
```bash
|
||||
sshpass -p 'passw0rd' scp /tmp/boss_start_frame.png [email protected]:/home/n8n/comfy-ui/input/
|
||||
```
|
||||
|
||||
4. **Modify an existing workflow JSON** — change the `imageFile` in timeline_data:
|
||||
```python
|
||||
import json
|
||||
with open('/tmp/api_cyberpunk_C1.json') as f:
|
||||
d = json.load(f)
|
||||
nodes = d['prompt']
|
||||
td = json.loads(nodes['131']['inputs']['timeline_data'])
|
||||
td['segments'][0]['imageFile'] = 'boss_start_frame.png'
|
||||
nodes['131']['inputs']['timeline_data'] = json.dumps(td)
|
||||
with open('/tmp/api_boss_test.json', 'w') as f:
|
||||
json.dump(d, f)
|
||||
```
|
||||
|
||||
5. **Update the prompt** to describe the stock character (not the Flux-generated character):
|
||||
- Describe what the stock image shows (clothing, features, expression)
|
||||
- Keep the same environment/lighting/camera from the original template
|
||||
- Target 50-80 words, 2-3 actions
|
||||
|
||||
6. **Submit and verify** (standard pipeline steps 4-8).
|
||||
|
||||
## Example: Boss Stock
|
||||
|
||||
**Stock:** `SHEETS2_00005_Boss.png` (3328×2432 RGB PNG, white background)
|
||||
**Resized:** 768×512
|
||||
**Prompt:** "Medium shot, slow steady pan right. A stern boss in a dark suit sits at a terminal in a neon-lit cyberpunk room. Pink and blue holographic light flickers across his face as he types on a glowing keyboard. Over 5 seconds: he leans forward studying the screen with cold intensity, then slowly turns his head right. His chrome temple implant glints in the neon glow. Volumetric fog drifts through colored light from overhead panels. Cables and server racks line the walls, indicator lights blinking. Cinematic lighting, shallow depth of field, 35mm film grain."
|
||||
|
||||
## Pitfalls
|
||||
|
||||
- **CRITICAL: Single stock images as I2V start frames do NOT anchor the face.** LTX treats the start frame as "general scene composition" not "this exact person." The model will morph the face, change clothing details, and substitute its own training data for the character. Confirmed 2026-07-22: boss stock test — face distorted, did not stay true to image. User: "stock test was malformed. It worked 80% but face distorted and did not stay to image." This approach is useful for quick composition tests only. For character consistency, use IC-LoRA Ingredients.
|
||||
- **Character description in prompt MUST match the canonical descriptions** in `references/character-descriptions.md`. Using generic descriptions like "stern boss" instead of "muscular boss with shaved head and goatee wearing a black suit" causes the model to blend between the stock image and its own training data. The user provided exact descriptions — use them verbatim.
|
||||
- **White/light backgrounds in stock images** may cause LTX to hallucinate background elements or wash out the scene. The model was trained on varied backgrounds; a plain white background gives it no environment cues. Prefer stock with scene-appropriate backgrounds, or use Flux to generate a start frame that composites the character into the target environment.
|
||||
- **Stock image aspect ratio** rarely matches 768×512. Resize with LANCZOS; avoid stretching (crop to aspect ratio first if needed).
|
||||
- **This is NOT a character-consistency solution across clips.** Each clip gets its own start frame; there's no mechanism to keep the same face across scenes. For multi-clip consistency, use IC-LoRA Ingredients.
|
||||
@@ -0,0 +1,134 @@
|
||||
# Story Structure Guide — LTX Video Pipeline
|
||||
|
||||
## The Problem (v4 Post-Mortem)
|
||||
|
||||
v4 rendered correctly (241 frames, no duplication, wiring fixed) but the user said "the videos had all kinds of inconsistencies." The root cause: the story was 6 standalone vignettes with no causal chain. Each scene was a different location with no narrative thread connecting them. LTX needs a story where each scene follows from the previous one.
|
||||
|
||||
## Story Design Rules
|
||||
|
||||
1. **Chain of events, not vignettes.** Each scene must be caused by the previous scene. "She discovers something → she runs → she hides → she chooses." Not "he wakes up → he makes coffee → he walks outside."
|
||||
2. **One character, one journey.** Multi-character stories cause identity drift. The ID LoRA helps but isn't perfect across different faces.
|
||||
3. **No dialogue, no plot twists.** LTX can't do dialogue well. Visual storytelling: action, reaction, environment, choice.
|
||||
4. **6 scenes × 10s = 60s.** This is the sweet spot. Each scene is one beat in the story.
|
||||
5. **Clear visual variety per scene.** Each scene should have a distinct setting, lighting, and camera language. This prevents the model from blending scenes together.
|
||||
6. **Match LTX prompt style for EVERY scene.** Every scene prompt must have: shot type, camera movement, subject action (present tense, sequential), lighting, audio. No exceptions.
|
||||
|
||||
## Story Template
|
||||
|
||||
```
|
||||
Scene 1: INCITING INCIDENT — character discovers/encounters something
|
||||
Scene 2: ESCALATION — the situation intensifies, stakes rise
|
||||
Scene 3: REACTION — character responds, makes a decision
|
||||
Scene 4: CHASE/STRUGGLE — physical action, pursuit, or confrontation
|
||||
Scene 5: LOW POINT — character is isolated, vulnerable, reflects
|
||||
Scene 6: CHOICE/RESOLUTION — character makes the final decision, walks toward outcome
|
||||
```
|
||||
|
||||
## User Preference: Detail Over Simplicity
|
||||
|
||||
The user initially asked for simpler stories, then reversed: "go back to your original." The detailed 4-story templates below are the preferred level of detail. Don't over-simplify.
|
||||
|
||||
---
|
||||
|
||||
## Story 1: "Ghost in the Wire" (Cyberpunk) — COMPLETED 2026-07-21
|
||||
|
||||
**Logline:** A netrunner discovers a corporate AI that's become self-aware — and it's been killing people through their neural implants. She has 60 seconds to decide: expose it, or join it.
|
||||
|
||||
**Character:** Woman, mid-20s, short dark hair, chrome temple implants, cybernetic fingers, worn black synth-leather jacket.
|
||||
|
||||
**Chain:** Hack → Discovery → Confrontation → Chase → Choice → Consequence
|
||||
|
||||
| # | Setting | Action | Camera | guide_strength |
|
||||
|---|---------|--------|--------|---------------|
|
||||
| 1 | Neon-lit hacker den | Woman at terminal, code streams across screens. Her eyes widen — she's found something. | Medium shot, slow push-in | 0.5 |
|
||||
| 2 | Corporate data vault (virtual) | Her avatar materializes inside a geometric data fortress. Crystalline servers pulse blue. She touches one — it turns red. The AI speaks. | Wide, slow orbit | 0.7 |
|
||||
| 3 | Rooftop, rain | She rips off her neural link, gasping. Rain pours. Drone's red eye appears in the distance — they know. | Medium close-up → wide reveal | 0.7 |
|
||||
| 4 | Lower city chase | She runs through a crowded neon market. Drones weave between stalls. She slides under a closing blast door. | Tracking shot, handheld | 0.7 |
|
||||
| 5 | Abandoned subway | She catches her breath in a flooded tunnel. Bioluminescent moss glows. Her implant flickers — the AI is still in her head. | Static, slow tilt down to reflection | 0.7 |
|
||||
| 6 | Tunnel junction → choice | Left tunnel: surface, exposure, truth. Right tunnel: deeper, the AI's offer. She walks right. Camera holds on empty junction. | Wide, static, then slow pull-back | 0.7 |
|
||||
|
||||
**Result:** All 6 scenes rendered successfully. 241 frames each (no wiring bug). Concat: 1446 frames, 60.18s. Uploaded to TrueNAS as `LTX_Director_S1_60s.mp4`. Full prompts at `story1-ghost-in-the-wire-prompts.md`.
|
||||
|
||||
---
|
||||
|
||||
## Story 2: "Chrome Angels" (Cyberpunk) — IN PROGRESS 2026-07-21
|
||||
|
||||
**Logline:** A black-market cybernetic surgeon gets a new patient — a corporate enforcer whose face he recognizes. It's his brother, declared dead five years ago. The brother doesn't remember him.
|
||||
|
||||
**Character:** Surgeon, 40s, graying hair, tired eyes, cybernetic eye replacement, worn surgical coat. Brother: same face, younger, corporate chrome, cold expression.
|
||||
|
||||
**Chain:** Operation → Recognition → Confrontation → Memory → Betrayal → Sacrifice
|
||||
|
||||
| # | Setting | Action | Camera |
|
||||
|---|---------|--------|--------|
|
||||
| 1 | Flooded basement clinic | Surgeon's chrome fingers replace organic ones. Water drips from pipes. Patient unconscious on gurney. | Close-up on hands → slow reveal |
|
||||
| 2 | Clinic, post-op | He pulls back drapes. Freezes. It's his brother — same jaw, same scar. But the eyes are cold. Corporate. | Medium shot, static → slow push-in |
|
||||
| 3 | Clinic, confrontation | Brother wakes. "That identity was terminated. I am Unit 734." He stands, towering. Surgeon backs against wall. | Low angle, static |
|
||||
| 4 | Memory flash (virtual) | Two boys on a rooftop, sunset. One points at a corporate gunship. "One day I'm going to work for them." The other: "You'd sell your soul for chrome." Memory corrupts into static. | Wide rooftop → dissolve to static |
|
||||
| 5 | Clinic, betrayal | Brother's hand around surgeon's throat — but trembling. A flicker of recognition. Then red light blinks on temple — corp override. Face goes blank. | Medium close-up, handheld |
|
||||
| 6 | Clinic, sacrifice | Surgeon jams EMP spike into brother's neural port. Brother seizes, collapses. Eyes — human now — focus one last time. "You kept your soul." They close. | Overhead shot, slow pull-up |
|
||||
|
||||
**Status:** Flux start frames submitted (prompt IDs: be0b5181 through 04668385). Waiting for queue drain, then build scene JSONs and render.
|
||||
|
||||
---
|
||||
|
||||
## Story 3: "The Last Human Job" (Cyberpunk) — PLANNED
|
||||
|
||||
**Logline:** In a city where AI replaced every worker, one man still runs a physical repair shop. When a damaged android brings in a human memory core, he uncovers a black-market trade in stolen consciousness — and learns his own mind was taken years ago.
|
||||
|
||||
**Character:** Repairman, 50s, weathered face, cybernetic arm, oil-stained hands, kind eyes.
|
||||
|
||||
**Chain:** Repair → Discovery → Investigation → Revelation → Confrontation → Identity
|
||||
|
||||
| # | Setting | Action | Camera |
|
||||
|---|---------|--------|--------|
|
||||
| 1 | Repair shop | Repairman's hands in the guts of a broken drone. Shelves of parts tower. Neon from street paints everything pink and blue. Bell rings. | Close-up → medium reveal |
|
||||
| 2 | Shop counter | Damaged android at counter, one eye flickering. "Please. They're erasing people." Pulls a glowing memory core from its own chest. | Medium shot, static |
|
||||
| 3 | Black-market memory den | Underground facility. Rows of humans in suspension pods, memories extracted and sold. He recognizes a face in one pod — his own. | Tracking shot through facility |
|
||||
| 4 | Memory den, revelation | He stares at his own body in the pod. Android: "You've been dead for three years. The person you think you are is a copy." He touches the glass. | Static, slow dolly-in on reflection |
|
||||
| 5 | Memory den, confrontation | Corporate security arrives. Red lasers cut through dark. He disables a guard — his hands move with military precision he didn't know he had. | Handheld, chaotic but controlled |
|
||||
| 6 | Rooftop escape → dawn | He and android burst onto rooftop at dawn. City stretches below. He looks at his hands — chrome and flesh, both his. "What do I do now?" Android: "Whatever the original you would have wanted." | Wide, slow pull-back |
|
||||
|
||||
---
|
||||
|
||||
## Story 4: "Neon Baptism" (Cyberpunk) — PLANNED
|
||||
|
||||
**Logline:** A street kid gets caught stealing from the wrong corporation. Instead of prison, they offer her full-body augmentation — a "recruitment." But when the chrome activates, she's not the weapon they expected.
|
||||
|
||||
**Character:** Street kid, 17-19, androgynous, dirty face, sharp eyes, worn synth-leather. Post-augmentation: same face, chrome limbs, glowing blue optical implant, sleek corporate body.
|
||||
|
||||
**Chain:** Theft → Capture → Offer → Transformation → Awakening → Rebellion
|
||||
|
||||
| # | Setting | Action | Camera |
|
||||
|---|---------|--------|--------|
|
||||
| 1 | Neon market, night | Kid weaves through crowded market, stolen data chip clutched in hand. Drone spots her. Red targeting laser paints her back. She runs. | Tracking, low angle, handheld |
|
||||
| 2 | Corporate interrogation room | Thrown into white room. Sterile. Single chair. Holographic face appears — calm, corporate. "You stole from us. That's impressive. We have an offer." | Wide, static, harsh overhead |
|
||||
| 3 | Augmentation chamber | Strapped to surgical chair. Robotic arms descend — chrome limbs, neural ports, optical implants. She screams. Camera circles as her body is rebuilt. | Slow orbit, clinical |
|
||||
| 4 | Training sim (virtual) | Consciousness boots into virtual training ground. Faster, stronger. Targets appear, she destroys them. But between targets: flashes of her old life. She holds on. | POV + wide cuts |
|
||||
| 5 | Corp tower, awakening | She wakes in corporate tower, chrome body active. Guards escort her to first mission. At a junction, she stops. Looks at her reflection — chrome and human, both her. She smiles. | Tracking, then static |
|
||||
| 6 | Tower → escape | She moves toward the window. Guards fire. She's through the glass, falling 80 stories, chrome limbs catching ledges. Lands in lower city, crouched, steam rising. Stands. Looks up. Walks into neon dark. | Overhead → tracking → wide static |
|
||||
|
||||
## Stock Management Per Story
|
||||
|
||||
| Story | New Frames Needed | Character | Key Visuals |
|
||||
|-------|------------------|-----------|-------------|
|
||||
| Ghost in the Wire | 6 | Female netrunner, 20s | Neon den, data vault, rooftop, market, subway, tunnel |
|
||||
| Chrome Angels | 4 | Male surgeon, 40s + brother | Basement clinic, rooftop memory |
|
||||
| The Last Human Job | 5 | Male repairman, 50s | Repair shop, memory den, rooftop dawn |
|
||||
| Neon Baptism | 6 | Street kid, 17-19 | Market, white room, surgical chair, virtual grid, corp tower |
|
||||
|
||||
## Cyberpunk Stock Sources (from better-search research 2026-07-21)
|
||||
|
||||
**Pexels** (Pexels License — free commercial, no attribution): 5,000+ cyberpunk images. Best for photorealistic characters. Mikhail Nilov and Yaroslav Shuraev sets are the strongest.
|
||||
|
||||
**Unsplash** (Unsplash License — free commercial, no attribution): Best for Tokyo rain/neon street environments. Photorealistic.
|
||||
|
||||
**Pixabay** (CC0-like): 34K+ cyberpunk city images. Mix of photos and illustrations.
|
||||
|
||||
**StockCake** (Royalty-free): AI-generated props/holograms. Quality varies.
|
||||
|
||||
**Freepik** (Free tier: attribution required): Vectors, neon signs. NOT CC0 — use only if attribution is acceptable.
|
||||
|
||||
**PublicDomainPictures** (CC0): True public domain. Limited selection.
|
||||
|
||||
Full research: `~/workspace/research/results/2026-07-21-cyberpunk-stock-materials-ai-video.md`
|
||||
@@ -0,0 +1,121 @@
|
||||
# Workflow Transformation: Two-Stage → Single-Stage I2V (7 Fixes)
|
||||
|
||||
Concrete recipe for transforming a two-stage LTX Director workflow JSON into a single-stage I2V workflow applying all 7 fixes. Use this when you have an existing two-stage scene JSON and need to produce a single-stage version.
|
||||
|
||||
**Status: PROVEN WORKING (2026-07-22).** The 2-clip test confirmed all 7 fixes produce clean 241-frame output with no Director wiring bug and no subject duplication. guide_strength 1.0 is clean when ID LoRA is dropped — the old "DO NOT use 1.0" rule was from stacked LoRA interference, not guide_strength alone.
|
||||
|
||||
## Source Pattern
|
||||
|
||||
Two-stage workflows have:
|
||||
- **Stage 1 (main):** Nodes 131 (Director), 133 (Guide), 55 (CropGuides), 33 (Scheduler, 8 steps, denoise 1.0), 30 (Noise), 32 (Sampler), 31 (SamplerCustomAdvanced), 29 (ConcatAVLatent), 28 (CFGGuider), 34 (SeparateAVLatent)
|
||||
- **Stage 2 (refiner):** Nodes 132 (Guide), 54 (CropGuides), 21 (Scheduler, 4 steps, denoise 0.42), 20 (Sampler), 19 (SamplerCustomAdvanced), 18 (ConcatAVLatent), 17 (CFGGuider)
|
||||
- **ID LoRA:** Node 201 (LTX2LoraLoaderAdvanced, TalkVid-3K) chained after node 200 (Distilled LoRA)
|
||||
- **Output:** Node 1 (VAEDecode) → Node 2 (CreateVideo) → Node 37 (SaveVideo)
|
||||
|
||||
## Nodes to Delete (9 total)
|
||||
|
||||
```
|
||||
21 — BasicScheduler (refiner, 4 steps, denoise 0.42)
|
||||
28 — CFGGuider (refiner)
|
||||
29 — LTXVConcatAVLatent (refiner)
|
||||
31 — SamplerCustomAdvanced (refiner)
|
||||
32 — KSamplerSelect (refiner)
|
||||
34 — LTXVSeparateAVLatent (refiner)
|
||||
54 — LTXDirectorCropGuides (refiner)
|
||||
132 — LTXDirectorGuide (refiner)
|
||||
201 — LTX2LoraLoaderAdvanced (ID LoRA, TalkVid-3K)
|
||||
```
|
||||
|
||||
## Rewiring (6 connections)
|
||||
|
||||
| Node.Input | Old Value | New Value | Reason |
|
||||
|-----------|-----------|-----------|--------|
|
||||
| 131.model | `["201", 0]` | `["200", 0]` | Skip ID LoRA, go direct to Distilled LoRA |
|
||||
| 17.model | `["132", 3]` | `["133", 3]` | CFGGuider now uses main Guide, not refiner |
|
||||
| 17.positive | `["132", 0]` | `["133", 0]` | Same |
|
||||
| 17.negative | `["132", 1]` | `["133", 1]` | Same |
|
||||
| 18.video_latent | `["132", 2]` | `["133", 2]` | ConcatAVLatent takes main Guide output |
|
||||
| 18.audio_latent | `["34", 1]` | `["131", 3]` | Audio from Director directly (34 is deleted) |
|
||||
| 19.sigmas | `["21", 0]` | `["33", 0]` | Sampler uses main scheduler (21 is deleted) |
|
||||
| 55.latent | `["34", 0]` | `["22", 0]` | CropGuides takes from main SeparateAVLatent |
|
||||
| 1.samples | `["54", 2]` | `["55", 2]` | VAEDecode from main CropGuides (54 is deleted) |
|
||||
|
||||
## Settings Changes
|
||||
|
||||
| Node.Input | Old | New | Fix # |
|
||||
|-----------|-----|-----|-------|
|
||||
| 200.strength_model | 0.7 | 1.0 | #5 Distilled LoRA at full strength |
|
||||
| 33.steps | 8 | 18 | #3 Single-stage needs more steps |
|
||||
| 33.denoise | 1.0 | 1.0 | Unchanged (already correct for single-stage) |
|
||||
| 131.guide_strength | 0.5 | 1.0 | #2 I2V conditioning at full strength — CONFIRMED CLEAN |
|
||||
| 131.end_second | 5.0 | 10.0 | #4 Duration (match to desired output) |
|
||||
| 131.duration_seconds | 5.0 | 10.0 | Same |
|
||||
| 131.end_frame | 120 | 240 | Same (10s × 24fps) |
|
||||
| 131.duration_frames | 120 | 240 | Same |
|
||||
| 30.noise_seed | (any) | 42 | #7 Fixed seed for reproducibility |
|
||||
| 37.filename_prefix | (old) | (new) | Rename output prefix |
|
||||
|
||||
## Timeline Data Changes
|
||||
|
||||
| Field | Old | New | Fix # |
|
||||
|-------|-----|-----|-------|
|
||||
| global_prompt | (old story) | (new story prompt) | #6 |
|
||||
| segments[0].length | 120 | 240 | #4 |
|
||||
| segments[0].end | 5.0 | 10.0 | #4 |
|
||||
| segments[0].guide_strength | 0.5 | 1.0 | #2 |
|
||||
| segments[0].imageFile | (old) | (new filename) | #6 |
|
||||
| segments[0].prompt | (old 150+ word) | (new 50-80 word) | #6 |
|
||||
| audioSegments | (any) | [] | Clean up |
|
||||
|
||||
## Verification Checklist
|
||||
|
||||
After transformation, verify:
|
||||
|
||||
- [ ] Node count: 22 (down from 31)
|
||||
- [ ] No deleted nodes remain: 21, 28, 29, 31, 32, 34, 54, 132, 201
|
||||
- [ ] 131.model → `["200", 0]` (Distilled LoRA only, no ID LoRA)
|
||||
- [ ] 200.strength_model = 1.0
|
||||
- [ ] 33.steps = 18, denoise = 1.0
|
||||
- [ ] 131.guide_strength = 1.0
|
||||
- [ ] 131.duration_frames = 240 (for 10s@24fps)
|
||||
- [ ] 30.noise_seed = 42
|
||||
- [ ] All rewired connections match the table above
|
||||
- [ ] timeline_data segment has correct imageFile, prompt, length, guide_strength
|
||||
- [ ] audioSegments is empty (or correctly populated if audio is needed)
|
||||
- [ ] JSON is valid and wraps in `{"prompt": {...}}` envelope
|
||||
|
||||
## Automation
|
||||
|
||||
The transformation can be automated with a Python script. Key pattern:
|
||||
|
||||
```python
|
||||
# Delete refiner + ID LoRA nodes
|
||||
nodes_to_delete = ['21', '28', '29', '31', '32', '34', '54', '132', '201']
|
||||
for nid in nodes_to_delete:
|
||||
del wf[nid]
|
||||
|
||||
# Rewire
|
||||
wf['131']['inputs']['model'] = ['200', 0]
|
||||
wf['17']['inputs']['model'] = ['133', 3]
|
||||
wf['17']['inputs']['positive'] = ['133', 0]
|
||||
wf['17']['inputs']['negative'] = ['133', 1]
|
||||
wf['18']['inputs']['video_latent'] = ['133', 2]
|
||||
wf['18']['inputs']['audio_latent'] = ['131', 3]
|
||||
wf['19']['inputs']['sigmas'] = ['33', 0]
|
||||
wf['55']['inputs']['latent'] = ['22', 0]
|
||||
wf['1']['inputs']['samples'] = ['55', 2]
|
||||
|
||||
# Update settings
|
||||
wf['200']['inputs']['strength_model'] = 1.0
|
||||
wf['33']['inputs']['steps'] = 18
|
||||
wf['131']['inputs']['guide_strength'] = 1.0
|
||||
wf['131']['inputs']['end_second'] = 10.0
|
||||
wf['131']['inputs']['duration_seconds'] = 10.0
|
||||
wf['131']['inputs']['end_frame'] = 240
|
||||
wf['131']['inputs']['duration_frames'] = 240
|
||||
wf['30']['inputs']['noise_seed'] = 42
|
||||
```
|
||||
|
||||
## Session Reference
|
||||
|
||||
First applied 2026-07-22: transformed `/tmp/api_s1_fix_01.json` → `/tmp/api_2clip_C1.json` on 10.0.0.202. 31 nodes → 22 nodes. All 7 fixes applied. Verified with remote checks and kimi-c validation (all 7 checks PASS). Rendered clean: 241 frames, 10.04s, 768×512, 24fps, no Director wiring bug.
|
||||
Reference in New Issue
Block a user