80 lines
3.8 KiB
Markdown
80 lines
3.8 KiB
Markdown
---
|
|
name: ltx-video-workflows
|
|
description: "Wire IC-LoRA and two-stage samplers in LTX-2.3 ComfyUI."
|
|
version: 1.0.0
|
|
author: Hermes Agent
|
|
license: MIT
|
|
platforms: [linux]
|
|
metadata:
|
|
hermes:
|
|
tags: [ltx, comfyui, video, ic-lora, workflow, music-video]
|
|
related_skills: [comfyui]
|
|
---
|
|
|
|
# LTX Video Workflows — ComfyUI Construction & Wiring
|
|
|
|
Build and modify LTX-2.3 ComfyUI workflows. Covers IC-LoRA Ingredients integration, two-stage sampler wiring, LTXVCropGuides, multi-scene music video patterns, and the RuneXX Music-Video-Creator template.
|
|
|
|
## When to Use
|
|
|
|
- Adding IC-LoRA to an existing LTX workflow
|
|
- Wiring two-stage samplers with IC-LoRA
|
|
- Building or modifying multi-scene music video workflows
|
|
- Debugging LTX workflow link/connection issues
|
|
- Porting workflow changes between machines (.202 ↔ DGX)
|
|
|
|
## IC-LoRA Two-Stage Wiring
|
|
|
|
The official two-stage IC-LoRA pattern (from `LTX-2.3_ICLoRA_Outpaint_Two_Stage_Distilled.json` in ComfyUI-LTXVideo repo). Three nodes required.
|
|
|
|
### Model Chain
|
|
```
|
|
UNETLoader → LoraLoaderModelOnly(distilled) → LTXICLoRALoaderModelOnly(IC @ 1.0) → SageAttention → ...bus... → CFGGuider(Pass1) & CFGGuider(Pass2)
|
|
```
|
|
IC-LoRA enters ONCE before SageAttention on the shared model bus. Do NOT fan IC output directly to CFGGuider(Pass2) — it must go through the full bus (Sage, ChunkFF, NAG, etc.).
|
|
|
|
### Conditioning Flow
|
|
```
|
|
LTXVConditioning → LTXAddVideoICLoRAGuideAdvanced → pos/neg → CFGGuider(Pass1)
|
|
→ pos/neg → LTXVCropGuides → pos/neg → CFGGuider(Pass2)
|
|
```
|
|
|
|
### Latent Flow
|
|
```
|
|
I2V latent → GuideAdvanced.latent(in) → GuideAdvanced.latent(out) → ConcatAV.video_latent → ConcatAV → Sampler(Pass1)
|
|
Sampler(Pass1) → SeparateAV.video_latent → CropGuides.latent(in) → CropGuides.latent(out) → Upscaler → I2V → ConcatAV → Sampler(Pass2)
|
|
```
|
|
|
|
### Critical Rules
|
|
- GuideAdvanced.latent MUST go through ConcatAV (not direct to sampler) — preserves audio path
|
|
- CropGuides.latent MUST come from SeparateAV.video_latent (5D tensor), never raw sampler output (AV NestedTensor)
|
|
- CropGuides.latent MUST go through upscaler chain, not direct to Pass2 sampler
|
|
- `latent_downscale_factor` from IC loader MUST be wired to GuideAdvanced
|
|
- GuideAdvanced.image MUST be connected (even if placeholder)
|
|
- GuideAdvanced.vae MUST be connected
|
|
|
|
### Common Mistakes
|
|
1. Skipping ConcatAV — orphans audio, breaks AV latent
|
|
2. Feeding raw sampler output to CropGuides — type mismatch (NestedTensor vs 5D)
|
|
3. Fanning IC model output directly to CFGGuider(Pass2) — bypasses Sage/NAG bus
|
|
4. Leaving GuideAdvanced.image unconnected — IC guidance fails silently
|
|
5. Missing LTXVCropGuides entirely — reference frames leak into Pass2, ghost/duplicate subject
|
|
|
|
## Workflow Porting (.202 ↔ DGX)
|
|
|
|
When porting parameter changes between machines:
|
|
- Compute field-level widget diff between base and target workflows
|
|
- Verify node ID parity before assuming identical structure
|
|
- DGX uses bf16 models + MultiGPU patcher; .202 uses fp8
|
|
- DGX Spark is single-GPU — MultiGPU patcher is dead weight but harmless
|
|
- Always set per-scene save_output=True before long renders (survives OOM crashes)
|
|
|
|
## Reference Workflows
|
|
|
|
- Official IC-LoRA examples: `ComfyUI-LTXVideo/example_workflows/2.3/`
|
|
- RuneXX Music-Video-Creator: `https://huggingface.co/RuneXX/LTX-2.3-Workflows`
|
|
- IC-LoRA Ingredients model: `ltx-2.3-22b-ic-lora-ingredients-0.9.safetensors` (1.3GB)
|
|
- **Sunwood-ai-labs/LTX23-ComfyUI-skill** — remote GPU bootstrap scripts, pre-exported API prompt (`ltx23-ti2v-audio-api-prompt.json`), experiment tracking manifests, recurrence guard documentation. Focused on Isi-dev's TI2V + Audio workflow. 5 stars, MIT license. `github.com/Sunwood-ai-labs/LTX23-ComfyUI-skill`
|
|
|
|
See `references/ic-lora-two-stage-wiring.md` for the full wiring reference with node IDs and link traces.
|