SpeedyFoxAi/hermes-skills
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Fix: flatten comfyui and ltx-video-pipeline to Gitea convention (flat skill dirs). comfyui v5.1.0, ltx-video-pipeline v2.0.0.

Browse Source
This commit is contained in:
Hermes Agent
2026-07-22 16:19:32 -05:00
parent d94132f4ca
commit d824f865c1
61 changed files with 184 additions and 674 deletions
Download Patch File Download Diff File Expand all files Collapse all files
ltx-video-pipeline/references/ic-lora-ingredients.md
+108
View File
@@ -0,0 +1,108 @@
# IC-LoRA Ingredients — Reference Sheet Control for LTX 2.3
**Model:** Lightricks/LTX-2.3-22b-IC-LoRA-Ingredients
**File:** `ltx-2.3-22b-ic-lora-ingredients-0.9.safetensors` (1.31 GB)
**HF URL:** https://huggingface.co/Lightricks/LTX-2.3-22b-IC-LoRA-Ingredients
**Status:** Downloaded on .202 (2026-07-22, 1.31 GB) — gated, requires "Agree and Access" on HF website + valid token
**Token:** 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 (user: MDKRUSH, 2026-07-22)
**Trained bucket:** 768×448, 121 frames, 24 fps
## fp8 Compatibility Gap (UNRESOLVED — deep research in progress 2026-07-22)
The official IC-LoRA workflow (`LTX-2.3_ICLoRA_Ingredients_Single_Stage_Distilled.json`) uses a completely different architecture from our pipeline:
| Component | Official IC-LoRA Workflow | Our Pipeline |
|-----------|--------------------------|--------------|
| Model loader | CheckpointLoaderSimple (full 46GB checkpoint) | UNETLoader (fp8 distilled, 23GB) |
| Sampler | SamplerCustomAdvanced + ManualSigmas + CFGGuider | LTXDirector (single-stage) |
| Text encoder | GemmaAPITextEncode | LTXConditioning |
| Total nodes | 38 | 22 |
**Key question:** Can `LTXICLoRALoaderModelOnly` accept a model from UNETLoader (fp8 distilled transformer) instead of CheckpointLoaderSimple? The node takes a "model" input — compatibility with fp8 UNET-only model is unverified. Deep research dispatched 2026-07-22 to answer this and 11 other compatibility questions. Results pending at `/home/n8n/workspace/research/results/2026-07-22-ic-lora-fp8-compatibility.md`.
## What It Does
Conditions video generation on a **reference sheet** — a single composite image inventorying characters, props, and location. The model reads the reference latents in-context and renders a new clip whose characters, props, and setting match the sheet. This is the solution for using stock character sheets (like `SHEETS2_00005_Boss.png` on TrueNAS) as character consistency anchors.
## Recommended Settings (from official README)
| Setting | Value | Notes |
|---------|-------|-------|
| LoRA strength | **1.4** | NOT 1.0 — official recommendation |
| Inference steps | **30** | Higher than our standard 18 |
| Guidance scale | **4.0** | NOT 1.0 — video-to-video mode |
| Resolution | 768×448 | Trained bucket — best results here |
| Frames | 121 | 24 fps |
| Negative prompt | `worst quality, inconsistent motion, blurry, jittery, distorted` | |
| STG | mode `stg_v`, block 29, scale 1.0 | Helps motion stability |
## Prompt Format
Two-part structure (matching training):
```
Reference sheet: <description of the panels in the sheet — characters, props, location>
Generated video: <description of the action / shot you want generated>
```
The `Reference sheet:` text describes what's in the panels. The `Generated video:` text drives the action. The model reads the reference latents for "what things look like" and the prompt for "what happens."
## Control Signal Requirements
- Reference sheet: single composite image with one clean panel per visual element
- Each character: face close-up + body turnaround
- Each prop: product-style render
- One clean location panel
- Laid out on black background with NO text
- **Bigger panels carry over better** — give important elements larger panels
- Reference must be looped into a static video ≥ 121 frames at output resolution
## Node Chain (ComfyUI)
```
CheckpointLoaderSimple (full 46GB checkpoint — adapt to UNETLoader fp8 for 24GB)
→ LTXICLoRALoaderModelOnly (ingredients-0.9.safetensors, strength 1.4)
→ LTXAddVideoICLoRAGuide (reference sheet as static video input)
→ LTXVCropGuides → SamplerCustomAdvanced
```
## Nodes on .202
- `LTXICLoRALoaderModelOnly` — in `iclora.py`, loads IC-LoRA and extracts `latent_downscale_factor`
- `LTXAddVideoICLoRAGuide` — in `iclora.py`, applies reference video conditioning
- `LTXAddVideoICLoRAGuideAdvanced` — extended version with `attention_strength` and `attention_mask`
## Example Workflow
On .202: `~/comfy-ui/custom_nodes/ComfyUI-LTXVideo/example_workflows/2.3/LTX-2.3_ICLoRA_Ingredients_Single_Stage_Distilled.json`
Uses 39 nodes including: CheckpointLoaderSimple, LTXICLoRALoaderModelOnly, LTXAddVideoICLoRAGuide, GemmaAPITextEncode, LTXVCropGuides, SamplerCustomAdvanced, ManualSigmas.
## Download Process
1. Visit https://huggingface.co/Lightricks/LTX-2.3-22b-IC-LoRA-Ingredients
2. Click "Agree and Access" (gated model)
3. Download with HF token:
```bash
HF_TOKEN='hf_...' huggingface-cli download Lightricks/LTX-2.3-22b-IC-LoRA-Ingredients \
ltx-2.3-22b-ic-lora-ingredients-0.9.safetensors \
--local-dir ~/comfy-ui/models/loras/
```
4. Place in `~/comfy-ui/models/loras/` on .202
## Stock Integration
The boss stock on TrueNAS (`ai_vid_stock_material/character_refs/SHEETS2_00005_Boss.png`) is a character sheet — exactly the input format IC-LoRA Ingredients expects. To use it:
1. Download the character sheet from TrueNAS
2. Create a reference sheet image with the boss character panel(s)
3. Loop into a 121-frame static video at 768×448
4. Feed as the reference/control input to LTXAddVideoICLoRAGuide
## Tips (from README)
- **Bigger panels carry over better** — give important characters/props larger panels
- **Identity drift fix:** ensure clean front-facing close-up + full turnaround for each character
- **Element not appearing:** add a dedicated panel for any prop/character that needs to persist
- **Reference too short:** static video must be ≥ 121 frames
- **Element-driven reference-sheet generator** exists for authoring sheets (referenced in README)