Add creative/comfyui v5.1.0 (official bundled) and media/ltx-video-pipeline v2.0.0 (custom LTX pipeline)

2026-07-22 16:17:52 -05:00
parent 7e81fdac31
commit d94132f4ca
60 changed files with 10421 additions and 0 deletions
@@ -0,0 +1,99 @@
+# Reference Sheet Creation for IC-LoRA Ingredients
+
+Proven workflow for creating IC-LoRA reference sheets from stock character images on TrueNAS. Used 2026-07-22 for the boss+woman cyberpunk test.
+
+## Steps
+
+### 1. Download stock from TrueNAS
+
+```bash
+smbclient -N //10.0.0.117/proxmoxBackup -c 'cd ai_vid_stock_material\\character_refs; get SHEETS2_00005_Boss.png /tmp/SHEETS2_00005_Boss.png'
+smbclient -N //10.0.0.117/proxmoxBackup -c 'cd ai_vid_stock_material\\character_refs; get cyberpunk_woman_neon_01.jpg /tmp/cyberpunk_woman_neon_01.jpg'
+```
+
+### 2. Extract panels from multi-panel character sheets
+
+The boss stock (3328×2432) is a 2-panel sheet on a bright background. Use brightness thresholding to find content regions:
+
+```python
+from PIL import Image
+import numpy as np
+
+img = Image.open("/tmp/SHEETS2_00005_Boss.png")
+arr = np.array(img)
+brightness = arr.mean(axis=2)
+is_content = brightness < 200  # content is darker than bright background
+
+# Find vertical regions (rows with >5% content pixels)
+row_content = is_content.mean(axis=1)
+content_rows = np.where(row_content > 0.05)[0]
+gaps = np.diff(content_rows)
+split_points = np.where(gaps > 20)[0]  # gaps >20px = panel boundary
+
+# Extract each panel with horizontal bounds
+regions = []
+start = content_rows[0]
+for sp in split_points:
+    end = content_rows[sp]
+    regions.append((start, end))
+    start = content_rows[sp + 1]
+regions.append((start, content_rows[-1]))
+
+for i, (y1, y2) in enumerate(regions):
+    region = is_content[y1:y2+1, :]
+    col_content = region.mean(axis=0)
+    content_cols = np.where(col_content > 0.02)[0]
+    x1, x2 = content_cols[0], content_cols[-1]
+    crop = img.crop((x1, y1, x2, y2))
+    crop.save(f"/tmp/boss_panel_{i}.png")
+```
+
+### 3. Composite reference sheet
+
+IC-LoRA requires: 768×448, black background, one panel per character, NO text.
+
+```python
+ref = Image.new("RGB", (768, 448), (0, 0, 0))
+
+# Place boss panels (left side)
+bp0 = boss_panels[0].copy()
+bp0.thumbnail((300, 200), Image.LANCZOS)
+ref.paste(bp0, (10, 10))
+
+bp1 = boss_panels[1].copy()
+bp1.thumbnail((300, 220), Image.LANCZOS)
+ref.paste(bp1, (10, 220))
+
+# Place woman panels (right side)
+w1 = woman1.copy()
+w1.thumbnail((200, 200), Image.LANCZOS)
+ref.paste(w1, (330, 10))
+
+w2 = woman2.copy()
+w2.thumbnail((200, 200), Image.LANCZOS)
+ref.paste(w2, (550, 10))
+
+ref.save("/tmp/ic_lora_reference_sheet.png")
+```
+
+### 4. Loop to 121-frame static video
+
+```bash
+ffmpeg -y -loop 1 -i /tmp/ic_lora_reference_sheet.png \
+  -c:v libx264 -t 5.04 -r 24 -pix_fmt yuv420p \
+  /tmp/ic_lora_reference_121f.mp4
+```
+
+### 5. Copy to .202 for ComfyUI
+
+```bash
+sshpass -p 'passw0rd' scp /tmp/ic_lora_reference_121f.mp4 [email protected]:~/comfy-ui/input/
+```
+
+## Pitfalls
+
+- **Bright backgrounds need thresholding.** The boss stock has a ~230 brightness background — content detection needs `brightness < 200`, not `< 30` (which is for black backgrounds).
+- **Panel gap detection is fragile.** The `gaps > 20` threshold works for the boss sheet but may need tuning for other sheets. Always inspect extracted panels before compositing.
+- **IC-LoRA requires black background.** The reference sheet MUST have a black background — bright backgrounds confuse the conditioning.
+- **Bigger panels = better carry-over.** Give important characters more space in the composite.
+- **Resolution must match trained bucket.** 768×448 is the IC-LoRA trained resolution. Other resolutions may work but are untested.