100 lines
3.4 KiB
Markdown
100 lines
3.4 KiB
Markdown
|
# Reference Sheet Creation for IC-LoRA Ingredients
|
|||
|
|
|
|||
|
|
Proven workflow for creating IC-LoRA reference sheets from stock character images on TrueNAS. Used 2026-07-22 for the boss+woman cyberpunk test.
|
|||
|
|
|
|||
|
|
## Steps
|
|||
|
|
|
|||
|
|
### 1. Download stock from TrueNAS
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
smbclient -N //10.0.0.117/proxmoxBackup -c 'cd ai_vid_stock_material\\character_refs; get SHEETS2_00005_Boss.png /tmp/SHEETS2_00005_Boss.png'
|
|||
|
|
smbclient -N //10.0.0.117/proxmoxBackup -c 'cd ai_vid_stock_material\\character_refs; get cyberpunk_woman_neon_01.jpg /tmp/cyberpunk_woman_neon_01.jpg'
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 2. Extract panels from multi-panel character sheets
|
|||
|
|
|
|||
|
|
The boss stock (3328×2432) is a 2-panel sheet on a bright background. Use brightness thresholding to find content regions:
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
from PIL import Image
|
|||
|
|
import numpy as np
|
|||
|
|
|
|||
|
|
img = Image.open("/tmp/SHEETS2_00005_Boss.png")
|
|||
|
|
arr = np.array(img)
|
|||
|
|
brightness = arr.mean(axis=2)
|
|||
|
|
is_content = brightness < 200 # content is darker than bright background
|
|||
|
|
|
|||
|
|
# Find vertical regions (rows with >5% content pixels)
|
|||
|
|
row_content = is_content.mean(axis=1)
|
|||
|
|
content_rows = np.where(row_content > 0.05)[0]
|
|||
|
|
gaps = np.diff(content_rows)
|
|||
|
|
split_points = np.where(gaps > 20)[0] # gaps >20px = panel boundary
|
|||
|
|
|
|||
|
|
# Extract each panel with horizontal bounds
|
|||
|
|
regions = []
|
|||
|
|
start = content_rows[0]
|
|||
|
|
for sp in split_points:
|
|||
|
|
end = content_rows[sp]
|
|||
|
|
regions.append((start, end))
|
|||
|
|
start = content_rows[sp + 1]
|
|||
|
|
regions.append((start, content_rows[-1]))
|
|||
|
|
|
|||
|
|
for i, (y1, y2) in enumerate(regions):
|
|||
|
|
region = is_content[y1:y2+1, :]
|
|||
|
|
col_content = region.mean(axis=0)
|
|||
|
|
content_cols = np.where(col_content > 0.02)[0]
|
|||
|
|
x1, x2 = content_cols[0], content_cols[-1]
|
|||
|
|
crop = img.crop((x1, y1, x2, y2))
|
|||
|
|
crop.save(f"/tmp/boss_panel_{i}.png")
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 3. Composite reference sheet
|
|||
|
|
|
|||
|
|
IC-LoRA requires: 768×448, black background, one panel per character, NO text.
|
|||
|
|
|
|||
|
|
```python
|
|||
|
|
ref = Image.new("RGB", (768, 448), (0, 0, 0))
|
|||
|
|
|
|||
|
|
# Place boss panels (left side)
|
|||
|
|
bp0 = boss_panels[0].copy()
|
|||
|
|
bp0.thumbnail((300, 200), Image.LANCZOS)
|
|||
|
|
ref.paste(bp0, (10, 10))
|
|||
|
|
|
|||
|
|
bp1 = boss_panels[1].copy()
|
|||
|
|
bp1.thumbnail((300, 220), Image.LANCZOS)
|
|||
|
|
ref.paste(bp1, (10, 220))
|
|||
|
|
|
|||
|
|
# Place woman panels (right side)
|
|||
|
|
w1 = woman1.copy()
|
|||
|
|
w1.thumbnail((200, 200), Image.LANCZOS)
|
|||
|
|
ref.paste(w1, (330, 10))
|
|||
|
|
|
|||
|
|
w2 = woman2.copy()
|
|||
|
|
w2.thumbnail((200, 200), Image.LANCZOS)
|
|||
|
|
ref.paste(w2, (550, 10))
|
|||
|
|
|
|||
|
|
ref.save("/tmp/ic_lora_reference_sheet.png")
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 4. Loop to 121-frame static video
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
ffmpeg -y -loop 1 -i /tmp/ic_lora_reference_sheet.png \
|
|||
|
|
-c:v libx264 -t 5.04 -r 24 -pix_fmt yuv420p \
|
|||
|
|
/tmp/ic_lora_reference_121f.mp4
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
### 5. Copy to .202 for ComfyUI
|
|||
|
|
|
|||
|
|
```bash
|
|||
|
|
sshpass -p 'passw0rd' scp /tmp/ic_lora_reference_121f.mp4 [email protected]:~/comfy-ui/input/
|
|||
|
|
```
|
|||
|
|
|
|||
|
|
## Pitfalls
|
|||
|
|
|
|||
|
|
- **Bright backgrounds need thresholding.** The boss stock has a ~230 brightness background — content detection needs `brightness < 200`, not `< 30` (which is for black backgrounds).
|
|||
|
|
- **Panel gap detection is fragile.** The `gaps > 20` threshold works for the boss sheet but may need tuning for other sheets. Always inspect extracted panels before compositing.
|
|||
|
|
- **IC-LoRA requires black background.** The reference sheet MUST have a black background — bright backgrounds confuse the conditioning.
|
|||
|
|
- **Bigger panels = better carry-over.** Give important characters more space in the composite.
|
|||
|
|
- **Resolution must match trained bucket.** 768×448 is the IC-LoRA trained resolution. Other resolutions may work but are untested.
|