SD 1.xããã³2.xã®ã¢ãã«ãåœãªããžããªã§åŠç¿ããLoRAãControlNetïŒv1.0ã®ã¿åäœç¢ºèªïŒãªã©ã«å¯Ÿå¿ãããDiffusersããŒã¹ã®æšè«ïŒç»åçæïŒã¹ã¯ãªããã§ããã³ãã³ãã©ã€ã³ããçšããŸãã
- Diffusers (v0.10.2) ããŒã¹ã®æšè«ïŒç»åçæïŒã¹ã¯ãªããã
- SD 1.xããã³2.x (base/v-parameterization)ã¢ãã«ã«å¯Ÿå¿ã
- txt2imgãimg2imgãinpaintingã«å¯Ÿå¿ã
- 察話ã¢ãŒããããã³ãã¡ã€ã«ããã®ããã³ããèªã¿èŸŒã¿ãé£ç¶çæã«å¯Ÿå¿ã
- ããã³ãã1è¡ãããã®çæææ°ãæå®å¯èœã
- å šäœã®ç¹°ãè¿ãåæ°ãæå®å¯èœã
fp16ã ãã§ãªãbf16ã«ã察å¿ã- xformersã«å¯Ÿå¿ãé«éçæãå¯èœã
- xformersã«ããçã¡ã¢ãªçæãè¡ããŸãããAutomatic 1111æ°ã®Web UIã»ã©æé©åããŠããªãããã512*512ã®ç»åçæã§ãããã6GBçšåºŠã®VRAMã䜿çšããŸãã
- ããã³ããã®225ããŒã¯ã³ãžã®æ¡åŒµããã¬ãã£ãããã³ãããéã¿ã¥ãã«å¯Ÿå¿ã
- Diffusersã®åçš®samplerã«å¯Ÿå¿ïŒWeb UIãããsampleræ°ã¯å°ãªãã§ãïŒã
- Text Encoderã®clip skipïŒæåŸããnçªç®ã®å±€ã®åºåãçšããïŒã«å¯Ÿå¿ã
- VAEã®å¥éèªã¿èŸŒã¿ã
- CLIP Guided Stable DiffusionãVGG16 Guided Stable DiffusionãHighres. fixãupscale察å¿ã
- Highres. fixã¯Web UIã®å®è£ ãå šã確èªããŠããªãç¬èªå®è£ ã®ãããåºåçµæã¯ç°ãªããããããŸããã
- LoRA察å¿ãé©çšçæå®ãè€æ°LoRAåæå©çšãéã¿ã®ããŒãžã«å¯Ÿå¿ã
- Text EncoderãšU-Netã§å¥ã®é©çšçãæå®ããããšã¯ã§ããŸããã
- Attention Coupleã«å¯Ÿå¿ã
- ControlNet v1.0ã«å¯Ÿå¿ã
- éäžã§ã¢ãã«ãåãæ¿ããããšã¯ã§ããŸããããããããã¡ã€ã«ãçµãããšã§å¯Ÿå¿ã§ããŸãã
- å人çã«æ¬²ãããªã£ãæ©èœããããã远å ã
æ©èœè¿œå æã«ãã¹ãŠã®ãã¹ããè¡ã£ãŠããããã§ã¯ãªãããã以åã®æ©èœã«åœ±é¿ãåºãŠäžéšæ©èœãåããªãå¯èœæ§ããããŸããäœãåé¡ãããã°ãç¥ãããã ããã
以äžã®ããã«å ¥åããŠãã ããã
python gen_img_diffusers.py --ckpt <ã¢ãã«å> --outdir <ç»ååºåå
> --xformers --fp16 --interactive--ckptãªãã·ã§ã³ã«ã¢ãã«ïŒStable Diffusionã®checkpointãã¡ã€ã«ããŸãã¯Diffusersã®ã¢ãã«ãã©ã«ãïŒã--outdirãªãã·ã§ã³ã«ç»åã®åºåå
ãã©ã«ããæå®ããŸãã
--xformersãªãã·ã§ã³ã§xformersã®äœ¿çšãæå®ããŸãïŒxformersã䜿ããªãå Žåã¯å€ããŠãã ããïŒã--fp16ãªãã·ã§ã³ã§fp16ïŒå粟床ïŒã§ã®æšè«ãè¡ããŸããRTX 30ç³»ã®GPUã§ã¯ --bf16ãªãã·ã§ã³ã§bf16ïŒbfloat16ïŒã§ã®æšè«ãè¡ãããšãã§ããŸãã
--interactiveãªãã·ã§ã³ã§å¯Ÿè©±ã¢ãŒããæå®ããŠããŸãã
Stable Diffusion 2.0ïŒãŸãã¯ããããã®è¿œå åŠç¿ã¢ãã«ïŒã䜿ãå Žåã¯--v2ãªãã·ã§ã³ã远å ããŠãã ãããv-parameterizationã䜿ãã¢ãã«ïŒ768-v-ema.ckptããã³ããããã®è¿œå åŠç¿ã¢ãã«ïŒã䜿ãå Žåã¯ããã«--v_parameterizationã远å ããŠãã ããã
--v2ã®æå®æç¡ãééã£ãŠãããšã¢ãã«èªã¿èŸŒã¿æã«ãšã©ãŒã«ãªããŸãã--v_parameterizationã®æå®æç¡ãééã£ãŠãããšè¶è²ãç»åã衚瀺ãããŸãã
Type prompt:ãšè¡šç€ºããããããã³ãããå
¥åããŠãã ããã
â»ç»åã衚瀺ããããšã©ãŒã«ãªãå ŽåãheadlessïŒç»é¢è¡šç€ºæ©èœãªãïŒã®OpenCVãã€ã³ã¹ããŒã«ãããŠãããããããŸãããpip install opencv-pythonãšããŠéåžžã®OpenCVãå
¥ããŠãã ããããŸãã¯--no_previewãªãã·ã§ã³ã§ç»åè¡šç€ºãæ¢ããŠãã ããã
ç»åãŠã£ã³ããŠãéžæããŠããäœããã®ããŒãæŒããšãŠã£ã³ããŠãéããæ¬¡ã®ããã³ãããå ¥åã§ããŸããããã³ããã§Ctrl+Zããšã³ã¿ãŒã®é ã«æéµãããšã¹ã¯ãªãããéããŸãã
以äžã®ããã«å ¥åããŸãïŒå®éã«ã¯1è¡ã§å ¥åããŸãïŒã
python gen_img_diffusers.py --ckpt <ã¢ãã«å> --outdir <ç»ååºåå
>
--xformers --fp16 --images_per_prompt <çæææ°> --prompt "<ããã³ãã>"--images_per_promptãªãã·ã§ã³ã§ãããã³ãã1ä»¶åœããã®çæææ°ãæå®ããŸãã--promptãªãã·ã§ã³ã§ããã³ãããæå®ããŸããã¹ããŒã¹ãå«ãå Žåã¯ããã«ã¯ã©ãŒããŒã·ã§ã³ã§å²ãã§ãã ããã
--batch_sizeãªãã·ã§ã³ã§ããããµã€ãºãæå®ã§ããŸãïŒåŸè¿°ïŒã
以äžã®ããã«å ¥åããŸãã
python gen_img_diffusers.py --ckpt <ã¢ãã«å> --outdir <ç»ååºåå
>
--xformers --fp16 --from_file <ããã³ãããã¡ã€ã«å>--from_fileãªãã·ã§ã³ã§ãããã³ãããèšè¿°ããããã¡ã€ã«ãæå®ããŸãã1è¡1ããã³ããã§èšè¿°ããŠãã ããã--images_per_promptãªãã·ã§ã³ãæå®ããŠ1è¡ãããçæææ°ãæå®ã§ããŸãã
ããã³ãããªãã·ã§ã³ïŒããã³ããå
ã§--xã®ããã«æå®ãåŸè¿°ïŒã§--nãæžããšã以éããã¬ãã£ãããã³ãããšãªããŸãã
ãŸãAUTOMATIC1111æ°ã®Web UIãšåæ§ã® () ã [] ã(xxx:1.3) ãªã©ã«ããéã¿ã¥ããå¯èœã§ãïŒå®è£
ã¯Diffusersã®Long Prompt Weighting Stable Diffusionããã³ããŒãããã®ã§ãïŒã
ã³ãã³ãã©ã€ã³ããã®ããã³ããæå®ããã¡ã€ã«ããã®ããã³ããèªã¿èŸŒã¿ã§ãåæ§ã«æå®ã§ããŸãã
ã³ãã³ãã©ã€ã³ããæå®ããŠãã ããã
-
--ckpt <ã¢ãã«å>ïŒã¢ãã«åãæå®ããŸãã--ckptãªãã·ã§ã³ã¯å¿ é ã§ããStable Diffusionã®checkpointãã¡ã€ã«ããŸãã¯Diffusersã®ã¢ãã«ãã©ã«ããHugging Faceã®ã¢ãã«IDãæå®ã§ããŸãã -
--v2ïŒStable Diffusion 2.xç³»ã®ã¢ãã«ã䜿ãå Žåã«æå®ããŸãã1.xç³»ã®å Žåã«ã¯æå®äžèŠã§ãã -
--v_parameterizationïŒv-parameterizationã䜿ãã¢ãã«ã䜿ãå Žåã«æå®ããŸãïŒ768-v-ema.ckptããã³ããããã®è¿œå åŠç¿ã¢ãã«ãWaifu Diffusion v1.5ãªã©ïŒã--v2ã®æå®æç¡ãééã£ãŠãããšã¢ãã«èªã¿èŸŒã¿æã«ãšã©ãŒã«ãªããŸãã--v_parameterizationã®æå®æç¡ãééã£ãŠãããšè¶è²ãç»åã衚瀺ãããŸãã -
--vaeïŒäœ¿çšããVAEãæå®ããŸããæªæå®æã¯ã¢ãã«å ã®VAEã䜿çšããŸãã
-
--interactiveïŒã€ã³ã¿ã©ã¯ãã£ãã¢ãŒãã§åäœããŸããããã³ãããå ¥åãããšç»åãçæãããŸãã -
--prompt <ããã³ãã>ïŒããã³ãããæå®ããŸããã¹ããŒã¹ãå«ãå Žåã¯ããã«ã¯ã©ãŒããŒã·ã§ã³ã§å²ãã§ãã ããã -
--from_file <ããã³ãããã¡ã€ã«å>ïŒããã³ãããèšè¿°ããããã¡ã€ã«ãæå®ããŸãã1è¡1ããã³ããã§èšè¿°ããŠãã ããããªãç»åãµã€ãºãguidance scaleã¯ããã³ãããªãã·ã§ã³ïŒåŸè¿°ïŒã§æå®ã§ããŸãã -
--W <ç»åå¹ >ïŒç»åã®å¹ ãæå®ããŸããããã©ã«ãã¯512ã§ãã -
--H <ç»åé«ã>ïŒç»åã®é«ããæå®ããŸããããã©ã«ãã¯512ã§ãã -
--steps <ã¹ãããæ°>ïŒãµã³ããªã³ã°ã¹ãããæ°ãæå®ããŸããããã©ã«ãã¯50ã§ãã -
--scale <ã¬ã€ãã³ã¹ã¹ã±ãŒã«>ïŒunconditionalã¬ã€ãã³ã¹ã¹ã±ãŒã«ãæå®ããŸããããã©ã«ãã¯7.5ã§ãã -
--sampler <ãµã³ãã©ãŒå>ïŒãµã³ãã©ãŒãæå®ããŸããããã©ã«ãã¯ddimã§ããDiffusersã§æäŸãããŠããddimãpndmãdpmsolverãdpmsolver+++ãlmsãeulerãeuler_aããæå®å¯èœã§ãïŒåŸãã®äžã€ã¯k_lmsãk_eulerãk_euler_aã§ãæå®ã§ããŸãïŒã -
--outdir <ç»ååºåå ãã©ã«ã>ïŒç»åã®åºåå ãæå®ããŸãã -
--images_per_prompt <çæææ°>ïŒããã³ãã1ä»¶åœããã®çæææ°ãæå®ããŸããããã©ã«ãã¯1ã§ãã -
--clip_skip <ã¹ãããæ°>ïŒCLIPã®åŸãããäœçªç®ã®å±€ã䜿ãããæå®ããŸããçç¥æã¯æåŸã®å±€ã䜿ããŸãã -
--max_embeddings_multiples <åæ°>ïŒCLIPã®å ¥åºåé·ãããã©ã«ãïŒ75ïŒã®äœåã«ããããæå®ããŸããæªæå®æã¯75ã®ãŸãŸã§ããããšãã°3ãæå®ãããšå ¥åºåé·ã225ã«ãªããŸãã -
--negative_scale: uncoditioningã®guidance scaleãåå¥ã«æå®ããŸããgcem156æ°ã®ãã¡ãã®èšäºãåèã«å®è£ ãããã®ã§ãã
-
--batch_size <ããããµã€ãº>ïŒããããµã€ãºãæå®ããŸããããã©ã«ãã¯1ã§ããããããµã€ãºã倧ãããšã¡ã¢ãªãå€ãæ¶è²»ããŸãããçæé床ãéããªããŸãã -
--vae_batch_size <VAEã®ããããµã€ãº>ïŒVAEã®ããããµã€ãºãæå®ããŸããããã©ã«ãã¯ããããµã€ãºãšåãã§ãã VAEã®ã»ããã¡ã¢ãªãå€ãæ¶è²»ãããããããã€ãžã³ã°åŸïŒstepã100%ã«ãªã£ãåŸïŒã§ã¡ã¢ãªäžè¶³ã«ãªãå ŽåããããŸãããã®ãããªå Žåã«ã¯VAEã®ããããµã€ãºãå°ããããŠãã ããã -
--xformersïŒxformersã䜿ãå Žåã«æå®ããŸãã -
--fp16ïŒfp16ïŒå粟床ïŒã§ã®æšè«ãè¡ããŸããfp16ãšbf16ãã©ã¡ããæå®ããªãå Žåã¯fp32ïŒå粟床ïŒã§ã®æšè«ãè¡ããŸãã -
--bf16ïŒbf16ïŒbfloat16ïŒã§ã®æšè«ãè¡ããŸããRTX 30ç³»ã®GPUã§ã®ã¿æå®å¯èœã§ãã--bf16ãªãã·ã§ã³ã¯RTX 30系以å€ã®GPUã§ã¯ãšã©ãŒã«ãªããŸããfp16ãããbf16ã®ã»ããæšè«çµæãNaNã«ãªãïŒçã£é»ã®ç»åã«ãªãïŒå¯èœæ§ãäœãããã§ãã
-
--network_moduleïŒäœ¿çšãã远å ãããã¯ãŒã¯ãæå®ããŸããLoRAã®å Žåã¯--network_module networks.loraãšæå®ããŸããè€æ°ã®LoRAã䜿çšããå Žåã¯--network_module networks.lora networks.lora networks.loraã®ããã«æå®ããŸãã -
--network_weightsïŒäœ¿çšãã远å ãããã¯ãŒã¯ã®éã¿ãã¡ã€ã«ãæå®ããŸãã--network_weights model.safetensorsã®ããã«æå®ããŸããè€æ°ã®LoRAã䜿çšããå Žåã¯--network_weights model1.safetensors model2.safetensors model3.safetensorsã®ããã«æå®ããŸããåŒæ°ã®æ°ã¯--network_moduleã§æå®ããæ°ãšåãã«ããŠãã ããã -
--network_mulïŒäœ¿çšãã远å ãããã¯ãŒã¯ã®éã¿ãäœåã«ããããæå®ããŸããããã©ã«ãã¯1ã§ãã--network_mul 0.8ã®ããã«æå®ããŸããè€æ°ã®LoRAã䜿çšããå Žåã¯--network_mul 0.4 0.5 0.7ã®ããã«æå®ããŸããåŒæ°ã®æ°ã¯--network_moduleã§æå®ããæ°ãšåãã«ããŠãã ããã -
--network_mergeïŒäœ¿çšãã远å ãããã¯ãŒã¯ã®éã¿ã--network_mulã«æå®ããéã¿ã§ãããããããŒãžããŸãã--network_pre_calcãšåæã«äœ¿çšã§ããŸãããããã³ãããªãã·ã§ã³ã®--amãããã³Regional LoRAã¯äœ¿çšã§ããªããªããŸãããLoRAæªäœ¿çšæãšåãçšåºŠãŸã§çæãé«éåãããŸãã -
--network_pre_calcïŒäœ¿çšãã远å ãããã¯ãŒã¯ã®éã¿ãçæããšã«ãããããèšç®ããŸããããã³ãããªãã·ã§ã³ã®--amã䜿çšã§ããŸããLoRAæªäœ¿çšæãšåãçšåºŠãŸã§çæã¯é«éåãããŸãããçæåã«éã¿ãèšç®ããæéãå¿ èŠã§ããŸãã¡ã¢ãªäœ¿çšéãè¥å¹²å¢å ããŸããRegional LoRAäœ¿çšæã¯ç¡å¹ã«ãªããŸã ã
次ã¯åäžããã³ããã§64æãããããµã€ãº4ã§äžæ¬çæããäŸã§ãã
python gen_img_diffusers.py --ckpt model.ckpt --outdir outputs
--xformers --fp16 --W 512 --H 704 --scale 12.5 --sampler k_euler_a
--steps 32 --batch_size 4 --images_per_prompt 64
--prompt "beautiful flowers --n monochrome"次ã¯ãã¡ã€ã«ã«æžãããããã³ãããããããã10æãã€ãããããµã€ãº4ã§äžæ¬çæããäŸã§ãã
python gen_img_diffusers.py --ckpt model.ckpt --outdir outputs
--xformers --fp16 --W 512 --H 704 --scale 12.5 --sampler k_euler_a
--steps 32 --batch_size 4 --images_per_prompt 10
--from_file prompts.txtTextual InversionïŒåŸè¿°ïŒããã³LoRAã®äœ¿çšäŸã§ãã
python gen_img_diffusers.py --ckpt model.safetensors
--scale 8 --steps 48 --outdir txt2img --xformers
--W 512 --H 768 --fp16 --sampler k_euler_a
--textual_inversion_embeddings goodembed.safetensors negprompt.pt
--network_module networks.lora networks.lora
--network_weights model1.safetensors model2.safetensors
--network_mul 0.4 0.8
--clip_skip 2 --max_embeddings_multiples 1
--batch_size 8 --images_per_prompt 1 --interactiveããã³ããå
ã§ã--nã®ããã«ããã€ãã³ãµãã€+ã¢ã«ãã¡ãããnæåãã§ããã³ããããåçš®ãªãã·ã§ã³ã®æå®ãå¯èœã§ãã察話ã¢ãŒããã³ãã³ãã©ã€ã³ããã¡ã€ã«ããããããããã³ãããæå®ããå Žåã§ãæå¹ã§ãã
ããã³ããã®ãªãã·ã§ã³æå®--nã®ååŸã«ã¯ã¹ããŒã¹ãå
¥ããŠãã ããã
-
--nïŒãã¬ãã£ãããã³ãããæå®ããŸãã -
--wïŒç»åå¹ ãæå®ããŸããã³ãã³ãã©ã€ã³ããã®æå®ãäžæžãããŸãã -
--hïŒç»åé«ããæå®ããŸããã³ãã³ãã©ã€ã³ããã®æå®ãäžæžãããŸãã -
--sïŒã¹ãããæ°ãæå®ããŸããã³ãã³ãã©ã€ã³ããã®æå®ãäžæžãããŸãã -
--dïŒãã®ç»åã®ä¹±æ°seedãæå®ããŸãã--images_per_promptãæå®ããŠããå Žåã¯ã--d 1,2,3,4ãã®ããã«ã«ã³ãåºåãã§è€æ°æå®ããŠãã ããã â»æ§ã ãªçç±ã«ãããWeb UIãšã¯åãä¹±æ°seedã§ãçæãããç»åãç°ãªãå ŽåããããŸãã -
--lïŒguidance scaleãæå®ããŸããã³ãã³ãã©ã€ã³ããã®æå®ãäžæžãããŸãã -
--tïŒimg2imgïŒåŸè¿°ïŒã®strengthãæå®ããŸããã³ãã³ãã©ã€ã³ããã®æå®ãäžæžãããŸãã -
--nlïŒãã¬ãã£ãããã³ããã®guidance scaleãæå®ããŸãïŒåŸè¿°ïŒãã³ãã³ãã©ã€ã³ããã®æå®ãäžæžãããŸãã -
--amïŒè¿œå ãããã¯ãŒã¯ã®éã¿ãæå®ããŸããã³ãã³ãã©ã€ã³ããã®æå®ãäžæžãããŸããè€æ°ã®è¿œå ãããã¯ãŒã¯ã䜿çšããå Žåã¯--am 0.8,0.5,0.3ã®ããã« ã«ã³ãåºåãã§ æå®ããŸãã
â»ãããã®ãªãã·ã§ã³ãæå®ãããšãããããµã€ãºãããå°ãããµã€ãºã§ããããå®è¡ãããå ŽåããããŸãïŒãããã®å€ãç°ãªããšäžæ¬çæã§ããªãããïŒãïŒããŸãæ°ã«ããªããŠå€§äžå€«ã§ããããã¡ã€ã«ããããã³ãããèªã¿èŸŒã¿çæããå Žåã¯ããããã®å€ãåäžã®ããã³ããã䞊ã¹ãŠãããšå¹çãè¯ããªããŸããïŒ
äŸïŒ
(masterpiece, best quality), 1girl, in shirt and plated skirt, standing at street under cherry blossoms, upper body, [from below], kind smile, looking at another, [goodembed] --n realistic, real life, (negprompt), (lowres:1.1), (worst quality:1.2), (low quality:1.1), bad anatomy, bad hands, text, error, missing fingers, extra digit, fewer digits, cropped, normal quality, jpeg artifacts, signature, watermark, username, blurry --w 960 --h 640 --s 28 --d 1
-
--image_pathïŒimg2imgã«å©çšããç»åãæå®ããŸãã--image_path template.pngã®ããã«æå®ããŸãããã©ã«ããæå®ãããšããã®ãã©ã«ãã®ç»åãé æ¬¡å©çšããŸãã -
--strengthïŒimg2imgã®strengthãæå®ããŸãã--strength 0.8ã®ããã«æå®ããŸããããã©ã«ãã¯0.8ã§ãã -
--sequential_file_nameïŒãã¡ã€ã«åãé£çªã«ãããã©ãããæå®ããŸããæå®ãããšçæããããã¡ã€ã«åãim_000001.pngããã®é£çªã«ãªããŸãã -
--use_original_file_nameïŒæå®ãããšçæãã¡ã€ã«åããªãªãžãã«ã®ãã¡ã€ã«åãšåãã«ãªããŸãã
python gen_img_diffusers.py --ckpt trinart_characters_it4_v1_vae_merged.ckpt
--outdir outputs --xformers --fp16 --scale 12.5 --sampler k_euler --steps 32
--image_path template.png --strength 0.8
--prompt "1girl, cowboy shot, brown hair, pony tail, brown eyes,
sailor school uniform, outdoors
--n lowres, bad anatomy, bad hands, error, missing fingers, cropped,
worst quality, low quality, normal quality, jpeg artifacts, (blurry),
hair ornament, glasses"
--batch_size 8 --images_per_prompt 32--image_pathãªãã·ã§ã³ã«ãã©ã«ããæå®ãããšããã®ãã©ã«ãã®ç»åãé æ¬¡èªã¿èŸŒã¿ãŸããçæãããææ°ã¯ç»åææ°ã§ã¯ãªããããã³ããæ°ã«ãªããŸãã®ã§ã`--images_per_promptPPãªãã·ã§ã³ãæå®ããŠimg2imgããç»åã®ææ°ãšããã³ããæ°ãåãããŠãã ããã
ãã¡ã€ã«ã¯ãã¡ã€ã«åã§ãœãŒãããŠèªã¿èŸŒã¿ãŸãããªããœãŒãé ã¯æååé ãšãªããŸãã®ã§ïŒ1.jpgâ2.jpgâ10.jpgã§ã¯ãªã1.jpgâ10.jpgâ2.jpgã®é ïŒãé ã0åããããªã©ããŠã察å¿ãã ããïŒ01.jpgâ02.jpgâ10.jpgïŒã
img2imgæã«ã³ãã³ãã©ã€ã³ãªãã·ã§ã³ã®--Wãš--Hã§çæç»åãµã€ãºãæå®ãããšãå
ç»åããã®ãµã€ãºã«ãªãµã€ãºããŠããimg2imgãè¡ããŸãã
ãŸãimg2imgã®å ç»åããã®ã¹ã¯ãªããã§çæããç»åã®å Žåãããã³ãããçç¥ãããšãå ç»åã®ã¡ã¿ããŒã¿ããããã³ãããååŸããã®ãŸãŸçšããŸããããã«ããHighres. fixã®2nd stageã®åäœã ããè¡ãããšãã§ããŸãã
ç»åããã³ãã¹ã¯ç»åãæå®ããŠinpaintingã§ããŸãïŒinpaintingã¢ãã«ã«ã¯å¯Ÿå¿ããŠããããåã«ãã¹ã¯é åã察象ã«img2imgããã ãã§ãïŒã
ãªãã·ã§ã³ã¯ä»¥äžã®éãã§ãã
--mask_imageïŒãã¹ã¯ç»åãæå®ããŸãã--img_pathãšåæ§ã«ãã©ã«ããæå®ãããšããã®ãã©ã«ãã®ç»åãé æ¬¡å©çšããŸãã
ãã¹ã¯ç»åã¯ã°ã¬ãŒã¹ã±ãŒã«ç»åã§ãçœã®éšåãinpaintingãããŸããå¢çãã°ã©ããŒã·ã§ã³ããŠãããšãªããšãªãæ»ããã«ãªããŸãã®ã§ãå§ãã§ãã
--textual_inversion_embeddingsãªãã·ã§ã³ã§äœ¿çšããembeddingsãæå®ããŸãïŒè€æ°æå®å¯ïŒãæ¡åŒµåãé€ãããã¡ã€ã«åãããã³ããå
ã§äœ¿çšããããšã§ããã®embeddingsãå©çšããŸãïŒWeb UIãšåæ§ã®äœ¿çšæ³ã§ãïŒããã¬ãã£ãããã³ããå
ã§ã䜿çšã§ããŸãã
ã¢ãã«ãšããŠãåœãªããžããªã§åŠç¿ããTextual Inversionã¢ãã«ãããã³Web UIã§åŠç¿ããTextual Inversionã¢ãã«ïŒç»ååã蟌ã¿ã¯é察å¿ïŒãå©çšã§ããŸã
--textual_inversion_embeddingsã®ä»£ããã«--XTI_embeddingsãªãã·ã§ã³ãæå®ããŠãã ãããäœ¿çšæ³ã¯--textual_inversion_embeddingsãšåãã§ãã
AUTOMATIC1111æ°ã®Web UIã«ããæ©èœã®é¡äŒŒæ©èœã§ãïŒç¬èªå®è£ ã®ããããããããããããç°ãªããããããŸããïŒãæåã«å°ããã®ç»åãçæãããã®ç»åãå ã«img2imgããããšã§ãç»åå šäœã®ç Žç¶»ãé²ãã€ã€å€§ããªè§£å床ã®ç»åãçæããŸãã
2nd stageã®stepæ°ã¯--steps ãš--strengthãªãã·ã§ã³ã®å€ããèšç®ãããŸãïŒsteps*strengthïŒã
img2imgãšäœµçšã§ããŸããã
以äžã®ãªãã·ã§ã³ããããŸãã
-
--highres_fix_scaleïŒHighres. fixãæå¹ã«ããŠã1st stageã§çæããç»åã®ãµã€ãºããåçã§æå®ããŸããæçµåºåã1024x1024ã§ãæåã«512x512ã®ç»åãçæããå Žåã¯--highres_fix_scale 0.5ã®ããã«æå®ããŸããWeb UIåºã®æå®ã®éæ°ã«ãªã£ãŠããŸãã®ã§ã泚æãã ããã -
--highres_fix_stepsïŒ1st stageã®ç»åã®ã¹ãããæ°ãæå®ããŸããããã©ã«ãã¯28ã§ãã -
--highres_fix_save_1stïŒ1st stageã®ç»åãä¿åãããã©ãããæå®ããŸãã -
--highres_fix_latents_upscalingïŒæå®ãããš2nd stageã®ç»åçææã«1st stageã®ç»åãlatentããŒã¹ã§upscalingããŸãïŒbilinearã®ã¿å¯Ÿå¿ïŒãæªæå®æã¯ç»åãLANCZOS4ã§upscalingããŸãã -
--highres_fix_upscalerïŒ2nd stageã«ä»»æã®upscalerãå©çšããŸããçŸåšã¯--highres_fix_upscaler tools.latent_upscalerã®ã¿å¯Ÿå¿ããŠããŸãã -
--highres_fix_upscaler_argsïŒ--highres_fix_upscalerã§æå®ããupscalerã«æž¡ãåŒæ°ãæå®ããŸããtools.latent_upscalerã®å Žåã¯ã--highres_fix_upscaler_args "weights=D:\Work\SD\Models\others\etc\upscaler-v1-e100-220.safetensors"ã®ããã«éã¿ãã¡ã€ã«ãæå®ããŸãã
ã³ãã³ãã©ã€ã³ã®äŸã§ãã
python gen_img_diffusers.py --ckpt trinart_characters_it4_v1_vae_merged.ckpt
--n_iter 1 --scale 7.5 --W 1024 --H 1024 --batch_size 1 --outdir ../txt2img
--steps 48 --sampler ddim --fp16
--xformers
--images_per_prompt 1 --interactive
--highres_fix_scale 0.5 --highres_fix_steps 28 --strength 0.5çŸåšã¯ControlNet 1.0ã®ã¿åäœç¢ºèªããŠããŸããããªããã»ã¹ã¯Cannyã®ã¿ãµããŒãããŠããŸãã
以äžã®ãªãã·ã§ã³ããããŸãã
-
--control_net_modelsïŒControlNetã®ã¢ãã«ãã¡ã€ã«ãæå®ããŸãã è€æ°æå®ãããšãããããstepããšã«åãæ¿ããŠå©çšããŸãïŒWeb UIã®ControlNetæ¡åŒµã®å®è£ ãšç°ãªããŸãïŒãdiffãšéåžžã®äž¡æ¹ããµããŒãããŸãã -
--guide_image_pathïŒControlNetã«äœ¿ããã³ãç»åãæå®ããŸãã--img_pathãšåæ§ã«ãã©ã«ããæå®ãããšããã®ãã©ã«ãã®ç»åãé æ¬¡å©çšããŸããCanny以å€ã®ã¢ãã«ã®å Žåã«ã¯ããããããããªããã»ã¹ãè¡ã£ãŠãããŠãã ããã -
--control_net_prepsïŒControlNetã®ããªããã»ã¹ãæå®ããŸãã--control_net_modelsãšåæ§ã«è€æ°æå®å¯èœã§ããçŸåšã¯cannyã®ã¿å¯Ÿå¿ããŠããŸãã察象ã¢ãã«ã§ããªããã»ã¹ã䜿çšããªãå Žåã¯noneãæå®ããŸãã cannyã®å Žå--control_net_preps canny_63_191ã®ããã«ãéŸå€1ãš2ã'_'ã§åºåã£ãŠæå®ã§ããŸãã -
--control_net_weightsïŒControlNetã®é©çšæã®éã¿ãæå®ããŸãïŒ1.0ã§éåžžã0.5ãªãååã®åœ±é¿åã§é©çšïŒã--control_net_modelsãšåæ§ã«è€æ°æå®å¯èœã§ãã -
--control_net_ratiosïŒControlNetãé©çšããstepã®ç¯å²ãæå®ããŸãã0.5ã®å Žåã¯ãstepæ°ã®ååãŸã§ControlNetãé©çšããŸãã--control_net_modelsãšåæ§ã«è€æ°æå®å¯èœã§ãã
ã³ãã³ãã©ã€ã³ã®äŸã§ãã
python gen_img_diffusers.py --ckpt model_ckpt --scale 8 --steps 48 --outdir txt2img --xformers
--W 512 --H 768 --bf16 --sampler k_euler_a
--control_net_models diff_control_sd15_canny.safetensors --control_net_weights 1.0
--guide_image_path guide.png --control_net_ratios 1.0 --interactiveããã³ãããããã€ãã®éšåã«åå²ããããããã®ããã³ãããç»åå
ã®ã©ã®é åã«é©çšããããæå®ã§ããæ©èœã§ããåå¥ã®ãªãã·ã§ã³ã¯ãããŸããããmask_pathãšããã³ããã§æå®ããŸãã
ãŸããããã³ããã§ANDãå©çšããŠãè€æ°éšåãå®çŸ©ããŸããæåã®3ã€ã«å¯ŸããŠé åæå®ãã§ãã以éã®éšåã¯ç»åå
šäœãžé©çšãããŸãããã¬ãã£ãããã³ããã¯ç»åå
šäœã«é©çšãããŸãã
以äžã§ã¯ANDã§3ã€ã®éšåãå®çŸ©ããŠããŸãã
shs 2girls, looking at viewer, smile AND bsb 2girls, looking back AND 2girls --n bad quality, worst quality
次ã«ãã¹ã¯ç»åãçšæããŸãããã¹ã¯ç»åã¯ã«ã©ãŒã®ç»åã§ãRGBã®åãã£ãã«ãããã³ããã®ANDã§åºåãããéšåã«å¯Ÿå¿ããŸãããŸããããã£ãã«ã®å€ããã¹ãŠ0ã®å Žåãç»åå šäœã«é©çšãããŸãã
äžèšã®äŸã§ã¯ãRãã£ãã«ãshs 2girls, looking at viewer, smileãGãã£ãã«ãbsb 2girls, looking backã«ãBãã£ãã«ã2girlsã«å¯Ÿå¿ããŸããæ¬¡ã®ãããªãã¹ã¯ç»åã䜿çšãããšãBãã£ãã«ã«æå®ããããŸããã®ã§ã2girlsã¯ç»åå
šäœã«é©çšãããŸãã
ãã¹ã¯ç»åã¯--mask_pathã§æå®ããŸããçŸåšã¯1æã®ã¿å¯Ÿå¿ããŠããŸããæå®ããç»åãµã€ãºã«èªåçã«ãªãµã€ãºããé©çšãããŸãã
ControlNetãšçµã¿åãããããšãå¯èœã§ãïŒçްããäœçœ®æå®ã«ã¯ControlNetãšã®çµã¿åãããæšå¥šããŸãïŒã
LoRAãæå®ãããšã--network_weightsã§æå®ããè€æ°ã®LoRAãããããANDã®åéšåã«å¯Ÿå¿ããŸããçŸåšã®å¶çŽãšããŠãLoRAã®æ°ã¯ANDã®éšåã®æ°ãšåãã§ããå¿
èŠããããŸãã
Diffusersã®Community Examplesã®ãã¡ãã®custom pipelineãããœãŒã¹ãã³ããŒã倿Žãããã®ã§ãã
éåžžã®ããã³ããã«ããçææå®ã«å ããŠã远å ã§ããå€§èŠæš¡ã®CLIPã§ããã³ããã®ããã¹ãã®ç¹åŸŽéãååŸããçæäžã®ç»åã®ç¹åŸŽéããã®ããã¹ãã®ç¹åŸŽéã«è¿ã¥ããããçæãããç»åãã³ã³ãããŒã«ããŸãïŒç§ã®ãã£ãããšããçè§£ã§ãïŒã倧ããã®CLIPã䜿ããŸãã®ã§VRAM䜿çšéã¯ããªãå¢å ãïŒVRAM 8GBã§ã¯512*512ã§ãå³ãããããããŸããïŒãçææéãæãããŸãã
ãªãéžæã§ãããµã³ãã©ãŒã¯DDIMãPNDMãLMSã®ã¿ãšãªããŸãã
--clip_guidance_scaleãªãã·ã§ã³ã«ã©ã®çšåºŠãCLIPã®ç¹åŸŽéãåæ ããããæ°å€ã§æå®ããŸããå
ã®ãµã³ãã«ã§ã¯100ã«ãªã£ãŠããŸãã®ã§ããã®ãããããå§ããŠå¢æžãããšè¯ãããã§ãã
ããã©ã«ãã§ã¯ããã³ããã®å
é 75ããŒã¯ã³ïŒéã¿ã¥ãã®ç¹æ®æåãé€ãïŒãCLIPã«æž¡ãããŸããããã³ããã®--cãªãã·ã§ã³ã§ãéåžžã®ããã³ããã§ã¯ãªããCLIPã«æž¡ãããã¹ããå¥ã«æå®ã§ããŸãïŒããšãã°CLIPã¯DreamBoothã®identifierïŒèå¥åïŒãã1girlããªã©ã®ã¢ãã«ç¹æã®åèªã¯èªèã§ããªããšæãããŸãã®ã§ãããããçããããã¹ããè¯ããšæãããŸãïŒã
ã³ãã³ãã©ã€ã³ã®äŸã§ãã
python gen_img_diffusers.py --ckpt v1-5-pruned-emaonly.ckpt --n_iter 1
--scale 2.5 --W 512 --H 512 --batch_size 1 --outdir ../txt2img --steps 36
--sampler ddim --fp16 --opt_channels_last --xformers --images_per_prompt 1
--interactive --clip_guidance_scale 100ããã¹ãã§ã¯ãªãCLIPã«å¥ã®ç»åãæž¡ãããã®ç¹åŸŽéã«è¿ã¥ãããçæãã³ã³ãããŒã«ããæ©èœã§ãã--clip_image_guidance_scaleãªãã·ã§ã³ã§é©çšéã®æ°å€ãã--guide_image_pathãªãã·ã§ã³ã§guideã«äœ¿çšããç»åïŒãã¡ã€ã«ãŸãã¯ãã©ã«ãïŒãæå®ããŠãã ããã
ã³ãã³ãã©ã€ã³ã®äŸã§ãã
python gen_img_diffusers.py --ckpt trinart_characters_it4_v1_vae_merged.ckpt
--n_iter 1 --scale 7.5 --W 512 --H 512 --batch_size 1 --outdir ../txt2img
--steps 80 --sampler ddim --fp16 --opt_channels_last --xformers
--images_per_prompt 1 --interactive --clip_image_guidance_scale 100
--guide_image_path YUKA160113420I9A4104_TP_V.jpgæå®ããç»åã«è¿ã¥ãããã«ç»åçæããæ©èœã§ããéåžžã®ããã³ããã«ããçææå®ã«å ããŠã远å ã§VGG16ã®ç¹åŸŽéãååŸããçæäžã®ç»åãæå®ããã¬ã€ãç»åã«è¿ã¥ããããçæãããç»åãã³ã³ãããŒã«ããŸããimg2imgã§ã®äœ¿çšããå§ãããŸãïŒéåžžã®çæã§ã¯ç»åããŒãããæãã«ãªããŸãïŒãCLIP Guided Stable Diffusionã®ä»çµã¿ãæµçšããç¬èªã®æ©èœã§ãããŸãã¢ã€ãã¢ã¯VGGãå©çšããã¹ã¿ã€ã«å€æããæåããŠããŸãã
ãªãéžæã§ãããµã³ãã©ãŒã¯DDIMãPNDMãLMSã®ã¿ãšãªããŸãã
--vgg16_guidance_scaleãªãã·ã§ã³ã«ã©ã®çšåºŠãVGG16ç¹åŸŽéãåæ ããããæ°å€ã§æå®ããŸãã詊ããæãã§ã¯100ãããããå§ããŠå¢æžãããšè¯ãããã§ãã--guide_image_pathãªãã·ã§ã³ã§guideã«äœ¿çšããç»åïŒãã¡ã€ã«ãŸãã¯ãã©ã«ãïŒãæå®ããŠãã ããã
è€æ°æã®ç»åãäžæ¬ã§img2img倿ããå
ç»åãã¬ã€ãç»åãšããå Žåã--guide_image_pathãš--image_pathã«åãå€ãæå®ããã°OKã§ãã
ã³ãã³ãã©ã€ã³ã®äŸã§ãã
python gen_img_diffusers.py --ckpt wd-v1-3-full-pruned-half.ckpt
--n_iter 1 --scale 5.5 --steps 60 --outdir ../txt2img
--xformers --sampler ddim --fp16 --W 512 --H 704
--batch_size 1 --images_per_prompt 1
--prompt "picturesque, 1girl, solo, anime face, skirt, beautiful face
--n lowres, bad anatomy, bad hands, error, missing fingers,
cropped, worst quality, low quality, normal quality,
jpeg artifacts, blurry, 3d, bad face, monochrome --d 1"
--strength 0.8 --image_path ..\src_image
--vgg16_guidance_scale 100 --guide_image_path ..\src_image `--vgg16_guidance_layerPã§ç¹åŸŽéååŸã«äœ¿çšããVGG16ã®ã¬ã€ã€ãŒçªå·ãæå®ã§ããŸãïŒããã©ã«ãã¯20ã§conv4-2ã®ReLUã§ãïŒãäžã®å±€ã»ã©ç»é¢šã衚çŸããäžã®å±€ã»ã©ã³ã³ãã³ãã衚çŸãããšããããŠããŸãã
-
--no_preview: 察話ã¢ãŒãã§ãã¬ãã¥ãŒç»åã衚瀺ããŸãããOpenCVãã€ã³ã¹ããŒã«ãããŠããªãå Žåããåºåããããã¡ã€ã«ãçŽæ¥ç¢ºèªããå Žåã«æå®ããŠãã ããã -
--n_iter: çæãç¹°ãè¿ãåæ°ãæå®ããŸããããã©ã«ãã¯1ã§ããããã³ããããã¡ã€ã«ããèªã¿èŸŒããšããè€æ°åã®çæãè¡ãããå Žåã«æå®ããŸãã -
--tokenizer_cache_dir: ããŒã¯ãã€ã¶ãŒã®ãã£ãã·ã¥ãã£ã¬ã¯ããªãæå®ããŸããïŒäœæ¥äžïŒ -
--seed: ä¹±æ°seedãæå®ããŸãã1æçææã¯ãã®ç»åã®seedãè€æ°æçææã¯åç»åã®seedãçæããããã®ä¹±æ°ã®seedã«ãªããŸãïŒ--from_fileã§è€æ°ç»åçæãããšãã--seedãªãã·ã§ã³ãæå®ãããšè€æ°åå®è¡ãããšãã«åç»åãåãseedã«ãªããŸãïŒã -
--iter_same_seed: ããã³ããã«ä¹±æ°seedã®æå®ããªããšãã--n_iterã®ç¹°ãè¿ãå ã§ã¯ãã¹ãŠåãseedã䜿ããŸãã--from_fileã§æå®ããè€æ°ã®ããã³ããéã§seedãçµ±äžããŠæ¯èŒãããšãã«äœ¿ããŸãã -
--diffusers_xformers: Diffuserã®xformersã䜿çšããŸãã -
--opt_channels_last: æšè«æã«ãã³ãœã«ã®ãã£ã³ãã«ãæåŸã«é 眮ããŸããå Žåã«ãã£ãŠã¯é«éåãããããšããããŸãã -
--network_show_meta: 远å ãããã¯ãŒã¯ã®ã¡ã¿ããŒã¿ã衚瀺ããŸãã





